Connect with us

IOT

A Raspberry Pi as a decent residential proxy #RaspberryPi #Networking #PiDay @Raspberry_Pi

Avatar

Published

on

A great use for a Raspberry Pi as a residential proxy server from WiringBits:

One of our projects uses web-scraping to scan several online stores to find discounts. Lately, we started supporting some stores that seem to block requests coming from common cloud providers (like AWS, DigitalOcean, etc), if you are curious, the websites are BestBuy and Costco Mexico.

A popular workaround to mitigate this problem is to pay for a proxy service to scrape these websites, sadly, we weren’t able to find a reliable provider that was within our small budget.

Hence, we ended up building our own residential proxy, right now being powered by an old Raspberry Pi model B. It’s worth adding that it wasn’t as simple as we expected, specially keeping the SSH tunnel available.

The code has been open sourced on GitHub.

I ended up investing more time than expected tweaking the necessary stuff to keep the proxy working reliable, the biggest problem was related the SSH tunnel.

If you see the actual project, it includes a systemd service to keep the tunnel opened with the necessary tweaks.

The tunnel command being:

  • /usr/bin/ssh -nNT -R 9999:localhost:9000 -o ConnectTimeout=10 -o ExitOnForwardFailure=yes -o ServerAliveInterval=180 ubuntu@cazadescuentos.net

What matter the most:

  • ExitOnForwardFailure=yes forces ssh to exit when there is a failure in the connection instead of silently staying running while the tunnel doesn’t work.
  • ServerAliveInterval=180 keeps sending a ping to the server to avoid the server closing the connection due to inactivity.

See the blog post for all the details. Nice work.

Simple proxy flow

Source: https://blog.adafruit.com/2020/06/12/a-raspberry-pi-as-a-decent-residential-proxy-raspberrypi-networking-piday-raspberry_pi/

IOT

Avatar

Published

on

Continue Reading

IOT

Avatar

Published

on

Continue Reading

IOT

Avatar

Published

on

Continue Reading

IOT

Avatar

Published

on

Continue Reading
Esports5 days ago

Free Fire World Series APK Download for Android

Esports3 days ago

C9 White Keiti Blackmail Scandal Explains Sudden Dismissal

Esports3 days ago

Overwatch League 2021 Day 1 Recap

Esports5 days ago

Dota 2: Top Mid Heroes of Patch 7.29

Esports3 days ago

Fortnite: Epic Vaults Rocket Launchers, Cuddlefish & Explosive Bows From Competitive

Esports4 days ago

Don’t Miss Out on the Rogue Energy x Esports Talk Giveaway!

Blockchain4 days ago

CoinSmart Appoints Joe Tosti as Chief Compliance Officer

Esports3 days ago

Gamers Club and Riot Games Organize Women’s Valorant Circuit in Latin America

Esports4 days ago

Fortnite: DreamHack Cash Cup Extra Europe & NA East Results

Blockchain5 days ago

Bitfinex-Hacker versenden BTC im Wert von 750 Millionen USD

Blockchain4 days ago

April Continuum Blockchain Legislation Summit ContinuumBlockLegs

Fintech5 days ago

Zip Co raises $400 million for international expansion

Blockchain3 days ago

15. BNB Burn: Binance zerstört Coins im Wert von 600 Mio. USD

Esports4 days ago

2021 Call of Duty Mobile World Championship Announced

Esports4 days ago

Position 5 Faceless Void is making waves in North American Dota 2 pubs after patch 7.29

Fintech4 days ago

Mambu research reveals global consumers are hesitant to use Open Banking

Esports3 days ago

LoL gameplay design director pulled, transferred to Riot’s MMO

Blockchain5 days ago

Ethereum in Blockchain Software Development

Esports4 days ago

A team debrief ahead of the 2021 Overwatch League season

Esports4 days ago

Call of Duty Mobile Season 3 Tokyo Escape: Oasis, Coastal

Trending