Residential Proxies for Web Scraping

Pricing for residential proxies fell again in 2026, but the spread between the best and worst networks widened. Four or five networks own the infrastructure that most of the market resells, which is why a dozen brands quote suspiciously similar per GB rates. The measurable differences show up in three places: the share of the pool that is actually online when you call the gateway, how quickly a flagged exit is retired, and whether the vendor will show you per-request logs when a crawl degrades.

Our working numbers for this category are a 620 – 1,300 ms median round trip, 96 – 99.9% on tier-1 targets, and pools advertised between 8M – 150M IPs. Treat the pool figure as marketing. What determines your success rate is the live concurrent subset in the specific country you target — a 100M-IP network with 4,000 live exits in Portugal will underperform a 10M-IP network that keeps 40,000 Portuguese peers online.

Billing is metered bandwidth, tiered by monthly commitment, priced in the $2 – $6 / GB range. Model your spend on payload, not page count: one JavaScript-heavy product page can pull 2–4 MB through the proxy, so a "cheap" per-unit rate turns expensive the moment you render assets you never parse.

We test every network on the same harness: ten regions, a fixed target set spanning search results, a major marketplace, a Cloudflare-protected page and a plain JSON API, 1,000 requests per region, 30-second timeout, three retries. Latency below is the end-to-end median through the gateway, not a ping to the front door.

In the current cycle Oxylabs returned the fastest median at 620 ms, while IPFly holds the lowest entry rate at $0.80/GB. That pairing is the whole decision in miniature: on tolerant targets the cheaper network costs you nothing measurable, and on hardened targets the faster, better-maintained pool pays for itself in retries you never issue.

Run your own trial before committing. A vendor that will not issue a 1–5 GB trial against your real targets is asking you to buy their marketing copy, and the two-week test costs less than a single bad month.

Nobody pays list price for residential proxies for web scraping. Published rates in the $2 – $6 / GB band assume no commitment. Once you can forecast monthly usage, the same vendor will typically move 30–60%, and will often throw in static IPs, a higher sticky-session TTL or a sandbox sub-account rather than cutting the headline rate — which is frequently the better trade.

Before signing, run the arithmetic on your own traffic. Block images, fonts, media and analytics beacons at the client; use conditional requests where the target honours ETags; and cap retries with exponential backoff so a broken selector cannot burn a month of budget overnight. Teams that instrument bytes-per-successful-record usually cut spend 40–70% without changing vendor.

• Measure cost per successful record, never cost per request
• Block non-essential asset types at the browser or client layer
• Cap and back off retries — silent retry storms are the top overspend cause
• Ask for unused-volume rollover before you ask for a discount
• Keep a second vendor provisioned at minimum spend as a failover

Every credible vendor in this category authenticates by username/password or IP whitelist against a single gateway host, and encodes routing options — country, city, ASN, session ID — inside the username. Rotation here is per request, or sticky 1–30 min. Get the session semantics right on day one: a rotating credential used for a logged-in flow produces a stream of re-authentication challenges that looks exactly like credential stuffing to the target.

Two configuration mistakes account for most "the proxies do not work" tickets. The first is local DNS resolution, which leaks your real resolver and often your region — use the proxy's own resolver or a SOCKS5h endpoint so lookups happen at the exit. The second is a timeout budget shorter than the pool's own connect time; with medians around 620 – 1,300 ms, a 5-second timeout will discard perfectly good exits and inflate your apparent failure rate.

Instrument from the start. Log the exit IP, country, HTTP status, byte count and elapsed time for every request into a table you can group by. Without that, you cannot tell a bad pool from a bad selector, and you will change vendor when you should have changed your parser.

Shortlist against evidence you can verify in a trial, not against a feature grid. Every network claims ethical sourcing, huge pools and 99.9% uptime; almost none publish the live-exit counts, subnet spread or ban-retirement policy that would let you check. Ask for those numbers in writing during the trial, and treat a refusal as an answer.

Weight the criteria to your workload. A team scraping public catalogues should optimise cost per successful record and concurrency ceiling. A team running marketplaces should optimise IP stability, replacement policy and support response time, and should be willing to pay several times more per unit for them.

• Live exits in your target countries — not the global pool headline
• Documented session control: rotation interval and maximum sticky TTL
• Concurrency ceiling in writing, plus what happens when you exceed it
• Subnet and ASN diversity, especially for residential ranges
• Ban handling: how fast a flagged exit is retired and replaced
• Per-request logs you can export for post-mortems
• Sourcing and compliance documentation (consent, opt-out, SOC 2 where relevant)
• Trial terms: volume, duration and whether unused units expire
• Support: named channel, response SLA, and an escalation path that is not a chatbot

The same pool behaves differently depending on what you point it at. These are the configurations we run for the workloads this category is bought for — each one is a starting point you should re-tune after a week of real traffic.

Marketplaces. Throughput comes from concurrency plus retry discipline. Start at 20–50 threads, watch p95 latency and block rate as you scale, and stop at the point where added concurrency raises failures faster than it raises records collected.

Travel. Stability is the whole strategy: static exits, one identity per IP, and a documented replacement path when an address is flagged. Log which identity uses which IP; the day you need to migrate, that mapping is the difference between an hour and a week.

Most failed proxy projects fail the same way: the network is fine, the integration is not. These are the errors we see most often in post-mortems, roughly in order of how much money they cost.

If a crawl degrades, change one variable at a time — first the target, then the fingerprint, then the pool. Swapping vendor while three things changed at once guarantees you learn nothing and repeat the problem on the new invoice.

• Skipping the trial because the vendor is well known, then discovering the geo you need is thin
• Never exporting logs, so every incident becomes a guess between pool, parser and target
• Buying on advertised pool size instead of live exits in the countries you actually target
• Using rotating credentials for logged-in flows, which reads as session hijacking to the target
• Leaving DNS resolution on the local machine and leaking the real region on every lookup
• Setting timeouts shorter than the network's own median connect time and blaming the pool

Using residential proxies for web scraping is lawful in most jurisdictions; what you do through them determines your exposure. Collecting publicly available data is broadly defensible, and courts in several jurisdictions have said so. Bypassing authentication, ignoring an explicit cease-and-desist, or collecting personal data without a lawful basis is a different matter entirely, and no proxy network insulates you from it.

Sourcing matters commercially, not just morally. Consumer IPs reach the pool through SDK partnerships and bandwidth-sharing apps, so ask how consent is obtained, whether peers can opt out, and what the compensation model is. Vendors with clean supply publish the answer; vendors without it change the subject. Under GDPR and CCPA the exit IP can itself be personal data, so keep retention short and document your lawful basis before a customer's procurement team asks.

Buy residential proxies for web scraping when your targets score the signal this category is strong at — marketplaces, travel, job boards, directories — and when the $2 – $6 / GB band is defensible against the value of the data or accounts involved. If your targets do not check IP reputation, you are paying a premium for a signal nobody is reading.

For most teams the shortlist is short: Oxylabs if you want the best-tested option in this category and can absorb $8.00/GB, IPFly if unit economics decide the project at $0.80/GB. Trial both against your own targets for two weeks, compare cost per successful record rather than cost per unit, and keep the loser provisioned at minimum spend as a failover.

Whatever you pick, revisit it every quarter. Pools rotate, anti-bot vendors ship new detection, and the network that led this cycle's benchmark is not automatically leading the next one.