We integrate Bright Data into your scrapers, source by source
When a site returns 403, shows CAPTCHAs or blocks by IP, we connect your scraper to Bright Data: Web Unlocker for HTTP requests, Scraping Browser for JavaScript pages and residential or ISP proxies for the rest. We enable it on the sources that need it and measure each one’s success rate, so usage matches the problem.
Bright Data is a web data access platform with proxy networks (residential, datacenter, ISP and mobile) and products that handle unblocking for you. Web Unlocker deals with CAPTCHAs, browser fingerprints and retries behind a single request. Scraping Browser is a remote browser you connect to over CDP from Playwright or Puppeteer. SERP API returns search engine results already structured, and Web Scraper API offers ready-made extractors for large domains. At Soamee we integrate it as one more strategy inside the scraper: each source decides through configuration whether it goes out directly or through Bright Data, with success and usage metrics per source.
The Bright Data products we connect
Bright Data has many products and not all of them fit every project. We choose per source, starting with the simplest one that solves the block.
Web Unlocker
For sources that block HTTP clients but have the content in the HTML. It works as a proxy or via API: the request goes out as before and Bright Data handles CAPTCHAs, fingerprints and retries. It fits into scrapers you already have without rewriting them.
Scraping Browser
A remote browser on Bright Data’s infrastructure. Your Playwright or Puppeteer code connects with connectOverCDP and nothing else changes: waiting for selectors, scrolling, clicking “load more”. And you no longer maintain Chromium on your server.
Residential, ISP and datacenter proxies
When the problem is the IP or geolocation: prices per country, regional content, per-address limits. We configure zones, country and sticky or rotating sessions depending on what each source requires.
SERP API
Results from Google, Bing and other search engines as JSON, by country and language. Useful for rank tracking, competitor analysis or giving an AI agent access to search.
Web Scraper API and Datasets
For large domains Bright Data already extracts (marketplaces, professional networks, business listings on maps) we trigger the extraction via API and collect the result instead of maintaining our own scraper. We compare it first with the cost of building it custom.
Usage control
Metrics per source: requests, success rate, volume and estimated cost. If a source no longer needs unblocking, it goes back to direct. And an alert fires when usage drifts from normal.
Which product for each block
This is how we decide for each source. If the first test solves it, we do not move to Bright Data.
| Symptom | First test | If that is not enough |
|---|---|---|
| 403 or redirect to a verification page with an HTTP client | Self-hosted headless browser with real browser headers | Web Unlocker |
| Content only appears after running JavaScript and there is anti-bot protection | Self-hosted Playwright | Scraping Browser |
| Works for a few requests, then 429 or an IP block | Slow down and respect Retry-After | Residential or ISP proxies with rotation |
| Content changes depending on the visitor’s country | No reasonable in-house alternative | Proxies pinned to a country |
| You need search engine results | No reasonable in-house alternative | SERP API |
Bright Data as one more strategy
We do not rewrite the scraper to use Bright Data. The transport is chosen in each source’s configuration, and the rest of the pipeline (normalization, deduplication, alerts) stays the same.
Playwright and Puppeteer connect to Scraping Browser without touching the extraction logic
Web Unlocker plugs in as one more proxy in clients like axios, fetch or got
is the unit: each site decides whether it goes out directly or through Bright Data
when a source’s success rate or usage falls outside the normal range
Before enabling Bright Data on a source we try the direct route. Many sites that look locked down only reject an HTTP library’s default User-Agent.
See the web scraping service →Integration in four steps
From diagnosing blocks to tracking usage, source by source.
Block diagnosis
We run each source directly and note what fails: HTTP status, CAPTCHA, empty content, IP block.
Product and zone
We choose the Bright Data product for each source and store zones, countries and credentials as environment secrets, never in the code.
Integration and tests
We connect the HTTP client or the browser, add smoke tests that go through Bright Data and measure the real success rate.
Usage tracking
A dashboard per source and alerts. Every month we review which sources could go back to direct.
Bright Data FAQ
Do I need my own Bright Data account? +
Yes. The account and billing stay in your name and we work with zone credentials you can revoke at any time. That way you do not depend on us to keep using it.
Web Unlocker or Scraping Browser? +
Web Unlocker if the content is in the response HTML: it is faster and plugs in as a proxy. Scraping Browser if you need to interact with the page (scrolling, clicks, waiting for a component to load). It is common to use both in the same project, each on different sources.
Can you integrate it into a scraper we already have? +
Yes. Web Unlocker and the proxies go in as HTTP client configuration, and Scraping Browser replaces the local browser launch with a remote connection. We review your code, connect it and add the per-source metrics.
Does using Bright Data make any scraping legal? +
No. Bright Data solves technical access, but the legal limits are the same: terms of use, database rights and the GDPR if personal data is involved. Bright Data also applies its own compliance policy and may refuse certain uses. We review this before starting.
How much does Bright Data cost? +
It depends on the product and the volume: depending on the product it charges per request, per GB or per result, and it publishes its rates. Before enabling anything we estimate each source’s usage with a real test, so the bill is not a surprise.
Are there alternatives to Bright Data? +
Yes: Oxylabs, Zyte, ScraperAPI and Decodo offer similar products. We work mostly with Bright Data because of its range of products, and since the transport is chosen per source, switching providers for one site does not mean touching the others.
You may also be interested in
Let’s review your blocked sources
Send us the URLs that return 403 or a CAPTCHA. We will tell you which Bright Data product each one needs, or whether adjusting the scraper is enough.
Let’s review your blocked sourcesTell us your challenge. We'll propose a solution.
No commitment. Within 24 hours, you'll receive a proposal with scope, timeline and budget. No fine print.