best 12 web scraping and browser apis

Firecrawl prices the way you would design it if you were being straightforward: one credit is one page, javascript rendering is included rather than surcharged, and search costs two credits per ten results. no render multiplier, no proxy tier lottery, no site-complexity band. among vendors that can answer the anchor question at all, it is not the cheapest per thousand, and it is by a distance the easiest to forecast.

the output is the point. this is a content-extraction tool, not a proxy — it returns clean markdown built for a model to read, and it handles rotating proxies, rate limits and js-blocked content underneath without exposing any of that as a billing surface. for rag pipelines and agent tooling that is the correct shape of product, and it explains the adoption: 170,358 github stars and a push to the repo the day we checked.

it states plainly that it only charges for successful requests, and it respects robots.txt for the FirecrawlAgent directive on crawls. there is an official mcp server, so an agent in Claude Code or Cursor can search and scrape without a client library. taken together that is the strongest combination of pricing clarity, billing ethics and agent-readiness here.

two real limits. the licence is AGPL-3.0 on the core repo — MIT only for the sdks — and agpl is a hard stop for plenty of companies that want to embed a scraper inside a closed product, so read it before you build on the self-hosted path. and the vendor publishes nothing about captcha solving or residential proxy rates, so against seriously defended targets you are buying on faith rather than on documentation.

at roughly $0.90 per thousand javascript-rendered pages, Scrapfly is the cheapest verified figure in this ranking, and it gets there with a credit table you can read: one credit for a datacenter request, five more for browser rendering, twenty-five if you route through the residential pool. every feature has a number next to it.

it also publishes a failed-scrape guarantee, which most of this category does not. failed scrapes are not billed — with a stated cliff. if more than 30 percent of your traffic fails inside one hour the protection switches off and usage becomes billable again. that is a reasonable defence against someone pointing a broken scraper at a wall, and it is also precisely when a struggling job starts costing money, so it is worth knowing the threshold exists before you find it.

the honest complication is anti-scraping protection. the vendor states that asp may dynamically upgrade the proxy pool to bypass anti-bot defences, and that this can affect the final cost. so the six-credit figure is the floor for an easy target, not a ceiling for a hard one — which is why Scrapfly itself points you at an interactive estimator rather than a flat number.

the entry plan allows five concurrent requests, the lowest of the proxy vendors here, and we could not confirm captcha handling, self-hosting or an mcp server from the vendor's own pages inside our fetch budget.

Bright Data bills per successfully delivered record and folds everything into that number: javascript rendering, automated proxy management, residential ips and captcha solving are all listed as included in the price rather than billed as multipliers. at $1.50 per thousand records with no monthly minimum, and $1.30 at scale, it is mid-pack on headline cost and near the top on what that cost actually covers.

the failure position is stated bluntly and it is the strongest here: pay only for what is successfully delivered, no charges for failed deliveries. against a well-defended target that clause is worth more than the difference between the top four vendors' per-thousand rates.

concurrency is unlimited on both the pay-as-you-go and scale plans, which is unusual — most of this field caps you at five to twenty-five parallel requests on entry tiers, or throttles by requests per second. for a large crawl that is a structural advantage rather than a feature bullet.

the weaknesses are about the buying experience rather than the product. the pricing page never breaks out what javascript rendering costs on its own, so you cannot model the render surcharge separately even though it is bundled. the scale plan's monthly price is set by an interactive slider rather than printed as a figure, so we cannot publish it. and the wider catalogue — residential, datacenter, mobile and isp proxies, plus datasets — sprawls across enough product pages that working out which one you are buying takes real effort.

when the job needs a real chromium — authenticating, clicking through a flow, handling a single-page app that never settles — a proxy api is the wrong tool and Browserbase is the best-built right one. Stagehand, its automation layer, is the most mature way to let a model drive a browser, and there is an official mcp server delivered over a hosted streamable http endpoint so an agent can use it without a client library.

the entry plan is $20 a month for 100 browser-hours and 25 concurrent browsers, with captcha solving and stealth mode included rather than metered — Steel and Hyperbrowser both charge per captcha solve. that bundling makes the effective cost easier to reason about than the raw hourly rate suggests.

but the unit is time, and that is the trade. a thousand pages costs whatever those thousand pages take to render, so a slow, javascript-heavy target costs several times what a fast one does for identical output. there is a Fetch api billed per call at $1 per thousand, which looks like the comparable number until you notice the docs never confirm it fully renders javascript — so we cannot use it as the anchor. proxy data is billed separately at $12 a gigabyte, the highest per-gigabyte rate in this ranking.

there is no self-hosted path. the vendor describes itself as cloud-first, with private cloud reserved for enterprise, so unlike Steel or the browser-use library there is no option to run it yourself if the economics or the data residency stop working.

Oxylabs does something none of its direct competitors bother with: it prints the javascript premium as a dollar figure rather than a credit multiplier, and it prints it flat. $1.35 per thousand successful rendered results, whatever the target site is. the non-rendered rates vary by target — $0.50 for Amazon, $1.00 for Google, $1.15 for everything else — but the moment you turn rendering on, the target stops mattering. compare that with Zyte, where the same switch costs between $1.01 and $16.08 depending on a tier assignment you do not control.

billing is per successful result throughout, with the plan tables split into rendered and non-rendered columns so the distinction is structural rather than a footnote. proxies and captcha handling are included as standard features rather than metered separately. the free trial is 2,000 results with no card.

the constraint is concurrency, and it is expressed unusually. rather than a thread count Oxylabs imposes a hard rate limit — 50 requests per second on the $49 Micro plan, 10 on the free trial. for steady high-volume crawling that is generous; for a bursty workload that wants to fire a thousand requests at once and wait, it is a wall you will hit, and it is not the kind of limit you can engineer around with more workers.

the other friction is organisational: residential proxy bandwidth lives on a separate product page from the scraper api at a $6 per gigabyte starting rate, so a full picture of what you will spend means reading two pricing pages that do not reference each other. we found no mcp server on the pages we checked.

ScrapingBee's credit table is four numbers and you can hold all of them at once: a plain request is one credit, javascript rendering is five and is the default, premium proxy is ten without rendering or twenty-five with, and stealth proxy is seventy-five. at $19.99 for 75,000 credits that puts a thousand rendered pages at about $1.33, second cheapest among vendors that can answer the question, on the cheapest entry plan in the ranking.

twenty-five concurrent requests on that entry plan is five times what Firecrawl or Scrapfly allow at their entry tiers, and there are 1,000 free credits to test with, no card required.

the billing definition deserves attention because it is quietly generous and quietly not. ScrapingBee states it charges only for successful requests, then defines success as http 200, 404 or 410. a page that no longer exists is billed as a delivered result. that is defensible — the fetch did happen and the answer is real information — but on a crawl of stale urls it is a line item you did not plan for, and it is the same choice ScraperAPI makes.

the ceiling is the stealth tier. against Cloudflare-class defences you are paying seventy-five credits a call, which is fifteen times the ordinary rendered rate and turns that $1.33 into roughly $20 per thousand. we also could not confirm captcha tooling, self-hosting or an mcp server from the vendor's own pages — its features page 404d during the check.

Steel is open source under Apache-2.0 with 7,521 stars and a push two days before we checked, and it self-hosts via docker. that matters more than it sounds in a category where Browserbase is cloud-only and Hyperbrowser publishes no repo at all: if the hosted economics stop working, or the data cannot leave your infrastructure, you have somewhere to go. Firecrawl is the only other self-hostable option here and its AGPL licence is far more restrictive than Steel's Apache-2.0.

the hosted pricing is honest about being usage-based. there is no monthly minimum on the Launch tier — $0.10 a browser-hour billed by the minute and rounded up, plus $30 of credits to start — and the $250 Scale plan buys better rates across the board along with 100 concurrent sessions.

the difficulty is that everything draws from a single dollar-denominated pool: browser hours, proxy bandwidth at $10 or $6 a gigabyte, captcha solves at $3 or $1 per thousand, and browser tool calls at $5 per thousand. four meters, one balance, no per-page price. working out what a workload costs means estimating each meter separately and adding them up, and the vendor markets captcha solving as built-in while still charging per solve.

the jump from the free Launch tier to the next flat plan is $250 a month, which is a large step for a small team that has outgrown pay-as-you-go but is nowhere near needing 100 concurrent sessions.

Apify's product is not really a scraping api, it is a marketplace. thousands of pre-built Actors — scrapers and agents for specific sites — that you rent and run on its infrastructure, plus the option to publish your own. when the target is a well-known site somebody has already solved, this is the fastest path from problem to working data in the whole ranking.

it is also unusually well set up for agents. the official mcp server at mcp.apify.com lets a model discover and run Actors from the store, reach storages and pull results, and the server component is open source as @apify/actors-mcp-server. Crawlee, the underlying open-source library, carries 25,453 stars under Apache-2.0. this is a company that ships real open source around a closed platform.

the pricing is where it falls down the ranking. one compute unit is one gigabyte of ram running for one hour, at $0.20 in overage on Starter. what a page costs therefore depends on which Actor you rented, how much memory it requests and how long it runs — three variables you do not control and the vendor does not publish. no per-page figure is derivable, and unlike the browser-hour vendors there is not even a clean hourly rate, because Actor rental prices sit on top of compute.

the platform itself is saas-only; only the mcp component self-hosts. residential proxy at $8 a gigabyte is the most expensive per-gigabyte rate of the proxy-selling vendors here, and the Starter tier's concurrency limit is not stated on the pricing page — the free tier says 25 and business says 256, with the tier most people actually buy left blank.

Hyperbrowser publishes the clearest credit arithmetic of any browser vendor here, and it does it by making credits meaningless as an abstraction: one credit is one tenth of a cent, stated on the page. browser sessions are 100 credits an hour billed per second, proxy is 10,000 credits a gigabyte, api requests are free. you can convert any of it to dollars without a calculator, which is more than can be said for Apify or Steel.

the $30 Startup plan includes 30,000 credits — about 300 browser-hours if you spend it all on sessions — with 25 concurrent browsers, automatic captcha solving and premium residential proxies included as plan features, with bandwidth still metered at $10 a gigabyte from the credit pool. that residential inclusion at the entry tier is genuinely better than Browserbase, which charges $12 a gigabyte on top.

it sits below its peers for what is absent rather than what is wrong. we found no open-source repository, no self-hosting path and no mcp server on the vendor's own pages. in a category where Steel is Apache-2.0 and self-hostable, Browserbase and Apify both ship mcp servers, and browser-use is MIT with 110,000 stars, being a pure hosted product with no agent integration and no exit route is a real competitive gap in 2026.

and the structural issue it shares with every browser-time vendor still applies: no published per-page rate, so a thousand pages costs whatever a thousand pages take.

the commercial terms here are the friendliest in the ranking and it is not close. no minimum commitment is required. if you exceed a commitment you took voluntarily, you are charged at your existing discounted rate rather than a penalty rate, which the vendor states plainly. there is $5 of free credit and thirty days to spend it. Zyte also stands behind Scrapy, which is BSD-3-Clause with 63,982 stars and remains the default python scraping framework.

then you look at the price. browser rendering is banded into five site-complexity tiers running from $1.01 to $16.08 per thousand. the cheapest band is the cheapest rendered price in this entire ranking. the most expensive is more than seventeen times Scrapfly's rate for the identical output. which band your target falls into is Zyte's assessment, not your choice, and the headline marketing figure of $0.06 per thousand refers to the Simple tier, non-rendered, on a $500 monthly commitment — three qualifiers deep.

the effect is that you cannot answer what this costs until you know your specific targets' tiers, which means the honest advice is to spend the $5 credit on your actual urls before planning anything. for a stable, known set of simple sites it may well be the cheapest option here. for a broad crawl across sites you have not classified, the exposure is unbounded in a way no other vendor's is.

the pricing page also does not publish concurrency limits, a failure-billing policy in explicit terms — the price is quoted per thousand successful responses, but the actual clause is not stated — or any compliance position. an mcp server was not mentioned.

nobody documents multipliers this thoroughly. rendering is 10 credits, premium proxy is 10, premium with rendering is 25, ultra premium with rendering is 75. Amazon costs 5 credits a request, Google and Bing 25, LinkedIn 30. bypassing Cloudflare, Datadome or PerimeterX adds 10 credits a scrape. every one of those is a whole number published in the docs, and you can price a workload precisely before you write any code — which is exactly what the rest of this category fails at.

there is an mcp server for prompt-driven scraping, 20 concurrent threads on the entry plan, and the pricing page is currently flying a banner announcing that ScraperAPI has acquired Traject Data.

the arithmetic is the problem. at $49 for 100,000 credits and 10 credits per rendered request, a thousand pages is about $4.90 — the most expensive verified figure in this ranking, roughly five times Scrapfly and three and a half times Oxylabs, for output that is not obviously five times better. and like ScrapingBee it counts 404s as billable successes, so crawls over stale urls cost real money.

the detail that costs it the most ranking positions is smaller and worse. ScraperAPI's marketing pricing page advertises a 7-day trial with 5,000 credits. its own documentation faq describes a permanent free plan of 1,000 credits with a maximum of five concurrent connections. those are two different offers on two pages owned by the same company, and neither acknowledges the other. for a vendor whose principal virtue is documentation precision, contradicting itself about what it gives away free is a bad look, and it is the reason to read every other figure twice.

the library deserves its reputation. browser-use is MIT-licensed with 109,980 github stars and a commit the day we checked, and it is the reason a lot of people think llm-driven browser automation is practical at all. pip install it, point a model at a browser, and it works. nothing else here has that kind of reach.

the cloud product wraps it with managed stealth browsers, proxies and credits. the Dev plan is $29 a month with 25 concurrent sessions, the free tier gives 10 agent tasks a month, and residential proxy is $5 a gigabyte — cheaper than Browserbase at $12, Steel at $10 and Hyperbrowser at $10. bring your own proxy and egress drops to $0.20 a gigabyte, the cheapest exit route in the ranking.

it ranks last for one specific reason. billing is per minute of browser time with credits reserved at session start against the requested timeout, and when we went to record the actual per-minute rate, the vendor's own pages returned figures that contradicted each other on whether the number was per minute or per hour. every other vendor here we could either price or definitively confirm publishes no price. this is the only one where the site states a rate and the rate does not hold still, and we are not going to publish a number we cannot pin down.

the practical consequence is that a real bill also includes llm token costs on top of browser time, and neither half is forecastable in advance. the honest recommendation is to use the MIT library, which is excellent and free, and treat the cloud tier as something to evaluate against a real workload with a spend cap set before you start.