SerpApi, the Austin company that Google is suing over the scraping of its search results, today released an Extract API that fetches any public web page and returns it as structured JSON, Markdown or raw HTML. The product can also map a website's URLs and crawl up to 100 pages in a single request, billed at one search credit per page extracted.

In Short

SerpApi used to sell only copies of search engine results pages, and today it started selling copies of the web pages those results point to. That matters to anyone who runs a website, because a developer can now pull that site's text, links, prices and images into an app or AI agent with one request. What changes is that a tool built for checking rankings now also works as a general-purpose crawler, and the documentation does not say how it treats a site's robots.txt rules or which name it uses when it visits.

What SerpApi released

The announcement went out on PRNewswire at 10:00 ET on October 13, 2026, under the headline "SerpApi Expands Beyond Search Results with New Extract API." A launch post on the company blog, written by developer advocate Corbin Davenport, and a full documentation page went live the same day.

According to SerpApi, the Extract API is "a new capability that helps developers retrieve real-time data from publicly available web pages in easy-to-use formats." The company framed it as a step past the product it is known for. "For years, we've helped developers find information on the web through search engine result page APIs," said Julien Khaleghy, chief executive and founder of SerpApi. "But finding a page is often just the first step. Developers increasingly need access to the content behind those results for AI agents, research workflows, and data-driven applications."

The new engine sits inside the existing platform rather than beside it. Requests go to the same endpoint used for every other SerpApi product, with the engine parameter set to extract: https://serpapi.com/search?engine=extract. Accounts, client libraries, integrations and search credits are shared across all engines, according to Davenport's post. The company says it now runs more than 100 specialised APIs, covering sources such as Google, Bing and YouTube.

Availability is immediate. According to the press release, the Extract API "is now available across all SerpApi accounts," including the free plan, which carries 250 search credits a month usable on any engine.

Three modes, one parameter

The documentation describes three modes, selected with the mode parameter.

Scrape, the default, fetches a single page. Map lists a website's URLs, drawn from the links on the starting page and from its sitemaps. Crawl follows links on the same website and extracts every page it visits, returning them as a pages array.

Crawl mode carries two limits. The max_pages setting defaults to 10 and tops out at 100, counting the starting page. The max_depth setting controls how many links deep the crawler goes from the start, with a default of 2 and a maximum of 3; a value of 0 extracts only the starting page. A separate include_subdomains switch, off by default, lets map and crawl modes stray from www.example.com to blog.example.com.

Map mode reads sitemaps by default. According to the documentation, the include_sitemap parameter tells the API "to also read the website's sitemaps, from /sitemap.xml and /robots.txt," and it is set to true unless a developer turns it off.

What comes back

The JSON response is long. After the standard search_metadata, search_parameters and search_information blocks, a scrape-mode result can contain 23 named sections: page_information, meta_tags, open_graph, twitter_card, dublin_core, json_ld, microdata, breadcrumbs, headings, inline_links, inline_images, inline_videos, inline_audios, inline_iframes, scripts, stylesheets, feeds, lists, tables, forms, sections, price_offers and text_content. Empty sections are omitted.

Several of those sections lift machine-readable markup straight out of the page. The json_ld and microdata fields return the structured data that sites add for search engines, which means a retailer's schema.org product markup arrives already parsed. The price_offers section, described in the documentation as normalised price output, is aimed at the same use.

Davenport's post walks through a response for the ArchWiki page on NVIDIA drivers. The page_information block reported a content size of 112,079 bytes, a content length of 30,100 and a word count of 4,098. Headings come back with their level and position. Every inline link carries its anchor text, a flag marking whether it is internal, and a field named serpapi_extract_link holding a ready-made API URL to extract that link next. Tables are returned as header and row arrays.

The full text of a page is available in text_content.full, as plain text. Setting include_markdown to true adds the page's main content as Markdown in text_content.markdown. That is separate from the output parameter, which switches the whole response between json, html and md. The HTML option returns the fetched page "up to 5 MB," according to the documentation, and Markdown output can also be requested through a /search.md endpoint or an Accept: text/markdown header.

Controls for developers

Developers can trim what the parser sees before it runs. The include_tags parameter takes a comma-separated list of CSS selectors to keep, such as "article, main, .post-content," and removes everything else inside the body. The exclude_tags parameter does the reverse, with the documentation's own example stripping "nav, footer, .ads, #cookie-banner."

Other settings cover timing and device. The timeout runs from 1,000 to 120,000 milliseconds, with a default of 30,000. A prefetch_delay of up to 10,000 milliseconds can be set before the first fetch. Requests can present as desktop, tablet ("currently using iPads," according to the documentation) or mobile. Enterprise customers can switch on ZeroTrace mode, which skips storing search parameters, files and metadata on SerpApi's servers.

No JavaScript, and what that means

One line in the documentation narrows the product considerably. Describing prefetch_delay, it says: "The Extract API doesn't run JavaScript, so this is not a rendering delay."

That matters because a large share of the modern web delivers its content through scripts. In June, Google analyst Gary Illyes wrote that AI agents and LLMs would have had an easier life on the Web 1.0, because pages that build themselves in the browser can look almost empty to a fetcher that reads only the first server response. A page rendered on the server will parse well through the Extract API; a single-page application that loads its prices or reviews through client-side code may return little beyond its shell.

SerpApi acknowledges the variability without quantifying it. "Because extraction relies on a generalized parser, website support may vary," according to the press release. Davenport's post makes the same point about the store example it uses: "The exact method of extracting prices will depend on the store, and how the Extract API is able to parse its contents."

Billing

Scrape and map requests are priced like any other SerpApi search. Crawl mode is billed per page. According to the documentation, it "consumes one search credit per successfully extracted page, including the starting page, so max_pages=10 can cost up to 10 credits." Failed pages and cached responses are not charged, and the search_metadata.credits_consumed field reports the cost of each call.

The documentation's crawl example shows the arithmetic. A request against the test site books.toscrape.com with max_pages set to 3 and max_depth set to 1 returned three pages and consumed three credits, in 0.85 seconds.

Caching shapes the bill as well. A cached result is served only when the query and every parameter match exactly, and the cache expires after one hour. Cached searches are free and do not count against monthly quotas, according to SerpApi. A no_cache flag forces a fresh fetch.

At the free tier, a single crawl at the 100-page maximum would use 40% of a month's 250 credits.

Redirects to dedicated engines

The Extract API does not treat every URL the same way. Search pages from services that already have a SerpApi engine, "such as Google, Bing, eBay or Walmart, are redirected to that engine and return its JSON structure," according to the documentation. The blog post extends this to individual items: a URL for "a YouTube video or Amazon product" will "automatically use that engine's API."

The press release describes the same behaviour more cautiously, saying that "some URLs covered by dedicated SerpApi engines may return the schema associated with that specialized engine." The difference is between "may" and "automatically," and the documents do not list which URL patterns trigger a redirect.

What the documentation does not say

The three documents are detailed on output fields and parameters. They are silent on the questions a website operator would ask first.

None of them describes how the fetcher identifies itself. No user agent string is published in the materials, and there is no mention of IP ranges that a site could allowlist or block. The only reference to robots.txt is the one in map mode, where the file is read as a source of sitemap locations. Whether the Extract API obeys disallow rules in robots.txt, in scrape or crawl mode, is not stated in any of the three documents. Nor do they mention crawl-delay directives, rate limits per target domain, or what happens when a site returns a CAPTCHA or a block page.

The press release does set one boundary in its wording: the product works on "publicly available web pages." The url parameter must be an absolute http or https address.

These gaps are not unusual for a developer-facing launch, but they are relevant to publishers. Cloudflare said earlier this year that it would block Training and Agent crawlers by default from September 15, 2026 on ad-supported pages of newly onboarded domains, after measuring crawl-to-referral ratios that reached nearly 50,000 for some operators. A fetcher sold explicitly for AI agents would fall into the kind of traffic that policy targets, if it can be identified.

Use cases, as SerpApi lists them

According to the press release, potential use cases include:

  • AI agents that need webpage content and supporting evidence
  • research platforms that collect information from public websites
  • product and market monitoring workflows
  • SEO and content discovery
  • internal knowledge enrichment and retrieval systems

The blog post gives two worked examples in JavaScript. The first combines the google_light engine with the Extract API: it fetches the top organic results for "best bluetooth headphones 2026" in the United States, then pulls each ranking page and prints its heading structure. Pages from recordingnow.com, RTINGS.com and loudnwireless.com appear in the sample output. It is, in effect, a content-gap audit of the first page of Google, assembled in about 37 lines of code.

The second example maps the Wi-Fi category on Ubiquiti's UniFi store with a 50-page cap, filters for product URLs, then reads each product's JSON-LD to print names and prices. The sample output lists a UniFi Express 7 two-pack at $379, a single unit at $199 and an Access Point E7 at $499. Davenport described crawling as useful for "web archival, e-commerce, competitive analysis, and much more."

For AI applications, the pitch rests on retrieval. SerpApi already offers a Model Context Protocol server, a command-line tool and other integrations, and the company packaged its MCP server as a one-click Claude Desktop extension in September. "With the Extract API, they can also fetch the actual pages behind each result," according to Davenport, which would let an agent ground its answer in a page rather than in a title and snippet. That is the basic pattern of retrieval-augmented generation. Markdown output, which the blog says carries "all the same important information, but with reduced token usage," targets the cost side of that pattern. Cloudflare made a similar argument in February when it said converting HTML to Markdown cut token usage by about 80%.

The litigation backdrop

None of SerpApi's three documents mentions its legal disputes. They are hard to separate from this launch, because the core accusation against the company concerns automated access to content that others want to control.

Google sued SerpApi on December 19, 2025 under the Digital Millennium Copyright Act, alleging that the company circumvented SearchGuard, the bot-detection system Google deployed in January 2025. The complaint said SerpApi handled hundreds of millions of automated queries a day and sought statutory damages of $200 to $2,500 per act of circumvention. SerpApi responded in January that it does not bypass authentication, access private accounts or retrieve non-public data.

On July 20, 2026, Chief Judge Yvonne Gonzalez Rogers dismissed Google's complaint, permanently for search results without copyrighted content and with leave to amend for copyrighted images in Knowledge Panels. Google filed an amended complaint built on Reddit licensing terms on August 10, and SerpApi moved on August 24 to end the case over three licensing contracts that Google had not produced.

A hearing on that motion had been set for 2:00 p.m. today in Oakland. It will not take place. According to the court docket for Case No. 4:25-cv-10826-YGR, the judge signed an order on October 5 vacating the hearing, which "will be reset or decided on the papers as appropriate." On October 12, Google filed a two-page statement of recent decision, without argument, drawing the court's attention to an October 2 order by Judge Corley in Ted Entertainment, Inc. v. Bytedance Inc., which denied a motion to dismiss a claim under Section 1201(a). So the motion that will decide whether Google's revived case proceeds is still pending on the day SerpApi widened its scraping business.

Other disputes remain open. Reddit sued SerpApi, Oxylabs, AWMProxy and Perplexity in October 2025 in New York, alleging that SerpApi accessed 784 million Google results pages containing Reddit content in the first six days of July 2025. SerpApi, for its part, is suing a former contractor and the rival SearchApi in Texas, a case in which the defendants have moved to dismiss.

Neither Google's case nor Reddit's concerns the Extract API directly. Google's claims are tied to SearchGuard on its own results pages, and the Extract API redirects Google search URLs to SerpApi's existing Google engine rather than parsing them generically. Still, the product announced today extends SerpApi's web scraping from a handful of search and marketplace sources to any public site, which enlarges the set of parties whose content it touches.

Why the marketing community is watching

For SEO practitioners, the immediate effect is consolidation. Rank tracking and page auditing have typically been separate purchases: one vendor for the results page, another crawler for the pages behind it. SerpApi's pitch is that a single key and a single credit balance now cover both. The company's competitors have taken different routes. Cloudflare's web search API for AI agents, released on October 2, prices queries from $0.25 to $7 per 1,000 requests and requires its search partners to run crawlers that meet Cloudflare's verified bot rules, including respecting robots.txt. Brave, selling its own index at $5 per 1,000 requests, has cited Google v. SerpApi as an example of the risk of building on scraped results.

The data supply for SEO tools has also been under strain. Google dropped the num=100 parameter in September 2025, multiplying the requests needed to track rankings, and in September 2026 SerpApi had to deploy a fix after Google's goto redirects hid destination URLs; it restored direct links on September 5. A product that reads third-party sites rather than Google's own pages is less exposed to that kind of countermeasure, though not to the defences individual sites deploy.

For publishers and retailers, the question runs the other way. The UniFi example shows how product names, variants and prices exposed through schema.org markup can be harvested in a few lines of code, at a cost of one credit per page. Publishers already contend with crawlers that take far more than they send back in traffic, and a further commercial fetcher whose user agent and robots.txt behaviour are not documented in its launch materials makes that harder to manage.

One more data point appeared on the documentation page itself. At the time it was captured, the Extract API page displayed an "API uptime" figure of 82.823%. The page does not say what period that covers or how it is measured. The example responses in the documentation carry timestamps of June 2, 2026 and September 28, 2026, which suggests the engine was running in some form well before today's launch, though SerpApi has not said so.

There is also a small discrepancy in SerpApi's own description of itself. The press release says the company was founded in 2017, while the documentation footer carries a copyright notice of "2016-2026."

Timeline

Summary

Who: SerpApi, LLC, an Austin, Texas search data API provider led by chief executive and founder Julien Khaleghy, whose developer advocate Corbin Davenport wrote the launch post. Google, Reddit and SearchApi are parties to separate litigation involving the company.

What: The Extract API, a new SerpApi engine that fetches public web pages and returns them as JSON with 23 possible sections, as Markdown or as raw HTML up to 5 MB. It has scrape, map and crawl modes, crawls up to 100 pages and three links deep per request, does not run JavaScript, and bills crawl mode at one search credit per successfully extracted page.

When: Today, October 13, 2026, at 10:00 ET, the same day a now-vacated hearing on SerpApi's motion to dismiss Google's amended DMCA complaint had been scheduled.

Where: Available globally to all SerpApi accounts through the https://serpapi.com/search endpoint, including the free plan with 250 credits a month; the Google litigation is in the Northern District of California.

Why: SerpApi says developers building AI agents, research tools and monitoring products need the content behind search results, not only the links. For marketers it merges SERP data and page crawling under one account; for publishers and retailers it adds a commercial crawler whose robots.txt behaviour and user agent are not described in the launch documents.