Firecrawl Tutorial 2026: Turn Any Site Into LLM-Ready Markdown

This Firecrawl tutorial takes one job, turning a store's product pages into data an LLM can use, through the five calls you need: scrape one page to markdown, map the site's URLs, batch scrape or crawl the pages, and extract fields with JSON mode. On the v2 API a plain page costs 1 credit and a JSON extraction costs 5, the free plan includes 1,000 credits a month, and the Python SDK installs with pip install firecrawl-py.

Checked October 2026 against Firecrawl's docs for scrape, map, crawl, JSON mode, Agent, billing and rate limits, plus the pricing page. This page is built from that documentation, not from benchmark runs: the code follows the official examples, and every credit total is arithmetic on the published rates.

By the end you will have a short Python script that lists a site's product URLs, pulls each page as clean markdown, and returns name, price and stock as typed JSON, plus a credit budget you can check before you run it. If you are still choosing between an API like this and writing your own Playwright scraper, start with our web scraping automation guide, which covers the build-it-yourself route.

What you need before you start

Nothing here needs a paid plan. The free plan's 1,000 credits cover every step in this tutorial several times over, as long as you keep the page limits small while you test.

Step 1: get a Firecrawl API key and install the SDK

Sign up at firecrawl.dev, open the dashboard and copy your key. Then install the Python package and put the key in an environment variable so it never lands in your code:

pip install firecrawl-py
export FIRECRAWL_API_KEY="fc-YOUR-API-KEY"

Expected result: pip show firecrawl-py prints a version. The SDK reads FIRECRAWL_API_KEY from the environment, or you can pass api_key when you create the client. Node users install with npm install firecrawl, and there is a CLI (npm install -g firecrawl, then firecrawl login) if you want to test from a terminal first.

Step 2: a Firecrawl Python example that scrapes one page

Start with a single page. One scrape tells you whether the content you care about survives the trip to markdown, and it costs 1 credit.

from firecrawl import Firecrawl

firecrawl = Firecrawl(api_key="fc-YOUR-API-KEY")

doc = firecrawl.scrape(
    "https://example.com/products/trail-shoe",
    formats=["markdown"],
)
print(doc.markdown)

Expected result: the page body as markdown, with navigation and footer stripped. Read it. If the price or the stock status is missing, the fix is here, not later: the content may sit outside the main content area, or load after a delay.

Scrape has sensible defaults, and most of the tuning is knowing which ones to change. These five come from the API reference:

One thing the cache does not do is save credits. Firecrawl's docs are explicit that a cached result still costs 1 credit per page; caching makes the response faster, nothing more.

Step 3: map the site and pick the pages

Map returns the URLs on a site without fetching their content. It draws mainly on the sitemap, and it costs 1 credit per call however many URLs come back, which makes it the cheapest way to find out what you are dealing with.

res = firecrawl.map(url="https://example.com", limit=500, sitemap="include")

product_urls = [link.url for link in res.links if "/products/" in link.url]
print(len(product_urls))

Expected result: a count of product URLs. The docs warn that map trades completeness for speed, so if the count looks low compared with the store's own category pages, use crawl for discovery instead. For a quick filter on the server side, map also accepts a search parameter that orders results by relevance to a term.

Step 4: batch scrape the list, or crawl the section

You now have two ways to fetch many pages. Batch scrape takes the list you just built. Crawl takes a starting URL and finds the pages itself. Batch is the predictable one, because you decide exactly which URLs get fetched.

job = firecrawl.batch_scrape(
    product_urls[:10],
    formats=["markdown"],
    poll_interval=2,
    wait_timeout=120,
)
for doc in job.data:
    print(doc.metadata.status_code, doc.metadata.source_url)

Expected result: ten lines, each with a status code and a URL. Check the status codes. Firecrawl reports two layers: the API call can succeed while the target page returned a 403 or 404, and that page still comes back as a document and still costs 1 credit. Stop retrying URLs that keep failing.

Crawl suits the case where you want everything under a path and do not trust the sitemap. The path filters are regular expressions on the URL pathname:

curl -s -X POST "https://api.firecrawl.dev/v2/crawl" \
  -H "Authorization: Bearer $FIRECRAWL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com",
    "limit": 100,
    "includePaths": ["^/products/.*"],
    "scrapeOptions": { "formats": ["markdown"], "onlyMainContent": true }
  }'

The response is a job ID. Poll GET /v2/crawl/<id> until the status is completed, or use the SDK's crawl method, which waits and handles pagination for you. Results stay available through the API for 24 hours. If you would rather be told than poll, crawl can post crawl.page and crawl.completed events to a webhook; our n8n webhook trigger guide shows how to catch one.

Step 5: extract structured product data with JSON mode

Markdown is right for retrieval and summarising. For a database row you want fields. JSON mode runs an LLM over the page and returns data that matches a schema you define, in the same scrape call.

from pydantic import BaseModel

class Product(BaseModel):
    name: str
    price: float
    currency: str
    in_stock: bool

result = firecrawl.scrape(
    "https://example.com/products/trail-shoe",
    formats=[{
        "type": "json",
        "schema": Product.model_json_schema(),
        "prompt": "Extract the product shown on this page.",
    }],
)
print(result.json)

Expected result: a dictionary with the four fields. In the v2 API the schema sits inside the format object; the older jsonOptions parameter from v1 no longer exists, which is the usual reason a copied snippet fails. You can also pass only a prompt and let the model choose the structure, but a schema is what keeps 500 pages consistent.

Two alternatives are worth knowing. For product pages specifically, the product format returns title, price, availability and variants without an LLM call or a schema, and the billing table lists no surcharge for it. And when you do not know which URLs hold the answer, the Agent endpoint takes a prompt and finds the pages itself; Firecrawl describes it as the successor to the older /extract endpoint, which still works.

Firecrawl API credit costs for the whole job

Credits are the same on every plan, so the budget is multiplication. Take an illustrative catalogue of 500 product pages. Mapping it costs 1 credit. Scraping all 500 to markdown costs 500. Running JSON mode on all 500 costs 2,500, because each page is 1 base credit plus 4 for the extraction.

That arithmetic decides the plan. The free plan's 1,000 credits cover 200 JSON extractions or 1,000 plain pages a month. Hobby's 5,000 cover 1,000 JSON pages. If your fields are standard product fields, the product format has no listed surcharge, which would put the same 500 pages at a fifth of the JSON-mode cost; confirm it on your first ten pages by reading the job's credits used. And once markdown is in your own storage, the remaining spend is the model you send it to; our LLM API pricing comparison has the per-token rates.

Firecrawl rate limits and the settings that matter

Two separate limits apply. Requests per minute are counted per endpoint and per team, and going over returns a 429. Concurrent browsers cap how many pages render at once; extra jobs wait in a queue instead of failing.

On the free plan, 10 scrape requests a minute means 500 single scrape calls take at least 50 minutes. That is the practical argument for batch scrape and crawl: one request starts the whole job, and Firecrawl works through it at your concurrency. The older extract endpoint shares Agent's limit.

Be a polite guest. Crawl's delay option waits between pages and forces concurrency to 1, and maxConcurrency caps parallel scrapes below your plan's ceiling. A small store's server does not need 25 browsers at once.

Firecrawl Docker: can you self-host it instead?

Yes. Firecrawl is open source under AGPL-3.0, and the repository includes a Docker Compose setup:

git clone https://github.com/firecrawl/firecrawl.git
cd firecrawl
# create .env first (see the self-host guide), then:
docker compose up --build -d

The API then answers on http://localhost:3002, and the same /v2/scrape calls work against it. The self-host guide is clear about the gaps: the default stack has no Fire-engine, so no advanced anti-bot handling, no screenshots and no page actions; Agent is a cloud feature; and JSON mode needs an LLM provider you wire up yourself. The quickstart also runs with API authentication off, so keep it off the public internet.

Troubleshooting: when a Firecrawl call fails

• HTTP 402. You are out of credits, or a crawl's limit is more than your balance allows. Lower the limit or wait for the monthly reset.
• HTTP 429. You passed the requests-per-minute limit for that endpoint. Back off, or move single scrapes into one batch job.
• Markdown is missing the data. Set only_main_content=False, or add a wait if the content loads late.
• Stale prices. The default cache window is two days. Pass max_age=0 for a fresh fetch.
• JSON fields come back empty. Check that the markdown contains them first; the extraction can only read what the scrape captured.
• A v1 snippet throws an error. Move the schema into formats as a json object and use the Firecrawl class.

Scraped data is only useful once something reads it. If you want an agent to call Firecrawl directly, it ships an MCP server, and our Model Context Protocol guide explains how that connection works. If you would rather call a model yourself, the Claude API tutorial picks up where this script ends, and the Supabase tutorial covers where to keep the rows.

Turning that pipeline into something customers log in to is a longer job. Our AI SaaS Builder program covers the part this tutorial stops at: the app, the database, auth and billing around the data.

Firecrawl tutorial: FAQ