> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/firecrawl/firecrawl/llms.txt
> Use this file to discover all available pages before exploring further.

# Rate Limits

> Understanding rate limits and usage guidelines for the Firecrawl API

The Firecrawl API implements rate limiting to ensure fair usage and maintain service quality for all users. This page explains how rate limits work and how to handle them in your application.

## Rate Limit Response

When you exceed the rate limit, the API will return a `429 Too Many Requests` status code with the following response:

```json theme={null}
{
  "success": false,
  "error": "Request rate limit exceeded. Please wait and try again later."
}
```

## Rate Limit Strategy

### Plan-Based Limits

Rate limits vary based on your subscription plan. Higher-tier plans receive increased rate limits to support larger-scale operations.

### Endpoint-Specific Limits

Different endpoints may have different rate limits:

* **Scraping endpoints** (`/scrape`, `/batch/scrape`) - Limited by requests per minute
* **Crawling endpoints** (`/crawl`) - Limited by concurrent crawls and pages per crawl
* **Search endpoint** (`/search`) - Limited by searches per minute
* **Extract endpoint** (`/extract`) - Limited by extraction requests per minute
* **Research endpoint** (`/deep-research`) - Limited by concurrent research operations

## Handling Rate Limits

### Best Practices

<AccordionGroup>
  <Accordion title="Implement Exponential Backoff">
    When you receive a 429 response, wait before retrying. Use exponential backoff to gradually increase wait times:

    ```python theme={null}
    import time
    import requests

    def scrape_with_retry(url, api_key, max_retries=5):
        base_wait = 1  # Start with 1 second
        
        for attempt in range(max_retries):
            response = requests.post(
                'https://api.firecrawl.dev/v1/scrape',
                headers={
                    'Authorization': f'Bearer {api_key}',
                    'Content-Type': 'application/json'
                },
                json={'url': url}
            )
            
            if response.status_code == 429:
                wait_time = base_wait * (2 ** attempt)
                print(f"Rate limited. Waiting {wait_time} seconds...")
                time.sleep(wait_time)
                continue
            
            return response.json()
        
        raise Exception("Max retries exceeded")
    ```
  </Accordion>

  <Accordion title="Use Batch Operations">
    Instead of making individual requests, use batch operations when scraping multiple URLs:

    ```bash theme={null}
    curl -X POST 'https://api.firecrawl.dev/v1/batch/scrape' \
      -H 'Authorization: Bearer fc-YOUR_API_KEY' \
      -H 'Content-Type: application/json' \
      -d '{
        "urls": [
          "https://example.com/page1",
          "https://example.com/page2",
          "https://example.com/page3"
        ]
      }'
    ```

    Batch operations count as a single request against your rate limit while processing multiple URLs.
  </Accordion>

  <Accordion title="Monitor Your Usage">
    Track your API usage to stay within limits:

    ```bash theme={null}
    curl -X GET 'https://api.firecrawl.dev/v1/team/credit-usage' \
      -H 'Authorization: Bearer fc-YOUR_API_KEY'
    ```

    Response:

    ```json theme={null}
    {
      "success": true,
      "data": {
        "remaining_credits": 1000
      }
    }
    ```
  </Accordion>

  <Accordion title="Use Webhooks for Async Operations">
    For crawling and batch operations, use webhooks instead of polling for status:

    ```json theme={null}
    {
      "url": "https://docs.firecrawl.dev",
      "webhook": {
        "url": "https://your-domain.com/webhook",
        "events": ["completed", "failed"]
      }
    }
    ```

    This reduces the number of status check requests you need to make.
  </Accordion>
</AccordionGroup>

## Credit System

In addition to rate limits, Firecrawl uses a credit-based system:

### Credit Consumption

* **Scrape**: 1 credit per page
* **Crawl**: 1 credit per page crawled
* **Batch Scrape**: 1 credit per URL
* **Search**: Credits vary based on scraping options
* **Extract**: Token-based pricing (separate from credits)
* **Map**: 1 credit per request

### Insufficient Credits

When you run out of credits, you'll receive a `402 Payment Required` response:

```json theme={null}
{
  "success": false,
  "error": "Payment required to access this resource."
}
```

To continue using the API, you'll need to upgrade your plan or purchase additional credits.

## Crawl-Specific Limits

### Concurrent Crawls

The number of simultaneous crawl operations you can run depends on your plan:

```bash theme={null}
# Check active crawls
curl -X GET 'https://api.firecrawl.dev/v1/crawl/active' \
  -H 'Authorization: Bearer fc-YOUR_API_KEY'
```

### Pages Per Crawl

You can limit the number of pages in a single crawl using the `limit` parameter:

```json theme={null}
{
  "url": "https://docs.firecrawl.dev",
  "limit": 100,
  "scrapeOptions": {
    "formats": ["markdown"]
  }
}
```

The default limit is 10,000 pages per crawl.

### Crawl Delays

Respect website rate limits by adding delays between requests:

```json theme={null}
{
  "url": "https://example.com",
  "delay": 2,
  "scrapeOptions": {
    "formats": ["markdown"]
  }
}
```

The `delay` parameter specifies the number of seconds to wait between scraping pages.

## Error Handling

### Rate Limit Headers

While not explicitly documented in all responses, monitor HTTP status codes to detect rate limiting:

```python theme={null}
response = requests.post(
    'https://api.firecrawl.dev/v1/scrape',
    headers=headers,
    json=data
)

if response.status_code == 429:
    # Rate limited - implement backoff
    pass
elif response.status_code == 402:
    # Insufficient credits
    pass
elif response.status_code == 200:
    # Success
    result = response.json()
```

### Server Errors

Occasional `500 Internal Server Error` responses may occur. These are different from rate limits and should be retried with exponential backoff:

```json theme={null}
{
  "success": false,
  "error": "An unexpected error occurred on the server."
}
```

## Optimizing API Usage

### 1. Use the Right Endpoint

* Use `/map` to discover URLs before crawling
* Use `/batch/scrape` for known URL lists
* Use `/crawl` for comprehensive site scraping

### 2. Filter Content Efficiently

Use crawl options to reduce unnecessary requests:

```json theme={null}
{
  "url": "https://firecrawl.dev",
  "includePaths": ["blog/.*"],
  "excludePaths": ["admin/.*", "login.*"],
  "limit": 50
}
```

### 3. Request Only Needed Formats

Specify only the formats you need:

```json theme={null}
{
  "url": "https://example.com",
  "formats": ["markdown"]
}
```

Avoid requesting multiple formats (HTML, screenshot, etc.) if you don't need them.

## SDK Rate Limit Handling

Our official SDKs include built-in rate limit handling:

### Python SDK

```python theme={null}
from firecrawl import Firecrawl

app = Firecrawl(api_key="fc-YOUR_API_KEY")

try:
    result = app.scrape("https://firecrawl.dev", formats=["markdown"])
    print(result.markdown)
except Exception as e:
    # SDK handles retries automatically
    print(f"Error: {e}")
```

### Node.js SDK

```javascript theme={null}
import Firecrawl from '@mendable/firecrawl-js';

const app = new Firecrawl({ apiKey: 'fc-YOUR_API_KEY' });

try {
  const result = await app.scrape('https://firecrawl.dev', {
    formats: ['markdown']
  });
  console.log(result.markdown);
} catch (error) {
  // SDK handles retries automatically
  console.error('Error:', error);
}
```

## Contact Support

If you're experiencing consistent rate limiting issues or need higher limits:

1. Review your usage patterns and optimize requests
2. Consider upgrading to a higher-tier plan
3. Contact [support@firecrawl.dev](mailto:support@firecrawl.dev) to discuss custom rate limits

For plan details and pricing, visit [firecrawl.dev/pricing](https://firecrawl.dev/pricing).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.