Get Crawl Errors

Retrieve errors that occurred during a crawl job

GET /v1/crawl//errors

Get detailed information about errors that occurred during a crawl job.

Authentication

This endpoint requires authentication using a Bearer token. Include your API key in the Authorization header:

Authorization: Bearer YOUR_API_KEY

Path Parameters

id · string · required

The ID of the crawl job (UUID format)

Response

errors · array

Errored scrape jobs and error details. Each error object contains:

Show error properties

id · string

The unique identifier of the failed scrape job

timestamp · string

ISO timestamp of when the failure occurred

url · string

The URL that failed to be scraped

error · string

Error message describing what went wrong

robotsBlocked · array

List of URLs that were attempted in scraping but were blocked by robots.txt

Example Request

cURL

curl -X GET https://api.firecrawl.dev/v1/crawl/123e4567-e89b-12d3-a456-426614174000/errors \
  -H 'Authorization: Bearer YOUR_API_KEY'

Example Response

{
  "errors": [
    {
      "id": "error-123",
      "timestamp": "2026-03-03T10:30:45.000Z",
      "url": "https://example.com/broken-page",
      "error": "404 Not Found"
    },
    {
      "id": "error-124",
      "timestamp": "2026-03-03T10:31:12.000Z",
      "url": "https://example.com/timeout-page",
      "error": "Request timeout after 30000ms"
    }
  ],
  "robotsBlocked": [
    "https://example.com/admin",
    "https://example.com/private"
  ]
}

Error Responses

402 Payment Required

{
  "error": "Payment required to access this resource."
}

429 Too Many Requests

{
  "error": "Request rate limit exceeded. Please wait and try again later."
}

500 Server Error

{
  "error": "An unexpected error occurred on the server."
}

Common Error Types

• 404 Not Found: The URL doesn’t exist or has been removed
• 403 Forbidden: Access to the URL is forbidden
• Timeout: The page took too long to load
• SSL/TLS errors: Certificate validation failed
• Network errors: Connection issues or DNS failures
• Robots.txt blocks: URLs blocked by the site’s robots.txt file (listed separately)

Use Cases

• Debug why certain pages weren’t crawled
• Identify problematic URLs in your crawl
• Monitor robots.txt restrictions
• Track down broken links or access issues
• Improve crawl configuration based on error patterns