GET /v1/crawl/
Retrieve the status and results of a crawl job using its ID.Authentication
This endpoint requires authentication using a Bearer token. Include your API key in theAuthorization header:
Path Parameters
string
required
The ID of the crawl job (UUID format)
Response
string
The current status of the crawl. Can be
scraping, completed, or failed.integer
The total number of pages that were attempted to be crawled.
integer
The number of pages that have been successfully crawled.
integer
The number of credits used for the crawl.
string
The date and time when the crawl will expire (ISO 8601 format).
string
The URL to retrieve the next 10MB of data. Returned if the crawl is not completed or if the response is larger than 10MB.
array
The crawled page data. Each item contains:
Example Request
Example Response
Error Responses
Pagination
If the response data exceeds 10MB, thenext field will contain a URL to retrieve the next batch of results. Continue fetching using the next URL until it becomes null.
Status Values
- scraping: The crawl is currently in progress
- completed: The crawl has finished successfully
- failed: The crawl encountered an error and stopped