Instead of creating duplicate scrapers for each delivery type, you can choose a delivery type per job using API.

hero-image.png

1

Update your scraper

Ensure that your scraper is updated to the latest version to improve success rate

update-available.png

2

Set a delivery type to Batch

type-batch.png on-completion.png

Following error message will be returned from Batch API when delivery type is set to Realtime.

Error
"error": "Cannot trigger a batch job with a real-time scraper." \
         "Use /trigger_immediate endpoint instead"
3

Trigger the scraper using a preferred API

dca/trigger

Trigger a scraper for batch collection method

curl "https://api.brightdata.com/dca/trigger?collector=ID_COLLECTOR&queue_next=1" \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer API_TOKEN" \
    -d '[{"url":"https://targetwebsite.com/product_id/"}]'

Batch responses begin with j_**** and real-time responses begin with z****

4

Receive data

Getting started with the APIInitiate a Batch job