1. Bright Data
  2. API documentation
  3. Data Collector API

Trigger a collector for batch collection method

API endpoint: POST /dca/trigger?collector={ID_COLLECTOR}
Parameters :
  • collector={ID_COLLECTOR} - A unique identification of collector
  • version=dev - (Optional) Trigger the development version of the collector
  • name={human_name} - (Optional) A human readable name for the batch
Choose trigger_behavior parameters:
  • queue_next=1 - (Default) If there’s already something in the crawl queue, push this job into the queue
  • queue=‘string’ - (Optional) Send another batch of requests that will start after the last one is finished
  • confirm_cancel=1 - (Optional) Cancel running job and run instead, submit the request and cancel running one
  • no_downloads=1 - (Optional) Disable media file download
  • deadline=1h - (Optional) Set job time deadline, job will be terminated after specified time
    • h : hours
    • m : minutes
    • s : seconds
Payload :
  • Header: Authorization Bearer Key -H "Authorization: Bearer API_TOKEN"
  • Header: Content-Type JSON/CSV/Multipart -H "Content-Type: application/json"
  • Body: raw JSON -d '[{"url":"https://targetwebsite.com/product_id/"}]'
Sample Response :
{
    "collection_id": "ID_DATASET",
    "start_eta": "2021-11-07T13:26:22.702Z"
}
 
Shell
curl "https://api.brightdata.com/dca/trigger?collector=ID_COLLECTOR&queue_next=1" -H "Content-Type: application/json" -H "Authorization: Bearer API_TOKEN" -d "[{\"url\":\"https://targetwebsite.com/product_id/\"}]"

Node.js

#!/usr/bin/env node
require('request-promise')({
	method: 'POST',
	url: 'https://api.brightdata.com/dca/trigger?collector=ID_COLLECTOR&queue_next=1',
	json: [{'url': 'https://targetwebsite.com/product_id/'}],
	headers: {'Authorization': 'Bearer API_TOKEN'},
}).then(function(data){ console.log(data); },
	function(err){ console.error(err); });

Java

package example;

import org.apache.http.HttpHost;
import org.apache.http.client.fluent.*;

public class Example {
  public static void main(String[] args) throws Exception {
    String body = "[{\"url\":\"https://targetwebsite.com/product_id/\"}]";
    String res = Executor.newInstance()
      .addHeader("Authorization", "Bearer API_TOKEN")
      .execute(Request.Post("https://api.brightdata.com/dca/trigger?collector=ID_COLLECTOR&queue_next=1"))
      .bodyString(body, ContentType.APPLICATION_JSON))
      .returnContent().asString();

    System.out.println(res)
  }
}

C#

using System;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;

public class Program {
  public static async Task Main() {
    var client = new HttpClient();
    var requestMessage = new HttpRequestMessage {
      Method = HttpMethod.Post,
      RequestUri = new Uri("https://api.brightdata.com/dca/trigger?collector=ID_COLLECTOR&queue_next=1"),
      Headers = {
        {"Authorization", "Bearer API_TOKEN"}
      },
      Content = new StringContent(JsonConvert.SerializeObject([
        new {
          url = "https://targetwebsite.com/product_id/"
        }
      ]), Encoding.UTF8, "application/json"))
    };
    var response = await client.SendAsync(requestMessage);
    var responseString = await response.Content.ReadAsStringAsync();
    Console.WriteLine(responseString);
  }
}

Python

#!/usr/bin/env python
print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests');
import requests

data = [{'url': 'https://targetwebsite.com/product_id/'}]
headers = {'Content-Type': 'application/json','Authorization': 'Bearer API_TOKEN'}
r = requests.post('https://api.brightdata.com/dca/trigger?collector=ID_COLLECTOR&queue_next=1', data=data, headers=headers)
print(r.content)

Was this article helpful?