Trigger a collector for batch collection method

Ori Goren-Bar 阳锐
Ori Goren-Bar 阳锐
  • Updated
API endpoint: POST /dca/trigger?collector={ID_COLLECTOR}
Parameters :
  • collector={ID_COLLECTOR} - A unique identification of collector
  • version=dev - (Optional) Trigger the development version of the collector
  • name={human_name} - (Optional) A human readable name for the batch
Choose trigger_behavior parameters:
  • queue_next=1 - (Default) If there’s already something in the crawl queue, push this job into the queue
  • queue=‘string’ - (Optional) Send another batch of requests that will start after the last one is finished
  • confirm_cancel=1 - (Optional) Cancel running job and run instead, submit the request and cancel running one
Payload :
  • Header: Authorization Bearer Key -H "Authorization: Bearer API_TOKEN"
  • Header: Content-Type JSON/CSV/Multipart -H "Content-Type: application/json"
  • Body: raw JSON -d '[{"url":"https://targetwebsite.com/product_id/"}]'
Sample Response :
{
    "collection_id": "ID_DATASET",
    "start_eta": "2021-11-07T13:26:22.702Z"
}
 
Shell
curl "https://api.brightdata.com/dca/trigger?collector=ID_COLLECTOR&queue_next=1" -H "Content-Type: application/json" -H "Authorization: Bearer API_TOKEN" -d "[{\"url\":\"https://targetwebsite.com/product_id/\"}]"

Node.js

#!/usr/bin/env node
require('request-promise')({
	method: 'POST',
	url: 'https://api.brightdata.com/dca/trigger?collector=ID_COLLECTOR&queue_next=1',
	json: [{'url': 'https://targetwebsite.com/product_id/'}],
	headers: {'Authorization': 'Bearer API_TOKEN'},
}).then(function(data){ console.log(data); },
	function(err){ console.error(err); });

Java

package example;

import org.apache.http.HttpHost;
import org.apache.http.client.fluent.*;

public class Example {
  public static void main(String[] args) throws Exception {
String body = "[{\"url\":\"https://targetwebsite.com/product_id/\"}]"; String res = Executor.newInstance() .addHeader("Authorization", "Bearer API_TOKEN")

.execute(Request.Post("https://api.brightdata.com/dca/trigger?collector=ID_COLLECTOR&queue_next=1"))
.bodyString(body, ContentType.APPLICATION_JSON))

.returnContent().asString();
System.out.println(res) }

}

C#

using System;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;

public class Program {
  public static async Task Main() {
    var client = new HttpClient();
    var requestMessage = new HttpRequestMessage {
      Method = HttpMethod.Post,
RequestUri = new Uri("https://api.brightdata.com/dca/trigger?collector=ID_COLLECTOR&queue_next=1"), Headers = { {"Authorization", "Bearer API_TOKEN"} },
Content = new StringContent(JsonConvert.SerializeObject([
new
{
url = "https://targetwebsite.com/product_id/"
}
]), Encoding.UTF8, "application/json")) }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } }

Python

#!/usr/bin/env python
print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests');
import requests

data = [{'url': 'https://targetwebsite.com/product_id/'}] headers = {'Content-Type': 'application/json','Authorization': 'Bearer API_TOKEN'} r = requests.post('https://api.brightdata.com/dca/trigger?collector=ID_COLLECTOR&queue_next=1', data=json.dumps(data), headers=headers) print(r.content)
Share this

Was this article helpful?

0 out of 0 found this helpful