1. 帮助中心
  2. API 文档资源
  3. 数据采集器 API

触发批量采集

API endpoint: POST /dca/trigger?collector={ID_COLLECTOR}
Parameters :
  • collector={ID_COLLECTOR} - A unique identification of collector
  • version=dev - (Optional) Trigger the development version of the collector
  • name={human_name} - (Optional) A human readable name for the batch
Choose trigger_behavior parameters:
  • queue_next=1 - (Default) If there’s already something in the crawl queue, push this job into the queue
  • queue=‘string’ - (Optional) Send another batch of requests that will start after the last one is finished
  • confirm_cancel=1 - (Optional) Cancel running job and run instead, submit the request and cancel running one
Payload :
  • Header: Authorization Bearer Key -H "Authorization: Bearer API_TOKEN"
  • Header: Content-Type JSON/CSV/Multipart -H "Content-Type: application/json"
  • Body: raw JSON -d '[{"url":"https://targetwebsite.com/product_id/"}]'
Sample Response :
{"collection_id":"ID_DATASET","start_eta":"2021-11-07T13:26:22.702Z"}
 
Shell
curl"https://api.brightdata.com/dca/trigger?collector=ID_COLLECTOR&queue_next=1"-H "Content-Type: application/json" -H "Authorization: Bearer API_TOKEN" -d "[{\"url\":\"https://targetwebsite.com/product_id/\"}]"

Node.js

#!/usr/bin/env noderequire('request-promise')({
	method: 'POST',
	url: 'https://api.brightdata.com/dca/trigger?collector=ID_COLLECTOR&queue_next=1',
	json: [{'url': 'https://targetwebsite.com/product_id/'}],
	headers: {'Authorization': 'Bearer API_TOKEN'},
}).then(function(data){ console.log(data); },
	function(err){ console.error(err); });

Java

packageexample;

importorg.apache.http.HttpHost;
importorg.apache.http.client.fluent.*;

public classExample {
  public static voidmain(String[] args) throwsException {
    String body = "[{\"url\":\"https://targetwebsite.com/product_id/\"}]";    String res = Executor.newInstance()
      .addHeader("Authorization", "Bearer API_TOKEN")
      .execute(Request.Post("https://api.brightdata.com/dca/trigger?collector=ID_COLLECTOR&queue_next=1"))
      .bodyString(body,ContentType.APPLICATION_JSON))
      .returnContent().asString();
    System.out.println(res)
  }
}

C#

usingSystem;
usingSystem.Net;
usingSystem.Net.Http;
usingSystem.Net.Http.Headers;

public classProgram {
  public static asyncTaskMain() {
    var client = newHttpClient();
    var requestMessage = newHttpRequestMessage {
      Method = HttpMethod.Post,
      RequestUri = newUri("https://api.brightdata.com/dca/trigger?collector=ID_COLLECTOR&queue_next=1"),
      Headers = {
        {"Authorization", "Bearer API_TOKEN"}
      },
      Content =newStringContent(JsonConvert.SerializeObject([
        new{
          url ="https://targetwebsite.com/product_id/"
        }
      ]), Encoding.UTF8,"application/json"))
    };
    var response = await client.SendAsync(requestMessage);
    var responseString = await response.Content.ReadAsStringAsync();
    Console.WriteLine(responseString);
  }
}

Python

#!/usr/bin/env python
print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests');
import requests

data = [{'url': 'https://targetwebsite.com/product_id/'}]
headers = {'Content-Type': 'application/json','Authorization': 'Bearer API_TOKEN'}
r = requests.post('https://api.brightdata.com/dca/trigger?collector=ID_COLLECTOR&queue_next=1', data=json.dumps(data), headers=headers)
print(r.content)

这篇文章有帮助吗？