Getting started with Web Scraper IDE

Self Serve Type Examples Time till response ETA Remarks General Support Questions   1 business hours up to 1 business day T1 online support Infrastructure  Service offline  Unable to initiate Collection  Delivery failure...

Data Collectors are automated tools that enable businesses to automatically collect all types of public online data on a mass scale, while heavily reducing in-house expenses on proxy maintenance and development. The Data Collector ...

Data Collector's IDE is its integrated development environment. The IDE is a Public web data on any scale at your fingertips, you can: Build your collector in minutes Debug and diagnose with ease Bring to production quickly Brows...

When collecting data, your “input” are the parameters you'll enter to run your collection with. This can include keywords, URL, search items, product ID, ASIN, profile name, check in and check out dates, etc.

The output is the data that you've collected from a platform based on your input parameters. You'll receive your data as JSON/NDJSON/CSV/XLSX.

Each free trial includes 100 records (note: 100 records does not mean 100 page loads).

You'll always receive a higher number of records than the inputs you've requested.

Number of followers, average number of likes for posts, level of engagement, account theme, social and demographic portrait of the audience, social listening: keywords/ brand mentions, sentiments, viral trends.

Yes, you can check if your public data was collected here:https://brightdata.com/check_your_data.

We never collect private data. We only collect publicly-available data.

Yes, we can collect data from large numbers of websites at the same time.

Yes you can, you can ask your account manager for help, or you can open a ticket related to the specific Data Collector by selecting 'Report an issue.' Then request that fields be added or removed from your Data Collector.

In cases where you don't know a specific URL, you can search for a term and get data based on that term.

With a discovery collector, you enter a URL(s) and collect all data from that page(s). You'll receive data without having to specify a specific product or keyword.

Yes, the code is in JS, and you can change it according to your requirements.

We have 3 option to initiate requests: Initiate by API - regular request, queue request and replace request. Initiate manually. Schedule mode

Click here to check our guide on how to start using the Data Collector

When you are sending more than one API request, a “queue request” means that you'd like your next request to start automatically after your first request is completed, and so on with all other requests.

CPM = 1000 page loads