This article reviews the interface components of our IDE that is especially designed for web scraping purposes, and the interface components of the control panel (dashboard) from which you can manage your scrapers (collectors). 

The IDE 

This is where you write your JavaScript code. Learn more about The basics of Web Scraping.

A : See more examples

- Examples of template code that our collector engineers built.

B : Add another step (stage)

- It is useful to add stages when you want to collect data from multiple pages. For example, in case you want to collect all products from Amazon search result page and details of all collected products. You can navigate the search result page on 1st stage to collect all product URLs and pass them to 2nd stage to navigate each product page.

- 'next_stage', 'run_stage' commands are available to interact between stages.

C : Help

- List of available functions, functions explanation and examples of usage. Learn more about Interaction functions and Parser functions.

- How to find a selector of data point you want to collect Learn more about Finding elements selectors

- How to use jQuery expressions Learn more about jQuery expressions 

D : Debugging Tabs

- Input : Define your input parameters and run a test (preview) with an input set

- Output : List of collected data

- Children : List of children which will be input sets of the next stage

- Run log : Code execution log

- Browser console : Collector browser console logs [browser > developer tool > 'console tab']

- Browser network : Collector browser network logs [browser > developer tool > 'network tab']

- Last errors : List of latest error information

E : Input

- Add input parameter : Define an input parameter including its name and type

- Add another input : Add the value of an input set to test

- Preview : Run a test with a selected input set

F : Code settings

- Error mode : Set a code behavior of the collector error case

- Take screenshot : Take screenshots during preview test. You will be able to check loaded pages during the test.

Dashboard - Collector action menu

The collector action menu allows performing different actions with the collector.

• Initiate by API - start a data collection without having to enter the control panel
• Initiate manually - Bright Data's control panel makes it easy to get started collecting data
• Run on schedule - select precisely when to collect the data you need
• Versions - review the modified versions of the collector
• Report an issue - You can use this form to communicate any problems you have with the platform, the collector, or the dataset results
• Copy link - copy the link of the collector to share it with your colleagues
• Tickets - view the status of your tickets
• Advanced options:
  • Edit the code - edit the collector's code within the IDE.
  • Disable collector - temporarily disable the collector, but you can reactivate it if needed.
  • Delete collector - permanently delete the collector.