Become a Bright Data professional
-
How to crawl a website without getting blocked or misled (cloaked)?
Why should I care?When a target website detects crawlers from a proxy (datacenter) IP, it typically Blocks the IP, or Presents the IP with purposely misleading information, or Throttle down the response rate How does the target w...
-
How to get an IP in a specific city
Why should I care?Example: If you are responsible for testing Yelp's city level service - you need to check the site from 10,000 different cities around the world.How to get an IP in a specific city?If you only use a traditional da...
-
How to accelerate your web scraping
Why should I care?If the number of requests you rotate through a single IP are higher than what target websites allow, the website you target will identify your IP and block or mislead you with false information. It means that your...
-
How to rotate your IP address
Why should I care?When many requests are sent to a website from the same IP, the website can tag the IP used as a crawler and send misleading information or block you. Periodically changing, or rotating, your IP address helps preve...
-
How do I avoid subnet block range?
What is a subnet block range?Smaller sections of a network are called subnets, which are useful for grouping hosts together and managing them all at once. Subnets are based on IP address, making it easy for websites or malicious us...
-
How to switch from API to Proxy Manager
Why should I care?Using the Proxy Manager offers advanced features that are not readily available in the API. Instead of having to manually code mechanisms for tasks like keeping an IP as long as possible or rotating your IP after ...
-
How to configure Bright Data raw API with iPhone
Guidelines for configuring Bright Data raw API with iPhone Open your device Settings app and go to Wi-Fi Tap the name of the Wi-Fi network you are connected to Scroll to the bottom and you will find a section for 'HTTP Proxy'. Thi...
-
How do I know if I'm getting cloaked?
Why should I care?Getting cloaked means that you're getting misleading information from the website you are scraping.Example: If you are collecting comparative competitive information to feed your automatic pricing algorithms, the ...