Getting Started With Bright Data

Learn how to quickly begin and achieve a high success rate using Bright Data
12 Min
Advanced
24-Oct-2018
Play Video

Learn how to quickly begin and achieve a high success rate

Agenda

  • Choosing the right IP type
  • Zone creation and price plans
  • Working with Bright Data browser extension, Proxy Manager and API
  • Quick setup to get high success rate
  • Targeting by country, city and ASN
  • Understanding Bright Data billing
  • How Bright Data keeps its network safe

Don’t want to watch the webinar, read it

First, let’s understand what network type is right for me.
There are three types of networks:
Data center, which is a machine generated IP, and therefore the most cost-effective, commonly used for mass crawling of non-sophisticated target websites.
The Residential Network is a real user PC IP which is commonly used for more sophisticated target websites when you need to crawl and see the web as a real user.
The Mobile Network is a real user mobile IP, used for targeting highly sophisticated websites.

How does it work?
Your browser sends a request, Bright Data’s network will route it through our Super Proxy peers to a real user PC, and then to your target website.
The response from the target website will be routed the same way back from the target website to the real user PC peer, to Bright Datas Super proxy, and to your crawler.

Here at the lower end of the page, you’ll find instructions on how to connect your crawler to Bright Data’s network in various coding syntax. In the “Other software” tab you’ll find examples of the value to enter in your crawler in order to work with Bright Data.

For a quick start, you can use Bright Data in your browser by installing the Bright Data Proxy Browser Extension by selecting next.
Once you add the extension to your browser, it will be added at the top browser bar.

You can choose any country to browse from, this will rotate between countries, or select a specific country. You can turn the extension off when not needed.
Here is an example using command line or crawler, which are also available in the API example tab.
In the API documentation, you’ll find rich examples of how to send proxy requests, IP persistence, and more.

It is highly recommended to use Proxy Manager with any of Bright Data’s products such as the browser extension, API or your crawler.
The Proxy Manager is an open source software and you can download it from the Proxy Manager tab to install on Windows, MAC, Linux and as a Docker image.

The Proxy Manager can be installed on multiple machines in your organization.
On this page, you’ll find very useful tools to work with Bright Data Proxy Manager.
Selecting “generate request” will enable you to test your Bright Data Proxy Manager with API requests.

Now selecting the “Proxy Manager commands”, Proxy Manager stands for Bright Data Proxy Manager.
You’ll find our swagger page with many API calls to your Bright Data Proxy Manager.
Keep in mind, once you are using API calls to Proxy Manager you don’t need to connect directly to the Super Proxy since the Proxy manager is between you and the super proxy.

Now that I know which network type I want to start with and the relevant Bright Data products, I’ll set my proxy Zone in the zones tab. Selecting ‘Add Zone’ button.

I can rename my Zone and select the network type as datacenter, residential or mobile.
For datacenter as network type, I may select Shared IP per usage with unlimited IPs which is the quickest and flexible way to start.

Exclusive IPs for specific domains you use will be kept.
The domain is the target website you are crawling, select your domains or choose unlimited domains.

Select your target country if needed and type of usage.
For advanced datacenter usage, you may select specific geo DB IPs you need and distribute them among many subnets.

Back to the Plan tab, now I’ll select Residential:
In residential first, I’ll select my Price plan.
As long as my traffic volume increases I can upgrade to cost-effective plans with a lower cost per gigabyte.
Selecting the city targeting will update the gigabyte price accordingly.

Most of Bright Data’s pro users start with the residential production plan, and when reaching one tera per month upgrade to the Plus plan.
Each of these plans has a monthly commitment that we’ll discuss later on.
Last, you may use high-quality mobile IPs in our mobile network, and select “shared” or your exclusive IPs.

When you want to start using the residential network and click save you’ll get a prompt to start the “Bright Data Know Your Customer” we refer to as the User Compliance Evaluation.
In the User Compliance Evaluation you’ll be asked to fill your company’s details and type of use.
Our compliance team will review the submitted form and will contact you if further clarification is needed.

Bright Data keeps its network safe and only for legitimate business use by reviewing and verifying each of the business’ onboard our network.
To shorten the qualification process with the Bright Data compliance team, please pay attention to all details you fill and supply the relevant documents. Once your User Compliance Evaluation form is approved you’ll get a 7 day free trial of the Bright Data residential network.

I’ll now go to the Proxy Manager, Always start here.
In case you see an upgrade message please make sure to select and update your local Proxy Manager with new features we are releasing occasionally and bug fixes.

I’ll create a proxy by selecting a new proxy.
Select the relevant Zone, I already created mine in the Bright Data dashboard in advance.
In the preset configuration, select the most relevant Proxy Manager set up to your type of use.

For example, long session IP is good when you are doing social media or ticketing and need the same IP throughout the session.

Selecting session IP per machine is good when working with many machines, simultaneously connected to the Proxy Manager and need a different IP for each one.

When you need to spread many requests across a large number of IPs, select Round-robin pool.
When request speed is important, select high-performance and the fastest IPs for your target site will be kept for you.

The random user agent and cookies preset are very handy when target sites check persistence and credibility.
The online-shopping preset is used on e-commerce websites and SEO, with a very specific setting to get high success rates of e-commerce crawling.

In the new Proxy you created, you may go to the Targeting tab and select your geo-location targeting by country and city.

In the rules tab, you can create a rule to replace the IP when getting a captcha, by selecting the HTML body element, write captcha and select action type “retry with new IP”.
To decrease your traffic and therefore your costs, you may remove pictures and videos from the response by creating a new rule.
Select URL, then select png, jpg and action type Null response.

Last, how Bright Data billing works.
I’ll go back to my Bright Data dashboard, and go to the billing tab.
Here I can see my current balance, add payment method and once the payment method is added, make a payment.

It is highly recommended to set auto-recharge on, this will charge your account with additional funds with the amount you specified, once account balance runs out.

To set auto-recharge select your credit card as Primary, this will enable the auto-recharge and will be the main payment method for the monthly billing cycle.
Now select the amount to recharge once your account balance runs out. You can always charge your account balance by selecting Add funds.

The residential network plan has a monthly commitment, which is the amount charged at the beginning of every month and ensures this amount of traffic to be used this month.

When you don’t use all of your monthly commitment traffic, next month you’ll be charged the monthly commitment in order to add your plan traffic for that month.

In case you deposit more than your monthly commitment plan and don’t use all of it, we’ll roll over the remaining funds to the next month and charge only the delta to reach your monthly commitment.
All of your transactions and invoices can be found in the lower end of this page, looking in the transaction table.

I hope this webinar was helpful, please contact us with any additional questions.
You can always look at our very detailed Frequently Asked Questions page.

Thank you and have a lovely day.

Resource download

Already have an account? Log in
  • Mais de 20,000+ clientes em todo o mundo
  • Número 1 na categoria de dados da web
  • 5,500+ reivindicações de patentes concedidas e contando
  • Usado por: Fortune 500 Companies, instituições acadêmicas, negócios de médio porte, ONGs

confiável por

Alimentado por nossa rede proxy

já tem uma conta? Inscrever-se

  • Mais de 20,000+ clientes em todo o mundo
  • Número 1 na categoria de dados da web
  • 5,500+ reivindicações de patentes concedidas e contando
  • Usado por: Fortune 500 Companies, instituições acadêmicas, negócios de médio porte, ONGs

confiável por

Alimentado por nossa rede proxy
Already have an account? Log in
  • Mais de 20,000+ clientes em todo o mundo
  • Número 1 na categoria de dados da web
  • 5,500+ reivindicações de patentes concedidas e contando
  • Usado por: Fortune 500 Companies, instituições acadêmicas, negócios de médio porte, ONGs

confiável por

Alimentado por nossa rede proxy

já tem uma conta? Inscrever-se

  • Mais de 20,000+ clientes em todo o mundo
  • Número 1 na categoria de dados da web
  • 5,500+ reivindicações de patentes concedidas e contando
  • Usado por: Fortune 500 Companies, instituições acadêmicas, negócios de médio porte, ONGs

confiável por

Alimentado por nossa rede proxy

já tem uma conta? Inscrever-se

  • Mais de 20,000+ clientes em todo o mundo
  • Número 1 na categoria de dados da web
  • 5,500+ reivindicações de patentes concedidas e contando
  • Usado por: Fortune 500 Companies, instituições acadêmicas, negócios de médio porte, ONGs

confiável por

Alimentado por nossa rede proxy

já tem uma conta? Inscrever-se

  • Mais de 20,000+ clientes em todo o mundo
  • Número 1 na categoria de dados da web
  • 5,500+ reivindicações de patentes concedidas e contando
  • Usado por: Fortune 500 Companies, instituições acadêmicas, negócios de médio porte, ONGs

confiável por

Alimentado por nossa rede proxy
  • Mais de 20,000+ clientes em todo o mundo
  • Número 1 na categoria de dados da web
  • 5,500+ reivindicações de patentes concedidas e contando
  • Usado por: Fortune 500 Companies, instituições acadêmicas, negócios de médio porte, ONGs

confiável por

Alimentado por nossa rede proxy

Web Scraper IDE - Contact Us

  • Mais de 20,000+ clientes em todo o mundo
  • Número 1 na categoria de dados da web
  • 5,500+ reivindicações de patentes concedidas e contando
  • Usado por: Fortune 500 Companies, instituições acadêmicas, negócios de médio porte, ONGs

confiável por

Alimentado por nossa rede proxy
  • Mais de 20,000+ clientes em todo o mundo
  • Número 1 na categoria de dados da web
  • 5,500+ reivindicações de patentes concedidas e contando
  • Usado por: Fortune 500 Companies, instituições acadêmicas, negócios de médio porte, ONGs

confiável por

Alimentado por nossa rede proxy
Already have an account? Log in

Sign-up is required to get the dataset sample

  • Mais de 20,000+ clientes em todo o mundo
  • Número 1 na categoria de dados da web
  • 5,500+ reivindicações de patentes concedidas e contando
  • Usado por: Fortune 500 Companies, instituições acadêmicas, negócios de médio porte, ONGs

confiável por

Alimentado por nossa rede proxy

Download Dataset Sample

Sign-up is required to get the dataset sample

Get a free sample
Bright Insights eCommerce Report

  • Mais de 20,000+ clientes em todo o mundo
  • Número 1 na categoria de dados da web
  • 5,500+ reivindicações de patentes concedidas e contando
  • Usado por: Fortune 500 Companies, instituições acadêmicas, negócios de médio porte, ONGs

confiável por

Alimentado por nossa rede proxy

já tem uma conta? Inscrever-se

  • Mais de 20,000+ clientes em todo o mundo
  • Número 1 na categoria de dados da web
  • 5,500+ reivindicações de patentes concedidas e contando
  • Usado por: Fortune 500 Companies, instituições acadêmicas, negócios de médio porte, ONGs

confiável por

Alimentado por nossa rede proxy

Join our Partner Program

Dataset Sample Request

  • Mais de 20,000+ clientes em todo o mundo
  • Número 1 na categoria de dados da web
  • 5,500+ reivindicações de patentes concedidas e contando
  • Usado por: Fortune 500 Companies, instituições acadêmicas, negócios de médio porte, ONGs

confiável por

Alimentado por nossa rede proxy
  • Mais de 20,000+ clientes em todo o mundo
  • Número 1 na categoria de dados da web
  • 5,500+ reivindicações de patentes concedidas e contando
  • Usado por: Fortune 500 Companies, instituições acadêmicas, negócios de médio porte, ONGs

confiável por

Alimentado por nossa rede proxy
  • Mais de 20,000+ clientes em todo o mundo
  • Número 1 na categoria de dados da web
  • 5,500+ reivindicações de patentes concedidas e contando
  • Usado por: Fortune 500 Companies, instituições acadêmicas, negócios de médio porte, ONGs

confiável por

Alimentado por nossa rede proxy
Already have an account? Log in

GLOBAL MARKET LEADER

RECOGNIZED BY

Crozdesk Award - Happiest Users - High :User Satisfaction Award 2020
Award from Crozdesk for Quality Choice Top Ranked Solution 2020
AWARD - Crozdesk Trusted Vendor High Market Presence 2020
Agende uma chamada
  • Mais de 20,000+ clientes em todo o mundo
  • Número 1 na categoria de dados da web
  • 5,500+ reivindicações de patentes concedidas e contando
  • Usado por: Fortune 500 Companies, instituições acadêmicas, negócios de médio porte, ONGs

confiável por

Alimentado por nossa rede proxy