Using a Residential Proxy

Avoid bot detection with a residential proxy

What is a residential proxy?

Some websites have mechanisms to detect and block bot traffic coming from datacenter IPs. Residential proxies are networks of residential IP addresses that proxy requests, making them look like regular human traffic. Airtop has support for using residential proxies to avoid detection.

Proxy Options

You can use a proxy in 2 different ways:

  1. Specifying that you want to use Airtop’s integrated proxy service when creating a session. With this option, you don’t have to do anything else and you’ll be charged for the bandwidth that your session uses.

  2. You can also bring your own proxy. If you’d like to use your own proxy, Airtop doesn’t charge for the data that your session uses, and you handle the billing for the proxy yourself with your provider of choice. We have a list of recommended proxy providers below.

How to use Airtop’s integrated proxy service

Using Airtop’s integrated proxy service is as simple as passing true to the proxy parameter when creating a session.

1await client.sessions.create({
2  configuration: {
3    proxy: true,
4  },
5});

Integrated Proxy Restrictions

The integrated proxy service has a few domains that it cannot access. If you try to access one of these domains, the session will fail. These domains are:

  • Some streaiming services like (e.g., Netflix)
  • Some banking and other financial institutions
  • Ticketing sites like Ticketmaster
  • Linkedin

If you need to access one of these domains, you can use a custom proxy instead. There are several providers listed below that don’t have these restrictions.

How to configure a custom proxy

You can specify a custom proxy in the sessions.create function. proxy can either take a string or an object with url, username, and password properties.

1await client.sessions.create({ 
2  configuration: {
3    proxy: 'https://username:password@my-proxy-provider.com',
4  },
5});
6
7await client.sessions.create({
8  configuration: {
9    proxy: {
10      url: 'https://my-proxy-provider.com',
11      username: 'my-proxy-username',
12      password: 'my-proxy-password',
13    },
14  },
15});

Limiting Proxy Usage to Specific Domains

You can limit proxy usage to specific domains by passing an array of { domainPattern, relay } objects to the proxy parameter. This gives you the flexibility to only use the proxy for certain domains, or to use a different proxy for each domain.

In this example, the first object in the array will apply to all requests to all subdomains of wikipedia.org, while the second object will apply to all other domains.

1await client.sessions.create({
2  configuration: {
3    proxy: [
4      { domainPattern: '*\.wikipedia\.org', relay: 'https://username:password@my-proxy-provider.com' },
5      { domainPattern: '*', relay: 'https://username:password@my-other-proxy-provider.com' },
6    ],
7  },
8});

In this example, only requests to subdomains of wikipedia.org will use the proxy, while all other requests will not use any proxy.

1await client.sessions.create({
2  configuration: {
3    proxy: [
4      { domainPattern: '*\.wikipedia\.org', relay: 'https://username:password@my-proxy-provider.com' },
5    ],
6  },
7});

domainPattern is a string where * is a wildcard that matches any string. You can also use the ? character to match a single character.

Different proxy providers have different pros and cons. These are 3 that we’ve tested and have had good results with, but you should do your own due diligence to see which one will work best for your use case.

  1. Oxylabs
  2. Smartproxy
  3. IPRoyal
Built with