Table of Contents
General definition of a proxy
Residential proxies (like any other proxy) act as a middle-man between a personal computer and another server or server-network. A proxy server fakes the signature of a personal computer. There are various use cases for using a proxy server. A few examples:
- The proxy user wants to access geo-targeted content
- The user wants to stay anonymous
- The user wants to get around blocking mechanisms
What is a residential proxy?
Generally, you have to differentiate between two different proxy types:
- Datacenter Proxies
- Residential Proxies
Residential Proxies are real residential IPs that are distributed to individuals by internet service providers (ISP). Each residential IP address is bound to a dedicated desktop or mobile device and contains information about the device’s ISP, location and Network.
The special thing about residential IPs is that they are usually perceived as the IP addresses of real people. Hence, they are ideal to access sites that are trying to minimise traffic from IP addresses that are related to data centers and scraping activities.
Residential proxies networks for web scraping
As written above, residential IPs are usually granted initial access to protective websites. However, these sites tend to continuously check the user behaviour throughout the user session.
Let’s say you are trying to scrape 100 pages from a website that has imposed advanced anti scraping measures. Using a residential proxy will allow you to scrape the first few pages. At that point, the server is going to notice that you are following a programmatic and unhuman behaviour. Hence, it might block your IP or confront you with a CAPTCHA.
This is where residential proxy networks come in. Residential proxy networks consist of a range of residential IP addresses. To prevent a webserver from noticing your programmatic user behaviour, the IP address is rotated after every request. This way, it is impossible for the webserver to block you from scraping content.
No more need for proxy management
Zenscrape offers a web scraping API that handles ip rotation and proxy management for you. Simply submit a URL to our API and we will return the HTML markup of the website. We offer a free plan with 1000 free requests / month.