Proxy error general error

• Перейти на CTF площадку Игры Кодебай

  Выполняйте задания по кибербезопасности в формате CTF и получайте бесценный опыт. Ознакомиться с подробным описанием можно тут.

• B правой части каждого сообщения есть стрелки ⇧ и ⇩. Не стесняйтесь оценивать ответы. Чтобы автору вопроса закрыть свой тикет, надо выбрать лучший ответ. Просто нажмите значок ✔ в правой части сообщения.

ClientTransportPlugin obfs4 exec /usr/bin/obfs4proxy
Bridge obfs4 198.244.141.200:29719 4947CC618E8729E99D807ABD66B5293DC416ECBE cert=WTgo6GrVKC8bjUPYO+2qDP19DyftOJjhW+4Kdi0qTC>
Bridge obfs4 135.125.217.239:52127 5A0CD52337DF2109B2E2B8C98D4E3BDD2A99EC29 cert=l8ppFuKqaHxJNBIao1z2O6K32nSYNceS7Hkp8STg5z>

Nov 29 13:07:23.420 [notice] Read configuration file "/etc/tor/torrc".
Nov 29 13:07:23.425 [notice] Opening Socks listener on 127.0.0.1:9050
Nov 29 13:07:23.425 [notice] Opened Socks listener connection (ready) on 127.0.0.1:9050
Nov 29 13:07:23.000 [notice] Parsing GEOIP IPv4 file /usr/share/tor/geoip.
Nov 29 13:07:23.000 [notice] Parsing GEOIP IPv6 file /usr/share/tor/geoip6.
Nov 29 13:07:23.000 [notice] Bootstrapped 0% (starting): Starting
Nov 29 13:07:23.000 [notice] Starting with guard context "default"
Nov 29 13:07:24.000 [notice] Bootstrapped 5% (conn): Connecting to a relay

11/29/22, 12:28:37.567 [WARN] Proxy Client: unable to connect OR connection (handshaking (proxy)) with 198.244.141.200:29719 ID=<none> RSA_ID=4947CC618E8729E99D807ABD66B5293DC416ECBE ("general SOCKS server failure")

Have you ever been frustrated by proxy error codes that you have received while using proxies, but you have no idea why? Proxy error codes are similar to HTTP status codes. By learning what these errors mean, you can smoothly perform your scraping activities and automate your IP settings as well. 

It is very likely to receive such errors if you do not adequately manage your proxies during crawling or scraping activities. Problems can occur either from your end (client-side) or from the server-side. You will learn the most common types of proxy errors, why you receive them, and how to solve them by further reading this article.

What is a Proxy Error?

A proxy error is an HTTP error status that you will receive as a response when a request sent to the web server via a proxy fails. To keep using the proxy, you have to find a solution no matter what the case is. The returned response during a request failure may seem a bit confusing. If you are conversant about HTTP status codes, understanding proxy errors is not an issue.
 

Status Code Classes

HTTP Status codes are displayed in three digits. They are grouped into five classes, such that the first digit of each error code depicts the class.

They are;

1. 1xx – Informational

2. 2xx – Success

3. 3xx – Redirection

4. 4xx – Client Error

5. 5xx – Server Error
 

Common Proxy Error Codes and Their
Solutions

1xx Informational Error Code

These are provisional responses that are rarely used. These requests are considered to be used when the server is processing the requests;

100 – Continue

The code indicates that a part of the request is received, and the client can continue sending the remaining request. Typically, the client first sends a request header with a statement “Expect: 100-continue” and receives the 100 status code from the server to send the request’s body. The “expect” in the initial request is to avoid sending further requests if the server rejects the initial request header.

101 – Switching Protocols

A web server sends a 101 status code when the browser requests to change the communication protocol during a transaction. When the client browser’s request asks the server to switch communication protocol and accepts it, it sends the HTTP status code “100 – Switching Protocols” as an acknowledgment.

102 – Processing (WebDAV)

The web server might need some considerable time to process complex requests. When a client’s browser sends a WebDAV request with multiple sub-requests involving complex requirements, the server takes some time to process and eventually sends this code “102 – Processing”. This code aims to avoid timeout errors on the client-side by informing the client that the server received the request and processing it.

103 – Early Hints

The code “103 – Early Hints” is received by the webserver when sending the HTTP status to the browser before processing the HTTP requests. The name indicates this is an early hint to the client’s browser that the server has not started processing the requests.
 

2xx Successful Status Code

Receiving codes between 200 and 299 means that your proxy received your HTTP request, sent it to your intended website, and got a response. 200 is the most received code, informing that the server has fulfilled the request successfully. Pay attention to any other 2xx code other than 200 OK as it may show an error.
 

Here are the most common 2xx status codes;

201 – Created

This status code indicates the server has completed the client’s received request and has created a new resource based on the received request. The initial request is acting as a trigger to generate a new response in the server. For example, when a server is generating a new response based on a user’s login details.

202 – Accepted

“202 – Accepted” is returned when the server receives the request from the client, but it has yet to process. That’s the only indication for accepting the request, but the result for processing the request will be known later on when the “actual processing” takes place.
 

203 – Non-Authoritative Information

The code “203 – Non-Authoritative Information” was received when the server successfully processed the request but returned the information from another resource to the requested client.
 

204 – No Content

The server cannot find any content for the received request; it sends a “204 – No Content” response code, indicating to the client that no content is returned.
 

205 – Reset Content

Similar to the 204 code above. A request was processed successfully by the server, but no content is returned. The only difference here is that the 205 code informs the client to reset the document view.
 

206 – Partial Content

A server will return this error code when it sends a part of the requested resource affected by the range mentioned in the request header. 

E.g., the client requests a range of files to download and uses multiple streams to download the complete content.
 

3xx – Redirection Error

3xx codes indicate that additional action is needed from the client-side to fulfill the request.

These status codes won’t be an issue when using a browser (for example, Google Chrome or Safari), but when using your own script. Scripts that you write, and when there’s no need to redirect the requests to other URLs, will come in handy.

As these actions can create infinite loops, web browsers usually don’t follow more than five consecutive redirections of the same request.
 

Some of the most common 3xx error codes are as follows;

300 – Multiple Choices

It occurs when a requested URL is pointing towards more than one resource. 

The user agent (crawler or web browser) cannot decide which page to fetch, and an HTTP code “300 – Multiple Choices “is received. 

Fix 300 error code by checking the HTTP headers and make sure the URL is pointing to a single resource – so that the user agent can access the page successfully.
 

301 – Resource Moved Permanently

This error is received when there’s a permanent redirection set to an original URL to forward the user agent to a different URL.

When a web server sends a “301 – Moved Permanently” status code as a user can’t see the original URL, search engines would only index the redirected URL. Most search engine crawlers and user agents can follow up to 5 redirects for a single URL. More than five redirects could result in an infinite loop, and browsers like Chrome will show a message like “Too Many Redirects”.

A 301 code is the most popular out of all 3xx server response codes.
 

302 – Resource Moved Temporarily

A code “302 – Moved Temporarily” is received when a temporary redirect is set to the original URL. It means the user agent is redirected to another URL once making the request.

303 – See Another Resource

A “303 – See Another Resource” is received when the requested resource is located in another URL address, and it should be requested by using a “GET” method rather than code. Note that the initially requested page will be indexed by the search engines only when the “200 – Success” code is received.

304 – Resource Not Modified

A server would respond with the “304 – Resource Not Modified” code if the requested resource hasn’t been modified since the request’s last time. 

In this case, the server would assume that there’s no need for sending the data again, since the client already has a copy of the requested resource, which was not modified.

Note that the “last time of modification” is mentioned in the request header line “If-Modified-Since” or “If-Match”.

Plus, if your web page has not changed since the last time the search engine(s) crawler has accessed your site, then returning the 304 code is recommended to speed up indexing and reduce the crawler load.

305 – Use proxy

The code “305 – Use Proxy” will appear if the requested resource can only be accessed through a proxy server. The proxy server’s address is also received in the response and displayed in the browser’s window. 

Some browsers like Internet Explorer will not display this response properly due to the security concern of displaying the proxy server.

306 – Switch Proxy

“306 – Switch proxy” code indicates the server should use the specified proxy for the following request(s).

307 – Temporary Redirection

Code “307 – Temporary Redirect” is received when a requested resource is moved temporarily to a different address, mentioned in the Location header of the request. This is just a temporary redirect, but the next request should access the original URL. BTW, only HTTP/1.1 protocol uses this status code.

308 – Permanent Redirect

A “308 – Permanent Redirection” code is an experimental code to indicate a permanent redirection (similar to 307, which is set as a temporary redirection). 307 and 308 codes are similar to the 302 and 301 codes, with the slight difference in not changing the HTTP method.

4xx Client Error Codes

The main types of HTTP proxy errors are 4xx and 5xx error codes. Receiving an error from the 4xx series indicates that the problem is from the client-side. It can be your request, browser, or the automation bot.

400 – Bad Request

It is a generic response indicating that there’s a problem with your sent request. Sometimes it can be that your proxy server or the target website is unable to parse your request. Problems can be malformed syntax, invalid formatting, or deceptive request routing.

401 – Unauthorized

“401 – Unauthorized” error code indicates that you are trying to access an unauthorized website where you have to authenticate yourself. 

The error is returned by the proxy server when the webserver requires authentication and authorization. Providing credentials will allow you to access the resource.

402 – Payment Required 

This response code is mostly set for future usage. Even though this status code is relatively rare and no standard convention exists, the aim for creating this code was for digital payment systems.

403 – Forbidden

403 indicates that your request is valid and understood by the proxy or the webserver but refuses to respond. It happens when you have no permission to view the resource.

404 – Not Found

This code is returned by the proxy server when the requested online resource is not available, even when the request is valid. Although 404 is primarily known as a “client error,” it results from dead links. It could mean that the URL has been taken down, incorrect, or changed without redirection.

405 – Method Not Allowed

405 is received when a request method is known by the server but has been disabled and can’t be used. For example, an API request may forbid 

“DELETE-ing” a resource. Both mandatory methods, GET and HEAD, must never be disabled and should not return this error code.

406 – Not Acceptable

A response is sent when the web server doesn’t find any content that conforms to the criteria given by the user agent after performing server-driven content negotiation.

407 – Proxy Authentication Required

A proxy indicates a 407 code when it requires authentication or when a tunnel fails to connect. It can happen when your scraper is not authenticated correctly with the proxy provider or when credentials are inaccurate. Another reason can be not whitelisting your IPs within the proxy settings.

Solving this error requires you to update your proxy settings by including whitelisted IPs and enter proper credentials. Plus, make sure that all the required information is included in the request as well.

408 – Request Timeout

This error code is received when a client hasn’t produced a request while the server is configured to wait or hold. The client may repeat the request without making modifications later at any time. 

If the 408 error is persistent, check the load created on your web server when detecting the errors. One more possibility might be connectivity problems.

409 – Conflict

The 409 – Conflict is usually unrelated to standard web server authority or security but to a specific application, conflicts not defined in the HTTP protocol itself.

The web server is responding with this error when considering the client’s requests as legitimate ones, but a request could not be completed due to a conflict with the current state of the resources. The response body usually includes sufficient information for users to recognize the conflict’s source and fix the error.

410 – Gone

The web server responds with this error code when the requested resource is no longer available via the server, will not be available again, and has no know forwarding address. This error is similar to the 404 error, but 410 is a permanent one.

411 – Length Required

This error code means that the server is refusing to accept the request without a defined content length. The client should repeat the request by adding a valid content-length header field, which contains the length of the message-body in the request’s message.

412 – Precondition Failed

The webserver responded with this error code when preconditions are given in one or more of the request-header fields that have been evaluated as false when tested on the server. 

This status code allows the client to place preconditions on current resources meta-information (header field data) and prevents the requested method from applying to a resource other than the initially intended one.

413 – Request Entity Too Large

The server’s refusing to process a request since the request entity is larger than what the server can process. The server could close the connection to prevent the client from sending more of the same request. 

“What constitutes ‘too large’ depends in part on the operation being attempted. For example, a request to upload large files (via the HTTP PUT method) may encounter limitations on upload file size set by the webserver.”

414 – Request-URL Too Long

The web server is responding with this error when refusing to service the request since the Request-URL is longer than the server can process. This rare condition is more likely to occur when a client has improperly converted “POST” requests to “GET” requests, with long query information once the client has descended into a URL redirections “black hole” (meant that a redirected URL prefix which is pointing a suffix of its own), or 

This error code could also be received when the server is attacked by a client’s attempts to exploit any security holes that are present in some servers, also using fixed-length buffers for Request-URL reading and manipulating. Typically, webservers are setting fairly generous genuine URLs limits on length. In case that a long URL is valid and you are still receiving a 414 error code, it means that the web server may need to be reconfigured to allow such URLs through.

415 – Unsupported Media Type

The webserver is refusing to complete the request since the entity of the request is in formats that are not supported by the requested resource for the requested method.

416 – Requested Range Not Satisfiable

416 status code is received when a server’s response usually returns with 416 if a request includes a “Range” request-header field. No range-specific values in this field overlap the current extents of selected resources, and the request didn’t have the If-Range request-header field. 

For example, if the resource is a file with 1000 bytes, and the Range requested is 500-1500, it cannot be sufficed as an acceptable range.

417 – Expectation Failed

The web server usually responds with this status code when the expectation received in an “Expect” request-header field cannot be fulfilled by the server or if the server is a proxy and has clear evidence showing that the next-hop server could not fulfill the request.

429 – Too Many Requests

This error is likely to be received when sending too many requests within a limited time frame while using the same IP address. Websites usually implement such restrictions to protect from attackers and to avoid overloading.

Using rotating proxies, setting delays between requests per IP and per a particular time frame can solve this error code.
 

5xx – Server Error

Series of 5xx is returned when the server receives the request successfully but cannot process the request or encounters a problem while processing it.

To solve all these 5xx errors, rotate the IPs, and change the proxy network and IP type. Using a residential proxy network is better in such cases to rotate IPs and ensure reliability.

You may receive error codes such as – 

500 – Internal Server 

Error code “500 – Internal Server “is received when the server encounters an unexpected condition, stopping it from responding to the request.

501 – Not Implemented

The “501 – Not Implemented” error is received when the server can’t provide the requested resource because of an unsupported or unrecognized method(s) used in the request.

502 – Bad Gateway

This error will frequently occur during data gathering when the server acts as a gateway or a proxy and receives an invalid response from another server. 

When super proxies refuse the internet connection or requests sent, IPs’ unavailability for chosen settings is detected as bots indicate 502 code.

503 – Services Unavailable

The “503 – Service Unavailable” code is received when a server receives the request at the same time other requests overload it, or it’s under planned downtime/maintenance. In this case, and if possible, check the status of the requested server.

504 – Gateway Timeout

“504 – Gateway Timeout” is received when a server (a) acts as an external gateway or a proxy and does not receive the response on time from the next server (b), further up in the request chain which tries to access to fulfill the request.

505 – HTTP Version Not Supported

A “505 – HTTP Version Not Supported” code is received when a server isn’t supporting the HTTP protocol version and used in the request message.

507 – Insufficient Space

“507 – Insufficient Storage” means that the server runs out of disk space and no more accommodates the request.

510 – Extensions are Missing

The server can’t process the request because an unsupported extension is requested, then the code “510 – Not Extended” is received.
 

Solving Common Proxy Error Codes

The easiest way to solve these proxy error codes is by using a proxy manager. A proxy manager is often a free and open-source software that automates proxy management to avoid such errors. To prevent the 407 error code, You have to choose a proxy port. Then, all the associated credentials and zone information are automatically updated. 

You can apply specific actions within your proxy manager when a rule you applied gets triggered. To avoid errors such as 403, rotating your IPs is essential; hence residential proxies are the best option. NetNut can play its part by integrating with your proxy tools to avoid these errors. Learn how to integrate NetNut to configure your proxy settings.

Check out our GitHub repositories as well for a better understanding of advanced usage with our proxies, code documentation, and walkthroughs.

To summarize, the basic steps that you can follow to solve these proxy errors are;

1. Switching to Residential Proxies

Although residential proxies are a bit more expensive, they provide a large pool of proxies, unlike data center proxies. Hence you can rotate your IPs and avoid getting blocked.

NetNut is a better example of a residential proxy service provider, using a dynamic P2P + ISP Proxy Network, offering both Rotating & Static Residential IPs.

2. Decrease the number of requests

Sending too many requests at the same time seems suspicious by any website. Set a slight delay between requests to avoid any errors.

3. Improve IP Rotation

As explained above, using a proxy management tool helps to achieve this task. Control your IP sessions to minimize requests made with the same IP address.

4. Have a Well-Performing Scraper

Following the above factors but using a poor-performing scraper can still give you errors. Hence, make sure that you have an advanced scraper to bypass obstacles implemented by websites.
 

Remember!

The first steps in overcoming these proxy errors are understanding the error code and why you get such code. Understanding such causes while implementing these techniques can let you perform your data gathering smoothly, minimizing errors.

Can’t figure out a solution to your error code? Ping us via email and our support experts can help you out!

Have you encountered an issue that isn’t listed here? Let us know!