Python error 10054

Python-сообщество

Уведомления

#1 Дек. 3, 2011 17:31:47

error: [Errno 10054]

у меня есть две программки — одна сервер, а вторая — клиент, я хочу чтобы от клиента сообщение передавалось сообщение серверу!
у меня первое сообщение передается( а вот на втором вылетает( подскажите пожалуйста, что я не так делаю?

собственно код клиента:
import socket

HOST = ‘127.0.0.1’
PORT = 50033

while True:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT))

message = getMessage()
key = getKey()

data = getTranslatedMessage(message, key)

s.send(data)
print ‘Send: ‘, data
data = s.recv(8192)
s.close()
print ’Received: ‘, data

первое сообщение идеально отправляется, а вот на втором выводит ошибку:
Traceback (most recent call last):
File “C:UsersKatushaDesktopKateKatetask3easy_client.py”, line 50, in
s.send(data)
error:

Отредактировано (Дек. 3, 2011 17:44:03)

#2 Дек. 3, 2011 18:30:44

error: [Errno 10054]

Надо попробовать вынести из цикла
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

#3 Дек. 3, 2011 18:56:59

error: [Errno 10054]

а это дело ъ на стороне клиента? или может на сервере что-то криво написано?

Отредактировано (Дек. 3, 2011 19:05:56)

#4 Дек. 3, 2011 18:59:44

error: [Errno 10054]

при переносе тоже ошибка(причем теперь не отправляется ни одно сообщение

Traceback (most recent call last):
File “C:UsersKatushaDesktopKateKatetask3easy_client.py”, line 44, in
s.connect((HOST, PORT))
File “C:Python27libsocket.py”, line 224, in meth
return getattr(self._sock,name)(*args)
File “C:Python27libsocket.py”, line 170, in _dummy
raise error(EBADF, ‘Bad file descriptor’)
error: Bad file descriptor

а если перетащить за цикл с s.connect
то пишет —
Traceback (most recent call last):
File “C:UsersKatushaDesktopKateKatetask3easy_client.py”, line 44, in
s.connect((HOST, PORT))
File “C:Python27libsocket.py”, line 224, in meth
return getattr(self._sock,name)(*args)
File “C:Python27libsocket.py”, line 170, in _dummy
raise error(EBADF, ‘Bad file descriptor’)
error: Bad file descriptor

сообщение ввести предлагает( но увы оно даже не отсылается( а в моем варианте одно проходило идеально

Отредактировано (Дек. 3, 2011 19:03:24)

#5 Дек. 3, 2011 23:06:16

error: [Errno 10054]

надо строки, создающие соединение, вынести из цикла
цикл сделать конечным
соединение в цикле не закрывать

Отредактировано (Дек. 3, 2011 23:06:44)

#6 Дек. 3, 2011 23:38:50

error: [Errno 10054]

здорого =) работает! только у меня не получилось его сделать конечным, а вот с while 1 все заработало) спасибо! я все строки вносила и выносила из цикла — а тупо про закрытие сессии не подумала!

Отредактировано (Дек. 3, 2011 23:55:25)

#7 Дек. 5, 2011 01:18:48

error: [Errno 10054]

while True — плохой стиль (бесконечная программа)
надо определить, когда она закончится, и написать соответствующее условие
например, посылает десять сообщений

Источник

Issues

Context Navigation

#21227 closed Bug (fixed)

Selenium tests terminate with [Errno 10054]

Description

When I run the test suite on Windows with the —selenium option, I consistently see errors like this:

Based on some ​ anecdotal ​ data, I tried putting a refresh() before the quit() in contrib.admin.tests.AdminSeleniumWebDriverTestCase._tearDownClassInternal() and that reliably solved the problem.

I have no idea why this works, but if it does it’s probably worth doing since it should be harmless.

Attachments (1)

Download all attachments as: .zip

Change History (16)

There are some other places where we call selenium.quit() — should those places be updated as well?

docs/topics/testing/overview.txt: cls.selenium.quit()
tests/view_tests/tests/test_i18n.py: cls.selenium.quit()

I think it’s worth doing this wherever we’re calling selenium.quit() , since it seems harmless and does solve the spurious failures. Here’s a patch for it: ​ https://github.com/django/django/pull/1806.

However, I hesitate to give this workaround canonical status by putting it in the documentation, since it’s basically a bug in another project, could be fixed at any time, and adds a little extra noise and cognitive dissonance to the documentation. It only seems to pop up in some environments, and the workaround can be easily found with a quick internet search. On the other hand, for people developing reusable applications, it might be nice to warn them about this since it might affect them even if they aren’t seeing the bug in their own test environments. Maybe the wiki is the right place for this?

Fixed #21227 — Added workaround for selenium test failures

Added a refresh() before quit() in the selenium tests, since this
solves the problem of spurious test failures in some environments.

@marfire That workaround slows down firefox pretty much and should Be unneeded; can you test if this sequence works better for you:

@marfire: Is this a hard failure or just output on the console? Cause essentially everything which is a subclass of Exception is captured and more or less ignored.

Источник

iBase.ru Forum

Форум по InterBase, Firebird и Yaffil

• Темы без ответов
• Активные темы
• Поиск
• Наша команда

INET/inet_error: send errno = 10054

Модераторы: kdv, dimitr

INET/inet_error: send errno = 10054

Сообщение DSKalugin » 14 фев 2005, 19:38

P4 Mon Feb 14 17:35:07 2005
SERVER/process_packet: broken port, server exiting

P4 Mon Feb 14 17:35:07 2005
INET/inet_error: send errno = 10054

P4 Mon Feb 14 17:35:07 2005
SERVER/process_packet: broken port, server exiting

клиентское приложение виснет, на сервере горит индикатор жосткого диска сплошным красным. Приходится через некоторое время срывать программу и подключаться по новой.

Недавно сменил архитектуру с СС на классик версия 1,52
вин2003. В программе изменений никаких не делал.
Может ли это быть из-за того что создал новый индекс во время работы клиентов?

Сообщение Merlin » 14 фев 2005, 19:54

Events используются?
Других ошибок в логе нет?
Есть привязка к какому-то конкретному действию в приложении?

10054 — это сервер заметил, что издох клиент, не более того. Шнурок оборвал, ресет нажал и т.п. В сочетании с events возникал всякий разный гемор, но вроде ЦПУ жрал, а не диск. Несколько раз провозглашалось, что наконец обнаружено и уборото, но проверка в поле всегда права. Ужор диска при слабом потреблении процессора может говорить о сборке горы мусора. Само создание индекса при клиентах безвредно, но появление нового индекса может вести к изменению планов выполнения каких-либо запросов, у которых раньше возможности его использовать не было, иной раз к фатально неудачному. Это в плане привязки к действиям. Ну и ещё — если он жутко неселективный, это может способствовать образованию горы мусора при массированных удалениях и апдейтах.

Сообщение DSKalugin » 14 фев 2005, 20:18

Эвентов не использую, приложения как уже говорил действительно срываю «снять задачу» из за беспробудного висяка

насчет мусора точно подметил
я поставил свипинтервал в 0 и вызываю
gfix -sweep каждую ночь по расписанию
попытка установить обратно автосборку

«C:Program FilesFirebird152bingfix.exe» -housekeeping 30000 -user «SYSDBA» -password «masterkey» C:ShopDBU96.GDB
говорит unavailable database

как это понимать?

Сообщение Merlin » 14 фев 2005, 20:39

DSKalugin писал(а): «C:Program FilesFirebird152bingfix.exe» -housekeeping 30000 -user «SYSDBA» -password «masterkey» C:ShopDBU96.GDB
говорит unavailable database

как это понимать?

Сообщение kdv » 14 фев 2005, 20:55

насчет unavailable database — это мистическое сообщение лично меня уже достало. ибо у меня на W2000 не воспроизводится, а у людей случается как на FB так и на IB 7.x.

насчет 10054 и т.п. — есть один четкий случай умирания IB 7.1 SP2 на Win2003 Server, с похожими симптомами. Сначала все ОК, потом начинаются 10054, и кончается все это 10093 и неработой IB. На Win2000 все замечательно с тем же IB и тем же приложением. Интересно было бы услышать, не имеет ли кто аналогичных проблем на W2003 + FB 1.5.2, ибо подозрения на какую-то очередную несовместимость с tcp.

Сообщение Merlin » 14 фев 2005, 22:01

Сообщение Дмитрий » 15 фев 2005, 09:23

Сообщение dimitr » 15 фев 2005, 11:53

Сообщение DSKalugin » 15 фев 2005, 12:14

По порядку теперь:
1-в прошлый четверг сменил сервер с Fb SS 1.5.2 на Fb CS 1.5.2
Причину перехода я тут обосновал (из-за глюка в УДФ на одной базе перегрузился ФБ и отключились аварийно другие БД) Хотел сделать независимые процессы. Но в результате эти отдельные стали работать медленней чем на СС, иногда притормаживать.

2-свипинтервал поставил в 0. Сборку мусора делал ежедневно по расписанию gfix -sweep

3-вчера вечером для ускорения выполнения разовой процедуры создал 5 индексов на работающей БД. Процедура занималась массовым обновлением в пределах 3 тыс записей и удалением «лишних».
Сразу после этого начались длительные висяки. Работать не возможно.
Сегодня сутра тоже висяки были железные. Работать не возможно. Загадки да и только. ИБЭксперт подключаться не захотел. Говорит

Unsuccessful execution caused by an unavailable resource.
unavailable database.

-ИБАналист тоже подобным образом ругнулся
-локальный gfix тоже не хочет выполнять ни одно действие (unavailable database).
-Зато удаленно получилось положить базу в даун и подключиться ибэкспертом.
-FirstAID протестировал и сказал все ок, единственное 5 удаленных индексов показал.

Вобщем решил задачу так. Вернул все на место
перегрузил ВинСерв2003 — не помогло.

-Удаленно ибэкспертом удалил эти злополучные индексы.
-сделал бэкап
-деинсталлировал Fb CS
-установил по новой но уже SS
-провелил БД gfix.exe -v -full
на экране ничего в логе нашол потом
P4 (Client) Tue Feb 15 10:27:22 2005
Control services error 1061
-вернул свипинтервал в 30000 (перед нулем было 20000)
-всех включил, жалоб нет на тормоза.

что это было, так и не понял. Но тяга к экспериментам пропала.

Источник

How Socket Error Codes Depend on Runtime and Operating System

Rider consists of several processes that send messages to each other via sockets. To ensure the reliability of the whole application, it’s important to properly handle all the socket errors. In our codebase, we had the following code which was adopted from Mono Debugger Libs and helps us communicate with debugger processes:

In the case of a failed connection because of a “ConnectionRefused” error, we are retrying the connection attempt. It works fine with .NET Framework and Mono. However, once we migrated to .NET Core, this method no longer correctly detects the “connection refused” situation on Linux and macOS. If we open the SocketException documentation, we will learn that this class has three different properties with error codes:

• SocketError SocketErrorCode : Gets the error code that is associated with this exception.
• int ErrorCode : Gets the error code that is associated with this exception.
• int NativeErrorCode : Gets the Win32 error code associated with this exception.

What’s the difference between these properties? Should we expect different values on different runtimes or different operating systems? Which one should we use in production? Why do we have problems with ShouldRetryConnection on .NET Core? Let’s figure it all out!

Digging into the problem

If we run it on Windows, we will get the same value on .NET Framework, Mono, and .NET Core:

10061 corresponds to the code of the connection refused socket error code in Windows (also known as WSAECONNREFUSED ). Now let’s run the same program on Linux:

As you can see, Mono returns Windows-compatible error codes. The situation with .NET Core is different: it returns a Windows-compatible value for SocketErrorCode (10061) and a Linux-like value for ErrorCode and NativeErrorCode (111). Finally, let’s check macOS:

Here, Mono is completely Windows-compatible again, but .NET Core returns 61 for ErrorCode and NativeErrorCode . In the IBM Knowledge Center, we can find a few more values for the connection refused error code from the Unix world (also known as ECONNREFUSED ):

• AIX: 79
• HP-UX: 239
• Solaris: 146

For a better understanding of what’s going on, let’s check out the source code of all the properties.

SocketErrorCode

These values correspond to the Windows Sockets Error Codes.

NativeErrorCode

In .NET Core, the native code is calculated in the constructor (see SocketException.cs#L20):

The Windows implementation of GetNativeErrorForSocketError is trivial (see SocketException.Windows.cs):

The Unix implementation is more complicated (see SocketException.Unix.cs):

TryGetNativeErrorForSocketError should convert SocketError to the native Unix error code. Unfortunately, there exists no unequivocal mapping between Windows and Unix error codes. As such, the .NET team decided to create a Dictionary that maps error codes in the best possible way (see SocketErrorPal.Unix.cs):

Once we have an instance of Interop.Error , we call interopErr.Info().RawErrno . The implementation of RawErrno can be found in Interop.Errors.cs:

Here we are jumping to the native function SystemNative_ConvertErrorPalToPlatform that maps Error to the native integer code that is defined in errno.h. You can get all the values using the errno util. Here is a typical output on Linux:

Note that errno may be not available by default in your Linux distro. For example, on Debian, you should call sudo apt-get install moreutils to get this utility. Here is a typical output on macOS:

Hooray! We’ve finished our fascinating journey into the internals of socket error codes. Now you know where .NET is getting the native error code for each SocketException from!

ErrorCode

Writing cross-platform socket error handling

There was a lot of work involved in tracking down the error code to check against, but in the end, our code is much more readable now. Adding to that, this method is now also completely cross-platform, and works correctly on any runtime.

Overview of the native error codes

We executed this program on Windows, Linux, and macOS. Here are the aggregated results:

Источник

i am trying to set up communication with an ethernet device connected to my computer. Connection seems to work ok, when i connect the socket to the device’s IP address and port, but when I try to send a command to the device, it returns the [error 10054 — connection was forcibly interrupted by the remote host]. The code I have so far is like this:

import socket
import time
import logging



logging.basicConfig(filename='conn.txt', filemode='w')


logging.warning('Starting...')
address = ('192.168.1.23', 23)
soc = socket.socket(socket.AF_INET, socket.SOCK_STREAM)


ok = soc.connect_ex(address)


if ok == 0:
    logging.warning('Connected to device ' + str(ok))
else:
    logging.warning('Connection failed ' + str(ok))



### send at command
command = 'AT+SENSOR_IDS'
cmd_to_send = command + 'rn'
cmd_to_send = cmd_to_send.encode('ascii')
logging.warning('Sending at command')

soc.sendall(cmd_to_send)
logging.warning('at command sent, waiting for echo')


# read echo
echo = soc.recv()
logging.warning('received echo: ' + echo)

print('finished')

When i try to «reconnect» with another soc.connect_ex(address), it tells me that the socket is in use.

——

EDIT

——

So because I don’t know much about the device thanks to the lack of documentation, I decided on simulating the problem just using an Echo server and a client example on a localhost. I have this code:

Server side:

import socket

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_address = ('localhost', 1001)
s.bind(server_address)

s.listen(1)

# looking for clients
while True:
    print('waiting for connection')
    connection, client_address = s.accept()
    # when client connects
    try:
        print('Connection from %s port %s' % client_address )

        # get all the data and echo it back
        while True:
            data = connection.recv(32)
            print('Received:   ' + str(data))

            if(len(data)==0):
                print('No more data from client')
                break
            else:
                print('sending data to client')

    finally:
        connection.close()

Client side:

import socket

address = ('127.0.0.1', 1001)
soc = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

ok = soc.connect_ex(address)

try:
    if ok == 0:
        print('Connected to device ' + str(ok))
    else:
        print('Connection failed ' + str(ok))
except Exception as e:
    soc.close()
    print("connection not succesful, error returned :", e)

try:
    cmd_to_send = 'AT+SENSOR_IDSrn'.encode('ascii')
    bytes_sent = soc.sendall(cmd_to_send, address)
    print('Client sent %s bytes' % str(bytes_sent))

    # read echo
    echo = soc.recv()
    print('received echo: ' + echo)
    soc.close()
except:
    try:
        soc.close()
        print('process finished, didn't send at command')
    except:
        print('Process finished socket exception occured')

The problem persists in that way, that the client does not manage to even send the command. It looks like the connection closes just after it has been established. I am new to sockets and I haven’t found any good source for learning about the way how things work with them, only a few basic examples, could somebody help me so that it works at least when simulating on the localhost?

Thank you for your help

Python socket.error: [Errno 10054] the remote host forced an existing connection to close. Solution to the problem:

The other day I used python to read web pages. Because of the heavy use of the urlopen operation on a site, it is considered an attack by that site. Sometimes downloads are no longer allowed. After causing urlopen(), requeste.read () remains stuck. Finally, errno 10054 is thrown.

The error is connection reset by peer. The reason may be socket timeout time is too long; It could also be that after request = urllib.request-urlopen (url), there is no request-close () operation. It could also be that there is no sleep for a few seconds, leading the site to conclude that this behavior is an attack.

The specific solution is as follows:

01.import socket 
02.import time 
03.timeout = 20  
04.socket.setdefaulttimeout(timeout)# Here for the whole socket The layer sets the timeout time. If used again in subsequent files socket , no need to set   
05.sleep_download_time = 10 
06.time.sleep(sleep_download_time) # Here the time is set by itself   
07.request = urllib.request.urlopen(url)# This is where the content is going to be read url  
08.content = request.read()# Read, usually an exception will be reported here   
09.request.close()# Remember to close it  

Because the read() operation after urlopen is actually calling some function of the socket layer. So set the socket default timeout time, you can let the network break itself. You don’t have to wait forever at read().

Of course, you can also write several try outside,except, for example:

try: 
  time.sleep(self.sleep_download_time) 
  request = urllib.request.urlopen(url) 
  content = request.read() 
  request.close() 
   
except UnicodeDecodeError as e: 
     
  print('-----UnicodeDecodeError url:',url) 
   
except urllib.error.URLError as e: 
  print("-----urlError url:",url) 
 
except socket.timeout as e: 
  print("-----socket timout:",url) 

Generally speaking, there is no problem. I tested thousands of web downloads before I said this. But if I downloaded thousands of them, and I tested them, ms would still jump out of this exception. It could be time. Sleep () is too short, or it could be a sudden network outage. I tested it with urllib.request-retrieve () and found that there were always failures as I kept downloading data.

The easy way to do this is to first refer to my article: (link: #). Do a checkpoint first. Then, while True the above bit of code that will run an exception. See the pseudocode below:

def Download_auto(downloadlist,fun,sleep_time=15): 
  while True:     
    try: #  A layer of outsourcing try  
      value = fun(downloadlist,sleep_time) #  Here, fun Is your download function, I pass in when the function pointer.  
      #  Only normal execution can exit.   
      if value == Util.SUCCESS: 
        break 
    except : #  If that happens 10054 or IOError or XXXError 
      sleep_time += 5 # More sleep 5 Second, re-execute the above download. Because of the checkpoint, the above program continues execution from where the exception was thrown. Prevents program interruption due to unstable network connection.  
      print('enlarge sleep time:',sleep_time) 

However, for the corresponding web page, and to do another processing:

#  Print download information   
def reporthook(blocks_read, block_size, total_size): 
  if not blocks_read: 
    print ('Connection opened') 
  if total_size < 0: 
    print ('Read %d blocks' % blocks_read) 
  else: 
    #  If you can't find it, the page doesn't exist, maybe totalsize is 0 Cannot calculate the percentage   
    print('downloading:%d MB, totalsize:%d MB' % (blocks_read*block_size/1048576.0,total_size/1048576.0)) 
    
 
def Download(path,url): 
#url = 'http://downloads.sourceforge.net/sourceforge/alliancep2p/Alliance-v1.0.6.jar' 
  #filename = url.rsplit("/")[-1]  
  try: 
  # python The built-in download function   
    urllib.request.urlretrieve(url, path, reporthook) 
  except IOError as e: #  If you can't find it, it's going to cause it IOError .  
    print("download ",url,"/nerror:",e) 
  print("Done:%s/nCopy to:%s" %(url,path)) 

If you still have a problem… Please note other solutions in the comments.

Python is a simple, minimalistic, and easy-to-comprehend programming language that is globally-accepted and universally-used today. Its simple, easy-to-learn syntax can sometimes lead Python developers – especially those who are newer to the language – into missing some of its subtleties and underestimating the power of the diverse Python language.

One of the most popular error messages that new developers encounter when using requests library in Python is the “Max retries exceeded with URL” (besides timeout errors). While it seems simple, sometimes this somewhat vague error message can make even advanced Python developers scratching their head for a few good hours.

This article will show you what causes “Max retries exceeded with URL” error and a few ways to debug it.

Max retries exceeded with URL is a common error, you will encounter it when using requests library to make a request. The error indicates that the request cannot be made successfully. Usually, the verbose error message should look like the output below

Traceback (most recent call last):
  File "/home/nl/example.py", line 17, in <module>
    page1 = requests.get(ap)
  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 55, in get
    return request('get', url, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 44, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 383, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 486, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 378, in send
    raise ConnectionError(e)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='localhost.com', port=443): Max retries exceeded with url: /api (Caused by <class 'socket.gaierror'>: [Errno -2] Name or service not known)
Code language: JavaScript (javascript)

Sometimes, the error message may look slightly different, like below :

requests.exceptions.ConnectionError(MaxRetryError("HTTPSConnectionPool(host='api.example.com', port=443): 
    Max retries exceeded with url: /api.json (
    Caused by <class 'socket.error'>: [Errno 10054] 
    An existing connection was forcibly closed by the remote host)",),)
Code language: HTML, XML (xml)

requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=8001): Max retries exceeded with url: /api (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x10f96ecc0>: Failed to establish a new connection: [Errno 61] Connection refused'))
Code language: JavaScript (javascript)

requests.exceptions.ConnectionError: HTTPConnectionPool(host='www.example.com', port=80): Max retries exceeded with url: /api (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x0000008EC69AAA90>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed'))
Code language: JavaScript (javascript)

requests.exceptions.SSLError: HTTPSConnectionPool(host='example.com', port=443): Max retries exceeded with url: /api (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:997)')))
Code language: JavaScript (javascript)

The error message usually begins with requests.exceptions.ConnectionError, which tell us that there is something bad happened when requests was trying to connect. Sometimes, the exception is requests.exceptions.SSLError which is obviously a SSL-related problem.

The exception then followed by a more detailed string about the error, which could be Failed to establish a new connection: [Errno 61] Connection refused, [Errno 11001] getaddrinfo failed, [Errno 10054] An existing connection was forcibly closed by the remote host or [Errno -2] Name or service not known. These messages were produced by the underlying system library which requests called internally. Based on these texts, we can further isolate and fix the problems.

Double-check the URL

There are a possibility that your requested URL wrong. It may be malformed or leading to a non-existent endpoint. In reality, this is usually the case among Python beginners. Seasoned developers can also encounter this error, especially when the URL is parsed from a webpage, which can be a relative URL or schemeless URL.

One way to further debug this is to prepare the URL in advance, then print it before actually making a connection.

# ...
url = soup.find("#linkout").href
print(url) # prints out "/api" which is a non-valid URL
r = requests.get(url)
Code language: PHP (php)

Unstable internet connection / server overload

The underlying problem may be related to your own connection or the server you’re trying to connect to. Unstable internet connection may cause packet loss between network hops, leading to unsuccessful connection. There are times the server has received so many requests that it cannot process them all, therefore your requests won’t receive a response.

In this case, you can try increasing retry attempts and disable keep-alive connections to see if the problems go away. The amount of time spent for each request will certainly increase too, but that’s a trade-off you must accept. Better yet, find a more reliable internet connection.

import requests
requests.adapters.DEFAULT_RETRIES = 5 # increase retries number
s = requests.session()
s.keep_alive = False # disable keep alive
s.get(url)
Code language: PHP (php)

Increase request timeout

Another way that you can avoid “Max retries exceeded with URL” error, especially when the server is busy handling a huge number of connections, is to increase the amount of time requests library waits for a response from the server. In other words, you wait longer for a response, but increase the chance for a request to successfully finishes. This method can also be applied when the server is in a location far away from yours.

In order to increase request timeout, simply pass the time value in seconds to the get or post method :

r = requests.get(url, timeout=3)

You can also pass a tuple to timeout with the first element being a connect timeout (the time it allows for the client to establish a connection to the server), and the second being a read timeout (the time it will wait on a response once your client has established a connection).

If the request establishes a connection within 2 seconds and receives data within 5 seconds of the connection being established, then the response will be returned as it was before. If the request times out, then the function will raise a Timeout exception:

requests.get('https://api.github.com', timeout=(2, 5))



  


                       
  



Code language: JavaScript (javascript)

Apply backoff factor

backoff_factor is an urllib3 argument, the library which requests relies on to initialize a network connection. Below is an example where we use backoff_factor to slow down the requests to the servers whenever there’s a failed one.

import requests
from requests.adapters import HTTPAdapter
from requests.packages.urllib3.util.retry import Retry

session = requests.Session()
retry = Retry(connect=3, backoff_factor=1)
adapter = HTTPAdapter(max_retries=retry)
session.mount('http://', adapter)
session.mount('https://', adapter)

session.get(url)
Code language: JavaScript (javascript)

According to urllib3 documentation, backoff_factor is base value which the library use to calculate sleep interval between retries. Specifically, urllib3 will sleep for {backoff factor} * (2 ^ ({number of total retries} - 1)) seconds after every failed connection attempt.

For example, If the backoff_factor is 0.1, then sleep() will sleep for 0.0s, 0.2s, 0.4s, … between retries. By default, backoff is disabled (set to 0). It will also force a retry if the status code returned is 500, 502, 503 or 504.

You can customize Retry to have even more granular control over retries. Other notable options are:

• total – Total number of retries to allow.
• connect – How many connection-related errors to retry on.
• read – How many times to retry on read errors.
• redirect – How many redirects to perform.
• _methodwhitelist – Set of uppercased HTTP method verbs that we should retry on.
• _statusforcelist – A set of HTTP status codes that we should force a retry on.
• _backofffactor – A backoff factor to apply between attempts.
• _raise_onredirect – Whether, if the number of redirects is exhausted, to raise a MaxRetryError, or to return a response with a response code in the 3xx range.
• raise_on_status – Similar meaning to _raise_onredirect: whether we should raise an exception, or return a response, if status falls in _statusforcelist range and retries have been exhausted.

We hope that the article helped you successfully debugged “Max retries exceeded with URL” error in Python requests library, as well as avoid encountering it in the future. We’ve also written a few other guides for fixing common Python errors, such as Timeout in Python requests, Python Unresolved Import in VSCode or “IndexError: List Index Out of Range” in Python. If you have any suggestion, please feel free to leave a comment below.