Error receive read timeout

[{«Product»:{«code»:»SS69YH»,»label»:»IBM SPSS Collaboration and Deployment Services»},»Business Unit»:{«code»:»BU059″,»label»:»IBM Software w/o TPS»},»Component»:»Not Applicable»,»Platform»:[{«code»:»PF025″,»label»:»Platform Independent»}],»Version»:»Not Applicable»,»Edition»:»»,»Line of Business»:{«code»:»LOB10″,»label»:»Data and AI»}}]

Nothing can be as annoying as getting timeout error when accessing an application.

One such error is FTP “Read timed out“. You are waiting for a successful FTP connection, but at the end it throws a timeout error.

Quite frustrating right? And, you know that the connection timed out, but where?

At Bobcares, we help website owners resolve these errors as part of our Dedicated Support Services for web hosts.

Today, we’ll discuss the top 4 reasons for ftp read timed out error and how we fix them.

What is FTP ‘Read Timed out’ error?

FTP Read Timed out error means that the client or server couldn’t read data from the source and has given up waiting for the information requested.

Users see the complete error message like this:

Failed to upload file
Establishing FTP connection failed: Read timed out
Read timed out

A Read timeout error explains a little about the error and the reason for the error, it just identifies that an error has occurred. So our Hosting Engineers analyze the FTP logs(/var/log/messages) to identify the origin of the issue.

FTP ‘Read Timed out’ error – Causes and Solutions

Let’s now see the main reasons for FTP read timed out error and how our Server Support Engineers fix them.

1) Firewall blocking Passive ports in server

The standard FTP ports are 20 and 21, and these ports should be opened in the server for proper functioning of FTP.

In addition to that, the FTP server should accept connections to Passive FTP ports which vary from server to server.

But, the problem is that most servers accept connections only on standard ports. And, if it’s not specifically set to access connections on Passive FTP ports, the incoming connections fail.

Consequently, users see FTP Read Timed out error.

Solution

Firstly, our Hosting Engineers confirm that the connectivity to the standard FTP ports 20 and 21 works well using the below command.

telnet hostname 20
telnet hostname 21

Secondly, we verify that the Passive port range is specified in the FTP configuration file.

For example, in a ProFTPd server, we un-comment the following directive in the configuration file /etc/proftpd.conf to specify the Passive port range.

PassivePorts 49152 65535

Finally, we open these passive port range in the server firewall.

For example, in a Linux server, we open the passive port range in firewall using the below command.

iptables -A INPUT -p tcp --match multiport --dports xxxxx:yyyyy -j ACCEPT

Here, xxxxx is the starting port, and yyyyy is the ending port in the Passive port range.

[Do you need help enabling Passive ports on your server? One of our Sever Experts can enable it for you in minutes.]

2) FTP client set to use Active mode

Another common reason for this error is that users enable Active mode in FTP client to transfer the files.

FTP transfers can happen in Passive and Active modes. However, active mode requires users to configure their PCs to connect to standard ports from the server.

In Active mode, the FTP client doesn’t make the actual connection to the server. Instead, it tells the server on which port it listens, and the server connects back to the specified port.

But this connection appears to be a cyber attack from the client side firewall, and hence it blocks such non-standard connections.

We’ve seen cases in which users have accidentally set their FTP mode to Active, resulting in FTP Read Timed out errors.

Solution

The solution here differs based on the FTP client software used by the users.

So, our Hosting Engineers first get the FTP client details used by the customer. And, we help users navigate their FTP client settings and enable Passive mode.

We always recommend users to enable Passive mode as the default option in their FTP clients.

[Not sure about the FTP settings to be used for your account? One of our Support Experts can help you here.]

3) Increase connection timeout limit

Usually, FTP Read Timed out error can occur when users try to upload a relatively large file. And, this problem is related to the internal timeout settings of the FTP client.

In other words, when users upload a large file, then the upload process may fail if it’s not completed within that predefined connection timeout limit.

Solution

In such cases, our Support Engineers help users to increase the Timeout values in their FTP clients accordingly.

Alternatively in some cases, we completely disable the Timeout value by setting it’s value as 0.

4) Intermediate firewall or routers block Passive ports

The majority of users are unaware of the firewall between their PC and server. This can be your network administrator or ISPs setting up a third-party firewall, intermediate firewall or a firewall on a router.

We’ve seen routers, proxies, etc. block connections through passive ports.

Solution

To resolve this, we ask users to turn off their gateways or routers to establish a direct connection. This helps us to determine if the block exists at the intermediate end.

Once, we’ve confirmed that the problem is with the intermediate devices, users can work with their network administrator or ISPs to set up the intermediate firewall to allow connections to passive ports.

Conclusion

In short, FTP Read Timed out error can happen due to server firewall settings, ftp client settings, and more. Today we’ve discussed the top 4 reasons of this error and how our Dedicated Support Engineers fix them.

var google_conversion_label = «owonCMyG5nEQ0aD71QM»;

The purpose of this article is to familiarize merchant services industry players with the mechanisms implemented in payment gateway software for the cases when transaction processing becomes impossible due to temporary loss of connection with the “back end” (for instance, a bank) or due to some other errors.

Problems with transaction processing might be caused by one of the two conceptually different reasons: timeouts and errors.

The first potential reason is a so-called timeout. Essentially, timeout means that there is a connection problem.

There are two types of timeouts sometimes referred to as ‘socket timeout (or connection timeout)’ and ‘read timeout’.

The key differences are as follows.

Connection timeout

Connection timeout usually occurs within 5 seconds. Connection timeout indicates that connection with the back end is impossible, and the server, to which the data needs to be transferred, cannot be reached. Issues with connection can be caused by DNS problems, server failure, Firewall rules blocking specific port, or some other reasons. In such cases the, a backup (or secondary) URL can be used (if available). If no connection can be established through either URL of a given service, further processing of transactions is impossible. It is important to note that no information is communicated to the host server when connection timeout occurs.

Read timeout

Read timeout usually occurs within 40 to 60 seconds. Read timeout occurs when the socket is open, connection to the host server is established, the request is sent, but the response from the server is not received on time, and cannot be read. Since the authorization request has already been sent, the transaction may have been processed (and approved), but, possibly, some error has occurred on the way back from the host server. As there is no clear response, there is a risk of charging the customer for the second time (if transaction is reattempted).

There are two approaches that are used by integrators to deal with situations like the one described above. These approaches are, generally, referred to as ‘authorization and capture’ (explicit or implicit capture) and ‘timeout reversal’.

Authorization-and-capture

The basic premise of authorization-and-capture approach is that any authorization processed has to be subsequently captured (confirmed) by an additional request. If timeout occurs, the submitting system does not send out the confirmation (since it never got the response), and, consequently, the host system reverses the authorization (because it has not been confirmed).

The capture operation can be executed in one of two ways: explicitly or implicitly. In case of explicit capture a separate ‘capture’ request is sent to the host server to confirm a previously successful authorization. In case of implicit capture, the reference number of the previously successful authorization that needs to be captured, is included as part of the subsequent authorization, or as part of the final settlement call.

In other words, in case of explicit capture the authorization request is submitted as one message and capture is sent as another separate message, while in case of implicit capture, authorization request is submitted as one message, while capture message (reference number of the successful authorization) is included in the authorization request of the next transaction. The final (end-of-the-day) message includes reference number of the last transaction which needs to be captured (confirmed).

Implicit capture is not recommended when time-initiated host capture is used.

Timeout reversal

Another approach is to explicitly reverse transaction with a host if a timeout has occurred. When a read timeout occurs, one or more attempts to reverse (or ‘void’) the authorization that timed out is made. When a transaction is submitted, and timeout occurs, it is assumed that the problem is temporary. In some cases up to three reversal attempts are made, each 40 seconds apart. It is assumed that within the next 160 seconds (40 seconds – initial waiting, plus 3 reversals) the problem of connection with the server will be resolved. If authorization is successful, subsequent reversals will void it (so that the customer will not be accidentally charged).

Conclusions

When a company integrates with some payment system (especially, if it plans to submit large transaction volumes in real time), it needs to consider and test the system’s processing strategy, keeping in mind the issues, related to read and connection timeouts.

(check apply)

• read the contribution guideline

Problem

i’m storing 570 ~ 600GB in 24 hours. (calculated by only string log size)
and my td-agent has two responsibility.

• storing string logs as files
• storing string logs as elasticsearch documents (of course, indicec has mapping)

file logs content is the same as elasticsearch documents. (1:1 match)
and my elasticsearch node is only 1(master, data, coordination all in one, this can be scaled out after day).

cpu usage avg is 10~20%.
8core, 64GB ram.

sometimes, when i push documents in elasticsearch via fluentd, ‘read timeout reached’ error caused.

and after a while, ‘retry succeed’ log has caused and input success.

but above error and warning is caused everytime.

in this time, buffer never flushed. just append as buffer files.

and after a while, buffer files too many created to input elasticsearch.

as a result, buffer files are infinitely increasing.

buffer files create speed >>>>> buffer files flush speed

it seems not just input-output data speed problem.

following is td-agent.log

2019-08-06 08:55:04 +0900 [warn]: #0 failed to flush the buffer. retry_time=0 next_retry_seconds=2019-08-06 08:55:05 +0900 chunk="58f676f133358450d1be0ce8155101fa" error_class=Fluent::Plugin::ElasticsearchOutput::RecoverableRequestFailure error="could not push logs to Elasticsearch cluster ({:host=>"127.0.0.1", :port=>9200, :scheme=>"http"}): read timeout reached"
  2019-08-06 08:55:04 +0900 [warn]: #0 suppressed same stacktrace
2019-08-06 08:55:05 +0900 [warn]: #0 failed to flush the buffer. retry_time=1 next_retry_seconds=2019-08-06 08:55:06 +0900 chunk="58f676f13352848f9efc996bbafcb211" error_class=Fluent::Plugin::ElasticsearchOutput::RecoverableRequestFailure error="could not push logs to Elasticsearch cluster ({:host=>"127.0.0.1", :port=>9200, :scheme=>"http"}): read timeout reached"
  2019-08-06 08:55:05 +0900 [warn]: #0 suppressed same stacktrace
2019-08-06 08:55:06 +0900 [warn]: #0 failed to flush the buffer. retry_time=0 next_retry_seconds=2019-08-06 08:55:07 +0900 chunk="58f650dc7d3fb60fde13ad3f6a8a39d6" error_class=Fluent::Plugin::ElasticsearchOutput::RecoverableRequestFailure error="could not push logs to Elasticsearch cluster ({:host=>"127.0.0.1", :port=>9200, :scheme=>"http"}): read timeout reached"
  2019-08-06 08:55:06 +0900 [warn]: #0 suppressed same stacktrace
2019-08-06 08:55:08 +0900 [warn]: #0 failed to flush the buffer. retry_time=1 next_retry_seconds=2019-08-06 08:55:09 +0900 chunk="58f650daeb7216733a89c42a421490cf" error_class=Fluent::Plugin::ElasticsearchOutput::RecoverableRequestFailure error="could not push logs to Elasticsearch cluster ({:host=>"127.0.0.1", :port=>9200, :scheme=>"http"}): read timeout reached"
  2019-08-06 08:55:08 +0900 [warn]: #0 suppressed same stacktrace
2019-08-06 08:55:16 +0900 [warn]: #0 failed to flush the buffer. retry_time=2 next_retry_seconds=2019-08-06 08:55:18 +0900 chunk="58f676f13352848f9efc996bbafcb211" error_class=Fluent::Plugin::ElasticsearchOutput::RecoverableRequestFailure error="could not push logs to Elasticsearch cluster ({:host=>"127.0.0.1", :port=>9200, :scheme=>"http"}): read timeout reached"
  2019-08-06 08:55:16 +0900 [warn]: #0 suppressed same stacktrace
2019-08-06 08:55:16 +0900 [warn]: #0 failed to flush the buffer. retry_time=3 next_retry_seconds=2019-08-06 08:55:20 +0900 chunk="58f676f133358450d1be0ce8155101fa" error_class=Fluent::Plugin::ElasticsearchOutput::RecoverableRequestFailure error="could not push logs to Elasticsearch cluster ({:host=>"127.0.0.1", :port=>9200, :scheme=>"http"}): read timeout reached"
  2019-08-06 08:55:16 +0900 [warn]: #0 suppressed same stacktrace
2019-08-06 08:55:20 +0900 [warn]: #0 retry succeeded. chunk_id="58f650dc7d3fb60fde13ad3f6a8a39d6"

in 08:55:04 ~ 08:55:16, ‘could not push logs to Elasticsearch cluster ({:host=>»127.0.0.1″, :port=>9200, :scheme=>»http»}): read timeout reached’ error has caused.

so, in this time, buffer has not been flushed.

in this time, destination index documents count was not increased.
(i check this via kibana)

this state is maintained until 08:55:20 and buffer files are continually created.

and 08:55:20, there is retry succeeded log has been occurred.

in this time, i checked buffer files directory, and buffer files was flushed in destination index
(i check this via kibana)

but buffer files flushed a liitle, and after a while, read timeout reached occurred.

Steps to replicate

• td-agent.conf

<match log.**>
        @type copy
        <store>
                @type forest
                subtype elasticsearch

                <template>
                         index_name ${tag}_%Y.%m.%d

                         <buffer tag, time>
                                @type file
                                path /log/${tag}.buffer

                               timekey_use_utc false
                               timekey_zone +0900

                               chunk_limit_size 51m # forward server's max chunk size is '50m', and forwarding every 1 second
                               flush_mode interval
                               flush_interval 1s

                               flush_thread_count 8

                               timekey 1m # chunks per hours ("3600" also available)
                        </buffer>

                        host 127.0.0.1
                        port 9200

#                       reconnect_on_error true  # default : false
#                       reload_on_failure true   # default : false
#                       reload_connections false # default : true
#                       request_timeout 10s # default : 5s

                        <secondary>
                                @type file
                                path /log/failed/${tag}.buffer
                        </secondary>
                </template>

        </store>
</match>

<source>
        @type forward
        port 24224
</source>

Expected Behavior or What you need to ask

how can i handle this error? any idea??

i tried request_timeoutand

reconnect_on_error true
reload_on_failure true
reload_connections false

but this options cant help me now.

request_timeout never given more then 10s. i will try more bigger value (example, 20s).
(but i am worried about this. i will try this later.)

and chunk_limit_size is 51m now, because log send server’s forward chunk limit size is 50m.
(forwarding server to receiving server flush_interval 1s,
receiving server to elasticsearch flush_interval 1s.)

i set receiving server’s chunk_limit_size as 4m before.
but if forwarding server’s data’s chunk size exceed 4m, my receiving server’s td-agent occurred following error.

2019-08-06 05:33:27 +0900 [warn]: #0 chunk bytes limit exceeds for an emitted event stream: 5665793bytes

(5665793bytes = 5.4MB.)

…

Using Fluentd and ES plugin versions

Fluentd : 1.3.3 (td-agent 3)
ES : 6.7

• OS version
  CentOS 7.6

• Bare Metal or within Docker or Kubernetes or others?
  neither Docker nor Kubernetes

• Fluentd v0.12 or v0.14/v1.0

  • paste result of fluentd --version or td-agent --version
    td-agent 1.3.3

• ES plugin 3.x.y/2.x.y or 1.x.y

  • paste boot log of fluentd or td-agent
  • paste result of fluent-gem list, td-agent-gem list or your Gemfile.lock
    elasticsearch (6.1.0)
    elasticsearch-api (6.1.0)
    elasticsearch-transport (6.1.0)
    fluent-plugin-elasticsearch (3.0.1)

• ES version (optional)
  ES 6.7

• ES template(s) (optional)