Winscp network error software caused connection abort

I have a scheduled job to perform secure file transfers daily through WinSCP. The flow starts with 2 ssrs file share subscriptions which drops 2 files in a local directory. Then the sql server agent job I scheduled kicks off a powershell script which initiates the winscp sftp transfer of the 2 files to the remote SFTP server.

The job and file transfers were all working fine and dandy for 2 months until randomly last week when the job started failing on some days (but not others). The logs show a cryptic network error «Software caused connection abort». I’ve checked winscp’s support docs here https://winscp.net/eng/docs/message_software_caused_connection_abort, here https://winscp.net/eng/docs/message_unexpected_close here https://winscp.net/eng/docs/faq_connection_refused and other sources here https://www.hostingride.in/content/solved-network-error-software-caused-connection-abort-winscp-error-putty-keepalives

Based on suggestions found I:

a) Added an exception to windows firewall to allow winscp through.

b) Enabled «Executing dummy protocol commands» under Keepalives in WinSCP Advanced Site Settings

c) Increased the server response timeout and seconds between keepalives to 360 seconds each

None of these fixed the problem. And oddly the file transfer is not consistently failing every day. Running the job repeatedly (same code) fails usually but occasionally succeeds (see powershell screenshot attached)

If anyone is familiar with how to fix this issue please let me know. See log entry and powershell screenshot attached.

. 2019-04-25 16:39:28.997 --------------------------------------------------------------------------
. 2019-04-25 16:39:28.998 Looking up host "REDACTED" for SSH connection
. 2019-04-25 16:39:29.002 Connecting to REDACTED port 22
. 2019-04-25 16:39:29.062 We claim version: SSH-2.0-WinSCP_release_5.13.1
. 2019-04-25 16:39:29.122 Network error: Software caused connection abort
. 2019-04-25 16:39:33.594 --------------------------------------------------------------------------
. 2019-04-25 16:39:33.594 WinSCP Version 5.13.1 (Build 8265) (OS 6.2.9200 - Windows Server 2012 Standard)
. 2019-04-25 16:39:33.594 Configuration: HKCUSoftwareMartin PrikrylWinSCP 2
. 2019-04-25 16:39:33.594 Log level: Normal
. 2019-04-25 16:39:33.594 Local account: REDACTEDebarnes
. 2019-04-25 16:39:33.594 Working directory: D:Div32_Daily_Reports
. 2019-04-25 16:39:33.594 Process ID: 205196
. 2019-04-25 16:39:33.595 Command-line: "D:Div32_Daily_ReportsWinSCP.exe" /console=5.13.1 /consoleinstance=_5176_503 "/script=SyncTo_bics_inventory_v2.txt" "/log=D:Div32_Daily_Reportswinscp_log_bics_inventory.txt" 
. 2019-04-25 16:39:33.595 Time zone: Current: GMT+0 (Coordinated Universal Time), No DST
. 2019-04-25 16:39:33.595 Login time: Thursday, April 25, 2019 4:39:33 PM
. 2019-04-25 16:39:33.595 --------------------------------------------------------------------------
. 2019-04-25 16:39:33.595 Script: Retrospectively logging previous script records:
> 2019-04-25 16:39:33.595 Script: option batch abort
< 2019-04-25 16:39:33.595 Script: batch           abort     
> 2019-04-25 16:39:33.595 Script: option confirm off
< 2019-04-25 16:39:33.595 Script: confirm         off       
> 2019-04-25 16:39:33.595 Script: open sftp://a427721x:***@REDACTED -hostkey="ssh-rsa 2048 ad:b3:c7:a1:ff:43:5c:71:d6:a7:50:6c:1a:c8:d1:70"
. 2019-04-25 16:39:33.595 --------------------------------------------------------------------------
. 2019-04-25 16:39:33.595 Session name: a427721x@REDACTED (Ad-Hoc site)
. 2019-04-25 16:39:33.595 Host name: REDACTED (Port: 22)
. 2019-04-25 16:39:33.595 User name: a427721x (Password: Yes, Key file: No, Passphrase: No)
. 2019-04-25 16:39:33.595 Tunnel: No
. 2019-04-25 16:39:33.595 Transfer Protocol: SFTP
. 2019-04-25 16:39:33.595 Ping type: Off, Ping interval: 30 sec; Timeout: 15 sec
. 2019-04-25 16:39:33.595 Disable Nagle: No
. 2019-04-25 16:39:33.595 Proxy: None
. 2019-04-25 16:39:33.595 Send buffer: 262144
. 2019-04-25 16:39:33.595 SSH protocol version: 2; Compression: No
. 2019-04-25 16:39:33.595 Bypass authentication: No
. 2019-04-25 16:39:33.595 Try agent: Yes; Agent forwarding: No; TIS/CryptoCard: No; KI: Yes; GSSAPI: Yes
. 2019-04-25 16:39:33.595 GSSAPI: Forwarding: No; Libs: gssapi32,sspi,custom; Custom: 
. 2019-04-25 16:39:33.595 Ciphers: aes,chacha20,blowfish,3des,WARN,arcfour,des; Ssh2DES: No
. 2019-04-25 16:39:33.595 KEX: ecdh,dh-gex-sha1,dh-group14-sha1,rsa,WARN,dh-group1-sha1
. 2019-04-25 16:39:33.595 SSH Bugs: Auto,Auto,Auto,Auto,Auto,Auto,Auto,Auto,Auto,Auto,Auto,Auto,Auto
. 2019-04-25 16:39:33.595 Simple channel: Yes
. 2019-04-25 16:39:33.595 Return code variable: Autodetect; Lookup user groups: Auto
. 2019-04-25 16:39:33.595 Shell: default
. 2019-04-25 16:39:33.595 EOL: LF, UTF: Auto
. 2019-04-25 16:39:33.595 Clear aliases: Yes, Unset nat.vars: Yes, Resolve symlinks: Yes; Follow directory symlinks: No
. 2019-04-25 16:39:33.595 LS: ls -la, Ign LS warn: Yes, Scp1 Comp: No
. 2019-04-25 16:39:33.595 SFTP Bugs: Auto,Auto
. 2019-04-25 16:39:33.595 SFTP Server: default
. 2019-04-25 16:39:33.595 Local directory: default, Remote directory: home, Update: Yes, Cache: Yes
. 2019-04-25 16:39:33.595 Cache directory changes: Yes, Permanent: Yes
. 2019-04-25 16:39:33.595 Recycle bin: Delete to: No, Overwritten to: No, Bin path: 
. 2019-04-25 16:39:33.595 DST mode: Unix
. 2019-04-25 16:39:33.595 --------------------------------------------------------------------------
. 2019-04-25 16:39:33.596 Looking up host "REDACTED" for SSH connection
. 2019-04-25 16:39:33.600 Connecting to REDACTED port 22
. 2019-04-25 16:39:33.660 We claim version: SSH-2.0-WinSCP_release_5.13.1
. 2019-04-25 16:39:33.720 Network error: Software caused connection abort
. 2019-04-25 16:39:37.840 --------------------------------------------------------------------------

Очень часто бывает что мы не особо активно пользуемся окном Putty, и оно висит у нас в фоне, после чего нас соединение с серверов сбрасывается или обрывается, тоже самое с sftp.

Network error: Software caused connection abort

Network error: The application caused the connection to be dropped

Для того чтобы решить проблему вылета с ssh/sftp можно поступить следующим образом:

Открываем файл /etc/ssh/ssh_config и в конец файла вставляем следующие строчки:

ServerAliveInterval 60
ServerAliveCountMax 10

Первый параметр определяет как часто ваш ПК с которого идёт сессия будет отправлять пакеты на сервер со статусом «я живой». Второй параметр означает сколько раз отправлять данный пакет если сервер не отвечает, в нашем случае после 300 секунд (5 минут) сессия оборвётся.

Также можно поступить немного иначе, можно сделать чтобы сам сервер также отправлял данные пакеты на ваш компьютер, для этого открываем файл /etc/ssh/sshd_config и также вставляем в конец файла следующие строчки:

ClientAliveInterval 60
ClientAliveCountMax 5

Первый параметр как и в первой ситуации означает какой интервал будет между пакетами со статусом «я живой», в нашем случае это 60 секунд, а второй параметр сколько будет отправляться пакетов если сервер не отвечает, после чего также разорвёт соединение.

После данных изменений требуется перезапустить службу sshd

service sshd restart

Решение на стороне клиента

Если вы используете SSH клиент Putty, то советуем также в настройках сессий выставить данный параметр. Это тот же самый параметр что и в первом случае, но устанавливается на стороне клиента. После чего у вас не будет сбрасываться соединение.

В случае с WinSCP сначала надо выбрать соединение для которого необходимо выставить keepalive пакеты, далее нажать «Еще», и открыть вкладку «Подключение» и выбрать «Пустые команды протокола», выставить значение на 30 секунд.

The server’s host key was not found in the cache

This error message occurs when WinSCP connects to a new SSH server. Every server identifies itself by means of a host key; once WinSCP knows the host key for a server, it will be able to detect if a malicious attacker redirects your connection to another machine. If you see this message, it means that WinSCP has not seen this host key before, and has no way of knowing whether it is correct or not. You should attempt to verify the host key by other means, such as asking the machine’s administrator. If you see this message and you know that your installation of WinSCP has connected to the same server before, it may have been recently upgraded to SSH protocol version 2. SSH protocols 1 and 2 use separate host keys, so when you first use SSH-2 with a server you have only used SSH-1 with before, you will see this message again. You should verify the correctness of the key as before.1)

Warning — Potential security breach!

This message, followed by “The server’s host key does not match the one WinSCP has is cache”, means that WinSCP has connected to the SSH server before, knows what its host key should be, but has found a different one. This may mean that a malicious attacker has replaced your server with a different one, or has redirected your network connection to their own machine. On the other hand, it may simply mean that the administrator of your server has accidentally changed the key while upgrading the SSH software; this shouldn’t happen but it is unfortunately possible. You should contact your server’s administrator and see whether they expect the host key to have changed. If so, verify the new host key in the same way as you would if it was new.

Network error: Connection refused

You may get this message when connecting to a server for following reasons:

• You are trying to use WinSCP for a purpose for which it is not designed. WinSCP needs a SSH or FTP server to be installed at the other end (on the machine you want to connect to). In particular, you cannot easily use it to connect to another Windows workstation, since Windows does not have an SSH or FTP server included by default. Please refer to FAQ.
• You are trying to use protocol that the server does not support. Particularly you are trying SFTP/SCP (over SSH), but the server supports FTP; or vice versa. Check selected protocol on login dialog. Note that WinSCP defaults to SFTP protocol, while most other similar applications default to FTP.
• The server is running on non-standard port. Please make sure you enter actual port number on login dialog.
• You may need to connect through proxy server, but you have not specified one on login dialog.
• Connection was blocked by firewall. Please refer to FAQ.

No connection could be made because the target machine actively refused it

The same as «Network error: Connection refused».

Network error: Connection timed out

All reasons and hints for «Network error: Connection refused» apply to this error too.

Network error: No route to host

All reasons and hints for «Network error: Connection refused» apply to this error too. It may also be worth trying again later as this error can be due to temporary network issue.

Timeout detected

If you are getting the error while logging in or while initiating file transfer, all reasons and hints for «Network error: Connection refused» apply to this error too.

Network error: Software caused connection abort

While Connecting

You may get this message when connecting to a server for following reasons:

• Connection was blocked by firewall.
• All reasons and hints for «Server unexpectedly closed network connection» apply to this error too.

In a Middle of Session

If you are getting the error in the middle of session, it means that Windows network code killed an established connection for some reason. For example, it might happen if you pull the network cable out of the back of an Ethernet-connected computer, a DHCP IP address renewal fails or changes the computer’s IP address, or if Windows has any other similar reason to believe the entire network has become unreachable.

Windows also generates this error if it has given up on the machine at the other end of the connection ever responding to it. If the network between your client and files server goes down and your client then tries to send some data, Windows will make several attempts to send the data and will then give up and kill the connection. In particular, this can occur even if you didn’t do anything, if you are using SSH-2 and WinSCP attempts a key re-exchange.

The problem can be caused also by the firewall. Try to disable it temporarily to see if the problem persists. Refer to FAQ.

It can also occur if you are using keepalives in your connection. Other people have reported that keepalives fix this error for them.

If you find DHCPNACK errors in the Event Viewer, your DHCP server may be briefly denying your IP address, causing your existing connections to fail. Where possible, this can be addressed by reserving a specific IP address on the DHCP server (e.g. cable modem/router), setting that as the static IP address, and disabling the DHCP client service.

Host does not exist

You may get this message when connecting to a server for following reasons:

• You may have typed a wrong hostname on Login dialog.
• Your domain name is new and is not fully distributed to DNS servers yet.
• Connection was blocked by firewall.
• Problem with DNS server.

Common mistake is to enter full URL (e.g. ftp://ftp.example.com) as hostname on Login dialog instead of actual hostname (e.g. ftp://ftp.example.com).

The requested name is valid, but no data of the requested type was found

All reasons and hints for «Host does not exist» apply to this error too.

General failure; Error code: 4

Particularly for “General failure”, the SFTP server should provide you more details, that you will see next to “Error message from server”. Unfortunately, SFTP server shipped with OpenSSH (or Sun SSH) does not. That is, why you see there just “Failure” (very useful).
Some situations when OpenSSH (Sun SSH) server issues this useless error message:

• Renaming file to name of already existing file.
• Creating directory that already exists.
• Moving remote file to different filesystem (HDD).
• Uploading file to full filesystem (HDD).
• Exceeding user disk quota.

If none of the above helps, check your permissions or contact server administrator.

Server unexpectedly closed network connection

While Connecting

If you get this error message while connecting to your server, it is most usually caused by the server not being able to run some process necessary to support your session. Possibilities are:

• Shell.
  • Your account may not be allowed to start a shell at all. With some servers (like OpenSSH or Sun SSH), you may need to be allowed to start a shell, even if using SFTP protocol.
  • Also some servers refuse to start a shell if your password has expired or your account was terminated.
  • Some shells do not work with non-interactive sessions. The same it true for some configurations (or profiles used) for otherwise working shells. This commonly exhibits with SCP protocol with associated error message «Error skipping startup message. Your shell is probably incompatible with the application (BASH is recommended).» Try to force bash shell explicitly on SCP/Shell tab of Login dialog. Using SFTP protocol instead of SCP is another option.
  • OpenSSH server may fail to start shell when chroot is configured, but not possible (e.g. due to permissions).
• SFTP server.
  • Your account may not be able to start SFTP server binary (e.g. /bin/sftp-server) or the binary is not present on your server. Your SSH server may also lack the SFTP subsystem.

In a Middle of Session

If you get this error message in a middle of the session, it is usually caused by some fatal error on server. The server error may possibly be initiated by error on client (WinSCP) side.

In both cases check log file of your server to see an actual reason, it closed connection for.

Connection has been unexpectedly closed. Server sent command exit status 1 (or 255 or 0)

All reasons and hints for «Server unexpectedly closed network connection» apply to this error too.

Network error: Connection reset by peer

All reasons and hints for «Server unexpectedly closed network connection» apply to this error too.

Host is not communicating for more than 15 seconds. Still waiting… Warning: Aborting this operation will close connection!

You get this message when WinSCP is waiting for response from the server for more than configured time. Note that this is not an error message, WinSCP still keeps waiting. If the server responds finally, the message goes away automatically. The message box is shown only to give you a chance to break the connection and reconnect, if you do not want to wait anymore. If the server never replies, it may be because of some fatal error on the server side. Also something may be interfering with the connection, preventing the server response from arriving or possibly even the original request to arrive at the server.

Access denied

You will get the error while authenticating when:

• You have entered incorrect password, used used non-authorized key, etc.
• The account you are trying to use cannot be logged in. This can be case even when you get “Access denied” only after entering password, as for security reasons, many servers do not reveal information about the accounts. Reasons for not being allowed to login include:
  • The account (username) you have entered on Login dialog does not exist at all.
  • The account is disabled.
  • The password has expired.
  • Number of parallel sessions allowed for the account has been exceeded.
  • You are trying to connect with super-user account (root), without having allowed that. Please read FAQ.

It may help to check log file of your server to see an actual reason, it denied you an access.

Permission denied

You do not have a sufficient permissions (access rights) to a resource, such as a file or directory, to perform the operation. You should contact the server administrator to resolve the problem. Access rights systems differ with operating system and the file server. With the most common combination of Unix-based system and OpenSSH:

• To see a directory contents you need to have read permissions to the directory;
• To read a file you need to have read permissions to the file;
• To write a file you need to have a write permissions to the file;
• To create or delete file you need have a write permissions to the directory;
• To change file or directory permissions you need to be its owner;
• To change file modification time you need to its owner.

Server sent disconnect message type 2 (protocol error): «Too many authentication failures for root»

This message is produced by an OpenSSH (or Sun SSH) server if it receives more failed authentication attempts than it is willing to tolerate. This can easily happen if you are using Pageant and have a large number of keys loaded into it, since these servers count each offer of a public key as an authentication attempt. This can be worked around by specifying the key that’s required for the authentication in the session configuration; WinSCP will ignore any other keys Pageant may have, but will ask Pageant to do the authentication, so that you don’t have to type your passphrase.

Unable to use this private key file, Couldn’t load private key, Key is of wrong type

If you see one of these messages, it often indicates that you’ve tried to load a key of an inappropriate type into WinSCP. You may have specified a key that’s inappropriate for the connection you’re making. The SSH-1 and SSH-2 protocols require different private key formats, and a SSH-1 key can’t be used for a SSH-2 connection (or vice versa). Alternatively, you may have tried to load an SSH-2 key in a “foreign” format (OpenSSH or ssh.com), in which case you need to import it into PuTTY’s native format.

Unexpected directory listing line ‘…’

You will get the error with SCP protocol, if output of ls command cannot be parsed by WinSCP. WinSCP expects listing in format:

<type><permissions> <inode> <owner> <group> 
<size> <timestamp> <filename>[ -> <target>]

Some common examples:

drw-r--r--  3 martinp users  4596 2007-06-06 11:18:33.000000000 +0200 private

lrwxrwxrwx  1 martinp users     4 Mar 24  2005 wiki -> dokuwiki

drwxr-xr-x+ 2 martinp users    96 Oct 26 14:58 httpdocs

If your listing does not correspond to some of the above:

• Try using SFTP protocol instead of SCP.
• Always make sure you are using the latest version of WinSCP, as support for different listing format is being added continuously.
• Alter output of ls command to match any of the supported formatting. WinSCP has few session options that may be used for that, including:
  • Listing command (e.g. use sed to modify output of ls command to match the WinSCP requirements);
  • Clear aliases (may help if the ls command is aliased to display non standard output) and

  Clear national variables (may help if your listing does not use english month names).

• Make sure you are using bash shell. If you do not want to set it as your default shell, force it for WinSCP sessions.

Received too large (… B) SFTP packet. Max supported packet size is 102400 B

The problem is typically caused by a message printed from some profile/logon script. It violates the SFTP protocol. Some of these scripts are executed even for non-interactive (no TTY) sessions, so they cannot print anything (nor ask user to type something). The number … represents the first four bytes read from the server. If your login scripts are printing words, this will be the first four characters cast into a number, and not an SFTP message at all. To fix the problem find out what command in your login script prints text. Once you find it move the command to the proper interactive script, or remove it entirely. The scripts are usually hidden (their name starts with dot) and are located in your home directory on the server. There are other possible sources of the message in addition to the profile script — some SSH servers print messages if they are unable to start the SFTP server, or encounter a fatal error. You should contact your server administrator. Another possibility is that the server is configured to only allow the SCP protocol and not the SFTP protocol, in such a way that SCP fallback mechanism of WinSCP does not work. The solution is to choose SCP protocol on the login dialog.

Command failed with return code 127 (or 255)

You will get the error with SCP protocol, if command necessary for facilitate operation you were trying to do does not exist on remote server or the shell cannot find it. If you are not an experienced Unix user, you should first try using SFTP protocol instead. If you are sure that the command exists on the remote server, make sure that WinSCP (or rather the shell) can find it. You may need to add path to the command to PATH environment variable. Also make sure that the startup script that sets PATH is actually executed for non-interactive sessions. You can also try to run the respective command from terminal (with the same account that you use with WinSCP), to verify that you can execute it. You may not have sufficient permissions, or the command dependencies may not be installed. Common situations, in which you may get the error:

• Transferring files fails, because path to scp command is not in PATH;
• Error appears while logging in, because your *nix distribution lacks groups command. You can instruct WinSCP not to use the command in session settings.

Error skipping startup message. Your shell is probably incompatible with the application (BASH is recommended)

This error is typically associated with another error. If there is no other error in the message, try searching a log file.

Server returned empty listing for directory

Check that you have read and execute permissions to the directory.

Timeout waiting for external console to complete the command

This error indicates that winscp.exe did not receieve an answer from winscp.com in time. Most commonly this happens when winscp.com has redirected output to a stream with limited buffer. If the stream is not being read from, the buffer gets eventually filled and winscp.com hangs, when trying to write into it. In turn the above most commonly happens when winscp.com is run from another program (such as .Net code), which redirect its output into stream. If the program then e.g. waits for winscp.com to finish without reading from the stream, the winscp.com eventually hangs, never finishing. See example of .Net or WSH code dealing with the problem.

Error listing directory

This is high-level error, and it is typically associated with another error, such as:

• Timeout detected;
• Unexpected directory listing line ‘…’.

Invalid access to memory

This error message is not useful for you as an end-user. It generally means that there is a bug in the software. Please report the bug.