Veeam error transmission pipeline hanged aborting process

Veeam Community discussions and solutions for: Transmission pipeline hanged, aborting process. of Microsoft Hyper-V

chuck.coggins

Novice
Posts: 5
Liked: never
Joined: Dec 14, 2018 1:17 am
Full Name: Charles R. Coggins III
Contact:

Transmission pipeline hanged, aborting process.

Case #04665111

Hello all,

I have been running into an issue with one of my VM backups for the last month.
I have Veeam running backups on about 23 VM’s and one of these VM’s just started to fail with the error: Processing Dynamics Error: Transmission pipeline hanged, aborting process.

I have been unable to search and figure out what is going on.

SCREENSHOT
https://imgur.com/a/KHs7Q72

It kind of sounds like there might be too much going on during this time and the backup is failing because of that.
However, I cannot figure out how to reschedule this backup.
There seems to be no way to change the time this backup begins.

SCREENSHOT
https://imgur.com/IE6MjUR

When I read the documentation it says you should be able to change this in the Job’s section, but there are no Jobs as you can see in this Screenshot.

SCREENSHOT
https://imgur.com/JtHvdPe

This software is just driving me nuts.
Veeam Backup and Replication: 9.5.4.2866


soncscy

Expert
Posts: 638
Liked: 307 times
Joined: Aug 04, 2019 2:57 pm
Full Name: Harvey Carel
Contact:

Re: Transmission pipeline hanged, aborting process.

Post

by soncscy » Feb 24, 2021 9:27 pm

Hi Charles,

the reason you can’t find a schedule is you’re doing VeeamZip backups as I see from your screenshots: https://helpcenter.veeam.com/docs/backu … ml?ver=110

If you’re not personally starting these by hand, then you must have some script running that does it, as there is no schedule for VeeamZIP, it’s either manual, or Powershell.

I can see you’re on HyperV…may I ask, did you install Veeam into the parent partition? (right on the HyperV host itself?) If so, this is a big «no no». The parent partition for HyperV is offlimits as per Microsoft: https://docs.microsoft.com/en-us/archiv … he-host-os

My guess is that indeed you’re just hammering your HyperV server badly, and yes, if you had a VBR server as a VM, it would likely reduce the stress of this.

Since I can see you’re on Community (free) edition, try using the Veeam Agent for Windows/Linux (depending on your guestOS) instead. It will be much less stressful I think on the host and likely you’ll get more stable backups.


chuck.coggins

Novice
Posts: 5
Liked: never
Joined: Dec 14, 2018 1:17 am
Full Name: Charles R. Coggins III
Contact:

Re: Transmission pipeline hanged, aborting process.

Post

by chuck.coggins » Feb 24, 2021 9:34 pm

Thank you for your reply Soncscy,

I will check the host server for some PowerShell scripts.
I did not set any of this up so I have just been doing my best with it.

The way we do our Hyper-V and Veeam Backups is we have a bare metal server that runs the main OS.
Then it runs all my servers via Hyper-V.
The bare metal server hosts the Veeam Agent (in almost all cases).

This VM and all the VMs on this server do not have Veeam installed on the Bare Metal server nor is it installed on any of the VM’s.
This is another thing that has been confusing me on how it is even being backed up in the first place. You might be right though and there might be scripts running causing this VeeamZIP to run on them.

I will take a look at this server tonight and Windows Task Scheduler and see if I can find anything going on.


veremin

Product Manager
Posts: 19898
Liked: 2154 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Transmission pipeline hanged, aborting process.

Post

by veremin » Feb 25, 2021 12:01 pm
1 person likes this post

There certainly VeeamZIP backups are running, this is indicated in backup view by corresponding subnode — Disk (VeeamZIP). VeeamZIP produces standalone full backup on demand. It does not have any schedule and the only option to run it on specific time is to use scripts. So, it’s a good idea to check Task Scheduler.

As to the issue itself, the root cause might vary from corrupted VM to broken connection to one of the backup components. So, keep working with the support team on identifying the real cause.

Thanks!


chuck.coggins

Novice
Posts: 5
Liked: never
Joined: Dec 14, 2018 1:17 am
Full Name: Charles R. Coggins III
Contact:

Re: Transmission pipeline hanged, aborting process.

Post

by chuck.coggins » Feb 25, 2021 11:15 pm

I was able to find the Powershell scripts running the VeeamZIP backups yesterday.
I took the script running the backup on those sets of VM’s and I switched them to run last night at 8:30 PM when no one would be on doing anything.
I still got the same error, but Hard Disk 1 was added to the end of the error this time.

«Processing finished with errors at 2/25/2021 1:16:01 AM Error: Transmission pipeline hanged, aborting process. Hard disk 1 (136.0 GB)»

So tonight I made a new script and put the Dynamics VM on it all by itself.
The rest of them run at 5 AM but Dynamics will back up on its own at 8:30 PM.
I guess we will see how that goes.


chuck.coggins

Novice
Posts: 5
Liked: never
Joined: Dec 14, 2018 1:17 am
Full Name: Charles R. Coggins III
Contact:

Re: Transmission pipeline hanged, aborting process.

Post

by chuck.coggins » Mar 04, 2021 7:17 pm

Quick Update:
Still have not been able to resolve the issue.
I put the backup on its own script during a slow time of the day when nothing is going on and I am still getting the error.
Each of the other VM’s on that server takes 4-6 mins to backup this one starts at 8:30 PM and fails at 1 AM.
I got an email from the support case from Veeam and they said due to a high volume of tickets they already archived this case so I think I am pretty much on my own.



PetrM

Veeam Software
Posts: 2561
Liked: 409 times
Joined: Aug 28, 2013 8:23 am
Full Name: Petr Makarov
Location: Prague, Czech Republic
Contact:

Re: Transmission pipeline hanged, aborting process.

Post

by PetrM » Jun 01, 2021 3:58 pm

Hello,

This is a generic error which is thrown when target Data Mover stops to retrieve data from a source one. This issue can happen due to many different reasons, the best way to figure out the real root cause is to let our support team to examine debug logs. Please don’t forget to share a support case ID.

Thanks!


Who is online

Users browsing this forum: No registered users and 5 guests

perjonsson1960

Expert
Posts: 344
Liked: 36 times
Joined: Jun 06, 2018 5:41 am
Full Name: Per Jonsson
Location: Sweden
Contact:

Transmission pipeline hanged

Folks,

Last week we «sucked in» our vCenter server into VMware. Before that it was a physical Windows server. After this change, every other day or so, we are getting this error during backup:

«Transmission pipeline hanged, aborting process.»

We have a large backup job that backs up 215 VMs, and sometimes one of the VMs (but not the same VM every time) gets this error. And B&R does a retry of that VM, and the retry succeeds.

This error never occured when we had the physical Windows server as the vCenter server. Does anybody know what this is all about?

PJ


Mike Resseler

Product Manager
Posts: 7762
Liked: 1171 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: Transmission pipeline hanged

Post

by Mike Resseler » May 15, 2019 6:41 am

Hi Per,

That message could mean a few things I think. In this case, I would look in the direction of the VM not being reachable or talking back. Maybe because the bigger issue here is the communication with VCenter where there is an issue or so. Difficult to say with only this one message.

I would suggest to open a support call (with the regular post the support case ID here and the outcome after that) so the engineers can look at the messages before and after this in the logs. They might (no promises as this typical message is very generic) see where the communication is lost or hanging

Cheers
Mike


HannesK

Veeam Software
Posts: 12494
Liked: 2400 times
Joined: Sep 01, 2014 11:46 am
Full Name: Hannes Kasparick
Location: Austria
Contact:

Re: Transmission pipeline hanged

Post

by HannesK » May 15, 2019 6:45 am

Hello,
I suggest contacting support for that. I just searched a little bit internally and there seem to be various reasons for that from corrupted VMs to network and resource issues. So it might be a coincidence between P2V of your VCenter.

Please post the case number here for reference.

Best regards,
Hannes


trevorgov

Novice
Posts: 3
Liked: 1 time
Joined: Sep 04, 2019 9:26 am
Full Name: Trevor Govender
Contact:

[MERGED] Error: Transmission pipeline hanged, aborting process. When backing up exchange 2016 server

Post

by trevorgov » Sep 04, 2019 9:45 am

Good day,

Please can someone assist, for the last 3 weeks now we are not able to backup our exchange server which is a vm on VMWare ESXi. The backup starts fine and runs till about 20% and then just hangs and eventually fails after 5 hours or so with the error, ‘Transmission pipeline hanged, aborting process.» We have 2 other jobs running on the same Veeam management PC and they run fine, I have tried using different proxies, different repositories, even created another management PC all return the same result. However if I do an agent backup from within Exchange 2016, that goes through fine.I have reported the case to Veeam support but I am just concerned that we are sitting ducks without a proper backup for the last 3 weeks.

Any suggestions would be much appreciated.

Case number : #03723972


foggy

Veeam Software
Posts: 20911
Liked: 2063 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Transmission pipeline hanged

Post

by foggy » Sep 04, 2019 10:48 am

Hi Trevor, if it is a production VM, I’d increase the case severity and also ask to escalate it to a higher tier, looks like there’s not much progress after almost two weeks of investigation.



foggy

Veeam Software
Posts: 20911
Liked: 2063 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Transmission pipeline hanged

Post

by foggy » Sep 04, 2019 4:41 pm

Severity 4 is the lowest one. To escalate the case, please use the ‘Talk to a Manager’ option on the support portal. Thanks!


trevorgov

Novice
Posts: 3
Liked: 1 time
Joined: Sep 04, 2019 9:26 am
Full Name: Trevor Govender
Contact:

Re: Transmission pipeline hanged

Post

by trevorgov » Sep 05, 2019 11:12 am
1 person likes this post

Perfect, thanks Foggy. They requested a whole load of logs but it has been escalated.



KOPFteam

Enthusiast
Posts: 53
Liked: 7 times
Joined: Sep 30, 2011 6:59 pm
Full Name: Florian Pürner
Contact:

Re: Transmission pipeline hanged

Post

by KOPFteam » Nov 01, 2019 2:28 pm

I had this problem solved by Veeam support yesterday (case 03838384).

They first suspected a problem with CBT. I disabled CBT for this VM + the used drives and deleted a few files in the datastore as specified in the VMware KB article. Didn’t help. Still «Transmission pipline hanged».

Then they asked me to activate nbdssl = «Enable host to proxy traffic encyption in Network mode (NBDSSL)» in the properties of the used proxy (in the Transport Mode section). And this did it! No more «Transmission pipline hanged»!

Problem solved :-)

Loving (and fighting with) Veeam for over a decade!


bdufour

Expert
Posts: 206
Liked: 41 times
Joined: Nov 01, 2017 8:52 pm
Full Name: blake dufour
Contact:

Re: Transmission pipeline hanged

Post

by bdufour » Nov 01, 2019 8:31 pm

before we reinvent the wheel, try vmotioning exchange to another host, and see if that solves the issue or not.


perjonsson1960

Expert
Posts: 344
Liked: 36 times
Joined: Jun 06, 2018 5:41 am
Full Name: Per Jonsson
Location: Sweden
Contact:

Re: Transmission pipeline hanged

Post

by perjonsson1960 » Nov 07, 2019 1:18 pm

The «Enable host to proxy traffic encyption in Network mode» didn’t solve it for me. I still get «Transmission pipline hanged, aborting process» once in a while. Maybe once a week, and not with the same VM every time, but all the affected VMs reside in the same Esxi host. Maybe there is a problem with that host. And sometimes I also get «Error: VSSControl: Failed to freeze guest over network, wait timeout» with VMs on that Esxi host.


veremin

Product Manager
Posts: 19898
Liked: 2154 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Transmission pipeline hanged

Post

by veremin » Nov 07, 2019 1:25 pm

Have you tried to resolve the issue via support team? They should be able to identify the root cause, based on the infrastructure and debug log investigation. Thanks!


perjonsson1960

Expert
Posts: 344
Liked: 36 times
Joined: Jun 06, 2018 5:41 am
Full Name: Per Jonsson
Location: Sweden
Contact:

Re: Transmission pipeline hanged

Post

by perjonsson1960 » Nov 07, 2019 3:07 pm

I tried to submit a case, but when it had been uploading the 2 GB zip file with logs for about an hour, and the page refreshed, and the «Next» button was grayed out, I got irritated and skipped the upload. But when I continued to submit the case, it failed, and no case was even created. Perhaps I will try again at another time.


perjonsson1960

Expert
Posts: 344
Liked: 36 times
Joined: Jun 06, 2018 5:41 am
Full Name: Per Jonsson
Location: Sweden
Contact:


perjonsson1960

Expert
Posts: 344
Liked: 36 times
Joined: Jun 06, 2018 5:41 am
Full Name: Per Jonsson
Location: Sweden
Contact:

Re: Transmission pipeline hanged

Post

by perjonsson1960 » Nov 15, 2019 1:08 pm

The solution from support was to split our large backup job with 215 VMs into two or three separate jobs, and I did that yesterday. As long as we had a physical vCenter server the large job worked well, but since we «sucked in» the vCenter server into our VxRail environment, the pipeline errors started. The original job now has 78 VMs, and the two new jobs have 67 and 70 VMs respectively.

The original job (incremental backups) went well. The second new job (full backups) got two «pipeline errors» and two «VDDK async operation errors». And the third new job (full backups) also went well. When the first retry was performed on the four affected VMs, it succeeded. The three jobs were run after one another, and no other jobs were running at the same time.

New case #03865617.


timgowen

Influencer
Posts: 17
Liked: never
Joined: Aug 22, 2017 8:34 am
Full Name: Tim Gowen
Contact:

Re: Transmission pipeline hanged

Post

by timgowen » Apr 20, 2021 1:18 pm

I have noted this issue on an Agent backup. I back up folders on a file server — all fine — but then the system backup — which backs up everything, fails with 20/04/2021 08:35:30 :: Transmission pipeline hanged, aborting process.


foggy

Veeam Software
Posts: 20911
Liked: 2063 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Transmission pipeline hanged

Post

by foggy » Apr 21, 2021 10:23 am

Hi Tim, please investigate with the help of our support team. Thanks!


Who is online

Users browsing this forum: No registered users and 11 guests

Содержание

  1. Veeam R&D Forums
  2. Pipeline timeout. Transmission pipeline hanged.
  3. Pipeline timeout. Transmission pipeline hanged.
  4. Re: Pipeline timeout. Transmission pipeline hanged.
  5. Re: Pipeline timeout. Transmission pipeline hanged.
  6. Who is online
  7. Veeam R&D Forums
  8. [ID #03954027] Transmission pipeline hanged
  9. [ID #03954027] Transmission pipeline hanged
  10. Re: [ID #03954027] Transmission pipeline hanged
  11. Who is online
  12. Veeam R&D Forums
  13. Full backup to tape fails with «Transmission pipeline hanged, aborting process»
  14. Full backup to tape fails with «Transmission pipeline hanged, aborting process»
  15. Re: Full backup to tape fails with «Transmission pipeline hanged, aborting process»
  16. Re: Full backup to tape fails with «Transmission pipeline hanged, aborting process»
  17. Re: Full backup to tape fails with «Transmission pipeline hanged, aborting process»
  18. Who is online
  19. Veeam R&D Forums
  20. Error: Transmission pipeline hanged, aborting process.
  21. Error: Transmission pipeline hanged, aborting process.
  22. Re: Error: Transmission pipeline hanged, aborting process.
  23. Who is online
  24. Veeam R&D Forums
  25. Veeam agent error — Transmission pipeline hanged, aborting process.
  26. Veeam agent error — Transmission pipeline hanged, aborting process.

Veeam R&D Forums

Technical discussions about Veeam products and related data center technologies

Pipeline timeout. Transmission pipeline hanged.

Pipeline timeout. Transmission pipeline hanged.

Post by JohnGG » Jun 28, 2016 3:01 am this post

I have been getting something very similar (Processing SQLServer Error: Transmission pipeline hanged, aborting process.), and also (Processing SQLServer Error: Shared memory connection was forcibly closed by peer. Failed to upload disk. Agent failed to process method . ). I replicate to a server first with ESXi on it and then replicate to a Synology 1815+ NAS and it only ever happens when replicating tot he NAS and with the sqlserver, mailserver and all others work fine.
Running latest version of Veeam as well.

Re: Pipeline timeout. Transmission pipeline hanged.

Post by veremin » Jun 28, 2016 6:56 am this post

Re: Pipeline timeout. Transmission pipeline hanged.

Post by JohnGG » Jun 28, 2016 10:27 pm this post

The thing is it doesnt do it all the time, maybe once a week.
Anyway i have updated the NAS and also changed the settings so it seeds to the nas and then remapped the servers to the nas.
Replicating to the server never misses a beat but replicating tot he nas can have issues so i will see when it fails again and then contact support.

Who is online

Users browsing this forum: Google [Bot] and 213 guests

  • Main
  • All times are UTC
  • Delete cookies
  • Members
  • The team
  • Contact us

DISCLAIMER: All feature and release plans are subject to change without notice.

Powered by phpBB® Forum Software © phpBB Limited

Источник

Veeam R&D Forums

Technical discussions about Veeam products and related data center technologies

[ID #03954027] Transmission pipeline hanged

[ID #03954027] Transmission pipeline hanged

Post by KamSandhu44 » Jan 15, 2020 8:54 am this post

I am having issue’s backing up two of my HyperV hosted VMs. I have the latest version of Veeam Backup & Replication (the community edition).

Two Vms are failing with Transmission pipeline hanged, aborting process. (Veeam.Backup.Common.CCppComponentException).

Snipet of the log: —

I am logging to a local NAS, my other seven VMs backup fine.

Any guidance would be great.

Re: [ID #03954027] Transmission pipeline hanged

Post by PetrM » Jan 15, 2020 6:18 pm this post

Generally, this message means that data transmission between Data Mover components is stopped by some reason but connection still exists.

According to log snippet, there is an issue with writing data to a NAS which seems to be overloaded:
— May be you run too many tasks in parallel to the NAS and it would make sense to reschedule backup of 2 problematic VMs
and backup these VMs in a separate job or to decrease concurrent tasks number at the level of repository settings
— There is a hardware issue with NAS itself which affects data write process

Anyway, let’s wait for results of troubleshooting from our support team.

Who is online

Users browsing this forum: No registered users and 4 guests

  • Main
  • All times are UTC
  • Delete cookies
  • Members
  • The team
  • Contact us

DISCLAIMER: All feature and release plans are subject to change without notice.

Powered by phpBB® Forum Software © phpBB Limited

Источник

Veeam R&D Forums

Technical discussions about Veeam products and related data center technologies

Full backup to tape fails with «Transmission pipeline hanged, aborting process»

Full backup to tape fails with «Transmission pipeline hanged, aborting process»

Post by glenn_ » Apr 12, 2021 8:29 am this post

We are trying to do a full backup of a Windows server and this is by now failing with the error «Transmission pipeline hanged, aborting process» and I don’t know why.

What could be the reason for this? Full backup was working for a long time without problems.

But with Veeam 11 I’m experiencing these issues.

Re: Full backup to tape fails with «Transmission pipeline hanged, aborting process»

Post by Natalia Lupacheva » Apr 12, 2021 9:10 am this post

This message can be caused by different reasons, so it’s hard to find the real issue without the logs.
Please keep working with Support and keep us informed.

Re: Full backup to tape fails with «Transmission pipeline hanged, aborting process»

Post by glenn_ » Apr 25, 2021 11:56 pm this post

Re: Full backup to tape fails with «Transmission pipeline hanged, aborting process»

Post by PetrM » Apr 26, 2021 11:13 am this post

The error message is quite generic and informs us that data traffic coming from source Data Mover to a target one is suddenly stopped, it can occur due to many different reasons. We provide support on a «best effort» basis for customers using Community licensed products. You may open a request again, every single attempt increases the chances to get technical support.

Who is online

Users browsing this forum: No registered users and 4 guests

  • Main
  • All times are UTC
  • Delete cookies
  • Members
  • The team
  • Contact us

DISCLAIMER: All feature and release plans are subject to change without notice.

Powered by phpBB® Forum Software © phpBB Limited

Источник

Veeam R&D Forums

Technical discussions about Veeam products and related data center technologies

Error: Transmission pipeline hanged, aborting process.

Error: Transmission pipeline hanged, aborting process.

Post by Wad4iPod » Apr 27, 2022 1:38 pm this post

Error: Transmission pipeline hanged, aborting process.
Windows 10 Pro (10.0.19044)
Veeam Agent for Windows Version 5.0.0.4301
Greetings,
We have one of eight Dell computers that fail to complete a full image backup. It will complete a volume of C: backup.
Near as I can tell the only difference in configuration on this particular pc in the hard drive partition includes an EFI. The partitions are such — EFI (System Partition), OS NTFS, Recovery Partition, Recovery Partition.
The failure seems to occur between after the EFI partition and at the start of the OS C: begins.
The failure occurs when the backup target is a NAS or local PC attached storage.

I have noted a few other posts with this error. The solutions were to log a support case. Does anyone have a resolution to this issue based on a support case or self-resolve?

There is a new Veeam for Windows Agent available (5.0.3). We have yet to install this and test to see if it will resolve this issue.

Re: Error: Transmission pipeline hanged, aborting process.

Post by PetrM » Apr 27, 2022 2:00 pm this post

Thanks for pasting your support case ID but I believe you should wait for the conclusion from your support engineer as the error is quite generic and just says that data flow has ceased. This error can have different root causes and a solution from another case could be useless in your case, anyway the precise examination of debug logs is needed.

Who is online

Users browsing this forum: No registered users and 8 guests

  • Main
  • All times are UTC
  • Delete cookies
  • Members
  • The team
  • Contact us

DISCLAIMER: All feature and release plans are subject to change without notice.

Powered by phpBB® Forum Software © phpBB Limited

Источник

Veeam R&D Forums

Technical discussions about Veeam products and related data center technologies

Veeam agent error — Transmission pipeline hanged, aborting process.

Veeam agent error — Transmission pipeline hanged, aborting process.

Post by jrbaumann » Apr 27, 2021 3:40 pm this post

I’ve been having some issues with Veeam agent since upgrading to VBR 10.0.1.4854. I’ve tried agent 3.02 and 4.0 and I’m having intermittent backup completion with both agents. The job may complete one day then go a few days with out completion. The errors we get are often different for each agent.

I opened case #04730406 with support, but we have not found a resolution.

The agents backup to our Exagrid repositories through a VPN to different data centers. Prior to the upgrade, had very little issues.

Jobs will complete until somewhere between 70-95% and then stop and error out. (Looking through the logs, it looks like it starts backing up to one IP address, then when it stops it looks like its trying to backup to a different ip address, and I’m not sure why)

We tried enabling backup cache, which didn’t make a difference

I was told by one rep that it could be that the repository was busy. i don’t think that is it as I’ve had all jobs paused and the dedup process completed on the SOBR and the job fails, but then the next day it completes. Very frustrating.

A couple of the errors i see are

— Transmission pipeline hanged, aborting process.
— An existing connection was forcibly closed by the remote host Failed to upload disk. Agent failed to process method .

Any help would be appreciated.

If you have any ideas, I’m willing to try as its been a couple weeks now since ive gotten a good backup on a couple of the servers.

Источник

У вас в логах следующие ошибки в задаче:

[30.04.2020 07:05:27] <36> Info [AP] (0225) output: —asyncNtf:disk_capacity: ‘161061273600’
[30.04.2020 07:05:27] <20> Info [AP] (f771) output: —asyncNtf:disk_capacity: ‘161061273600’
[30.04.2020 07:05:27] <19> Info [AP] (a0b9) output: —asyncNtf:disk_capacity: ‘161061273600’
[30.04.2020 07:05:28] <24> Info [AP] (0225) output: —size: 161061273600
[30.04.2020 07:05:28] <18> Info [AP] (0225) output: —pex:0;267386880;0;0;0;32;97;0;0;0;0;0;132326931287340000
[30.04.2020 07:05:42] <30> Error (4fd5) error: ChannelError: ConnectionReset
[30.04.2020 07:05:42] <38> Info [RemoteAgentSystemSession] Performing reconnection…
[30.04.2020 07:05:42] <38> Info Reconnecting backup client, host ‘192.168.11.5’.
[30.04.2020 07:05:42] <38> Info [ProxyAgent] Starting client agent session, id ‘8605ed3c-b173-445c-a644-0d757e1ee3a9’, host ‘192.168.11.5’, agent id ‘ea3e9510-c2f5-4c35-86ab-d3511b558ea3’, IPs ‘192.168.11.5:2507’, PID ‘6756’
[30.04.2020 07:05:42] <38> Info [SocketAgentService] Connecting to agent ‘192.168.11.5’ (‘192.168.11.5:2507’)
[30.04.2020 07:05:42] <38> Info [NetSocket] Connecting to ‘192.168.11.5:2507’.
[30.04.2020 07:05:42] <62> Error [ReconnectableSocket][PacketAsyncReceiver] Exception on [2d0656ab-85e0-4cc6-b3db-8b51f964e732].
[30.04.2020 07:05:42] <62> Error Удаленный хост принудительно разорвал существующее подключение (System.Net.Sockets.SocketException)
[30.04.2020 07:05:42] <62> Error в Veeam.Backup.Common.Reconnect.CPacketAsyncReceiver.PushData(CSocketAsyncEventArgsEx e)
[30.04.2020 07:05:42] <62> Info [ReconnectableSocket] Retryable error on [2d0656ab-85e0-4cc6-b3db-8b51f964e732].
[30.04.2020 07:05:42] <62> Info [ReconnectableSocket] Reconnecting [2d0656ab-85e0-4cc6-b3db-8b51f964e732] (retryable error)…
[30.04.2020 07:05:42] <62> Info [NetSocket] Connecting to ‘192.168.11.5:2507’.
[30.04.2020 07:05:43] <38> Error Failed to connect to agent’s endpoint ‘192.168.11.5:2507’. Host: ‘192.168.11.5’.
[30.04.2020 07:05:43] <38> Error Подключение не установлено, т.к. конечный компьютер отверг запрос на подключение 192.168.11.5:2507 (System.Net.Sockets.SocketException)
[30.04.2020 07:05:43] <38> Error в System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
[30.04.2020 07:05:43] <38> Error в System.Net.Sockets.Socket.Connect(EndPoint remoteEP)
[30.04.2020 07:05:43] <38> Error в Veeam.Backup.Common.CNetSocket.Connect(IPEndPoint remoteEp)
[30.04.2020 07:05:43] <38> Error в Veeam.Backup.AgentProvider.CAgentEndpointConnecter.ConnectToAgentEndpoint(ISocket socket, IAgentEndPoint endPoint)
[30.04.2020 07:05:43] <38> Info Disposing BaseAgentProtocol [0x1361a0e]
[30.04.2020 07:05:43] <38> Info Disposing CSocketAgentService [0x22dfb24], sessionId []
[30.04.2020 07:05:43] <38> Error [RemoteAgentSystemSession] Keep-alive thread has failed.
[30.04.2020 07:05:43] <38> Error Unable to reestablish connection to agent. (System.Exception)
[30.04.2020 07:05:43] <38> Error в Veeam.Backup.AgentProvider.CRemoteAgentSystemSession.ReconnectCycle(CancellationToken token)
[30.04.2020 07:05:43] <38> Error в Veeam.Backup.AgentProvider.CRemoteAgentSystemSession.KeepAliveThreadProc()

Получается, соединение было внезапно разорвано. Кроме того, перед тем, как соединение разорвалось, в логе Target Agent, я нашел следующие ошибки:

[30.04.2020 07:04:23] < 2480> srv| ERR |bad allocation
[30.04.2020 07:04:23] < 2480> srv| >> |—tr:Failed to unserialize buffer with data.
[30.04.2020 07:04:23] < 2480> srv| >> |—tr:Failed to unserialize data block.
[30.04.2020 07:04:23] < 2480> srv| >> |—tr:Failed to unserialize FIB block.
[30.04.2020 07:04:23] < 2480> srv| >> |—tr:Failed to receive data block with multi-channel download stream.
[30.04.2020 07:04:23] < 2480> srv| >> |Unable to retrieve next block transmission command. Number of already processed blocks: [128].

Ошибка bad allocation обычно возникает из-за нехватки памяти на одном из серверов, на котором поднялся агент.

В вашем случае, ошибка возникла на сервере, где был target agent. Согласно логам, он поднялся на сервере «REMOTE-DC2»:

[30.04.2020 07:01:03] <36> Info Initializing shared repository client for repository ‘REMOTE-DC2’.
[30.04.2020 07:01:03] <36> Info Starting target agent ( sharing mode: ‘SharedAgent’, host: ‘on repository’.).
[30.04.2020 07:01:12] <36> Info Setting repository ‘REMOTE-DC2’ (‘7f1aef43-231d-43fb-a5d7-854e49542ff5’) credentials for backup client.

который является Windows сервером:

[29.04.2020 22:19:32] <01> Info RepositoryID: 7f1aef43-231d-43fb-a5d7-854e49542ff5, Type: WinLocal, TotalSpace: 32212118396928, FreeSpace: 21094426656768, ConcurrentTaskLimit: {Enabled: True, Number: «4»}, RWrateLimit: {Enabled: False, MBs: «0»}, OnRotatedDrive: False, AlignFileBlocks: False, DecompressBeforeStoring: False, PerVMbackup: False, vPowerNFS: True, ProxyAffinitySet: False, IsReFsSytheticEnabled: True, IsReFsSytheticAvailable: False, ClusterSize: 8192, IsDedupEnabled: False, ServerID: a8ad2735-7961-41fd-9a67-7a744350bc50, OS Name: Microsoft Windows Server 2016 Standard 64-bit, OS version: 10.0.14393 build:14393, CoresCount: 2, CPUCount: 2, RAMTotalSize: 2146947072

На момент запуска задания, на сервере «REMOTE-DC2» оставалось меньше 1 ГБ памяти, в то время как общая память составляет 2 Гб:

[30.04.2020 07:01:07] < 4312> Total physical memory installed: [2047 MB], available: [856 MB].

Минимальное количество памяти для Veeam — репозитория составляет 4 ГБ + 2 ГБ (или 4 Гб, если 64-битная ОС установлена) за каждую дополнительную задачу, выполняющуюся параллельно.
Как я вижу, у вас 4 ядра на репозитории и стоит выполнение 4 параллельных заданий. В таком случае, рекомендуемая память 4 + 2*4 = 12 Гб (или 4 + 4*4 = 20 Гб, если 64-битная ОС установлена).
В итоге — на сервере «REMOTE-DC2» надо увеличить память как минимум до 12 Гб (или 20 Гб). Можно конечно попробовать поиграть со SWAP, но не думаю что поможет.
Так же советую посмотреть вот эту статью

We use Veeam Agent v5 (and v4, same error) for our backups and found we have some Windows 2019 physical servers using a Volume level backup that have recently failed in their Operating System backup with the following error.

The text of the error gives a little bit more to work with.

31/03/2021 10:00:43 AM :: Error: The device is not ready. Asynchronous read operation failed Failed to upload disk. Agent failed to process method {DataTransfer.SyncDisk}. Exception from server: The device is not ready. Asynchronous read operation failed Unable to retrieve next block transmission command. Number of already processed blocks: [0]. Failed to download disk ‘1140af5b-d567-46db-9b74-11c85ca1fb25’.  

So we logged a job with Veeam and their answer was to log a job with Microsoft as it was a server error. Time to have a deeper look before we went down that path.

This was a bit strange, the jobs were happy until a round of Windows Updates done some time in February. Looking at the volume picker below showed what the job was trying to back up.

If we removed the Recovery volume from the selected volumes, there is an error about an incomplete backup but the job runs successfully with the remaining two volumes. Including the Recovery volume into the backup causes it to fail again. A backup of only the Recovery volume also had the same failure.

A hunt of the Veeam forums showed that a few others had the same issue.

Hunting around with PowerShell provided this from the Recovery partition

(Get-Partition).Where({$_.type -eq "Recovery"}) | FormatList

UniqueId : {00000000-0000-0000-0000-100000000000}600508B1001C480977D3EE56F92B41FB
AccessPaths : {\?Volume{62480ffe-130d-4cfd-a21f-e44b531d5a34}}
DiskNumber : 4
DiskPath : \?scsi#disk&ven_hpe&prod_logical_volume#5&25daa67a&0&000100#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}
DriveLetter :
Guid : {62480ffe-130d-4cfd-a21f-e44b531d5a34}
IsActive : False
IsBoot : False
IsHidden : False
IsOffline : True
IsReadOnly : False
IsShadowCopy : False
IsDAX : False
IsSystem : False
NoDefaultDriveLetter : True
Offset : 1048576
OperationalStatus : Offline
PartitionNumber : 1
Size : 499 MB
Type : Recovery

Operational Status is Offline, how about we look at that. So with

(Get-Partition).Where({$_.type -eq "Recovery"}) | Set-Partition -IsOffline $False

We set the Recovery partition to being online, run a backup job with the Recovery volume included and it works. Huzzah!

Then a reboot to test and we found that the Offline setting returns to $True after a reboot and the Veeam Agent job would fail. Setting a PowerShell script with the above line in it and making it run as a Scheduled Task on Startup ensured the Recovery volume was online. Not sure if that is best practice, but it makes the job run and our backup engineer happy.

Понравилась статья? Поделить с друзьями:
  • Veeam error code 0x00000005
  • Veeam error 1064 an exception occurred in the service when handling the control request
  • Veeam connecting to guest os via vix error
  • Veeam backup error log
  • Vectors must be the same lengths ошибка матлаб