Veeam error nfc storage connection is unavailable storage

Challenge

A job fails with an error related to NFC (Network File Copy) connectivity. For example:

Cause

The cause of most NFC errors fall into 4 primary categories:

DNS
(The proxy or Veeam server cannot resolve the ESXi host)
Port (902)
(The proxy can resolve the IP, but port 902 is blocked)
Permissions*
(The account specified in [Backup Infrastructure] for the vCenter does not have permissions)
File Locks
(The file Veeam is trying to read is locked within the vSphere environment)

*If the account that Veeam Backup & Replication is using to communicate with the VMware Environment has granular permissions set, please confirm all permissions are set according to the Granular Permissions Guide.

Due to the nature of NFC as a client<->server type connection, there is also potential that all things may be correct on the Veeam client-side, and there exists an issue within the VMware Infrastructure itself.

Solution

Troubleshooting Tip

When investigating this issue, it is advisable to reconfigure the job to use a specific proxy. This is done to isolate the issue to a single proxy being the culprit. This also makes it easier to identify which server the logs will be found on.

Logs to Investigate

To investigate the potential of DNS, Ports, Permissions, or File Lock issues, you will need to review the Agent process logs.

Navigate to the following location on the VMware Backup Proxy:
```
%programdata%VeeamBackup
```
Open the folder that matches the name of the job that is having an issue.
Within this folder, you will need to find the agent log for the specific VM.
- For a Backup Job:
```
Agent.<JobName>.Source.<VMName>.log
```
- For a Replication Job:
```
Agent.<JobName>.Source.<VMName>.log
```
```
Agent.<JobName>.Target.<VMName>.log
```
Search for the following:
```
Creating NFC download stream
```
Here you will want to look for the following items that could point to the cause of the error.
1. DNS
  If there is an issue with DNS, the following error will be found.
```
nfc| Resolving host name (esx1) to IP address...
nfc| Resolving host name (esx1) to IP address... Failed.
```
2. Port
  If there is an issue with ports, there will be an error like this:
```
nfc| Establishing connection with the host [esx1]. Port: [902]. Failed.
```
3. Permissions
  If there is an issue with permissions, there will be an error after the following:
```
nfc| Sending authd message: [SESSION ID NUMBER].
nfc| Sending authd message: [Name of Host].
nfc| Waiting for the authd reply message...
```
4. File Locks
  If the file is locked, there will be an error like this:
```
nfc | Starting file download. File: [[datastore] Test/Test.vmx]. Failed.
ERR |Failed to start file downloading from host [esx1]. File path: [[datastore] Test/Test.vmx].
ERR |DiskLib error: [13].The file is locked or in use – File open failed: File not open
--tr:Failed to start file downloading. VMFS path: [[datastore] Test/Test.vmx].
```

Troubleshooting

DNS Issues _{^{(Most Common)}}

The proxy must be able to resolve the hostname or FQDN of the ESXi hosts in the Virtual Infrastructure.

Port Issues

Further testing can be done using the following PowerShell command. The destination should be the hostname/FQDN/IP shown in the logs for the host.

If the Proxy cannot reach port 902, check all applicable Firewalls between the server and the ESXi host.

Permissions Issues

When a vCenter Server or standalone ESXi host is added to the [Backup Infrastructure] section of Veeam Backup & Replication, the user assigns an account to be used. That account should have sufficient privileges to all objects related to the VM being backed up.

If the account that Veeam Backup & Replication is using to communicate with the VMware Environment has granular permissions set please confirm all permissions are set according to the Granular Permissions Guide.

Isolation Tips

Try a different proxy. (Isolates issue to specific Veeam Proxy)
Try vMotioning the VM to a different host. (Isolates issue to specific ESXi host)
Try Storage vMotioning the VM to a different datastore. (Isolates possible missing permission at datastore level)
Try adding the ESXi host the VM is on as a standalone host to Veeam, then create a backup job to pull the VM from the standalone host.
(This will help isolate if the Management Agents need to be restarted on the ESXi host)
Remember to remove the job and standalone host from Veeam’s configuration after the issue is resolved.

More Information

DNS Resolution of the target host
Port 902 is open to/from Backup Server/Proxy to ESX(i) host
Permissions to download files via ESX(i) host and/or vCenter

An issue with Port 902 may represent an issue with a firewall on the ESXi host, Veeam Proxy, or the connection between the two.

To submit feedback regarding this article, please click this link: Send Article Feedback
To report a typo on this page, highlight the typo with your mouse and press CTRL + Enter.

Источник

pirx: Veeam Legend; Posts: 441; Liked: 52 times; Joined: Dec 20, 2015 6:24 pm; Contact:

Error: NFC storage connection is unavailable

Hi,

(Case 03389599)

I’ve a fresh Veeam 9.5 U4 test setup (no license) with one W2K16 VM running all Veeam components. I’ve added a NetApp CIFS Shares as backup repository which I can access from Veeam VM, creating files and rescannig of Repo is workig too. If I start a VeeamZIP backup of a VM it always fails with:

Code: Select all

30.01.2019 12:49:28 :: Error: NFC storage connection is unavailable. Storage: [stg:datastore-11,nfchost:host-17,conn:192.168.0.20]. Storage display name: [vsan-x00-edgemgmt].
Failed to create NFC download stream. NFC path: [nfc://conn:192.168.0.20,nfchost:host-17,stg:datastore-11@aa8f495c-eeab-5467-802b-48df371c7cd0/x00-jumphost-linux.vmx].

The VM is stored on a 4 node cluster vSAN datastore, but I get the same error backing up a VM on a standalone server with local raid’ed storage. 192.168.0.20 is my vCenter, vCenter and ESXi host are all running latest 6.5U2 version. Veeam settings a more or less defaults. I’ve found some threads about this error message, but I’ve problems finding the root of the problem. From Netapp I can resolve the ESXi server names.

pirx: Veeam Legend; Posts: 441; Liked: 52 times; Joined: Dec 20, 2015 6:24 pm; Contact:

Re: Error: NFC storage connection is unavailable

Post

by pirx » Jan 31, 2019 8:26 am

The setup is currently very simple, only one Veeam B&R VM.

vCenter Login is working:

[30.01.2019 16:50:40] < 4788> vim| [SOAP] Successfully logged in ( server: [192.168.0.20], user: [Administrator@vsphere.local], sessionKey: [525a3156-1789-6f95-bb44-5ca496d945d6] )

But DNS lookup of ESXi host seems to be broken.

[30.01.2019 16:50:40] < 4788> nfc| Connecting to NFC session. Target host: [x00-esxem-til-024.dhc-dev.internal]. Storage: [vsan-x00-edgemgmt]. VI SOAP connection ID: [192.168.0.20].
[30.01.2019 16:50:40] < 4788> nfc| NFC service: [vpxa-nfc]. Port: [902].
[30.01.2019 16:50:40] < 4788> nfc| Opening NFC session with the specified ticket [52 f0 b7 c2 95 a8 b2 df-89 e9 90 6a eb 6d 9c a9]…
[30.01.2019 16:50:40] < 4788> nfc| Establishing connection with the host [x00-esxem-til-024.dhc-dev.internal]. Port: [902].
[30.01.2019 16:50:40] < 4788> nfc| Resolving host name (x00-esxem-til-024.dhc-dev.internal) to IP address…
[30.01.2019 16:50:52] < 4788> nfc| Resolving host name (x00-esxem-til-024.dhc-dev.internal) to IP address… Failed.
[30.01.2019 16:50:52] < 4788> nfc| Establishing connection with the host [x00-esxem-til-024.dhc-dev.internal]. Port: [902]. Failed.
[30.01.2019 16:50:52] < 4788> nfc| Opening NFC session with the specified ticket [52 f0 b7 c2 95 a8 b2 df-89 e9 90 6a eb 6d 9c a9]… Failed.
[30.01.2019 16:50:52] < 4788> nfc| Connecting to NFC session. Target host: [x00-esxem-til-024.dhc-dev.internal]. Storage: [vsan-x00-edgemgmt]. VI SOAP connection ID: [192.168.0.20]. Failed.
[30.01.2019 16:50:52] < 4788> ERR |Failed to initiate NFC session. Target host: [x00-esxem-til-024.dhc-dev.internal]. VI connection ID: [192.168.0.20]. Storage MOID: [vsan-x00-edgemgmt].

But nslookup for the host from the Veema VM is working:

nslookup.exe x00-esxem-til-024.dhc-dev.internal
Server: x00-pfsense.dhc-dev.internal
Address: 192.168.0.10

Name: x00-esxem-til-024.dhc-dev.internal
Address: 192.168.1.14

Same from vC:

# nslookup x00-esxem-til-024.dhc-dev.internal.
Server: 127.0.0.1
Address: 127.0.0.1#53

Name: x00-esxem-til-024.dhc-dev.internal
Address: 192.168.1.14

And from NetApp:

vserver services name-service getxxbyyy getaddrinfo -node cluster-prodit-01 -vserver svm1 -hostname x00-esxem-til-024.dhc-dev.internal
Host name: x00-esxem-til-024.dhc-dev.internal
Canonical Name: x00-esxem-til-024.dhc-dev.internal
IPv4: 192.168.1.14

So where is this DNS lookup performed?

pirx: Veeam Legend; Posts: 441; Liked: 52 times; Joined: Dec 20, 2015 6:24 pm; Contact:

Re: Error: NFC storage connection is unavailable

Post

by pirx » Jan 31, 2019 9:09 am
2 people like this post

Problem solved. DNS lookup on cli was working but the Veeam VM is multihomed and the interface with the private IP had the wrong DNS server setting…

HannesK: Veeam Software; Posts: 12494; Liked: 2400 times; Joined: Sep 01, 2014 11:46 am; Full Name: Hannes Kasparick; Location: Austria; Contact:

jbishir: Lurker; Posts: 1; Liked: 1 time; Joined: Nov 20, 2019 9:23 pm; Full Name: jbishir; Contact:

Re: Error: NFC storage connection is unavailable

Post

by jbishir » Nov 20, 2019 9:29 pm
1 person likes this post

I wanted to add what fixed my issue in case anyone else gets this error and has tried all of the DNS and firewall stuff.

When I got this error in Veeam, I also found out that I could not open my ESXi 6.5u3’s web interface in any interner browser from the Veeam server. I found that my management vmk nic’s MTU didnt match the vSwitch’s MTU that the vmk was assigned to. Once I updated them to match, I was able to open ESXi in my internet browser and the NFC Storage error went away in Veeam instantly. Why did the MTU’s not match, not sure, but it all started when I updated my ESXi 6.5u1 server to 6.5u3.

ITGuy: Novice; Posts: 3; Liked: never; Joined: Jan 07, 2020 8:03 pm; Full Name: Eliot Sinclair; Contact:

Re: Error: NFC storage connection is unavailable

Post

by ITGuy » Jan 07, 2020 8:18 pm

I had another issue which was causing this. All 3 options here were checked and correct but upon logging into vCentre I was presented with:

vmware esx cannot synchronize host «incorrect username and password»
Cannot synchronize host : Cannot complete login due to an incorrect user name or password

I have no idea how this happened but followed this from VMware and all is fixed:
https://kb.vmware.com/s/article/2014275

ottoste: Lurker; Posts: 1; Liked: never; Joined: Oct 07, 2020 3:20 pm; Full Name: Ste Otto; Contact:

Re: Error: NFC storage connection is unavailable

Post

by ottoste » Oct 07, 2020 3:43 pm

pirx wrote: ↑Jan 30, 2019 3:40 pm
Hi,

(Case 03389599)

I’ve a fresh Veeam 9.5 U4 test setup (no license) with one W2K16 VM running all Veeam components. I’ve added a NetApp CIFS Shares as backup repository which I can access from Veeam VM, creating files and rescannig of Repo is workig too. If I start a VeeamZIP backup of a VM it always fails with:
Code: Select all
30.01.2019 12:49:28 :: Error: NFC storage connection is unavailable. Storage: [stg:datastore-11,nfchost:host-17,conn:192.168.0.20]. Storage display name: [vsan-x00-edgemgmt].
Failed to create NFC download stream. NFC path: [nfc://conn:192.168.0.20,nfchost:host-17,stg:datastore-11@aa8f495c-eeab-5467-802b-48df371c7cd0/x00-jumphost-linux.vmx].
The VM is stored on a 4 node cluster vSAN datastore, but I get the same error backing up a VM on a standalone server with local raid’ed storage. 192.168.0.20 is my vCenter, vCenter and ESXi host are all running latest 6.5U2 version. Veeam settings a more or less defaults. I’ve found some threads about this error message, but I’ve problems finding the root of the problem. From Netapp I can resolve the ESXi server names.

In my case I’ve configured Veeam 9.5 to the vSphere, but I still get the same error. Analyzing the log file it says «nfc| Establishing connection with the host [esx1]. Port: [902]. Failed.», but the firewall is not blocking this port. I suspect that Veeam needs to communicate also with the ESXI nodes and not only with vSphere. Is that correct?

da-michael: Lurker; Posts: 1; Liked: never; Joined: Jun 29, 2021 2:21 pm; Full Name: DA Michael; Contact:

Re: Error: NFC storage connection is unavailable

Post

by da-michael » Jun 29, 2021 2:36 pm

I had the same errors. My issues wasn’t DNS related, as my vcenter is connected via an IP address, but I had an unexpected power loss and my main esx box removed itself from the vcenter management. As a result, Veeam was able to reach vcenter, but vcenter wasn’t able to reach esxi. Once I readded that server to inventory, the backups worked again.

Andreas Neufert: VP, Product Management; Posts: 6256; Liked: 1301 times; Joined: May 04, 2011 8:36 am; Full Name: Andreas Neufert; Location: Germany; Contact:

Re: Error: NFC storage connection is unavailable

Post

by Andreas Neufert » Jun 29, 2021 3:24 pm

Hi DA Michael,

thanks for sharing. Please know that even if you add the ESXi host by IP address to the vcenter, the VMware APIs always work with the full qualified domain names.
If you register a ESXi host by IP address to the vcenter it will transfer the hostname and as well the set domain to the vcenter which is used later. Of cause Veeam has added some fallbacks for more reliable processing, but ideally DNS should work well in any backup environment if it is Veeam Servers or VMware infrastructure. Protocolls like VMware VAIO which we use for Veeam Continous Data Protection do not work at all without fully configured and working DNS.

Who is online

Users browsing this forum: No registered users and 11 guests

Источник

Содержание

Veeam R&D Forums
Error: Client error: NFC storage connection is unavailable
Error: Client error: NFC storage connection is unavailable
[MERGED] Jobs failing after migrating to Cisco UCS host
Re: Error: Client error: NFC storage connection is unavailab
Re: Error: Client error: NFC storage connection is unavailab
Who is online
Veeam R&D Forums
NFC storage connection is unavailable
NFC storage connection is unavailable
Veeam R&D Forums
Error: NFC storage connection is unavailable
Error: NFC storage connection is unavailable
Re: Error: NFC storage connection is unavailable
Re: Error: NFC storage connection is unavailable
Re: Error: NFC storage connection is unavailable
Re: Error: NFC storage connection is unavailable
Re: Error: NFC storage connection is unavailable
Re: Error: NFC storage connection is unavailable
Re: Error: NFC storage connection is unavailable
Re: Error: NFC storage connection is unavailable
Re: Error: NFC storage connection is unavailable
Re: Error: NFC storage connection is unavailable
Re: Error: NFC storage connection is unavailable
Re: Error: NFC storage connection is unavailable
Who is online
Veeam R&D Forums
NFC storage connection is unavailable.
NFC storage connection is unavailable.
Re: NFC storage connection is unavailable.
Re: NFC storage connection is unavailable.
Re: NFC storage connection is unavailable.
Veeam error nfc storage connection is unavailable storage

Veeam R&D Forums

Technical discussions about Veeam products and related data center technologies

Error: Client error: NFC storage connection is unavailable

Post by Cragdoo » Jun 28, 2012 11:21 am 1 person likes this post

I thought I would share another possible workaround for the above error. I was getting the following error when trying to backup a new environment

27/06/2012 12:42:35 :: Error: Client error: NFC storage connection is unavailable. Storage: [stg:datastore-188,nfchost:host-221,conn:prd-vc1]. Storage display name: [ZONE-OS].
Failed to create NFC download stream. NFC path: [nfc://conn:prd-vc1,nfchost:host-221,stg:datastore-188@gry-sc2/gry-sc2.vmx].

so a quick search of the KB articles came up with http://www.veeam.com/KB1198 , I tried everything in the article but still the error message persisted.

Was almost at wits end , so took a final look in the log files and notice the job was trying to use a 169.x.x.x address as the proxy. A colleague then pipped up something about checking the Network binding (in Windows 208 R2). The perculiar thing this is a hidden menu , so steps take were

1. Open Network and Sharing Center
2. Click on ‘Change Adapter Settings’
3. Press ‘ALT’ key to bring up menu
4. Click on ‘Advanced’
5. Select ‘Advanced Settings’

You will now be able to change the order of connections. In my case I had to put the connection in the management VLAN at the top of the list.

Thought I would share on here to save some time.

[MERGED] Jobs failing after migrating to Cisco UCS host

Post by Bucksears » Jul 25, 2014 6:03 pm this post

Veeam case ID 00607168

We migrated from individual Dell ESX hosts to the Cisco UCS blade chassis last weekend. Now our Veeam backup jobs are failing.
The only thing that was migrated was the physical host; the VM storage, backup storage (EMC Data Domain), permissions and Veeam job info did not change.

I’m getting the ubiquitous «Error: Client error: NFC storage connection is unavailable. Storage: Storage display name: [ ]. Failed to create NFC download stream. NFC path: [.vmx].»

I verified everything in the KB1198 article.
Went to all of the Veeam proxies/backup server and was able to ping all ESX hosts; everything resolves correctly (so it’s not a DNS issue). I added the ESX host info to the hosts file on all Veeam proxies/backup server.
Port 902 is open between the ESX hosts and Veeam infrastructure.

Anyone experience this when moving to Cisco UCS?

Re: Error: Client error: NFC storage connection is unavailab

Post by veremin » Jul 28, 2014 8:06 am this post

Have you tried to check the log file and see whether there are any issues reported that are similar to the one described by Craig? And if so, perform the workaround mentioned above.

Re: Error: Client error: NFC storage connection is unavailab

Post by ian0x0r » Sep 15, 2016 10:27 am this post

Thought I would post my two pence on here as none of the above solutions worked for me in this situation.

This error occurred for me when a VM was moved to local storage. The Veeam proxies were set to use SAN processing mode only with out failover to network mode. As there was no way to process the VM, it failed with the NFC error above.

Who is online

Users browsing this forum: Google [Bot] and 207 guests

Main
All times are UTC
Delete cookies
Members
The team
Contact us

DISCLAIMER: All feature and release plans are subject to change without notice.

Источник

Veeam R&D Forums

Technical discussions about Veeam products and related data center technologies

NFC storage connection is unavailable

Post by nokogerra » Feb 22, 2017 4:04 am this post

Hello there.
There is a Veeam B&R 8.0.0.817 in our branch office and few standalone ESXi hosts licensed with vSphere Enterprise license (don`t ask me why its vShpere Ent without vCenter).
Here are the hosts:
1) 5.0.0 469512
2) 5.5.0 1331820
3) 5.0 3568722 (appeared few days ago)

There are no problems with backups on hosts 1 and 2, but if we try to backup vms on host3 we got an error:

So I`ve tried some things:
1. Check if ports 902 tcp and udp are open on ESXi.
2. Check if ESXi host 3 is added to Veeam B&R by ip address. It is so, but I make the corresponding record in hosts file on veeam proxy (it is a management server also) to be sure, so there is definitely no problems with name resolving.
3. hosts is added to Veeam with root credentials, so there is no problem with permissions (tried to readd it — still no luck with backup).
4. I`ve tried to install vclient on Veeam proxy and to connect to ESXi via the client and to download the vmx file. This experiment was successful.
5. I also tried to rename datastore, so it named «211_DAS» now, and name of vm is listed as «/vmfs/volumes/211_DAS/roads22/roads22.vmx» now.

During the attemt of backing up the vm, there are messages in vSphere client: «manipulate file paths».

Its strange, but I couldnt find anything corresponding «nfc» in vm backup logs.

Im curious, is it possible that Veeam 8.0.0.817 doesnt support ESXi 5.5u3 (3568722)? I couldnt find any compatibility guide. May be we should upgrade our Veeam version?

Источник

Veeam R&D Forums

Technical discussions about Veeam products and related data center technologies

Error: NFC storage connection is unavailable

Post by pirx » Jan 30, 2019 3:40 pm this post

Re: Error: NFC storage connection is unavailable

Post by HannesK » Jan 31, 2019 7:19 am this post

Hello,
I have seen it very often and from my customer experience it was always a firewall / networking / (split)DNS issue https://www.veeam.com/kb1198

The backup server cannot connect to the ESX hosts or VCenter.

Best regards,
Hannes

Re: Error: NFC storage connection is unavailable

Post by pirx » Jan 31, 2019 8:26 am this post

The setup is currently very simple, only one Veeam B&R VM.

vCenter Login is working:

nslookup.exe x00-esxem-til-024.dhc-dev.internal
Server: x00-pfsense.dhc-dev.internal
Address: 192.168.0.10

Name: x00-esxem-til-024.dhc-dev.internal
Address: 192.168.1.14

# nslookup x00-esxem-til-024.dhc-dev.internal.
Server: 127.0.0.1
Address: 127.0.0.1#53

Name: x00-esxem-til-024.dhc-dev.internal
Address: 192.168.1.14

Re: Error: NFC storage connection is unavailable

Post by pirx » Jan 31, 2019 9:09 am 2 people like this post

Re: Error: NFC storage connection is unavailable

Post by HannesK » Jan 31, 2019 9:55 am this post

Re: Error: NFC storage connection is unavailable

Post by vmtech123 » Aug 02, 2019 4:55 pm 2 people like this post

Just wanted to bump this and say thanks.

Had the NFC error. scanned this thread. Hostname ping failed. Was a DNS issue.

Re: Error: NFC storage connection is unavailable

Post by jbishir » Nov 20, 2019 9:29 pm 1 person likes this post

I wanted to add what fixed my issue in case anyone else gets this error and has tried all of the DNS and firewall stuff.

Re: Error: NFC storage connection is unavailable

Post by ITGuy » Jan 07, 2020 8:18 pm this post

I had another issue which was causing this. All 3 options here were checked and correct but upon logging into vCentre I was presented with:

vmware esx cannot synchronize host «incorrect username and password»
Cannot synchronize host : Cannot complete login due to an incorrect user name or password

I have no idea how this happened but followed this from VMware and all is fixed:
https://kb.vmware.com/s/article/2014275

Re: Error: NFC storage connection is unavailable

Post by ottoste » Oct 07, 2020 3:43 pm this post

pirx wrote: ↑ Jan 30, 2019 3:40 pm Hi,

Re: Error: NFC storage connection is unavailable

Post by HannesK » Oct 08, 2020 6:34 am this post

Re: Error: NFC storage connection is unavailable

Post by Zew » Apr 16, 2021 8:49 pm this post

Came across this issue today. Veeam’s KB: https://www.veeam.com/kb1198 was an amazing resource to troubleshooting the issue.

As other’s here mine was related to DNS. For further details feel free to see my blog post on my findings. https://zewwy.ca/index.php/2021/04/16/v . ad-stream/

Cheers, stay safe everyone!

Re: Error: NFC storage connection is unavailable

Post by da-michael » Jun 29, 2021 2:36 pm this post

Re: Error: NFC storage connection is unavailable

Post by Andreas Neufert » Jun 29, 2021 3:24 pm this post

Who is online

Users browsing this forum: No registered users and 10 guests

Main
All times are UTC
Delete cookies
Members
The team
Contact us

DISCLAIMER: All feature and release plans are subject to change without notice.

Источник

Veeam R&D Forums

Technical discussions about Veeam products and related data center technologies

NFC storage connection is unavailable.

Post by c0t0d0s0 » Oct 05, 2015 1:58 pm this post

We are demoing hardware from Nutanix and have run into a goofy problem. When we have a VM running on the Nutanix cluster we cannot back it up with Veeam. If we move it to a traditional server with traditional storage we can backup just fine. Same vCenter, same Veeam.

Error is:
10/4/2015 4:11:08 PM :: Processing mc1-txsw-ci Error: NFC storage connection is unavailable. Storage: [stg:datastore-590,nfchost:host-587,conn:mc1-vcenter60. ]. Storage display name: [nutanix].
Failed to create NFC download stream. NFC path: [nfc://conn:mc1-vcenter60. ,nfchost:host-587,stg:datastore-590@mc1-txsw-ci/mc1-txsw-ci.vmx].

Re: NFC storage connection is unavailable.

Post by PTide » Oct 05, 2015 2:27 pm this post

Please try this article to troubleshoot the issue. If nothing helps please open a case with support and post your case ID here.

Re: NFC storage connection is unavailable.

Post by c0t0d0s0 » Oct 05, 2015 3:10 pm this post

Looks like an SSL problem:

[05.10.2015 01:24:51] nfc| Initializing new SSL connection.
[05.10.2015 01:24:51] nfc| Establishing connection with the SSL server. Failed.
[05.10.2015 01:24:51] nfc| Initializing new SSL connection. Failed.
[05.10.2015 01:24:51] nfc| Opening NFC session with the specified ticket [52 4b ef 38 ba 08 0a 87-45 f1 dc 3c 1c d2 38 8e]. Failed.

Re: NFC storage connection is unavailable.

Post by c0t0d0s0 » Oct 05, 2015 3:19 pm this post

Источник

Mark as New
Bookmark
Subscribe
Mute
Subscribe to RSS Feed
Permalink
Print
Report Inappropriate Content

after updating one host to the latest build 6.7.0 (Build 9214924) VEEAM BACKUP Cannot detect the storages with the below error.

processing DC01 Error: NFC storage connection is unavailable.

I have restarted the management agents but the issue is still exisiting.

Can someone please point me to right direction how to get this fixed ?

Mark as New
Bookmark
Subscribe
Mute
Subscribe to RSS Feed
Permalink
Print
Report Inappropriate Content

With ESXi 6.0 U1 (and above), SSLv3 support is disabled by default for all services and ports. If the host was running a version prior to this, this might be the problem. More details in the article KB2063: After applying update 1 for vSphere 6.0, Jobs fail with NFC storage unavailable —

Resolution-1 —> Upgrade Veeam to the latest version.

Resolution-2 —> Enable SSLv3 on all the hosts. Steps as below —

1. Create a backup copy of the /etc/vmware/config file

2. Edit the /etc/vmware/config file to append the following line at the end of the file:

Note: If you have the line vmauthd.ssl.noSSLv3 = true in the file, change it to vmauthd.ssl.noSSLv3 = false

Example of /etc/vmware/config file:

3. Restart the rhttpproxy service with the command:

If this does not resolve the issue, share one of the failed VM agent log file from the Veeam proxy VM. It will be under the location .

Please consider marking this answer as «correct» or «helpful» if you think your questions have been answered.

Источник

На чтение 11 мин Просмотров 1 Опубликовано 27.12.2021

Содержание

Error:client error: nfc storage connection is unavailable. failed to create nfc downstream
Kb2063: after applying update 1 for vsphere 6.0, jobs fail with nfc storage unavailable
Veeam error nfc storage connection | .ua
Veeam: nfc storage connection is unavailable.failed to create nfc download stream. – zewwy’s info tech talks

Error:client error: nfc storage connection is unavailable. failed to create nfc downstream

After installing Veeam at a clients site and setting it up to replicate a 2 host cluster and san to a single host with storage it tested OK in the test environment. The servers where then IP`d and put into the correct sub net with the relevant gateways etc.

Then the replication job was made live and left to run over night. Next morning I checked only to find this error below.

error:client error: NFC Storage connection is unavailable. Failed To Create NFC Downstream

Now I knew that IP`s had been changed so I tested the following.

From the Veeam server.

Pinged its gateway – OK.
Pinged its DNS servers – OK.
Pinged the source VM hosts – FAIL
Pinged the storage – OK.

The the Veeam side failed on pinging the source hosts, I checked all my local settings which were ok. I went onto my hosts and checked the Local IP settings. It turns out that there was incorrect setting for the hosts gateway. I changed accordingly and the pings were fine.The hosts could not talk back to the Veeam server correctley. Then I set the replication away and it completed successfully.

The point of this post is the error:client error: NFC Storage connection is unavailable. Failed To Create NFC Downstream is generally an IP misconfiguration error or a communications error. So look there first and test the simple tings like pings.

Tags: Storage

Allen is an IT Consultant and holds the following accreditations. MCSA, MCSE, MCTS, MCITP, CCA, CCSP, VCP 4,5, 6 and HP ASE, AIS — Network Infrastructure.

Kb2063: after applying update 1 for vsphere 6.0, jobs fail with nfc storage unavailable

This form is only for KB Feedback/Suggestions, if you need help with the software open a support case

Veeam error nfc storage connection | .ua

Solution
Caution: These steps expose the security vulnerabilities with SSLv3. This issue is resolved in VMware View 6.2, available at VMware Downloads. For more information, see VMware Horizon 6 version 6.2 Release Notes.

The SSLv3 support can be enabled for these ports and services:

CIM Port 5989
Authd Service Port 902
Enabling support for SSLv3 on CIM Port 5989 in ESXi
Create a backup copy of the /etc/sfcb/sfcb.cfg file.

Edit the /etc/sfcb/sfcb.cfg file to append the following line at the end of the file:

enableSSLv3: true

Note: If you have the line enableSSLv3: false in the file, change it to enableSSLv3: true

For Example:

cat /etc/sfcb/sfcb.cfg
# Generated by sfcb-config.py. Do not modify this header.
# VMware ESXi 6.0.0 build-3029758
#
basicAuthLib: sfcBasicPAMAuthentication
certificateAuthLib: sfcCertificateAuthentication
cimXmlFdHardLimit: 1024
cimXmlFdSoftLimit: 512
.
.
.
threadStackSize: 524288
useChunking: true
sslCipherList: HIGH:!DES-CBC3-SHA!CAMELLIA128-SHA!CAMELLIA256-SHA
enableSSLv3: true

Restart the SFCBD service with the command:

/etc/init.d/sfcbd-watchdog restart
Enabling support for SSLv3 on Authd service 902 in ESXi
Create a backup copy of the /etc/vmware/config file
Edit the /etc/vmware/config file to append the following line at the end of the file:

vmauthd.ssl.noSSLv3 = “false”

Note: If you have the line vmauthd.ssl.noSSLv3 = “true” in the file, change it to vmauthd.ssl.noSSLv3 = “false”

For Example:

cat /etc/vmware/config
libdir = “/usr/lib/VMware”
authd.proxy.nfc = “vmware-hostd:ha-nfc”
authd.proxy.nfcssl = “vmware-hostd:ha-nfcssl”
authd.proxy.vpxa-nfcssl = “vmware-vpxa:vpxa-nfcssl”
authd.proxy.vpxa-nfc = “vmware-vpxa:vpxa-nfc”
authd.fullpath = “/sbin/authd”
vmauthd.ssl.noSSLv3 = “false”

Restart the rhttpproxy service with the command:

/etc/init.d/rhttpproxy restart
Additional Information
For the related Veeam Knowledge Base article, see http://www.nfcphones.ru/kb2063.

Follow these steps to enable SSLv3 protocol on hostd service for ESXi 6.0 U1b later.

By default SSLv3 is disabled. If you want to enable SSLv3, set the setting to empty by using the below command:

esxcli system settings advanced list -o /UserVars/VMAuthdDisabledProtocols

Path: /UserVars/VMAuthdDisabledProtocols
Type: string
Int Value: 0
Default Int Value: 0
Min Value: 0
Max Value: 0
String Value:sslv3
Default String Value: sslv3
Valid Characters: *
Description: VMAuthd disabled protocols. Choices are sslv3, tlsv1, tlsv1.1, tlsv1.2. By default sslv3 is disabled. If no protocol is specified, all protocols are enabled.

If SSLv3 is disabled, To enable SSLv3 is run the following command:

esxcli system settings advanced set -o /UserVars/VMAuthdDisabledProtocols -s “”

Restart the rhttpproxy services by running the following command:

/etc/init.d/rhttpproxy restart

Run the following command to get a list of enabled protocols for hostd:

esxcli system settings advanced list -o /UserVars/VMAuthdDisabledProtocols

Path: /UserVars/VMAuthdDisabledProtocols
Type: string
Int Value: 0
Default Int Value: 0
Min Value: 0
Max Value: 0
String Value:
Default String Value: sslv3
Valid Characters: *
Description: VMAuthd disabled protocols. Choices are sslv3, tlsv1, tlsv1.1, tlsv1.2. By default sslv3 is disabled. If no protocol is specified, all protocols are enabled.

[collapse]

Veeam: nfc storage connection is unavailable.failed to create nfc download stream. – zewwy’s info tech talks

I’ll keep this post short as well.

Run replication Job… ERROR. Check error, huh haven’t this one before…

4/16/2021 11:22:16 AM :: Processing VMName Error: NFC storage connection is unavailable. Storage: [stg:datastore-3,nfchost:host-2,conn:vcenter]. Storage display name: [ESXi-Local-Datastore].
Failed to create NFC upload stream. NFC path: [nfc://conn:vCenter,nfchost:host-23982,stg:datastore-23983@VMName_replica/VMName.vmx].

To note on this, I did some changes, I changed a route between sites (as I needed to reduce a entire network from being improperly routed, but some of services still required access from the main location, thus some dedicated /32 routes were put in their place).

I had also just patched the host in question and was testing jobs after the patch. Since I wasn’t exactly sure which was the cause. I decided to do regular troubleshooting to get more details to the root cause.

I love Veeam, they got a nice KB to help with this. So I followed along, checking the main Veeam server log areas didn’t have the log file in question, so was pretty confident it was still using the proxy at the alt site.

Checking the log as mentioned by the KB, sure enough the same error line showed up which indicated it was DNS related. Checking the proxie’s DNS settings…. DOH. It was using a server within the routes I had removed, and didn’t create a dedicated /32 for, as I wasn’t expecting any systems here to need to communicate to that subnet.

Now that I know what the issue is… this feels familiar… oh yeah the Veeam Soap Fault issue I had to deal with…

The funny part about this is… 1) it’s the same server/proxy 2) Again DNS related 3) Again going to stick with host file to avoid dependencies on DNS servers

In my case the error showed the ESXi server by the hostname WAS fully qualified, but access to a DNS server to resolve it was unavailable. As soon as I saw this I had two options:

Create a route to allow the Proxy to reach required DNS servers (which won’t be available in a DR case) OR
Just add a static record in the Proxy host file. (DNS server not required, but if hostname or IP changes needs to be adjusted manually here)

As you can see I have the exactly 2 options similar to my first post, the difference is now it was fully dependent on DNS. Since this is a self hosted instance of a Veeam proxy, there’s a good chance DNS access might not be available when time comes, so this option was chosen.

It’s important to note that when these types of choices are made it is well documented WHY they were made.

So in this case… to resolve it I added a record in the Proxy’s local host file

172.x.x.x     ESXi.domain.postfix

You may notice that the ESXi hostname is not within the initial error, it only tells your the datastore, the Veeam logs will indicate which lookup failed. More than likely the hostname look up for the ESXi host in which the VM will be created on.

I really hope this post helps someone. Honestly I just followed the Veeam KB which was a great source reference to troubleshooting the issue. Your case maybe different, depending on the root cause your resolution maybe different then what was discussed here.

Cheers stay safe everyone.

I’ll keep this post short as well.

Run replication Job… ERROR. Check error, huh haven’t this one before…

4/16/2021 11:22:16 AM :: Processing VMName Error: NFC storage connection is unavailable. Storage: [stg:datastore-3,nfchost:host-2,conn:vcenter]. Storage display name: [ESXi-Local-Datastore].
Failed to create NFC upload stream. NFC path: [nfc://conn:vCenter,nfchost:host-23982,stg:datastore-23983@VMName_replica/VMName.vmx].

Now that I know what the issue is… this feels familiar… oh yeah the Veeam Soap Fault issue I had to deal with…

The funny part about this is… 1) it’s the same server/proxy 2) Again DNS related 3) Again going to stick with host file to avoid dependencies on DNS servers

In my case the error showed the ESXi server by the hostname WAS fully qualified, but access to a DNS server to resolve it was unavailable. As soon as I saw this I had two options:

Create a route to allow the Proxy to reach required DNS servers (which won’t be available in a DR case) OR
Just add a static record in the Proxy host file. (DNS server not required, but if hostname or IP changes needs to be adjusted manually here)

It’s important to note that when these types of choices are made it is well documented WHY they were made.

So in this case… to resolve it I added a record in the Proxy’s local host file

172.x.x.x     ESXi.domain.postfix

Cheers stay safe everyone.

Источник

Приветствую, перестали бэкапиться несколько виртуальных машин, у всех заданий в логах veeam backup такая ошибка

05.01.2022 20:55:50 :: Processing BITRIX Error: DiskLib error: [13].The file is locked or in use — File open failed: File not open
Failed to create NFC download stream. NFC path: [nfc://conn:vcenter,nfchost:host-3989,stg:datastore-9392@BITRIX/BITRIX.vmx].

Приветствую, перестали бэкапиться несколько виртуальных машин, у всех заданий в логах veeam backup такая ошибка

Проверьте что esxi хост жив на котором крутится виртуальная машина BITRIX.vmx

Проверьте что esxi хост жив на котором крутится виртуальная машина BITRIX.vmx

Были проблемы с vmware. Один из узлов кластера умер. После его восстановления все заработало.

Добрый день, наткнулся на эту тему. У меня точно такая же проблема, но перезагрузкой esxi не ограничивается дело. Сбоят задания veeam

Processing AIS_STORE Error: DiskLib error: [13].The file is locked or in use — File open failed: File not open —tr:Failed to start file downloading. VMFS path: [[EMC_6] AIS_STORE/AIS_STORE.vmx]. —tr:NFC storage connection is unavailable. Storage: [stg:datastore-10437,nfchost:host-3991,conn:vcenter]. Storage display name: [EMC_6]. Failed to create NFC download stream. NFC path: [nfc://conn:vcenter,nfchost:host-3991,stg:datastore-10437@AIS_STORE/AIS_STORE.vmx]. —tr:Unable to open source file [nfc://conn:v

Подскажите куда копать…

Еще заметил что какая то непонятная виртуальная машина в vsphere7 в веб клиенте. Вот успел заскринить.

Еще заметил что какая то непонятная виртуальная машина в vsphere7 в веб клиенте. Вот успел заскринить.

Посмотреть вложение 11326

Что такое виртуальная машина vCLS?
Виртуальная машина vCLS — это, по сути, «устройство» или «сервисная» виртуальная машина, которая позволяет кластеру vSphere продолжать работу в случае, если vCenter Server становится недоступным. Он будет поддерживать работоспособность и службы этого кластера.

Откуда взялась виртуальная машина vCLS?
Виртуальная машина vCLS появится после обновления до vSphere 7.0 Update 1. Я предполагаю, что она была развернута в процессе обновления.

Он не отображается в стандартном представлении кластера, хостов и виртуальных машин, но появляется при просмотре списков виртуальных машин объектов vSphere, списков виртуальных машин хранилища и т. д.

Добрый день, наткнулся на эту тему. У меня точно такая же проблема, но перезагрузкой esxi не ограничивается дело. Сбоят задания veeam

Подскажите куда копать…

Напишите подробнее версию vSphere до сборки и что за серверы, какое железо.

Думаю тут дело либо в утечке памяти или думать куда то в сторону поддержки и совместимости железа и версии vsphere

Думаю тут дело либо в утечке памяти или думать куда то в сторону поддержки и совместимости железа и версии vsphere

Возможно, мы как раз недавно обновились до vsphere 7. Серверы cisco ucs b200.
Заметил что перед глюками на esxi появляется предупреждение:

Quick stats on X.X.X.X is not up-to-date

Думаю тут дело либо в утечке памяти или думать куда то в сторону поддержки и совместимости железа и версии vsphere

Hypervisor:VMware ESXi, 7.0.2, 17867351
Model:UCSB-B200-M4
Processor Type:Intel(R) Xeon(R) CPU E5-2670 v3 @ 2.30GHz

В общем вот так. При этом пинг и KVM до esxi есть, виртуалки которые на нем — работают, доступны по rdp.

Посмотри что в логах vmkernel, подключись к этому esxi и зайди в troubleshooting options

В основном — Failed to load NAS plugin(s).
Could not expand environment variable HOME.

Как будто HDD умирают в esxi. Считать что то не может.

Как будто HDD умирают в esxi. Считать что то не может.

У меня там нет ни HDD ни SSD, там внутри USB flash на который установлен esxi 7

А если заново его присоединить ? Выбрать и — Connect

А если заново его присоединить ? Выбрать и — Connect

Cannot contact the specified host (esxi3). The host may not be available on the network, a network configuration problem may exist, or the management services on this host may not be responding.

Попробуйте зайти непосредственно на хост через https\esxi

Попробуйте зайти непосредственно на хост через https\esxi

Когда захожу вижу

503 Service Unavailable (Failed to connect to endpoint: [N7Vmacore4Http16LocalServiceSpecE:0x000000655dc05070] _serverNamespace = / action = Allow _port = 8309)

Попробуйте выбрать вот эту галку при перезапуске агентов

Источник

After upgrading a client’s Veeam solution to version 7 patch 4 I came across this error message for the replication jobs that were set up and it had me a little confused:

Processing configuration Error: Client error: NFC storage connection is unavailable. Storage: [stg:datastore-22,nfchost:host-21,conn:172.1.1.1]. Storage display name: [Temp].
Failed to create NFC download stream. NFC path: [nfc://conn:172.1.1.1,nfchost:host-21,stg:datastore-22@Server_replica/Server.vmx].

There is a knowledge base article on Veeam’s website regarding this — http://www.veeam.com/kb1198 but given nothing had changed regarding ports opening/closing etc I was sceptical that this was the root cause. Especially since the replication jobs had worked an hour earlier, before the patch was applied.

After much head scratching and restarting of servers etc I found the resolution to the issue. Within the VMware vSphere client there is an option to rescan the storage attached to the host to help pick up new volumes, changes, etc, once I ran that on the host that the backup storage is attached to the replica jobs completed successfully!!

rescan

Am still unsure as to why the error occurred or why that fixed the issue but it did, one worth remembering!!

Veeam

Subscribe to our newsletter

Get the latest posts delivered right to your inbox.

Now check your inbox and click the link to confirm your subscription.

Please enter a valid email address

Oops! There was an error sending the email, please try later.

Recommended for you

No posts found

Apparently there are no posts at the moment, check again later.

Great! You’ve successfully subscribed.

Great! Next, complete checkout for full access.

Welcome back! You’ve successfully signed in.

Success! Your account is fully activated, you now have access to all content.

Источник

Challenge

Cause

Solution

Logs to Investigate

Troubleshooting

DNS Issues (Most Common)

Port Issues

Permissions Issues

Isolation Tips

More Information

Error: NFC storage connection is unavailable

Re: Error: NFC storage connection is unavailable

Re: Error: NFC storage connection is unavailable

Re: Error: NFC storage connection is unavailable

Re: Error: NFC storage connection is unavailable

Re: Error: NFC storage connection is unavailable

Re: Error: NFC storage connection is unavailable

Re: Error: NFC storage connection is unavailable

Who is online

Veeam R&D Forums

Error: Client error: NFC storage connection is unavailable

Error: Client error: NFC storage connection is unavailable

[MERGED] Jobs failing after migrating to Cisco UCS host

Re: Error: Client error: NFC storage connection is unavailab

Re: Error: Client error: NFC storage connection is unavailab

Who is online

Veeam R&D Forums

NFC storage connection is unavailable

NFC storage connection is unavailable

Veeam R&D Forums

Error: NFC storage connection is unavailable

Error: NFC storage connection is unavailable

Re: Error: NFC storage connection is unavailable

Re: Error: NFC storage connection is unavailable

Re: Error: NFC storage connection is unavailable

Re: Error: NFC storage connection is unavailable

Re: Error: NFC storage connection is unavailable

Re: Error: NFC storage connection is unavailable

Re: Error: NFC storage connection is unavailable

Re: Error: NFC storage connection is unavailable

Re: Error: NFC storage connection is unavailable

Re: Error: NFC storage connection is unavailable

Re: Error: NFC storage connection is unavailable

Re: Error: NFC storage connection is unavailable

Who is online

Veeam R&D Forums

NFC storage connection is unavailable.

NFC storage connection is unavailable.

Re: NFC storage connection is unavailable.

Re: NFC storage connection is unavailable.

Re: NFC storage connection is unavailable.