Содержание
- Особенности обновления NetBackup до 7.6 в связке с VMware
- Порядок настройки NetBackup до 7.6 в связке с VMware.
- 1. В web версии VMware vSphere Client создаем нового локального пользователя.
- 2. Через «толстого клиента» VMware vSphere Client проверяем ранее созданного пользователя.
- 3. Создаем новую роль пользователей.
- 4. Разрешаем управление кластером пользователю vmbackup с правами «Backup user for NetBackup»
- 5. Вносим изменения в параметрах подключения netbackup к вируальной среде.
- Err error opening the snapshot disks using given transport mode san status 23
- Err error opening the snapshot disks using given transport mode san status 23
- Err error opening the snapshot disks using given transport mode san status 23
- Err error opening the snapshot disks using given transport mode san status 23
Особенности обновления NetBackup до 7.6 в связке с VMware
После обновления NetBackup до версии 7.6, последний перестает снимать снапшеты с виртуальных машин VMware.
В консоли можно наблюдать следующие ошибки:
ERR — Error opening the snapshot disks using given transport mode: hotadd:nbd:nbdssl Status 23
Error bpbrm (pid=####) from client vmclient01: ERR — Error opening the snapshot disks using given transport mode: Status 23
замена «transport mode» проблему не решает.
По данным ошибкам неявно гуглится «KB Article с номером TECH214315«. Особое внимание в статье следует уделить правам, которые необходимо дать NetBackup для работы в VMware :
This VDDK version requires slightly different datastore level permissions for our NetBackup credential account in vCenter. If the credential role is not already an admin in vCenter and is set granularly (or with a stock VCB role), backups for the virtual clients
т.е. если 7.5 версии достаточно было немного расширенного стандартного «vmware consolidated backup«,
In the Edit Role screen, place a check mark on the Extension privilege to include all three extension privileges: Register extension, Unregister extension, Update extension.
In the Edit Role screen, expand the Global privilege group and select the following privileges: Log event, Manage custom attributes, Set custom attribute.
то 7.6 версия хочет сразу админа.
Если более внимательно изучить KB Article, то можно найти статью «What are the minimum permissions needed to properly backup and restore using vStorage api?«, которая более детально описывает настройку роли пользователя для работы связки. Разрешения в виде xml файла доступны по ссылке: vcenter-netbackup-permissions.xml.
Порядок настройки NetBackup до 7.6 в связке с VMware.
1. В web версии VMware vSphere Client создаем нового локального пользователя.
Для логина вам необходим пользователь с правами Single Sign-On administrator@vsphere.local (vSphere 5.5)
Создавать будем пользователя с именем «vmbackup«.
Напрашивается вопрос, почему не воспользоваться доменным пользователем? Best practices показывает, что в случае наличия проблем в инфраструктуре, сети или контроллерах домена, — вам будет сложно восстановиться из бекапа без дополнительных действий.
2. Через «толстого клиента» VMware vSphere Client проверяем ранее созданного пользователя.
Использование толстого клиента продиктовано удобством администрирования, при желании можно по аналогии продолжить администрирование через web интерфейс.
3. Создаем новую роль пользователей.
Название на ваше усмотрение, мне приглянулось «VMware Consolidated Backup user for NetBackup». Производим настройку согласно «What are the minimum permissions needed to properly backup and restore using vStorage api?»
4. Разрешаем управление кластером пользователю vmbackup с правами «Backup user for NetBackup»
5. Вносим изменения в параметрах подключения netbackup к вируальной среде.
Необходимые нам поля находятся в credentials — virtual machine server:
Источник
Err error opening the snapshot disks using given transport mode san status 23
02-10-2015 01:48 PM
i’m running netbackup 7.6.0.3 with 5220 appliance.
this message only appears for one vm backup. I can backup another vm on the same datastore without a problem, so it’s not an issue for the appliance to see the lun.
I can also backup a vm in the same policy. what else can I look into? I added nbd to the policy and it worked. I don’t see any issues with the name or dns.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-22-2015 09:55 AM
nbd transport mode is when sending data across WAN , which work fine. If you are using SAN transfer, ensure the same LUN acting as you datastore is also visible (Zoned) to the media server (backup host) which is writing this job. Thank you.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-10-2015 05:38 PM
If it were my own situation, I’d first try removing the VM from the policy and then add back in.
Try checking on the VM name — display name, hostname, DNS name all OK.
Compared to the «good VM» do you see any differences like, RDM, OS, VMware tools installed or not, size of the allocated vmdk, too much CPU/resource pool size, thin/thick provisioned disks.
Perhaps if you share the full progress log from the activity monitor job detail window it may help everyone understand more in detail on what is happening.
Источник
Err error opening the snapshot disks using given transport mode san status 23
09-05-2012 09:18 PM
We are using NBU 7 and backup VM.
All VM stored in SAN. Netbackup host can see all LUN of SAN. But when running backup, it show the error ERR — Error opening the snapshot disks using given transport mode: Status 23
How can we fix it?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-05-2012 09:32 PM
SAN storage type is SATA. and FC
Previous, VM in FC backup normally. VM in Sata backup failed.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-06-2012 10:41 AM
If a combination of the transport modes is given, NetBackup will try all of them one by one until gaining successful access to the data.So why can’t you try selecting both SAN and NBU as your transport mode.
If SAN is selected in purpose, then you will have try with different option.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-06-2012 06:17 PM
Could you please explain more, how can we do it on NBU?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-07-2012 12:33 AM
I am sorry it was typo mistake, it should be NBD (Network Block Device).
In this method, VMware Network File Copy (NFC) protocol is used such that the VMDK object looks like a block device to the backup host which can be read through network. You need one NFC connection for each VMDK file being backed up. Thus backup is streamed from ESX/ESXi system’s VMkernel port to VMware backup host. If your ESX/ESXi hosts have VMKernel ports (known as the management ports) with dedicated uplinks, your virtual machine traffic links are not affected.
Источник
Err error opening the snapshot disks using given transport mode san status 23
05-21-2010 12:05 PM
We are using Netbackup 7 and are attempting to backup a vmware client via fiber when we receive the following error:
«Error opening the snapshot disks using given transport mode: Status 23.»
The VMware guide states the following for this error:
Select a different transfer type and retry the backup.»
The response is pretty obvious . The transfer type can easily be changed from the policy, although we have it set to SAN for a reason.
Our Configuration:
Master server — Windows 2003
Media server / Host backup server — Windows 2008
Both media and vmware host are in two different zones, although the shared snapshot disk is located on the SAN which both have access to.
Any recommendations? Is there a port that needs to be opened on my media server to allow the message to work?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-26-2010 11:10 AM
Adding the host ESX machine to the same zone as your Host-Backup server does not help. You must zone in the physical disks on the SAN LUN so that your Host-Backup server can see them in the Disk Management snap-in. In order to do this, Add the LUN to a new (or existing) SAN zone within the SAN fabric.
After doing so, tell the Host-Backup machine to scan for new devices in the device manager snap-in. Then Check your Disk Management Snap-in to verify the disks are showing up. They should show up as basic discs without drive letters, just as if the SAN was sharing them to the machine to be mapped. (Do NOT map them).
Re-run the job, and it should now work as the Host-Backup server has full access to the physical hard disks that the snapshot resides on.
** My issue was that:
A) The Veritias / Symantec guide for configuring VM-ware did a good job at leaving the topology out.
B) Once I finally managed to get support, they told me to only zone the physical ESX machine and the Host-Backup server together.
C) Once I finally insisted to speak with a better tech, I was informed that I needed to share the physical Hard Disks of where the snapshot actually resides on with the Host-Backup Server. After doing so, the backup worked flawlessly. (15GB in under 10min from FC disk to LTO4 via 2/4 GB Fiber Channel.. Not to mention the new ability of being able to backup Virtual Machines a lot easier)
Источник
Err error opening the snapshot disks using given transport mode san status 23
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-22-2017 11:17 AM
Master : Linux VM
Media : NetBackup Appliance 5230
Client : VM
Transport : SAN
VM image backup getting failed in SAN mode, but working fine with NBD mode.
Data store are mapped and visible to backup host.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-22-2017 01:03 PM
For me, its does not make sense. If you can make backup by NBD transport mode and cannot make using SAN, there is a problem with the Luns that were presented to VM backup host.
When this happen the backup job failed with an error message similar to this one:
From Detailed Status:
10/21/2013 11:18:28 — Info bptm (pid=14932) using 1048576 data buffer size
10/21/2013 11:18:28 — Info bptm (pid=14932) setting receive network buffer to 1048576 bytes
10/21/2013 11:18:28 — Info bptm (pid=14932) using 512 data buffers
10/21/2013 11:18:29 — Info bptm (pid=14932) start backup
10/21/2013 11:18:32 — begin writing
10/21/2013 11:18:34 — Error bpbrm (pid=14922) from client winclient.local: ERR — Error opening the snapshot disks using given transport mode: Status 23
10/21/2013 11:18:37 — Error bptm (pid=14932) media manager terminated by parent process
10/21/2013 11:18:42 — Info backup2 (pid=14932) StorageServer=PureDisk:backup2; Report=PDDO Stats for (backup2): scanned: 3 KB, CR sent: 0 KB, CR sent over FC: 0 KB, dedup: 100.0%
10/21/2013 11:18:42 — Info bpbkar (pid=0) done. status: 6: the backup failed to back up the requested files
10/21/2013 11:18:42 — end writing; write time: 0:00:10
the backup failed to back up the requested files (6)
Make sure LUNs are accessible to the VMware Backup Host and check the Troubleshotting below:
Troubleshooting for some common transport mode related failures
Backups/Restores failing with status 6 or status 13 or status 11 with following indication in Activity monitor might indicate that there is some issue with transport modes:-
- ERR — Error opening the snapshot disks using given transport mode: Status 23 indicates that there was some problem in accessing the vmdk using given transport mode.
Here are some tips on handling this kind of error:
- If you are using NBD, make sure the VMware Backup Host has connectivity to ESX server hosting the virtual machine.
- If you are using SAN, please make sure that the datastore LUNs are accessible to VMware Backup Host.
- If you are using HotAdd, please make sure that your backup host is Virtual Machine and following conditions are satisfied:
- The VM should not contain IDE disks.
- Ensure that there are sufficient SCSI controllers attached on the Backup Host VM.
- The Backup Host VM has access to datastores where VM being backed up resides.
- The Backup Host VM and VM being backed up should be under the same datacenter.
- If the previous backup failed, it might have left some disks of the backup VM attached to Backup Host. These disks need to be manually removed before attempting the next backup.
- If a non-default port for vCenter is in use, then that port needs to be defined while adding vCenter credentials to NetBackup or Backup Exec.
- If using NBD/NBDSSL/HotAdd, please make sure the VMware Backup Host is able to communicate to port 902 of ESX server hosting the VM.
- If using SAN for restores, please make sure datastore LUNs are accessible to the VMware Backup Host and in an online state.
- If using HotAdd for restore, please make sure that SAN policy on the Backup Host is set to OnlineAll .
- If using SAN for restore, make sure that the size of VMDK is multiple of datastore block size. Otherwise, the write of the last block will fail. In this case, a workaround would be to use NBD for restore.
- Please make sure that the you assign necessary privileges to the user configured in NetBackup or Backup Exec to log on to vSphere.
Источник
После обновления NetBackup до версии 7.6, последний перестает снимать снапшеты с виртуальных машин VMware.
В консоли можно наблюдать следующие ошибки:
ERR - Error opening the snapshot disks using given transport mode: hotadd:nbd:nbdssl Status 23
или
Error bpbrm (pid=####) from client vmclient01: ERR - Error opening the snapshot disks using given transport mode: Status 23
замена «transport mode» проблему не решает.
По данным ошибкам неявно гуглится «KB Article с номером TECH214315«. Особое внимание в статье следует уделить правам, которые необходимо дать NetBackup для работы в VMware :
This VDDK version requires slightly different datastore level permissions for our NetBackup credential account in vCenter. If the credential role is not already an admin in vCenter and is set granularly (or with a stock VCB role), backups for the virtual clients
т.е. если 7.5 версии достаточно было немного расширенного стандартного «vmware consolidated backup«,
In the Edit Role screen, place a check mark on the Extension privilege to include all three extension privileges: Register extension, Unregister extension, Update extension.
In the Edit Role screen, expand the Global privilege group and select the following privileges: Log event, Manage custom attributes, Set custom attribute.
то 7.6 версия хочет сразу админа.
Если более внимательно изучить KB Article, то можно найти статью «What are the minimum permissions needed to properly backup and restore using vStorage api?«, которая более детально описывает настройку роли пользователя для работы связки. Разрешения в виде xml файла доступны по ссылке: vcenter-netbackup-permissions.xml.
Порядок настройки NetBackup до 7.6 в связке с VMware.
1. В web версии VMware vSphere Client создаем нового локального пользователя.
Для логина вам необходим пользователь с правами Single Sign-On administrator@vsphere.local (vSphere 5.5)
Создавать будем пользователя с именем «vmbackup«.
Напрашивается вопрос, почему не воспользоваться доменным пользователем? Best practices показывает, что в случае наличия проблем в инфраструктуре, сети или контроллерах домена, — вам будет сложно восстановиться из бекапа без дополнительных действий.
2. Через «толстого клиента» VMware vSphere Client проверяем ранее созданного пользователя.
Использование толстого клиента продиктовано удобством администрирования, при желании можно по аналогии продолжить администрирование через web интерфейс.
3. Создаем новую роль пользователей.
Название на ваше усмотрение, мне приглянулось «VMware Consolidated Backup user for NetBackup». Производим настройку согласно «What are the minimum permissions needed to properly backup and restore using vStorage api?»
4. Разрешаем управление кластером пользователю vmbackup с правами «Backup user for NetBackup»
5. Вносим изменения в параметрах подключения netbackup к вируальной среде.
Необходимые нам поля находятся в credentials — virtual machine server:
6. Делаем тестовую политику и проверяем успешность выполнения.
Вы можете оставить комментарий ниже.
A VMware Backup Host can access Virtual Machine data from datastores using four different methods – SAN, LAN(NBD), HotAdd, NBDSSL. These methods are referred to as VMware Transport modes. This article talks about these transport modes, the best practices around them and troubleshooting tips for some commonly seen errors related to transport modes in NetBackup.
For both Backup and Restore operations, NetBackup and Backup Exec allow choosing any of the four transport modes or a combination of these. If a combination of the transport modes is given, NetBackup and Backup Exec will try all of them one by one until gaining successful access to the data of the Virtual Machine.
Details on each of the transport modes
1. SAN: The SAN transport mode requires the VMware Backup Host to reside on a physical machine with access to Fibre Channel or iSCSI SAN containing the virtual disks to be accessed. This is an efficient data path because no data needs to be transferred through the production ESX/ESXi host.
In this mode, vStorage APIs obtain information from the vCenter server or ESX/ESXi host about the layout of VMFS LUNs and, using this information, reads data directly from the SAN or iSCSI LUN where the VMDK resides.
Best practices around SAN:
For using SAN, make sure that datastore LUNs are accessible to the VMware Backup Host.
SAN transport is usually the best choice for backups when running on a physical VMware Backup Host. However, it is disabled inside virtual machines, so use HotAdd instead on a virtual VMware Backup Host.
SAN transport is not always the best choice for restores. It offers the best performance on thick disks, but the worst performance on thin disks, because of the way vStorage APIs work. For thin disk restore, LAN(NBD) is faster.
For SAN restores, disk size should be a multiple of the underlying VMFS block size, otherwise the write to the last fraction of a disk will fail. For example, if virtual disk has a 1MB block size and the datastore is 16.3MB large, the last 0.3MB will not get written. The workaround in this case would be to use NBD for restores of such Virtual Machines.
When using SAN transport or hot-add mode on a Windows Server 2008/2008 R2 VMware Backup Host, make sure to set:
SAN policy to onlineAll
SAN disk as read-only, except during restores
2. LAN (NBD): In this mode, the ESX/ESXi host reads data from storage and sends it across a network to the VMware Backup Host. As its name implies, this transport mode is not LAN‐free, unlike SAN transport.
LAN transport offers the following advantages:
The ESX/ESXi host can use any storage device, including local storage or NAS.
The VMware Backup server could be a virtual machine, so you can use a resource pool and scheduling capabilities of VMware vSphere to minimize the performance impact of backup. For example, you can put the VMware Backup Host in a different resource pool than the production ESX/ESXi hosts, with lower priority for backup.
If the ESX/ESXi host and VMware Backup Host are on a private network, you can use unencrypted data transfer,which is faster and consumes fewer resources than NBDSSL. If you need to protect sensitive information, you have the option of transferring virtual machine data in an encrypted form using NBDSSL.
Best Practices when using LAN:
Since the data in this case is read by ESX/ESXi server from storage and then sent to VMware Backup Host, It is must to have network connectivity between ESX/ESXi server and VMware Backup Host. If the VMware Backup Host has connectivity to vCenter server but not the ESX/ESXi server- snapshots will succeed but vmdk read/write operations will fail.
The VMware Backup Host will need the ability to connect to TCP port 902 on ESX/ESXi hosts while using NBD/NBDSSL for backup/restores.
VMware uses Network File Copy (NFC) protocol to read VMDK using NBD transport mode. You need one NFC connection for each VMDK file being backed up. There is a limit on the number of NFC connections that can be made per ESX/vCenter server. These limits differ in different versions of vSphere – please refer to the NetBackup for VMware Admin Guide (linked below) for these limits. Backup/Restore operations using NBD might hang if this limit is reached.
3. HotAdd: When running VMware Backup Host on a Virtual Machine, vStorage APIs can take advantage of the SCSI Hot-add capability of the ESX/ESXi server to attach the VMDKs of a Virtual Machine being backed up to the VMware Backup Host. This is referred to as HotAdd transport mode.
Running the VMware Backup server on a virtual machine has two advantages: it is easy to move a virtual machine around and it can also back up local storage without using the LAN, although this incurs more overhead on the physical ESX/ESXi host than when using SAN transport mode.
Best practices when using HotAdd:
HotAdd works only with virtual machines with SCSI disks and is not supported for backing up virtual machines with IDE disks.
A single SCSI controller can have a maximum of 15 disks attached. To run multiple concurrent jobs totally more than 15 disks it is necessary to add more SCSI controllers to the HotAdd host. The maximum number of 4 SCSI controllers can be added to a HotAdd host, so a total of 60 devices are supported at the maximum.
HotAdd requires the VMware Backup Host to have access to datastores where the Virtual Machine being backed up resides. This essentially means:
ESX where the VMware backup host is running should have access to datastores where the Virtual Machine being backed up resides.
Both the VMware backup host and Virtual Machine being backed up should be under the same datacenter.
HotAdd cannot be used if the VMFS block size of the datastore containing the virtual machine folder for the target virtual machine does not match the VMFS block size of the datastore containing the VMware Backup Host virtual machine. For example, if you back up virtual disk on a datastore with 1MB blocks, the VMware Backup Host must also be on a datastore with 1MB blocks.
Restores using HotAdd on a Windows Server 2008 proxy require setting the SAN policy to onlineAll
If you are converting a physical machine to a virtual machine with the intention of using hottadd to back up the virtual machine, do not use IDE controllers for any disks that are used during the conversion process.
The VMware Backup Host will need the ability to connect to TCP port 902 on ESX/ESXi hosts while using HotAdd for backup/restores.
4. NBDSSL: NBDSSL is the same as NBD, except that NBDSSL uses SSL to encrypt all data passed over the TCP/IP connection.
Troubleshooting for some common transport mode related failures
Backups/Restores failing with status 13 or status 11 with following indication in Activity monitor might indicate that there is some issue with transport modes:-
ERR – Error opening the snapshot disks using given transport mode: Status 23 indicates that there was some problem in accessing the vmdk using given transport mode.
Here are some tips on handling this kind of error:
If you are using NBD, make sure the VMware Backup Host has connectivity to ESX server hosting the virtual machine.
If you are using SAN, please make sure that the datastore LUNs are accessible to VMware Backup Host.
If you are using HotAdd, please make sure that your backup host is Virtual Machine and following conditions are satisfied:
The VM should not contain IDE disks.
Ensure that there are sufficient SCSI controllers attached on the Backup Host VM.
The Backup Host VM has access to datastores where VM being backed up resides.
The Backup Host VM and VM being backed up should be under the same datacenter.
If the previous backup failed, it might have left some disks of the backup VM attached to Backup Host. These disks need to be manually removed before attempting the next backup.
If a non-default port for vCenter is in use, then that port needs to be defined while adding vCenter credentials to NetBackup or Backup Exec.
If using NBD/NBDSSL/HotAdd, please make sure the VMware Backup Host is able to communicate to port 902 of ESX server hosting the VM.
file read failed indicates that there might be problem in reading the VMDK using the given transport mode.
file write failed indicates that there might be some problem in writing to the VMDK using the given transport mode.
If using SAN for restores, please make sure datastore LUNs are accessible to the VMware Backup Host and in an online state.
If using HotAdd for restore, please make sure that SAN policy on the Backup Host is set to OnlineAll.
If using SAN for restore, make sure that the size of VMDK is multiple of datastore block size. Otherwise, the write of the last block will fail. In this case, a workaround would be to use NBD for restore.
Please make sure that the you assign necessary privileges to the user configured in NetBackup or Backup Exec to log on to vSphere.
Encountered issues restoring via “SAN” failed during the boot up, error symptom is “operating system not found”, tried again with “NBD”, it works.
http://www.symantec.com/business/support/index?page=content&id=tech183072
Issue
A VMware Backup Host can access Virtual Machine data from datastores using four different methods – SAN, LAN(NBD), HotAdd, NBDSSL. These methods are referred to as VMware Transport modes. This article talks about these transport modes, the best practices around them and troubleshooting tips for some commonly seen errors related to transport modes in NetBackup and Backup Exec.
Solution
For both Backup and Restore operations, NetBackup and Backup Exec allow choosing any of the four transport modes or a combination of these. If a combination of the transport modes is given, NetBackup and Backup Exec will try all of them one by one until gaining successful access to the data of the Virtual Machine.
Details on each of the transport modes
1. SAN: The SAN transport mode requires the VMware Backup Host to reside on a physical machine with access to Fibre Channel or iSCSI SAN containing the virtual disks to be accessed. This is an efficient data path because no data needs to be transferred through the production ESX/ESXi host.
In this mode, vStorage APIs obtain information from the vCenter server or ESX/ESXi host about the layout of VMFS LUNs and, using this information, reads data directly from the SAN or iSCSI LUN where the VMDK resides.
Best practices around SAN:
- For using SAN, make sure that datastore LUNs are accessible to the VMware Backup Host.
- SAN transport is usually the best choice for backups when running on a physical VMware Backup Host. However, it is disabled inside virtual machines, so use HotAdd instead on a virtual VMware Backup Host.
- SAN transport is not always the best choice for restores. It offers the best performance on thick disks, but the worst performance on thin disks, because of the way vStorage APIs work. For thin disk restore, LAN(NBD) is faster.
- For SAN restores, disk size should be a multiple of the underlying VMFS block size, otherwise the write to the last fraction of a disk will fail. For example, if virtual disk has a 1MB block size and the datastore is 16.3MB large, the last 0.3MB will not get written. The workaround in this case would be to use NBD for restores of such Virtual Machines.
- When using SAN transport or hot-add mode on a Windows Server 2008/2008 R2 VMware Backup Host, make sure to set:
- SAN policy to onlineAll
- SAN disk as read-only, except during restores
2. LAN (NBD): In this mode, the ESX/ESXi host reads data from storage and sends it across a network to the VMware Backup Host. As its name implies, this transport mode is not LAN‐free, unlike SAN transport.
LAN transport offers the following advantages:
- The ESX/ESXi host can use any storage device, including local storage or NAS.
- The VMware Backup server could be a virtual machine, so you can use a resource pool and scheduling capabilities of VMware vSphere to minimize the performance impact of backup. For example, you can put the VMware Backup Host in a different resource pool than the production ESX/ESXi hosts, with lower priority for backup.
- If the ESX/ESXi host and VMware Backup Host are on a private network, you can use unencrypted data transfer,which is faster and consumes fewer resources than NBDSSL. If you need to protect sensitive information, you have the option of transferring virtual machine data in an encrypted form using NBDSSL.
Best Practices when using LAN:
- Since the data in this case is read by ESX/ESXi server from storage and then sent to VMware Backup Host, It is must to have network connectivity between ESX/ESXi server and VMware Backup Host. If the VMware Backup Host has connectivity to vCenter server but not the ESX/ESXi server- snapshots will succeed but vmdk read/write operations will fail.
- The VMware Backup Host will need the ability to connect to TCP port 902 on ESX/ESXi hosts while using NBD/NBDSSL for backup/restores.
- VMware uses Network File Copy (NFC) protocol to read VMDK using NBD transport mode. You need one NFC connection for each VMDK file being backed up. There is a limit on the number of NFC connections that can be made per ESX/vCenter server. These limits differ in different versions of vSphere — please refer to the NetBackup for VMware Admin Guide(linked below) for these limits. Backup/Restore operations using NBD might hang if this limit is reached.
3. HotAdd: When running VMware Backup Host on a Virtual Machine, vStorage APIs can take advantage of the SCSI Hot-add capability of the ESX/ESXi server to attach the VMDKs of a Virtual Machine being backed up to the VMware Backup Host. This is referred to as HotAdd transport mode.
Running the VMware Backup server on a virtual machine has two advantages: it is easy to move a virtual machine around and it can also back up local storage without using the LAN, although this incurs more overhead on the physical ESX/ESXi host than when using SAN transport mode.
Best practices when using HotAdd:
- HotAdd works only with virtual machines with SCSI disks and is not supported for backing up virtual machines with IDE disks.
- A single SCSI controller can have a maximum of 15 disks attached. To run multiple concurrent jobs totally more than 15 disks it is necessary to add more SCSI controllers to the HotAdd host. The maximum number of 4 SCSI controllers can be added to a HotAdd host, so a total of 60 devices are supported at the maximum.
- HotAdd requires the VMware Backup Host to have access to datastores where the Virtual Machine being backed up resides. This essentially means:
- ESX where the VMware backup host is running should have access to datastores where the Virtual Machine being backed up resides.
- Both the VMware backup host and Virtual Machine being backed up should be under the same datacenter.
- HotAdd cannot be used if the VMFS block size of the datastore containing the virtual machine folder for the target virtual machine does not match the VMFS block size of the datastore containing the VMware Backup Host virtual machine. For example, if you back up virtual disk on a datastore with 1MB blocks, the VMware Backup Host must also be on a datastore with 1MB blocks.
- Restores using HotAdd on a Windows Server 2008 proxy require setting the SAN policy toonlineAll
- If you are converting a physical machine to a virtual machine with the intention of using hottadd to back up the virtual machine, do not use IDE controllers for any disks that are used during the conversion process.
- The VMware Backup Host will need the ability to connect to TCP port 902 on ESX/ESXi hosts while using HotAdd for backup/restores.
4. NBDSSL: NBDSSL is the same as NBD, except that NBDSSL uses SSL to encrypt all data passed over the TCP/IP connection.
Troubleshooting for some common transport mode related failures
Backups/Restores failing with status 6 or status 13 or status 11 with following indication in Activity monitor might indicate that there is some issue with transport modes:-
- ERR — Error opening the snapshot disks using given transport mode: Status 23 indicates that there was some problem in accessing the vmdk using given transport mode.
Here are some tips on handling this kind of error:- If you are using NBD, make sure the VMware Backup Host has connectivity to ESX server hosting the virtual machine.
- If you are using SAN, please make sure that the datastore LUNs are accessible to VMware Backup Host.
- If you are using HotAdd, please make sure that your backup host is Virtual Machine and following conditions are satisfied:
- The VM should not contain IDE disks.
- Ensure that there are sufficient SCSI controllers attached on the Backup Host VM.
- The Backup Host VM has access to datastores where VM being backed up resides.
- The Backup Host VM and VM being backed up should be under the same datacenter.
- If the previous backup failed, it might have left some disks of the backup VM attached to Backup Host. These disks need to be manually removed before attempting the next backup.
- If a non-default port for vCenter is in use, then that port needs to be defined while adding vCenter credentials to NetBackup or Backup Exec.
- If using NBD/NBDSSL/HotAdd, please make sure the VMware Backup Host is able to communicate to port 902 of ESX server hosting the VM.
- file read failed indicates that there might be problem in reading the VMDK using the given transport mode.
- file write failed indicates that there might be some problem in writing to the VMDK using the given transport mode.
- If using SAN for restores, please make sure datastore LUNs are accessible to the VMware Backup Host and in an online state.
- If using HotAdd for restore, please make sure that SAN policy on the Backup Host is set to OnlineAll.
- If using SAN for restore, make sure that the size of VMDK is multiple of datastore block size. Otherwise, the write of the last block will fail. In this case, a workaround would be to use NBD for restore.
- Please make sure that the you assign necessary privileges to the user configured in NetBackup or Backup Exec to log on to vSphere.
+7
- Byte
- 49 replies
Hi,
We are having a problem with our Netbackup setup with HotAdd mode to backup our VMware, i.e. if the Proxy Server takes more than 4 VMs to backup at the same time, loads of VMs would fail with the below error messages.
“Error opening the snapshot disks using given transport mode: hotadd Status 23”
In other words, this Proxy Server can backup no more than 4 VMs at the same time.
OK, I admit that this is a problem in our Netbackup environment but I feel that I may get some help by posting this question here in Commvault forum because
- Firstly, the reason why we are testing this HotAdd transport mode within Netbackup environment is because we intend to deploy the same technology in the coming months in our another backup environment which is Commvault.
- Also, at the moment, the Netbackup support seems to be at their wits’ end (which does make me feel a bit disappointed). But, on the other hand, the issue seems to be more connected to the VMware environment and its associated configuration / setup of the Proxy Server rather than the backup software itself. In other words, the problem could be commonly and widely known by all the major backup vendors.
- Last but not least, you guys on this forum do have a much quicker and more informative response, to be honest.
So, if your guys are still happy for me to continue, then I’ll give you the details of our environment
- Number of datacentre: 1
- Datacenter vSphere version: 6.7
- The version of ESXi Server hosting the Proxy Server: 6.7
- Datastore hosting the Proxy Server and other VMs: VMFS 6
- Total number of VMs to backup: 560
The configuration of the VMware backup Proxy Server
- OS: Windows 2019
- MEM: 32 GB
- CPU: 16 cores
- Local disks: C (100GB with 68GB free) and D (40GB with 35 GB free)
- The size of the datastore hosting this proxy Server: 4TB with 950GB free
- The number of SCSI cards: 4
- The type of SCSI cards: VMware Paravirtual
- The VDDK version (embedded within the Netbackup): 6.7.3
- PCI device: 1 passing through PCI which is zoned with a tape library
Just to summarise, this VMware Proxy Server is also a Netbackup Media Server, which reads data directly from its HotAdd mounted VMDK, and then writes data directly to the FC connected tape library.
My observation is
- During the backup, even with simultaneously backup VMs to be set at 16, the CPU and MEM would hardly be stressed.
- However, from its disk management GUI, with a setting of more than 4 VMs to backup at the same time, it would constantly scanning the SCSI cards to mount/unmount those VM’s VMDK disks, which would freeze the Proxy Server. In fact, the more VMs to backup at the same time, the busier its SCSI cards would be, which makes me wonder if this is the root cause of the problem ?
Has anyone seen the similar issue under a Commvault environment with a CommVault Proxy Server setup in this way ?
Many thanks,
Kelvin
icon
Best answer by Mike Struening 11 October 2021, 23:04
View original
Problem
A VMware Backup Host can access Virtual Machine data from
datastores using four different methods – SAN, LAN(NBD), HotAdd, NBDSSL.
These methods are referred to as VMware Transport modes.
This article talks about these transport modes, the best practices
around them and troubleshooting tips for some commonly seen errors
related to transport modes in NetBackup and Backup Exec.
Solution
For both Backup and Restore operations, NetBackup and Backup Exec
allow choosing any of the four transport modes or a combination of
these. If a combination of the transport modes is given, NetBackup and
Backup Exec will try all of them one by one until gaining successful
access to the data of the Virtual Machine.
Details on each of the transport modes
1. SAN: The SAN transport mode requires the VMware
Backup Host to reside on a physical machine with access to Fibre Channel
or iSCSI SAN containing the virtual disks to be accessed. This is an
efficient data path because no data needs to be transferred through the
production ESX/ESXi host.
In this mode, vStorage APIs obtain information from the vCenter
server or ESX/ESXi host about the layout of VMFS LUNs and, using this
information, reads data directly from the SAN or iSCSI LUN where
the VMDK resides.
Best practices around SAN:
- For using SAN, make sure that datastore LUNs are accessible to the VMware Backup Host.
- SAN
transport is usually the best choice for backups when running on a
physical VMware Backup Host. However, it is disabled inside virtual
machines, so use HotAdd instead on a virtual VMware Backup Host. - SAN
transport is not always the best choice for restores. It offers the
best performance on thick disks, but the worst performance on thin
disks, because of the way vStorage APIs work. For thin disk restore,
LAN(NBD) is faster. - For SAN restores, disk size should be a
multiple of the underlying VMFS block size, otherwise the write to the
last fraction of a disk will fail. For example, if virtual disk has a
1MB block size and the datastore is 16.3MB large, the last 0.3MB will
not get written. The workaround in this case would be to use NBD for
restores of such Virtual Machines. - When using SAN transport or hot-add mode on a Windows Server 2008/2008 R2 VMware Backup Host, make sure to set:
- SAN policy to onlineAll
- SAN disk as read-only, except during restores
2. LAN (NBD): In this mode, the ESX/ESXi host reads
data from storage and sends it across a network to the VMware Backup
Host. As its name implies, this transport mode is not LAN‐free, unlike
SAN transport.
LAN transport offers the following advantages:
- The ESX/ESXi host can use any storage device, including local storage or NAS.
- The
VMware Backup server could be a virtual machine, so you can use a
resource pool and scheduling capabilities of VMware vSphere to minimize
the performance impact of backup. For example, you can put the VMware
Backup Host in a different resource pool than the production ESX/ESXi
hosts, with lower priority for backup. - If the ESX/ESXi host and
VMware Backup Host are on a private network, you can use unencrypted
data transfer,which is faster and consumes fewer resources than NBDSSL.
If you need to protect sensitive information, you have the option of
transferring virtual machine data in an encrypted form using NBDSSL.
Best Practices when using LAN:
- Since the data in this case is read by ESX/ESXi server from
storage and then sent to VMware Backup Host, It is must to have network
connectivity between ESX/ESXi server and VMware Backup Host. If the
VMware Backup Host has connectivity to vCenter server but not the
ESX/ESXi server- snapshots will succeed but vmdk read/write operations
will fail. - The VMware Backup Host will need the ability to
connect to TCP port 902 on ESX/ESXi hosts while using NBD/NBDSSL for
backup/restores. - VMware uses Network File Copy (NFC) protocol to
read VMDK using NBD transport mode. You need one NFC connection for
each VMDK file being backed up. There is a limit on the number of NFC
connections that can be made per ESX/vCenter server. These limits differ
in different versions of vSphere — please refer to the NetBackup for VMware Admin Guide (linked below) for these limits. Backup/Restore operations using NBD might hang if this limit is reached.
3. HotAdd: When running VMware Backup Host on a
Virtual Machine, vStorage APIs can take advantage of the SCSI Hot-add
capability of the ESX/ESXi server to attach the VMDKs of a Virtual
Machine being backed up to the VMware Backup Host. This is referred to
as HotAdd transport mode.
Running the VMware Backup server on a virtual machine has two
advantages: it is easy to move a virtual machine around and it can also
back up local storage without using the LAN, although this incurs more
overhead on the physical ESX/ESXi host than when using SAN transport
mode.
Best practices when using HotAdd:
- HotAdd works only with virtual machines with SCSI disks and is not supported for backing up virtual machines with IDE disks.
- A
single SCSI controller can have a maximum of 15 disks attached. To run
multiple concurrent jobs totally more than 15 disks it is necessary to
add more SCSI controllers to the HotAdd host. The maximum number of 4
SCSI controllers can be added to a HotAdd host, so a total of 60 devices
are supported at the maximum. - HotAdd requires the VMware Backup
Host to have access to datastores where the Virtual Machine being
backed up resides. This essentially means:
- ESX where the VMware backup host is running should have access to datastores where the Virtual Machine being backed up resides.
- Both the VMware backup host and Virtual Machine being backed up should be under the same datacenter.
- HotAdd cannot be used if the VMFS block size of the
datastore containing the virtual machine folder for the target virtual
machine does not match the VMFS block size of the datastore containing
the VMware Backup Host virtual machine. For example, if you back up
virtual disk on a datastore with 1MB blocks, the VMware Backup Host must
also be on a datastore with 1MB blocks. - Restores using HotAdd on a Windows Server 2008 proxy require setting the SAN policy to onlineAll
- If
you are converting a physical machine to a virtual machine with the
intention of using hottadd to back up the virtual machine, do not use
IDE controllers for any disks that are used during the conversion
process. - The VMware Backup Host will need the ability to connect
to TCP port 902 on ESX/ESXi hosts while using HotAdd for
backup/restores
4. NBDSSL: NBDSSL is the same as NBD, except that NBDSSL uses SSL to encrypt all data passed over the TCP/IP connection.
NFC Session Limits:
NBD employs the VMware network file copy (NFC) protocol. NFC Session Connection Limits shows limits on the number of network connections for various host types. VixDiskLib_Open() uses one connection for every virtual disk that it accesses on an ESX/ESXi host. VixDiskLib_Clone()
also requires a connection. It is not possible to share a connection
across disks. These are host limits, not per process limits, and do not
apply to SAN or HotAdd.
Host Platform |
When Connecting |
Limits You To About |
vSphere 4 |
to an ESX host |
9 connections directly, 27 connections through vCenter Server |
vSphere 4 |
to an ESXi host |
11 connections directly, 23 connections through vCenter Server |
vSphere 5 |
to an ESXi host |
Limited by a transfer buffer for all NFC connections, enforced by the host; the sum of all NFC connection buffers to an ESXi host cannot exceed 32MB. 52 connections through vCenter Server, including the above per-host limit. |
Troubleshooting for some common transport mode related failures
Backups/Restores failing with status 6 or status 13 or status 11 with
following indication in Activity monitor might indicate that there is
some issue with transport modes:-
- ERR — Error opening the snapshot disks using given transport mode: Status 23 indicates that there was some problem in accessing the vmdk using given transport mode.
Here are some tips on handling this kind of error:
- If you are using NBD, make sure the VMware Backup Host has connectivity to ESX server hosting the virtual machine.
- If you are using SAN, please make sure that the datastore LUNs are accessible to VMware Backup Host.
- If you are using HotAdd, please make sure that your backup host is Virtual Machine and following conditions are satisfied:
- The VM should not contain IDE disks.
- Ensure that there are sufficient SCSI controllers attached on the Backup Host VM.
- The Backup Host VM has access to datastores where VM being backed up resides.
- The Backup Host VM and VM being backed up should be under the same datacenter.
- If
the previous backup failed, it might have left some disks of the backup
VM attached to Backup Host. These disks need to be manually removed
before attempting the next backup.
- If a non-default port for vCenter is in use, then
that port needs to be defined while adding vCenter credentials to
NetBackup or Backup Exec. - If using NBD/NBDSSL/HotAdd, please
make sure the VMware Backup Host is able to communicate to port 902 of
ESX server hosting the VM.
- file read failed indicates that there might be problem in reading the VMDK using the given transport mode.
- file write failed indicates that there might be some problem in writing to the VMDK using the given transport mode.
- If using SAN for restores, please make sure datastore LUNs are accessible to the VMware Backup Host and in an online state.
- If using HotAdd for restore, please make sure that SAN policy on the Backup Host is set to OnlineAll.
- If
using SAN for restore, make sure that the size of VMDK is multiple of
datastore block size. Otherwise, the write of the last block will
fail. In this case, a workaround would be to use NBD for restore. - Please
make sure that the you assign necessary privileges to the user
configured in NetBackup or Backup Exec to log on to vSphere.
Далее, в меню выбора опций восстанавливаемой машины ставим галку – «Remove backing information for devices like DVD/CD-ROM……»;
также советую выбрать формат диска «Thick Provision Lazy Zeroed».
Далее, выбираем хранилище. Следующее окно самое интересное, выбор сетевых интерфейсов, ОБЯЗАТЕЛЬНО убрать все галки со всех интерфейсов, иначе восстановление упадет с ошибкой 2820 VMware policy restore error,
т.к. старые сетевые интерфейсы все равно не смогут подключиться к distributed switch. А если у вас standard switch то и тогда все равно нужно убрать все сетевые интерфейсы, но с подключением проблем не возникнет(просто используем свитч по-умолчанию VM Network).
Далее, проводим предварительную проверку наших вводных данных, все проверки должны быть пройдены, если нет проверяем конкретный пункт. После чего жмем «Начать восстановление».
Процедура настройки после восстановления:
Как было указано в условиях, к хосту на который мы восстановили vcenter, необходимо подключить еще один линк.
Этот линк добавляем в standard switch — VM Network, заходим на виртуалку и назначаем любой IP на эту сетевую карту.
Перезагружаем vcenter. После перезагрузки vcenter должен увидеть хосты, если нет делаем reconnect. Теперь у vcenter станут доступны наши distributed switch.
Нужно проверить находится ли старая сетевка в distributed switch, если нет добавляем. Далее в свойствах vcenter добавляем новую сетевую карту из distributed switch, после проверки временный линк можно отключать.