Добрый день! Уважаемые читатели и гости популярного блога, о Vmware ESXI и настройке серверов pyatilistnik.org. Относительно недавно, я вам рассказывал, как создавать свои оповещения в vCenter 5.5 (тригеры). Там я показывал, как редактируются встроенные тригеры. Сегодня на одном из моих датасторов, выскочило предупреждение: Datastore usage on disk, сам LUN имеет размеры 4 ТБ и на нем было свободно более 800 гигабайт. Данное оповещение сообщает, что у меня начинает заканчиваться свободное место на дисковом массиве, но в виду того, что свободно 800 гигабайт, это оповещение мне кажется лишним. Я вам покажу, как его поправить в vCenter 6.5 сервере.
Как выглядит предупреждение Datastore usage on disk
Вот так вот на вкладке «Summary» выглядит предупреждение.
Если вы более подробно посмотрите сообщение, то вы увидите, что это стандартное оповещение (alarm) используемое в мониторинге гипервизоров.
Варианты решения проблемы
Данное уведомление очень полезное, так как системный администратор будет в курсе, что у него заканчивается место, хотя уверен, что он об этом узнает из другой системы мониторинга, например, Zabbix. Но если у вас ситуация как у меня, когда на датасторе полно место и вы не хотите, чтобы предупреждение мозолило вам глаза, то у вас два варианта, точнее три:
- Освободить свободное место на нужном датасторе ESXI хоста, не всегда представляется возможным
- Полностью отключить оповещение, не самый лучший вариант
- Отредактировать настройки, и изменить значения срабатывания тригера, наш выбор
Для того, чтобы изменить параметры, нужного вам задания по мониторингу, выберите в корне ваш vCenter 6.5 сервер, раздел «Monitor«, вкладка Issues, в которой выберите пункт «Alarm Definitions«. В поисковой строке введите «Datastore usage on disk». В итоге у вас будет выполнен фильтр по данному имени.
Щелкните по нему правой кнопкой мыши и из контекстного меню выберите пункт «Edit», для его редактирования.
Откройте пункт «Triggers», в нем вы увидите значения при которых он будет срабатывать. По умолчанию, это 75% занятого места, это предупреждение и 85% это критический alarm.
Щелкните по ним и измените значения на свои.
Я выставил 85% для предупреждения и 95% для критического сигнала. Сохраняем изменения, и ручками очищаем текущие уведомления, через функцию «reset to Green».
Второй метод, полного отключения политики оповещения по тригеру «Datastore usage on disk». Делается это, через VMware PowerCLI. Подключитесь, через команду Connect-viserver -Server имя сервера, если будет ошибка проверки сертификата (Invalid server certificate), то ее решение в скобках.
Запросим текущее состояние политики с помощью команды:
Get-AlarmDefinition -Name ‘Datastore usage on disk’
Как видите она включена и имеет значение «True». Давайте ее отключим, для этого есть две команды, которые я подсмотрел на (https://kb.vmware.com/s/article/2076157):
Get-AlarmDefinition -Name ‘Datastore usage on disk’ | Set-AlarmDefinition -Enabled:$false
для включения политики, выполните:
Get-AlarmDefinition -Name ‘Datastore usage on disk’ | Set-AlarmDefinition -Enabled:$true
Посмотреть все политики оповещения связанные с датастором, можно вот так:
Get-AlarmDefinition -Name ‘*Datastore*’ | ft -autosize
Из графического интерфейса можно выключить тригер, сняв галку «Enable this alarm»
Старайтесь не доводить ваши датасторы, до уведомления «Datastore usage on disk»
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
So you want to reduce the vmdk size of the guest os? If that’s what your looking to do you could utilize VMware converter to redo the disk layout and lower the disk size. You could also utilize sdelete and storage vmotion to move it to another datastore as a thin provisoned disk. I’ve never done the latter but seen it mentioned several times here in the forums.
If you found this or any other post helpful please consider the use of the Helpful/Correct buttons to award points
Twitter: http://twitter.com/mittim12
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I would check to make sure that you don’t have any VMs that are currently running on snapshots. This discrepancy could be caused by this.
If you found this at all helpful please award points by using the correct or helpful buttons! Thanks!
James B. | Blog: http://www.vSential.com | Twitter: @vSential
—
If you found this helpful then please awards helpful or correct points accordingly. Thanks!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Ok, so just want to clarify and you let me know if this is right.
We have a thin provisioned vmdk of around 85 GB on a 86 GB data store. The vmdk has grown to around 65 GB or so and the datastore is around 20GB of free space. You look inside the VM and notice that 78 GB or so is free and your wondering why the datastore is out of space and how to fix it?
If you found this or any other post helpful please consider the use of the Helpful/Correct buttons to award points
Twitter: http://twitter.com/mittim12
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
hi mittim12.
very close. i know why it grew, a LOG file got very large (@ the OS level) forcing the datastore to grow as well (@ the VM level).
i am 90% sure that was thr root cause.
but i would like to know how to fix it, or i should ignore it.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
If you want to reclaim the space your options are to use VMware converter or to use sdelete inside the VM to zero out the free space and storage vMotion that vmdk to another datastore while specifiying thin disk.
If you found this or any other post helpful please consider the use of the Helpful/Correct buttons to award points
Twitter: http://twitter.com/mittim12
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
do you recommend sDelete with both -c and -z or ONLY -z?
thank you.
Usage: sdelete
sdelete -z
-c Zero free space (good for virtual disk optimization).
-p passes Specifies number of overwrite passes.
-s Recurse subdirectories.
-q Don’t print errors (quiet).
-z Cleanse free space.
-
kayvonp@re-wa.org
- Influencer
- Posts: 14
- Liked: 1 time
- Joined: Aug 13, 2018 1:52 pm
- Full Name: KTHP
- Contact:
‘Datastore usage on disk’ Alarm in VMware for Veeam Proxy
I am getting a strange error for one of my Veeam proxies. I don’t completely understand it.
In VMware, I have two datastores that were created by Veeam — one for my Veeam server and one for a backup proxy. I have a second backup proxy that does not have a datastore, though to my knowledge the two proxies are identical. I don’t understand what these datastores do — can anyone explain?
The error I get is for used space on the datastore for the proxy. It has reached >75% used. The one for my Veeam server, however, is only using about .5% of its space.
I tried searching online for the error, but all I found was errors about space filling up on the actual backup drives. These datastores aren’t where backups are stored, but rather datastores created by Veeam in VMware.
-
PTide
- Product Manager
- Posts: 6249
- Liked: 687 times
- Joined: May 19, 2015 1:46 pm
- Contact:
Re: ‘Datastore usage on disk’ Alarm in VMware for Veeam Proxy
Post
by PTide » Nov 16, 2018 5:03 pm
Hi,
I can’t see it very clear from my side of the screen, but I think that those must be NFS datastores that are in fact vPower NFS shares. Does it look any similar to this picture? If yes, then I would check if there are any rouge Instant Restore or SureBackup VMs running in your environment.
Thanks!
-
kayvonp@re-wa.org
- Influencer
- Posts: 14
- Liked: 1 time
- Joined: Aug 13, 2018 1:52 pm
- Full Name: KTHP
- Contact:
-
PTide
- Product Manager
- Posts: 6249
- Liked: 687 times
- Joined: May 19, 2015 1:46 pm
- Contact:
Re: ‘Datastore usage on disk’ Alarm in VMware for Veeam Proxy
Post
by PTide » Nov 16, 2018 6:01 pm
Have you checked what occupies the space on the datastores? If there is nothing on the actual datastore, then I would suggest to check it from the NFS server side, maybe something is taking up storage space on the disk where your NFS folder resides.
Thanks!
-
kayvonp@re-wa.org
- Influencer
- Posts: 14
- Liked: 1 time
- Joined: Aug 13, 2018 1:52 pm
- Full Name: KTHP
- Contact:
Re: ‘Datastore usage on disk’ Alarm in VMware for Veeam Proxy
Post
by kayvonp@re-wa.org » Nov 16, 2018 6:04 pm
I browsed it in VSphere and it appears empty.
«I would suggest to check it from the NFS server side, maybe something is taking up storage space on the disk where your NFS folder resides.»
How do I go about this? I scanned the disk on the proxy server itself and didn’t find anything. Is it located elsewhere?
-
PTide
- Product Manager
- Posts: 6249
- Liked: 687 times
- Joined: May 19, 2015 1:46 pm
-
Contact:
Re: ‘Datastore usage on disk’ Alarm in VMware for Veeam Proxy
Post
by PTide » Nov 16, 2018 6:09 pm
You should check disks on the servers that have been assigned «Mount server» roles. When you setup a repository you are offered to choose where your mount server should reside, and where write cache location should be (default value is C:ProgramDataVeeamBackupNfsDatastore). You need to review your settings and check how much space those location have.
Thanks!
-
kayvonp@re-wa.org
- Influencer
- Posts: 14
- Liked: 1 time
- Joined: Aug 13, 2018 1:52 pm
- Full Name: KTHP
- Contact:
Re: ‘Datastore usage on disk’ Alarm in VMware for Veeam Proxy
Post
by kayvonp@re-wa.org » Nov 16, 2018 6:24 pm
That’s kinda what I thought it was, but this particular server isn’t used as a mount server for anything. It’s only used as a proxy. The server that does act as a mount server is clear.
I do have a C:ProgramDataVeeamBackupNfsDatastore folder on that server, but it is empty.
-
PTide
- Product Manager
- Posts: 6249
- Liked: 687 times
- Joined: May 19, 2015 1:46 pm
- Contact:
Re: ‘Datastore usage on disk’ Alarm in VMware for Veeam Proxy
Post
by PTide » Nov 16, 2018 6:31 pm
Then, if the disk where the NfsDatatores directories reside at are not in shortage of space, please contact our support team directly so they can dig into the problem, because the described behaviour does not look expected. Once you do please post your case ID here for future reference.
Thanks!
Who is online
Users browsing this forum: No registered users and 43 guests
One of the complaints of vCenter 2.x was the limited number of alarms that could be configured. With vCenter 4.x VMware addressed this by giving admins the ability to create alarms for just about any event on any object.
There are several alarms that are configured out of box and defined at the topmost vCenter server level.
One of the pre-configured alarms that seems to always be in a warnings state is the “Datastore usage on disk” alarm. 95% of the time this alarm is in a warning state because of the local VMFS data store that is required in ESX 4 for the Service Console OS to live in.
I still have yet to understand why VMware decided to put the COS in a VMDK. If anyone knows why this is beneficial I would love to here from you.
Anyway, back to the “Datastore usage on disk” alarm issue.
I have found that the following procedure is the best way to prevent this alarm on the local VMFS datastores.
- Go to the Home –>Inventory—>Datastores view in the vSphere Client
- Right-click on your datacenter and create two new folders, one named LOCAL-COS-DATASTORES and one named SAN-DATASTORES.
- Drag and drop the local VMFS datastores into the LOCAL-COS-DATASTORES folder and the SAN datastores into the SAN-DATASTORES folder.
- Go to the alarm definition for “Datastore usage on disk” and disable it by clearing the Enable this alarm box.
- Create a new alarm at the LOCAL-COS-DATASTORES level named “Datastore usage on LOCAL Disks”.
- Configure the “Datastore usage on LOCAL Disks” trigger to alarm a warning at 90% and alert at 95%.
- Create a new alarm at the SAN-DATASTORES level named “Datastore usage on SAN Disks”.
- Configure the “Datastore usage on SAN Disks” trigger to alarm a warning at 70% and alert at 85%.
Now you should have a lot fewer datastore warning alarms.