Ubuntu ext4 fs error

I have a Dell XPS 15 9550. I’ve been running Ubuntu 16.10 on it for four months with no dramas.

Two days ago, I upgraded to Ubuntu 17.04. About an hour after upgrading, my hard-drive remounted into read-only mode. When I jumped to a tty screen, this appeared:

[ 746.341551] EXT4-fs error (device nvme0n1p7): ext4_find_entry:1463: inode #525023: comm NetworkManager: reading directory iblock 0
[ 746.343318] EXT4-fs error (device nvme0n1p7): ext4_find_entry:1463: inode #524289: comm pool: reading directory iblock 0
[ 746.356125] EXT4-fs error (device nvme0n1p7): ext4_find_entry:1463: inode #11272213: comm systemd-udevd: reading directory iblock 0
[ 746.356139] EXT4-fs error (device nvme0n1p7): ext4_find_entry:1463: inode #11272210: comm systemd-udevd: reading directory iblock 0
[ 746.356332] EXT4-fs error (device nvme0n1p7): ext4_find_entry:1463: inode #11272193: comm systemd-udevd: reading directory iblock 0
[ 746.356338] EXT4-fs error (device nvme0n1p7): ext4_find_entry:1463: inode #11272825: comm systemd-udevd: reading directory iblock 0
[ 746.356400] EXT4-fs error (device nvme0n1p7): ext4_find_entry:1463: inode #11272210: comm systemd-udevd: reading directory iblock 0
[ 746.474632] EXT4-fs error (device nvme0n1p7): ext4_find_entry:1463: inode #524539: comm unity-settings-: reading directory iblock 0
[ 746.992814] EXT4-fs error (device nvme0n1p7): ext4_find_entry:1463: inode #5506108: comm BrowserBlocking: reading directory iblock 0
[ 746.304451] EXT4-fs error (device nvme0n1p7): ext4_find_entry:1463: inode #5506117: comm BrowserBlocking: reading directory iblock 0

Here’s what fdisk -l shows:

Disk /dev/nvme0n1: 477 GiB, 512110190592 bytes, 1000215216 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 3CD27380-DAC8-48DC-910A-D084CE857DA3

Device             Start        End   Sectors   Size Type
/dev/nvme0n1p1      2048    1026047   1024000   500M EFI System
/dev/nvme0n1p2   1026048    1288191    262144   128M Microsoft reserved
/dev/nvme0n1p3   1288192  487948287 486660096 232.1G Microsoft basic data
/dev/nvme0n1p4 972302336  973223935    921600   450M Windows recovery environmen
/dev/nvme0n1p5 973223936  998094847  24870912  11.9G Windows recovery environmen
/dev/nvme0n1p6 998094848 1000204287   2109440     1G Windows recovery environmen
/dev/nvme0n1p7 487948288  939046911 451098624 215.1G Linux filesystem
/dev/nvme0n1p8 939046912  972302335  33255424  15.9G Linux swap

Partition table entries are not in disk order.

I rebooted, and continued to get the error around once an hour. So I reinstalled Ubuntu 17.04 from scratch. However I am still getting the same issue.

I tried running fsck by creating a /forcefsck file (I created a wrapper shell script that adds the -v flag and outputs stdout to a file). Here’s the result:

fsck.fat 4.0 (2016-05-06)                               
Checking we can access the last sector of the filesystem
Boot sector contents:                                   
System ID "MSDOS5.0"                                    
Media byte 0xf8 (hard disk)                             
       512 bytes per logical sector                     
      4096 bytes per cluster                            
      6206 reserved sectors                             
First FAT starts at byte 3177472 (sector 6206)          
         2 FATs, 32 bit entries                         
    508416 bytes per FAT (= 993 sectors)                
Root directory start at cluster 2 (arbitrary size)      
Data area starts at byte 4194304 (sector 8192)          
    126976 data clusters (520093696 bytes)              
63 sectors/track, 255 heads                             
      2048 hidden sectors                               
   1024000 sectors total                                
Reclaiming unconnected clusters.                        
Checking free cluster summary.                          
/dev/nvme0n1p1: 212 files, 15526/126976 clusters    

I tried booting from a live USB and running e2fsck -p /dev/nvme0n1p7 as suggested here (https://askubuntu.com/a/768813/679041). It didn’t give any errors.

I also tried to run smartctl -t long /dev/nvme0n1p7 however the results seem to indicate that the tool doesn’t work with my particular SSD:

smartctl 6.6 2016-05-31 r4324 [x86_64-linux-4.10.0-19-generic] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:                       PM951 NVMe SAMSUNG 512GB
Serial Number:                      S29PNX0H611013
Firmware Version:                   BXV77D0Q
PCI Vendor/Subsystem ID:            0x144d
IEEE OUI Identifier:                0x002538
Controller ID:                      1
Number of Namespaces:               1
Namespace 1 Size/Capacity:          512,110,190,592 [512 GB]
Namespace 1 Utilization:            254,982,533,120 [254 GB]
Namespace 1 Formatted LBA Size:     512
Local Time is:                      Mon Apr 17 17:45:48 2017 AEST
Firmware Updates (0x06):            3 Slots
Optional Admin Commands (0x0017):   Security Format Frmw_DL *Other*
Optional NVM Commands (0x001f):     Comp Wr_Unc DS_Mngmt Wr_Zero Sav/Sel_Feat
Maximum Data Transfer Size:         32 Pages

Supported Power States
St Op     Max   Active     Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +     6.00W       -        -    0  0  0  0        5       5
 1 +     4.20W       -        -    1  1  1  1       30      30
 2 +     3.10W       -        -    2  2  2  2      100     100
 3 -   0.0700W       -        -    3  3  3  3      500    5000
 4 -   0.0050W       -        -    4  4  4  4     2000   22000

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 +     512       0         0

=== START OF SMART DATA SECTION ===
Read NVMe SMART/Health Information failed: NVMe Status 0x2002

Any idea of why this issue might be occuring and how I might solve it? Thanks!

Страницы: [1]   Вниз

0 Пользователей и 1 Гость просматривают эту тему.

Страницы: [1]   Вверх

Уже второй раз (первый был несколько дней назад) возникают ошибки ФС и диск перемонтируется в ро:

[ 6974.205725] EXT4-fs error (device sda2): ext4_iget:4761: inode #7699735: comm TaskSchedulerFo: bad extra_isize 65535 (inode size 256)
[ 6974.210925] Aborting journal on device sda2-8.
[ 6974.213421] EXT4-fs (sda2): Remounting filesystem read-only
[ 6974.214980] EXT4-fs error (device sda2): ext4_journal_check_start:61: Detected aborted journal
[ 6995.041017] systemd-journald[281]: Failed to write entry (26 items, 852 bytes), ignoring: Read-only file system
[ 6995.041254] systemd-journald[281]: Failed to write entry (26 items, 835 bytes), ignoring: Read-only file system
[ 6995.041313] systemd-journald[281]: Failed to write entry (26 items, 1059 bytes), ignoring: Read-only file system
[ 6995.041364] systemd-journald[281]: Failed to write entry (26 items, 852 bytes), ignoring: Read-only file system
[ 6995.041599] systemd-journald[281]: Failed to write entry (26 items, 852 bytes), ignoring: Read-only file system
[ 6995.041652] systemd-journald[281]: Failed to write entry (26 items, 852 bytes), ignoring: Read-only file system
[ 6995.041715] systemd-journald[281]: Failed to write entry (26 items, 852 bytes), ignoring: Read-only file system

Диск:

> sudo hdparm -i /dev/sda
[sudo] пароль для alex: 

/dev/sda:

 Model=SanDisk SD8TB8U256G1001, FwRev=X4120101, SerialNo=170617804405
 Config={ }
 RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=0
 BuffType=unknown, BuffSize=unknown, MaxMultSect=1, MultSect=off
 (maybe): CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=500118192
 IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
 PIO modes:  pio0 pio1 pio2 pio3 pio4 
 DMA modes:  mdma0 mdma1 mdma2 
 UDMA modes: udma0 udma1 udma2 udma3 udma4 udma5 *udma6 
 AdvancedPM=yes: unknown setting WriteCache=enabled
 Drive conforms to: unknown:  ATA/ATAPI-4,5,6,7

 * signifies the current active mode

Сам я в этом смарте ничерта не понимаю:

> sudo smartctl -A /dev/sda 
smartctl 6.6 2016-05-31 r4324 [x86_64-linux-4.15.0-39-generic] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART Attributes Data Structure revision number: 4
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  5 Reallocated_Sector_Ct   0x0032   100   100   ---    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   100   100   ---    Old_age   Always       -       4435
 12 Power_Cycle_Count       0x0032   100   100   ---    Old_age   Always       -       460
170 Unknown_Attribute       0x0032   100   100   ---    Old_age   Always       -       0
171 Unknown_Attribute       0x0032   100   100   ---    Old_age   Always       -       0
172 Unknown_Attribute       0x0032   100   100   ---    Old_age   Always       -       0
173 Unknown_Attribute       0x0032   100   100   ---    Old_age   Always       -       7
174 Unknown_Attribute       0x0032   100   100   ---    Old_age   Always       -       96
178 Used_Rsvd_Blk_Cnt_Chip  0x0032   100   100   ---    Old_age   Always       -       0
180 Unused_Rsvd_Blk_Cnt_Tot 0x0033   100   100   010    Pre-fail  Always       -       100
184 End-to-End_Error        0x0033   100   100   097    Pre-fail  Always       -       0
187 Reported_Uncorrect      0x0032   100   100   ---    Old_age   Always       -       0
194 Temperature_Celsius     0x0022   067   048   ---    Old_age   Always       -       33 (Min/Max 13/48)
199 UDMA_CRC_Error_Count    0x0032   100   100   ---    Old_age   Always       -       0
233 Media_Wearout_Indicator 0x0033   098   100   001    Pre-fail  Always       -       16278028
234 Unknown_Attribute       0x0032   100   100   ---    Old_age   Always       -       4297
241 Total_LBAs_Written      0x0030   253   253   ---    Old_age   Offline      -       3367
242 Total_LBAs_Read         0x0030   253   253   ---    Old_age   Offline      -       5468
249 Unknown_Attribute       0x0032   100   100   ---    Old_age   Always       -       1973

Вот что это за End-to-End_Error? 97 это плохо да? Это может быть следствием проблем со шлейфом или это однозначно сам диск?

После перезагрузки система не грузится и просит fsck. После проверки начинает работать нормально.

UPD:
Проблема проявляется на всех ядрах от 4.15 до 4.19 включительно.

Пока всегда только с диском /dev/sda2 (но он и используется интенсивнее). На данный момент диски смонтированы так:

UUID=27652258-937c-4be5-b12b-83ade6d5ff80	/               	ext4    errors=remount-ro,discard,commit=60		0       1
UUID=f1e0b59c-9f19-427c-acb2-f3f45d2eca55	/home/alex/misc         ext4    errors=remount-ro,noatime,discard	0       1
UUID=5667-4C56					/boot/efi       vfat    umask=0077			0       1
/home/alex/misc/swapfile			none            swap    sw				0       0

tmpfs						/tmp				tmpfs	rw,noatime,nosuid,mode=01777,size=2g						0	0
tmpfs						/var/tmp			tmpfs	rw,size=1g									0	0
tmpfs						/var/cache/apt/archives		tmpfs	rw,noatime,nosuid,size=1g							0	0

commit на /dev/sda2 и перенос swapfile на /dev/sda3 сделал недавно с целью увеличить интенсивность его использования и попробовадь получить ошибку на нем, чтобы убедится, что проблема свойственно железу или непосредственно дику, а не конкретному разделу.

Сервис Lenovo с помощью встроенного тестировния выявил неисправность планки RAM, которую надо сказать к чести Lenovo заменили в течении недели у меня на дому и мне даже не пришлось никуда ехать.

Тест железа встроенный прогнал 4 раза — никаких ошибок ниразу не вылезло. Следующим этапом по совету сервисника обновил BIOS (была и правда очень старая версия). Потом скачал SanDisk Dashboard и проверил диск им (пришлось венду на флэшку ради этого вкорячить), в том числе расширенное тестирование SMART. Прошивка диска последняя.

Проблема сохраняется.

UPD:
С момента переустановки прошел месяц. Полет нормальный. Нужно констатировать следующее — источником проблем стала оперативная память, что привело к повреждению данных записываемых на диск, а это в свою очередь повлекло все остальные последствия. Считаю что сервис Lenovo отработал оперативно — от момента обращение в чат, на сайте производителя, до замены планки памяти прошло 6 дней. Учитывая погодные условия и то что я не в ДС считаю это хорошей реакцией + мне не пришлось никуда ехать — специалист СЦ, приехал для выполнения работ ко мне, в тот же день когда в СЦ поступила деталь, не смотря на то, что к этому времени рабочий день уже завершился.

All times are GMT +1. The time now is 05:39 AM.

vBulletin �2000 — 2023, Jelsoft Enterprises Ltd. Ubuntu Logo, Ubuntu and Canonical � Canonical Ltd. Tango Icons � Tango Desktop Project.

User contributions on this site are licensed under the Creative Commons Attribution Share Alike 4.0 International License. For details and our forum data attribution, retention and privacy policy, see here

The most terrible thing a Linux could have may be some errors reported by the file system. It is usually caused by power outages or bad blocks/sectors within the hard drive. Eventually, as a Linux user, you may at least have suffered from file system errors once, where you have to reinstall the OS and for sure to lose all the existing data. This post is the summary of the recent file system errors happened on my Fedora 20. May give some hints.

0. File system errors seem to be inevitable

There was sth wrong with my Fedora 20 after a recent kernel update to 3.14.4. After megasas kernel module (a kernel module for LSI RAID cards) init’d, the whole system fell into emergency mode (single user mode). However, a manually CTRL-D would force the system going into the normal mode. I hunted for the changes of megasas which has been stable for years. Then I was looking forward to the next kernel release which may fix this weird issue. Interestingly, things were getting more weird.

After a ‘yum clean all‘ (trying to fix the broken dependency but not working anyway), I was not able to run ‘yum update’ anymore, always with the same error from Python: OSError [Errono 5]: Input/Output error. Tried to reboot the system and rebuild the yum cache manually but still got the same error. While I was wondering what happened within yum, there was sth going wrong within my file system.

1. Use tools to figure out what happens

1.1 dmesg

Please do use this powerful tool to check kernel’s complain and I found sth like below:
May 23 08:28:29 daveti kernel:EXT4-fs (dm-0): warning: mounting fs with errors, running e2fsck is recommended
May 23 08:28:29 daveti kernel:EXT4-fs (dm-0): re-mounted. Opts: (null)
May 23 08:28:29 daveti systemd: Started Remount Root and Kernel File Systems.
May 23 08:28:29 daveti systemd: Starting Configure read-only root support…
May 23 08:28:29 daveti systemd: Starting Import network configuration from initramfs…
May 23 08:28:29 daveti systemd: Starting Load/Save Random Seed…
May 23 08:28:29 daveti systemd: Starting Local File Systems (Pre).
May 23 08:28:29 daveti systemd: Reached target Local File Systems (Pre).
May 23 08:28:29 daveti systemd: Started Load/Save Random Seed.
May 23 08:28:29 daveti systemd: Started Configure read-only root support.
May 23 08:28:29 daveti systemd: Started Import network configuration from initramfs.
May 23 08:28:29 daveti systemd: Starting Emergency Shell…
May 23 08:28:29 daveti systemd: Started Emergency Shell.
May 23 08:28:29 daveti systemd: Starting Emergency Mode.
May 23 08:28:29 daveti systemd: Reached target Emergency Mode.
May 23 08:28:29 daveti systemd-fsck: /dev/sda1: clean, 410/128016 files, 244603/512000 blocks
May 23 08:28:29 daveti systemd: Started File System Check on /dev/mapper/fedora_dyn–184–171–61–7-home.
May 23 08:28:29 daveti systemd: Started File System Check on /dev/disk/by-uuid/c96cf320-40a7-42d5-9406-bf80c866faa6.
May 23 08:28:29 daveti systemd: Mounting /boot…
May 23 08:28:29 daveti systemd: Mounting /home…
May 23 08:33:08 daveti kernel:[ 301.924281] EXT4-fs (dm-0): error count: 45
May 23 08:33:08 daveti kernel:[ 301.924299] EXT4-fs (dm-0): initial error at 1400170854: __ext4_get_inode_loc:3919: inode 2238830: block 8913590
May 23 08:33:08 daveti kernel:[ 301.924307] EXT4-fs (dm-0): last error at 1400799080: __ext4_get_inode_loc:3919: inode 2238830: block 8913590
May 23 08:33:08 daveti kernel:EXT4-fs (dm-0): error count: 45
May 23 08:33:08 daveti kernel:EXT4-fs (dm-0): initial error at 1400170854: __ext4_get_inode_loc:3919: inode 2238830: block 8913590
May 23 08:33:08 daveti kernel:EXT4-fs (dm-0): last error at 1400799080: __ext4_get_inode_loc:3919: inode 2238830: block 8913590

What does this tell us? First, it tells my file system (EXT4) screws. Second, it tells the reason why the system falling into emergency mode (because of file system errors). Last, it tells some screwed inode operations.

1.2 smartctl

If your hard drive support SMART, try ‘smartctl -a /dev/yourDevice‘. Mine’s not. Let me know if this tool helps.

1.3 badblocks

This tool is used to see if there is any bad blocks within the drive – ‘badblocks /dev/yourDevice‘. NOTE, first it does NOT fix these bad blocks and second nothing from badblocks does not mean the file system in a good state. Again, dmesg!

2. Rescue mode?

We probably all know the right solution for this case – umount the bad disk and run fsck! The point is how to do that if you are not able to umount the disk even in single user mode when the root file system is screwed. If you have a second machine, you can unplug the issued hard drive and plug it into the other machine and run fsck there. If not, what else shall we do?

It is said that most Linux distribution installation ROMs have the so-called rescue mode where you can try to fix issues on the current system. I tried Fedora 20 live DVD and booted into ‘Try’ mode where a live image is loaded into the memory as the active OS. Unfortunately, I am still not able to umount the root file system for certain reasons.

3. Workaround

Fortunately, I have got a Ubuntu 14.04 LTS live DVD ROM. Booting from this system, I am able to umount all the partitions and run ‘fsck -yvf /dev/yourPartition‘ to fix all the bad blocks. After a reboot, my Fedora 20 runs smoothly into the Gnome login.

4. At last

File system errors may be inevitable for Linux (and eventually for all OSes). Once it happens, try to fix it ASAP. While I was trying to figure out how to umount the root file system, the number of errors (bad blocks) reported by EXT4 was increasing indeed! And please UMOUNT your disk/partition before fsck, otherwise you will hate Linux…