(I know this is an old question, I came across this problem myself and got my FS back to life without ddrescue, so I’ll share the expericence for anyone else encountering this)
Ext filesystems store backups of the superblock — for an occasion just like this one.
First, determine the locations of the backups (make sure you have the -n option! otherwise this will wipe the file system with a new one):
mke2fs -n /dev/sdxx
This is a test run (i.e. no write) of FS creation routine. It will let you know the offsets of where it would put superblock backups if it were creating a filesystem. If you know your FS block size to be something other than 4096, you must also specify the parameter -b {blocksize} to get the right numbers.
For 4096-sized blocks, first backup superblock will be at 32768. If the following operations fail with the same error message about bad superblock/short read, try the next superblock backup from the list mke2fs gave you.
Next, you can either mount the filesystem using the backup superblock like this
mount -o sb=32768 /dev/sdxx /mnt/sdxx
then explore the FS from your file manager, copy undamaged files etc.
Or, to actually fix the FS, you can run fsck with backup superblock like this
e2fsck -fy -b 32768 /dev/sdxx
Here -f makes it scan the disk even if it’s not dirty and -y answers yes to all enquiries about fixing stuff. The -b option, aside from specifying the backup superblock, will make it update the original superblock with info from backup.
After this, you should have your filesystem back.
If e2fsck fails to write the main superblock
If the superblock got corrupted because it’s on a bad sector e2fsck will finish the run, attempt to update the superblock, and give you the following error message:
Error writing block 1 (Attempt to write block from filesystem resulted in short write)
Obviously, the main superblock isn’t updated and the whole e2fsck run is in vain.
You need to hint the disk to remap that sector — by writing zeroes to it. Thanks to @Keith for pointing this out: the next command can make a lot of mess if mistyped, so triple-check it before running. Here’s the magic:
dd if=/dev/zero of=/dev/sdxx bs=4096 count=1 seek=0
This will write 1 4096-sized block of zeroes to sdxx at offset 0. Don’t forget to account for different block size if that’s the case for you.
After that you’ll be able to write to superblock (which will be on a different physical sector transparently to you). Now, you run the e2fsck command above again and it should succeed writing the superblock allowing you to mount the FS normally.
Goes without saying Now you should backup the critical data to another physical drive and, if you still plan on using the filesystem, run
e2fsck -fccy /dev/sdxx
PS Kudos to @Nemo on this find: in case all of the backups of the superblock of your FS are corrupted, mke2fs/mkfs has the -S option that will re-create the superblock and group descriptors as if creating a new filesystem, without touching anything else. But you absolutely have to be sure your blocksize is right and the man page says you should run e2fsck after it and there are no guarantees about data being left for rescuing. Read the man page and throw a plus on this answer.
Heya everyone, I ran into a problem which has been frustrating me and I would appreciate any help possible to fix it.
I started getting the errors below and I would like to recover because the drive has some some stuff which I really need and don’t have anywhere else..
Figures this would happen while I was waiting for my backup drive to come in..
I started getting errors on my drive at start up.
When I try double clicking on the drive to mount it.. I get the following error message:
Error mounting /dev/sda at /run/media/wolf/Back up: Command-line `mount -t «ext4» -o «uhelper=udisks2,nodev,nosuid» «/dev/sda» «/run/media/wolf/Back up»‘ exited with non-zero exit status 32: mount: wrong fs type, bad option, bad superblock on /dev/sda,
missing codepage or helper program, or other error
When I run dmesg as suggested I get:
dmesg tail
[ 15.257708] ata1.00: exception Emask 0x0 SAct 0x1 SErr 0x0 action 0x0
[ 15.257712] ata1.00: irq_stat 0x40000008
[ 15.257714] ata1.00: failed command: READ FPDMA QUEUED
[ 15.257718] ata1.00: cmd 60/08:00:d8:00:00/00:00:00:00:00/40 tag 0 ncq 4096 in
res 41/40:00:dc:00:00/00:00:00:00:00/40 Emask 0x409 (media error) <F>
[ 15.257719] ata1.00: status: { DRDY ERR }
[ 15.257720] ata1.00: error: { UNC }
[ 15.262611] ata1.00: configured for UDMA/133
[ 15.262620] sd 0:0:0:0: [sda] Unhandled sense code
[ 15.262621] sd 0:0:0:0: [sda]
[ 15.262622] Result: hostbyte=0x00 driverbyte=0x08
[ 15.262623] sd 0:0:0:0: [sda]
[ 15.262624] Sense Key : 0x3 [current] [descriptor]
[ 15.262626] Descriptor sense data with sense descriptors (in hex):
[ 15.262627] 72 03 11 04 00 00 00 0c 00 0a 80 00 00 00 00 00
[ 15.262632] 00 00 00 dc
[ 15.262634] sd 0:0:0:0: [sda]
[ 15.262634] ASC=0x11 ASCQ=0x4
[ 15.262636] sd 0:0:0:0: [sda] CDB:
[ 15.262636] cdb[0]=0x28: 28 00 00 00 00 d8 00 00 08 00
[ 15.262641] end_request: I/O error, dev sda, sector 220
[ 15.262654] ata1: EH complete
[ 17.106788] ata1.00: exception Emask 0x0 SAct 0x1 SErr 0x0 action 0x0
[ 17.106791] ata1.00: irq_stat 0x40000008
[ 17.106793] ata1.00: failed command: READ FPDMA QUEUED
[ 17.106797] ata1.00: cmd 60/08:00:48:03:00/00:00:00:00:00/40 tag 0 ncq 4096 in
res 41/40:00:4f:03:00/00:00:00:00:00/40 Emask 0x409 (media error) <F>
[ 17.106798] ata1.00: status: { DRDY ERR }
[ 17.106799] ata1.00: error: { UNC }
[ 17.111616] ata1.00: configured for UDMA/133
[ 17.111626] sd 0:0:0:0: [sda] Unhandled sense code
[ 17.111627] sd 0:0:0:0: [sda]
[ 17.111628] Result: hostbyte=0x00 driverbyte=0x08
[ 17.111629] sd 0:0:0:0: [sda]
[ 17.111630] Sense Key : 0x3 [current] [descriptor]
[ 17.111632] Descriptor sense data with sense descriptors (in hex):
[ 17.111633] 72 03 11 04 00 00 00 0c 00 0a 80 00 00 00 00 00
[ 17.111637] 00 00 03 4f
[ 17.111639] sd 0:0:0:0: [sda]
[ 17.111640] ASC=0x11 ASCQ=0x4
[ 17.111641] sd 0:0:0:0: [sda] CDB:
[ 17.111642] cdb[0]=0x28: 28 00 00 00 03 48 00 00 08 00
[ 17.111646] end_request: I/O error, dev sda, sector 847
[ 17.111657] ata1: EH complete
[ 18.922588] ata1.00: exception Emask 0x0 SAct 0x1 SErr 0x0 action 0x0
[ 18.922590] ata1.00: irq_stat 0x40000008
[ 18.922592] ata1.00: failed command: READ FPDMA QUEUED
[ 18.922595] ata1.00: cmd 60/08:00:48:03:00/00:00:00:00:00/40 tag 0 ncq 4096 in
res 41/40:00:4f:03:00/00:00:00:00:00/40 Emask 0x409 (media error) <F>
[ 18.922596] ata1.00: status: { DRDY ERR }
[ 18.922597] ata1.00: error: { UNC }
[ 18.927618] ata1.00: configured for UDMA/133
[ 18.927623] sd 0:0:0:0: [sda] Unhandled sense code
[ 18.927624] sd 0:0:0:0: [sda]
[ 18.927625] Result: hostbyte=0x00 driverbyte=0x08
[ 18.927626] sd 0:0:0:0: [sda]
[ 18.927627] Sense Key : 0x3 [current] [descriptor]
[ 18.927629] Descriptor sense data with sense descriptors (in hex):
[ 18.927630] 72 03 11 04 00 00 00 0c 00 0a 80 00 00 00 00 00
[ 18.927634] 00 00 03 4f
[ 18.927636] sd 0:0:0:0: [sda]
[ 18.927637] ASC=0x11 ASCQ=0x4
[ 18.927638] sd 0:0:0:0: [sda] CDB:
[ 18.927639] cdb[0]=0x28: 28 00 00 00 03 48 00 00 08 00
[ 18.927643] end_request: I/O error, dev sda, sector 847
[ 18.927644] quiet_error: 30 callbacks suppressed
[ 18.927645] Buffer I/O error on device sda, logical block 105
[ 18.927654] ata1: EH complete
[ 24.365612] fuse init (API version 7.22)
[ 1871.730523] ata1.00: exception Emask 0x0 SAct 0x3 SErr 0x0 action 0x0
[ 1871.730528] ata1.00: irq_stat 0x40000001
[ 1871.730532] ata1.00: failed command: READ FPDMA QUEUED
[ 1871.730538] ata1.00: cmd 60/00:00:e0:00:00/01:00:00:00:00/40 tag 0 ncq 131072 in
res 41/40:00:00:00:00/00:00:00:00:00/00 Emask 0x9 (media error)
[ 1871.730542] ata1.00: status: { DRDY ERR }
[ 1871.730544] ata1.00: error: { UNC }
[ 1871.730547] ata1.00: failed command: READ FPDMA QUEUED
[ 1871.730552] ata1.00: cmd 60/80:08:60:00:00/00:00:00:00:00/40 tag 1 ncq 65536 in
res 41/40:00:dc:00:00/00:00:00:00:00/40 Emask 0x409 (media error) <F>
[ 1871.730555] ata1.00: status: { DRDY ERR }
[ 1871.730558] ata1.00: error: { UNC }
[ 1871.736056] ata1.00: configured for UDMA/133
[ 1871.736073] sd 0:0:0:0: [sda] Unhandled sense code
[ 1871.736076] sd 0:0:0:0: [sda]
[ 1871.736078] Result: hostbyte=0x00 driverbyte=0x08
[ 1871.736080] sd 0:0:0:0: [sda]
[ 1871.736082] Sense Key : 0x3 [current] [descriptor]
[ 1871.736086] Descriptor sense data with sense descriptors (in hex):
[ 1871.736087] 72 03 11 04 00 00 00 0c 00 0a 80 00 00 00 00 00
[ 1871.736096] 00 00 00 00
[ 1871.736101] sd 0:0:0:0: [sda]
[ 1871.736102] ASC=0x11 ASCQ=0x4
[ 1871.736105] sd 0:0:0:0: [sda] CDB:
[ 1871.736106] cdb[0]=0x28: 28 00 00 00 00 e0 00 01 00 00
[ 1871.736114] end_request: I/O error, dev sda, sector 224
[ 1871.736118] quiet_error: 5 callbacks suppressed
[ 1871.736120] Buffer I/O error on device sda, logical block 28
[ 1871.736128] Buffer I/O error on device sda, logical block 29
[ 1871.736131] Buffer I/O error on device sda, logical block 30
[ 1871.736133] Buffer I/O error on device sda, logical block 31
[ 1871.736135] Buffer I/O error on device sda, logical block 32
[ 1871.736137] Buffer I/O error on device sda, logical block 33
[ 1871.736140] Buffer I/O error on device sda, logical block 34
[ 1871.736142] Buffer I/O error on device sda, logical block 35
[ 1871.736145] Buffer I/O error on device sda, logical block 36
[ 1871.736147] Buffer I/O error on device sda, logical block 37
[ 1871.736180] sd 0:0:0:0: [sda] Unhandled sense code
[ 1871.736183] sd 0:0:0:0: [sda]
[ 1871.736184] Result: hostbyte=0x00 driverbyte=0x08
[ 1871.736186] sd 0:0:0:0: [sda]
[ 1871.736188] Sense Key : 0x3 [current] [descriptor]
[ 1871.736190] Descriptor sense data with sense descriptors (in hex):
[ 1871.736192] 72 03 11 04 00 00 00 0c 00 0a 80 00 00 00 00 00
[ 1871.736201] 00 00 00 dc
[ 1871.736205] sd 0:0:0:0: [sda]
[ 1871.736206] ASC=0x11 ASCQ=0x4
[ 1871.736208] sd 0:0:0:0: [sda] CDB:
[ 1871.736210] cdb[0]=0x28: 28 00 00 00 00 60 00 00 80 00
[ 1871.736217] end_request: I/O error, dev sda, sector 220
[ 1871.736231] ata1: EH complete
[ 1873.546301] ata1.00: exception Emask 0x0 SAct 0x1 SErr 0x0 action 0x0
[ 1873.546307] ata1.00: irq_stat 0x40000008
[ 1873.546310] ata1.00: failed command: READ FPDMA QUEUED
[ 1873.546317] ata1.00: cmd 60/08:00:d8:00:00/00:00:00:00:00/40 tag 0 ncq 4096 in
res 41/40:00:dc:00:00/00:00:00:00:00/40 Emask 0x409 (media error) <F>
[ 1873.546320] ata1.00: status: { DRDY ERR }
[ 1873.546322] ata1.00: error: { UNC }
[ 1873.551138] ata1.00: configured for UDMA/133
[ 1873.551153] sd 0:0:0:0: [sda] Unhandled sense code
[ 1873.551157] sd 0:0:0:0: [sda]
[ 1873.551159] Result: hostbyte=0x00 driverbyte=0x08
[ 1873.551161] sd 0:0:0:0: [sda]
[ 1873.551163] Sense Key : 0x3 [current] [descriptor]
[ 1873.551166] Descriptor sense data with sense descriptors (in hex):
[ 1873.551168] 72 03 11 04 00 00 00 0c 00 0a 80 00 00 00 00 00
[ 1873.551177] 00 00 00 dc
[ 1873.551181] sd 0:0:0:0: [sda]
[ 1873.551182] ASC=0x11 ASCQ=0x4
[ 1873.551185] sd 0:0:0:0: [sda] CDB:
[ 1873.551186] cdb[0]=0x28: 28 00 00 00 00 d8 00 00 08 00
[ 1873.551195] end_request: I/O error, dev sda, sector 220
[ 1873.551220] ata1: EH complete
[ 1875.478731] ata1.00: exception Emask 0x0 SAct 0x8 SErr 0x0 action 0x0
[ 1875.478736] ata1.00: irq_stat 0x40000008
[ 1875.478740] ata1.00: failed command: READ FPDMA QUEUED
[ 1875.478747] ata1.00: cmd 60/80:18:80:00:00/01:00:00:00:00/40 tag 3 ncq 196608 in
res 41/40:00:dc:00:00/00:00:00:00:00/40 Emask 0x409 (media error) <F>
[ 1875.478750] ata1.00: status: { DRDY ERR }
[ 1875.478753] ata1.00: error: { UNC }
[ 1875.483194] ata1.00: configured for UDMA/133
[ 1875.483224] sd 0:0:0:0: [sda] Unhandled sense code
[ 1875.483228] sd 0:0:0:0: [sda]
[ 1875.483230] Result: hostbyte=0x00 driverbyte=0x08
[ 1875.483232] sd 0:0:0:0: [sda]
[ 1875.483234] Sense Key : 0x3 [current] [descriptor]
[ 1875.483237] Descriptor sense data with sense descriptors (in hex):
[ 1875.483239] 72 03 11 04 00 00 00 0c 00 0a 80 00 00 00 00 00
[ 1875.483248] 00 00 00 dc
[ 1875.483252] sd 0:0:0:0: [sda]
[ 1875.483254] ASC=0x11 ASCQ=0x4
[ 1875.483256] sd 0:0:0:0: [sda] CDB:
[ 1875.483258] cdb[0]=0x28: 28 00 00 00 00 80 00 01 80 00
[ 1875.483266] end_request: I/O error, dev sda, sector 220
[ 1875.483304] ata1: EH complete
[ 1877.294507] ata1.00: exception Emask 0x0 SAct 0x1 SErr 0x0 action 0x0
[ 1877.294512] ata1.00: irq_stat 0x40000008
[ 1877.294515] ata1.00: failed command: READ FPDMA QUEUED
[ 1877.294521] ata1.00: cmd 60/08:00:d8:00:00/00:00:00:00:00/40 tag 0 ncq 4096 in
res 41/40:00:dc:00:00/00:00:00:00:00/40 Emask 0x409 (media error) <F>
[ 1877.294524] ata1.00: status: { DRDY ERR }
[ 1877.294527] ata1.00: error: { UNC }
[ 1877.299993] ata1.00: configured for UDMA/133
[ 1877.300004] sd 0:0:0:0: [sda] Unhandled sense code
[ 1877.300006] sd 0:0:0:0: [sda]
[ 1877.300008] Result: hostbyte=0x00 driverbyte=0x08
[ 1877.300011] sd 0:0:0:0: [sda]
[ 1877.300012] Sense Key : 0x3 [current] [descriptor]
[ 1877.300015] Descriptor sense data with sense descriptors (in hex):
[ 1877.300017] 72 03 11 04 00 00 00 0c 00 0a 80 00 00 00 00 00
[ 1877.300026] 00 00 00 dc
[ 1877.300030] sd 0:0:0:0: [sda]
[ 1877.300032] ASC=0x11 ASCQ=0x4
[ 1877.300034] sd 0:0:0:0: [sda] CDB:
[ 1877.300036] cdb[0]=0x28: 28 00 00 00 00 d8 00 00 08 00
[ 1877.300043] end_request: I/O error, dev sda, sector 220
[ 1877.300046] quiet_error: 61 callbacks suppressed
[ 1877.300049] Buffer I/O error on device sda, logical block 27
[ 1877.300066] ata1: EH complete
[ 1879.143628] ata1.00: exception Emask 0x0 SAct 0x1 SErr 0x0 action 0x0
[ 1879.143633] ata1.00: irq_stat 0x40000008
[ 1879.143636] ata1.00: failed command: READ FPDMA QUEUED
[ 1879.143643] ata1.00: cmd 60/08:00:48:03:00/00:00:00:00:00/40 tag 0 ncq 4096 in
res 41/40:00:4f:03:00/00:00:00:00:00/40 Emask 0x409 (media error) <F>
[ 1879.143646] ata1.00: status: { DRDY ERR }
[ 1879.143648] ata1.00: error: { UNC }
[ 1879.148135] ata1.00: configured for UDMA/133
[ 1879.148148] sd 0:0:0:0: [sda] Unhandled sense code
[ 1879.148151] sd 0:0:0:0: [sda]
[ 1879.148153] Result: hostbyte=0x00 driverbyte=0x08
[ 1879.148155] sd 0:0:0:0: [sda]
[ 1879.148157] Sense Key : 0x3 [current] [descriptor]
[ 1879.148160] Descriptor sense data with sense descriptors (in hex):
[ 1879.148162] 72 03 11 04 00 00 00 0c 00 0a 80 00 00 00 00 00
[ 1879.148171] 00 00 03 4f
[ 1879.148175] sd 0:0:0:0: [sda]
[ 1879.148177] ASC=0x11 ASCQ=0x4
[ 1879.148179] sd 0:0:0:0: [sda] CDB:
[ 1879.148181] cdb[0]=0x28: 28 00 00 00 03 48 00 00 08 00
[ 1879.148189] end_request: I/O error, dev sda, sector 847
[ 1879.148192] Buffer I/O error on device sda, logical block 105
[ 1879.148205] ata1: EH complete
[ 1880.967752] ata1.00: exception Emask 0x0 SAct 0x1 SErr 0x0 action 0x0
[ 1880.967757] ata1.00: irq_stat 0x40000008
[ 1880.967760] ata1.00: failed command: READ FPDMA QUEUED
[ 1880.967767] ata1.00: cmd 60/08:00:48:03:00/00:00:00:00:00/40 tag 0 ncq 4096 in
res 41/40:00:4f:03:00/00:00:00:00:00/40 Emask 0x409 (media error) <F>
[ 1880.967770] ata1.00: status: { DRDY ERR }
[ 1880.967772] ata1.00: error: { UNC }
[ 1880.972154] ata1.00: configured for UDMA/133
[ 1880.972167] sd 0:0:0:0: [sda] Unhandled sense code
[ 1880.972170] sd 0:0:0:0: [sda]
[ 1880.972172] Result: hostbyte=0x00 driverbyte=0x08
[ 1880.972174] sd 0:0:0:0: [sda]
[ 1880.972176] Sense Key : 0x3 [current] [descriptor]
[ 1880.972179] Descriptor sense data with sense descriptors (in hex):
[ 1880.972181] 72 03 11 04 00 00 00 0c 00 0a 80 00 00 00 00 00
[ 1880.972190] 00 00 03 4f
[ 1880.972194] sd 0:0:0:0: [sda]
[ 1880.972195] ASC=0x11 ASCQ=0x4
[ 1880.972198] sd 0:0:0:0: [sda] CDB:
[ 1880.972199] cdb[0]=0x28: 28 00 00 00 03 48 00 00 08 00
[ 1880.972207] end_request: I/O error, dev sda, sector 847
[ 1880.972211] Buffer I/O error on device sda, logical block 105
[ 1880.972230] ata1: EH complete
[ 2136.512550] ata1.00: exception Emask 0x0 SAct 0x1 SErr 0x0 action 0x0
[ 2136.512556] ata1.00: irq_stat 0x40000008
[ 2136.512559] ata1.00: failed command: READ FPDMA QUEUED
[ 2136.512566] ata1.00: cmd 60/08:00:d8:00:00/00:00:00:00:00/40 tag 0 ncq 4096 in
res 41/40:00:dc:00:00/00:00:00:00:00/40 Emask 0x409 (media error) <F>
[ 2136.512570] ata1.00: status: { DRDY ERR }
[ 2136.512572] ata1.00: error: { UNC }
[ 2136.517302] ata1.00: configured for UDMA/133
[ 2136.517315] sd 0:0:0:0: [sda] Unhandled sense code
[ 2136.517318] sd 0:0:0:0: [sda]
[ 2136.517320] Result: hostbyte=0x00 driverbyte=0x08
[ 2136.517322] sd 0:0:0:0: [sda]
[ 2136.517324] Sense Key : 0x3 [current] [descriptor]
[ 2136.517327] Descriptor sense data with sense descriptors (in hex):
[ 2136.517329] 72 03 11 04 00 00 00 0c 00 0a 80 00 00 00 00 00
[ 2136.517337] 00 00 00 dc
[ 2136.517342] sd 0:0:0:0: [sda]
[ 2136.517343] ASC=0x11 ASCQ=0x4
[ 2136.517346] sd 0:0:0:0: [sda] CDB:
[ 2136.517347] cdb[0]=0x28: 28 00 00 00 00 d8 00 00 08 00
[ 2136.517355] end_request: I/O error, dev sda, sector 220
[ 2136.517367] ata1: EH complete
[ 2136.517488] EXT4-fs (sda): can't read group descriptor 26
Then I checked with fsck:
fsck.ext4 /dev/sda
fsck.ext4: Attempt to read block from filesystem resulted in short read while trying to open /dev/sda
Could this be a zero-length partition?
I followed that up with :
smartctl -a /dev/sda
=== START OF INFORMATION SECTION ===
Model Family: Western Digital Caviar Black
Device Model: WDC WD1002FAEX-00Z3A0
Serial Number: WD-WMATR1249216
LU WWN Device Id: 5 0014ee 207499f43
Firmware Version: 05.01D05
User Capacity: 1,000,204,886,016 bytes [1.00 TB]
Sector Size: 512 bytes logical/physical
Device is: In smartctl database [for details use: -P show]
ATA Version is: ATA8-ACS (minor revision not indicated)
SATA Version is: SATA 3.0, 6.0 Gb/s (current: 3.0 Gb/s)
Local Time is: Sun Dec 22 22:16:55 2013 EST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
General SMART Values:
Offline data collection status: (0x82) Offline data collection activity
was completed without error.
Auto Offline Data Collection: Enabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: (16680) seconds.
Offline data collection
capabilities: (0x7b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 2) minutes.
Extended self-test routine
recommended polling time: ( 193) minutes.
Conveyance self-test routine
recommended polling time: ( 5) minutes.
SCT capabilities: (0x3037) SCT Status supported.
SCT Feature Control supported.
SCT Data Table supported.
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x002f 195 195 051 Pre-fail Always - 475
3 Spin_Up_Time 0x0027 178 173 021 Pre-fail Always - 4066
4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 538
5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0
7 Seek_Error_Rate 0x002e 100 253 000 Old_age Always - 0
9 Power_On_Hours 0x0032 099 099 000 Old_age Always - 924
10 Spin_Retry_Count 0x0032 100 100 000 Old_age Always - 0
11 Calibration_Retry_Count 0x0032 100 100 000 Old_age Always - 0
12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 496
192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 59
193 Load_Cycle_Count 0x0032 200 200 000 Old_age Always - 477
194 Temperature_Celsius 0x0022 110 106 000 Old_age Always - 37
196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0
197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 14
198 Offline_Uncorrectable 0x0030 200 200 000 Old_age Offline - 10
199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0
200 Multi_Zone_Error_Rate 0x0008 200 200 000 Old_age Offline - 12
SMART Error Log Version: 1
No Errors Logged
SMART Self-test log structure revision number 1
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
# 1 Short offline Completed: read failure 90% 916 220
SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay. Has anyone seen this before?
Any help would be really appreciated.
Cheers,
Last edited by steady_drop (2013-12-26 04:09:29)
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and
privacy statement. We’ll occasionally send you account related emails.
Already on GitHub?
Sign in
to your account
Closed
tpecholt opened this issue
Jul 6, 2016
· 20 comments
Comments
I am playing with http client and SSL. And I am having problem with getting the response from the server after calling beast::http::async_read.
When I examined the code in read.ipp: parse_op::operator() I found the there is nothing to handle «ssl short read errors». I am not SSL expert but from my previous attempts with ASIO I remember ASIO often finished read operations on SSL socket by receiving this kind of error. I think it could be interpreted as error::eof.
When I made following changes to read.ipp I can get correct http response in my handler. Without this change SSL short read error is propagated to my handler but response doesn’t contain parsed response although it was received successfully. cpp-netlib library also contains special code to handle it.
The problem with this patch is that it requires <boost/asio/ssl/error.hpp> which requires <openssl/conf.h>. That means that users of Beast will need to have OpenSSL installed and configured even if they are not using SSL sockets. I’m trying to think of a workaround…
An SSL short read error occurs when the TCP connection closes unexpectedly, that is, without the exchange of SSL shutdown notifications. There is an important distinction between the proper shutdown of the SSL layer and the unexpected shutdown of the TCP connection without proper SSL close notification. Pretending you had a normal close when you actually had an unexpected one can create vulnerabilities (consider an HTTP 0.9 connection where the end of a resource is marked by the normal close of the connection). Unexpected shutdown of the TCP connection without an SSL close notification is an error condition, not an expected condition.
@tpecholt I have researched the issue and I believe that the Beast behavior is correct. A SSL «short read» means that the SSL close_notify message was never received (see http://stackoverflow.com/questions/25587403/boost-asio-ssl-async-shutdown-always-finishes-with-an-error scenario PartyA invokes shutdown() after PartyB closes connection without negotiating shutdown).
Your patch «eats» this error message and prevents the caller from knowing that the connection was not closed gracefully. In fact, to prevent vulnerabilities the application must always check the error code returned from SSL operations.
I believe what is happening here, is that the remote end of your program’s connection is not gracefully closing the connection via SSL. In other words, it is not properly invoking the SSL shutdown handshake mechanism. Or it is closing the socket before the shutdown completes (note, we are talking about the SSL shutdown here and not the TCP/IP shutdown).
When the remote end does not shut down correctly, correct application behavior is to discard the partial HTTP message not to make a best effort to finish it because the contents of the buffers may be undefined. Its not possible to distinguish the SSL short read resulting from improper shutdown, versus a short read resulting from the connection being severed.
I suggest that you investigate the software on the other end of the connection to be certain it is correctly performing the SSL shutdown handshake.
I don’t think this is a good idea, but if there is a consensus that this is a problem, the least awful solution I can think of is this: On any TCP error, if the implementation can prove that the only thing the other side could possibly have done at that point is close the connection or start a new request, consider the connection to have closed normally.
This seems safe to me and doesn’t require decoding the short read error. But it seems like a partial workaround for a broken peer, and since it’s only partial, it doesn’t avoid the need to fix the peer anyway. So there doesn’t seem to be much point.
Sorry, I feel like I keep asking dumb questions:
system info:
using beast version 115, boost 1.65.1
code used:
using the example code here
i changed the lines
// The SSL context is required, and holds certificates ssl::context ctx{ssl::context::sslv23_client}; // This holds the root certificate used for verification load_root_certificates(ctx);
to:
ssl::context ctx{ssl::context::tlsv11_client};
so i’m just not validating the cert for tests, and using tls 1.1.
problem:
i get a short read from google.com:443/ (oddly not from yahoo.com:443/). I’m also getting a short read from our kestrel server running on dotnet core 2.0, which sends a connection-close in the header.
the short read occurs during shutdown, obviously.
should i try calling shutdown again?
should i just wait some amount of time for shutdown to complete or give up if that time expires?
i can try looking into what google/kestrel are doing and contact them?
Google is famous for abruptly closing TLS connections without going through the handshake, it «optimizes» their products at «large scales.» As long as you know your code is right, I would not worry about it. Chrome does this to my HTTP servers as well, and I know my code is right, because when I use my HTTP client against my HTTP server the shutdown handshake is graceful.
I know my code is right, because when I use my HTTP client against my HTTP server the shutdown handshake is graceful.
that’s why i pointed out that your code always worked with yahoo, and never with google, figured there was some relevance.
good to know about the optimizations being a reason for that. i’d be interested in reading their engineering blog discussing the reasoning.
i’m still trying to decide if i need to do anything with a short read error, because right now it’s handled nowhere. i guess i’ll have to read the tls/ssl spec and see if i can generate a list of when it can happen and for what reasons so i can determine where to handle it or where to ignore it.
thanks for always being so quick with replies!
if the short read happens at the end of a session I guess you could just ignore it
What is the suggested workaround if the «short read» does not happen during shutdown, but rather while reading the response of a GET request?
I ran into this with the http_client_async_ssl.cpp example from Boost 1.67, adding only a connection «close» header field in the request and running it with «www.google.com 443 /» as command line arguments.
The current Beast implementation (Boost 1.67) seems to always return an empty response body and header in this case (along with the short read error_code), even if the response data was received (as indicated by bytes_transferred and validated by debugging into read.ipp).
Shouldn’t the read operation rather return all data it has received and let the application decide how to react to the error_code and what to do with the data (keep or discard)?
Or should it be treated like the EOF case in read.ipp (see the «caller sees EOF on next read» comment in the file)?
With the current behaviour, I don’t really see how to implement a generic GET request that also works with servers that «optimize» SSL shutdown (incl. popular ones from Google)?
Special-casing such known URLs or retrying different request combinations (like removing the connection close header in the failure case) is not a feasible option.
Thanks!
That does sound like a corner case which could use investigation. I will look into it, thanks!
@vinniefalco do you have plans to fix this for the next boost release?
Do you have any suggestion to work around this issue for the time being?
It may be a corner case that SSL shortcuts lead to an error in this case; but I would argue that this exposes a more general issue that beast may discard data it has already received / parsed when it encounters an error. I think it would be more sensible and flexible to pass the data to the caller along with the error code and let the caller decide what to do.
This issue has been open for a while with no activity, has it been resolved?
Could you also possibly mention where the «short read» error is located in code to help people handle it while it’s out there as an issue? I’m noticing this while hitting a server behind CloudFlare:
uncaught exception of type boost::system::system_error: short read
Edit: For others who might need it, the error details are:
asio.ssl:335544539 in the category:id format.
Come to think of it, I believe that this might be fixable in Beast. If we get a «short read» and the parser has not yet received any bytes, and there are no bytes in the dynamic buffer, then we can safely convert this to an «end of file» signal. Supporting evidence:
«…if your communication protocol is self-terminated, then you may simply ignore the lack of close_notify»
https://security.stackexchange.com/questions/91435/how-to-handle-a-malicious-ssl-tls-shutdown
I need a way to detect the «short read» error without including OpenSSL, any ideas?
This is what I’m thinking:
bool
is_ssl_short_read(error_code const& ec)
{
if(ec.value() != 335544539)
return false;
if(string_view{ec.category().name()} != "asio.ssl")
return false;
BOOST_ASSERT(ec.message() == "short read");
return true;
}
Actually scratch that, I have a better way. Only took 3 years to figure this out, patch incoming.
vinniefalco
added a commit
to vinniefalco/beast
that referenced
this issue
Feb 23, 2019
fix boostorg#38
This improves the behavior when encountering a short read:
* Any stream error encountered during a read is converting into
`http::error::partial_message` if some data was received but
the message is incomplete.
* Examples squelch SSL short read errors from the logs.
Hi all,
I have an issue trying to make and HTTPS POST request from my LUA scnene in Fibaro HC.
I am getting «short read» error with the URL «https://api2.magicair.tion.ru/idsrv/oauth2/token»
This is my scene:
=============================================
—[[
%% properties
%% events
%% globals
—]]
local authData = «username=XXX&password=YYY»
function doTest()
print(‘Test!’)
local requestUrl = «https://api2.magicair.tion.ru/idsrv/oauth2/token»
local httpClient = net.HTTPClient()
httpClient:request(requestUrl, {
options = {
method = «POST»,
headers = {
[«Content-Type»] = «application/x-www-form-urlencoded»,
[«Content-Length»] = tostring(authData:len())
},
data = authData,
timeout = 5000
},
success = function (response) fibaro:debug (response.data) end,
error = function (err) fibaro:debug («Error: » .. err) end
})
setTimeout(doTest, 5*1000)
end
print(‘Script start…’)
doTest()
=============================================
I am getting the following in my log:
[DEBUG] 15:16:52: Script start…
[DEBUG] 15:16:52: Test!
[DEBUG] 15:16:52: Error: short read
I have been trying to play with headers, data format, timeouts and all parameters I could find, however I always get the same error with the provided URL.
Other HTTPS URLS I tried with GET method works just fine.
Also I have been trying to do the same with cURL and it works fine with the exact same set of headers and payload (with full credentials string, of course).
So the issue is somehow connected with this specific URL: «https://api2.magicair.tion.ru/idsrv/oauth2/token»
The Internet is saying that short read problem in HTTPClient request might be related to ciphering protocols mismatch between the client and server.
So there could be some parameter regulating this?
2 main versions, which I see:
1. Some tricky param is required to form the correct HTTPS request.
2. Some bug / missing feature in HTTPClient implementation in HC2.
Anyway I am stuck and need the help from the community and HC developers.
Has anyone seen something similar? Any clues?
Please help!
Many thanks in advance.
PS Using the latest FW 4.51 of Fibaro Come Center 2