Tar unexpected eof in archive tar error is not recoverable exiting now

unexpected EOF in .tar.gz file from get command #1791

I just tried to get a large file from ipfs using the get command using both the archive and compress options:

However when I try to unarchive it I get the following error:

Also when trying to use python’s tarfile capabilities I also get errors that look like: IOError: CRC check failed 0x3b94367b != 0x1a9b23adL , if that is any help.

The text was updated successfully, but these errors were encountered:

I just tried the same thing without —compress and I’m still getting the same error.

Which version are you using? Should have been fixed in master since 23dd82f

Ah ok, I’m using ipfs version 0.3.8-dev . Not sure if I pulled after that commit, I’ll try updating and get back to you.

Ok it is all fine now, thanks.

@rht @amstocker
I’m using 0.3.11-dev and I’m still getting errors.
For example: QmSXfUAh4HfMNgBcDisZ7tbA4cRMXzuJ3BdrjAz1T5TTjw
(it’s this file, decompressed with xz and added with ipfs tar add )

This is an issue of documentation. Right now, the ipfs tar add command creates objects that can only be easily used by the ipfs tar cat command.

Further integration of ipfs tar created objects into the other commands is definitely planned, Its just taking a bit of thought to figure out how best to do it in a way that doesnt privelege one format unnecessarily. (thats not to say ipfs tar doesnt already privelege the tar format, but thats the sort of thing we want to avoid whenever we can)

Further integration of ipfs tar created objects into the other commands is definitely planned, Its just taking a bit of thought to figure out how best to do it in a way that doesnt privelege one format unnecessarily.

I see two possible ways to go:

Option (1): Transparent archives

In this case ipfs tar would not influence any other part of ipfs project: it would just be a quick way to import/export files without writing the uncompressed archive on disk.

Option (2): Smart Chunking

• ipfs tar add detects the file boundaries in the tar file and applies a sort of pre-chunking (i.e. file headers are standalone chunks, and all the files are chunked indipendently, but keeping the flatness of the tar archive)
• ipfs tar cat does not exist
• ipfs get would be completely unaware of the existence of tar archives and happily download the archive as a normal file (at most, it will notice an odd chunking)

Option (1) would discard by default all the extra content that doesn’t fit in ipfs (owners, permissions, xattrs, etc.), but they could be optionally added as a special node in the dag and smartly restored by tar cat.

I’m not sure if this has been resolved, in part, by any other tickets, so forgive me if my comment isn’t relevant.

Источник

qmrestore error using lzo compression

jonemmons

Guest

I’m running ProxMox 2.1 and I have been backing up a CentOS VM for a few weeks now and I thought everything was going ok until I had an issue and tried to restore from a recent backup. I ran the qmrestore command and immediately got this error message:
new volume ID is ‘local:104/vm-104-disk-1.raw’
restore data to ‘/var/lib/vz/images/104/vm-104-disk-1.raw’ (214748364800 bytes)
lzop: : Compressed data violation
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now
lzop: : Compressed data violation
tar: Unexpected EOF in archive
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now
201849344 bytes copied, 1 s, 192.00 MiB/s
starting cleanup
temporary volume ‘local:104/vm-104-disk-1.raw’ sucessfuly removed
TASK ERROR: command ‘zcat -f|tar xf /mnt/pve/VMBACKUP/dump/vzdump-qemu-102-2012_06_09-12_00_01.tar.lzo ‘—to-command=/usr/lib/qemu-server/qmextract —storage local» failed: exit code 2

I had this error before when I tried to qmrestore a OpenSuse machine but was able to fix it by doing another backup and not selecting any compression. I can’t do that for this vm because the disk are just way to large. I was able to fix the vm manually but I would really like to figure out why the backups are failing to restore.
pveversion -v
pve-manager: 2.1-1 (pve-manager/2.1/f9b0f63a)
running kernel: 2.6.32-11-pve
proxmox-ve-2.6.32: 2.0-66
pve-kernel-2.6.32-11-pve: 2.6.32-66
lvm2: 2.02.95-1pve2
clvm: 2.02.95-1pve2
corosync-pve: 1.4.3-1
openais-pve: 1.1.4-2
libqb: 0.10.1-2
redhat-cluster-pve: 3.1.8-3
resource-agents-pve: 3.9.2-3
fence-agents-pve: 3.1.7-2
pve-cluster: 1.0-26
qemu-server: 2.0-39
pve-firmware: 1.0-15
libpve-common-perl: 1.0-27
libpve-access-control: 1.0-21
libpve-storage-perl: 2.0-18
vncterm: 1.0-2
vzctl: 3.0.30-2pve5
vzprocps: 2.0.11-2
vzquota: 3.0.12-3
pve-qemu-kvm: 1.0-9
ksm-control-daemon: 1.1-1

Thanks in advance.

Marvin

Active Member

I am running into the exact same issue. I believe the problem occurs when there is a disk in the archive that is larger than the root partition. I think that the tar script in the restore procedure uses the root partition as its extraction area. Purely speculative. I have not been able to solve this.

My root partition = local hardware RAID 72Gb
Image lvm partition = local hardware RAID 2.5Tb
Backup partition = iSCSI 250Gb

Some of my VM’s have disks that are 250Gb in size (although only 1/3 used or less.) Those are the ones that fail with «tar: Unexpected EOF in archive». It does not matter if i compress the backup or not, it cannot be restored.

Источник

can’t restore KVM from one host to another — tar: Unexpected EOF in archive

yaboc

Active Member

I have two Proxmox hosts

box1 pve-manager: 2.1-1 (pve-manager/2.1/f9b0f63a)

box2 pve-manager: 2.2-24 (pve-manager/2.2/7f9cfa4c)

and i can’t move the KVM (25GB in size) over to the newer box (box2).

command vzdump 101 —compress lzo —dumpdir /back completes successfully on box 1 and then i can SCP both files
vzdump-qemu-101-2012_11_12-17_03_26.log vzdump-qemu-101-2012_11_12-17_03_26.tar.lzo to box 2

but when i perform qmrestore vzdump-qemu-101-2012_11_12-17_03_26.tar.lzo 101 on box 2

extracting archive ‘/home/vzdump-qemu-101-2012_11_12-17_03_26.tar.lzo’
extracting ‘qemu-server.conf’ from archive
extracting ‘vm-disk-ide0.raw’ from archive
Formatting ‘/var/lib/vz/images/101/vm-101-disk-1.raw’, fmt=raw size=32768
new volume ID is ‘local:101/vm-101-disk-1.raw’
restore data to ‘/var/lib/vz/images/101/vm-101-disk-1.raw’ (26843545600 bytes)
lzop: : Compressed data violation
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now
lzop: : Compressed data violation
tar: Unexpected EOF in archive
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now
868219392 bytes copied, 15 s, 55.13 MiB/s
starting cleanup
temporary volume ‘local:101/vm-101-disk-1.raw’ sucessfuly removed
command ‘zcat -f|tar xf /home/vzdump-qemu-101-2012_11_12-17_03_26.tar.lzo ‘—to-command=/usr/lib/qemu-server/qmextract» failed: exit code 2

I know that there’s one thread that mentions the cause for this could be that the / partition which restore uses as workspace to extract the KVM is lower than the size of KVM, but in this case KVM is 25GB and my df -h on box2 is

Filesystem Size Used Avail Use% Mounted on
/dev/mapper/pve-root 95G 2.5G 88G 3% /
/dev/mapper/pve-data 1.7T 254G 1.5T 15% /var/lib/vz

Any ideas what could be happening over there ? Thanks in advance.

Источник

qmrestore error using lzo compression

jonemmons

Guest

I’m running ProxMox 2.1 and I have been backing up a CentOS VM for a few weeks now and I thought everything was going ok until I had an issue and tried to restore from a recent backup. I ran the qmrestore command and immediately got this error message:
new volume ID is ‘local:104/vm-104-disk-1.raw’
restore data to ‘/var/lib/vz/images/104/vm-104-disk-1.raw’ (214748364800 bytes)
lzop: : Compressed data violation
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now
lzop: : Compressed data violation
tar: Unexpected EOF in archive
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now
201849344 bytes copied, 1 s, 192.00 MiB/s
starting cleanup
temporary volume ‘local:104/vm-104-disk-1.raw’ sucessfuly removed
TASK ERROR: command ‘zcat -f|tar xf /mnt/pve/VMBACKUP/dump/vzdump-qemu-102-2012_06_09-12_00_01.tar.lzo ‘—to-command=/usr/lib/qemu-server/qmextract —storage local» failed: exit code 2

I had this error before when I tried to qmrestore a OpenSuse machine but was able to fix it by doing another backup and not selecting any compression. I can’t do that for this vm because the disk are just way to large. I was able to fix the vm manually but I would really like to figure out why the backups are failing to restore.
pveversion -v
pve-manager: 2.1-1 (pve-manager/2.1/f9b0f63a)
running kernel: 2.6.32-11-pve
proxmox-ve-2.6.32: 2.0-66
pve-kernel-2.6.32-11-pve: 2.6.32-66
lvm2: 2.02.95-1pve2
clvm: 2.02.95-1pve2
corosync-pve: 1.4.3-1
openais-pve: 1.1.4-2
libqb: 0.10.1-2
redhat-cluster-pve: 3.1.8-3
resource-agents-pve: 3.9.2-3
fence-agents-pve: 3.1.7-2
pve-cluster: 1.0-26
qemu-server: 2.0-39
pve-firmware: 1.0-15
libpve-common-perl: 1.0-27
libpve-access-control: 1.0-21
libpve-storage-perl: 2.0-18
vncterm: 1.0-2
vzctl: 3.0.30-2pve5
vzprocps: 2.0.11-2
vzquota: 3.0.12-3
pve-qemu-kvm: 1.0-9
ksm-control-daemon: 1.1-1

Thanks in advance.

Marvin

Active Member

I am running into the exact same issue. I believe the problem occurs when there is a disk in the archive that is larger than the root partition. I think that the tar script in the restore procedure uses the root partition as its extraction area. Purely speculative. I have not been able to solve this.

My root partition = local hardware RAID 72Gb
Image lvm partition = local hardware RAID 2.5Tb
Backup partition = iSCSI 250Gb

Some of my VM’s have disks that are 250Gb in size (although only 1/3 used or less.) Those are the ones that fail with «tar: Unexpected EOF in archive». It does not matter if i compress the backup or not, it cannot be restored.

Источник

gzip — unexpected end of file #144

Sometimes I get the following error:
gzip: stdin: unexpected end of file
/bin/tar: Child returned status 1
/bin/tar: Exiting with failure status due to previous errors
! Failed to unpack Plack-Test-ExternalServer-0.01.tar.gz: no directory
! Failed to fetch distribution Plack-Test-ExternalServer-0.01

I am not quite sure if that’s not problem with my installation/OS/gzip actually. It happens only sometimes and if I run second time cpanm Some::Module the install process completes without errors.

The text was updated successfully, but these errors were encountered:

Closing. Reopen if there’s a way to reproduce reliably. Otherwise I think it’s a problem with the unstable internet downloads.

I’m probably missing something, but the manual shows a usage of «cpanm —mirror

/minicpan —mirror-only».
It seems that wget is used to try and fetch the archive.

When not using «—mirror-only», I’m getting «Fetching nothing/authors/id/P/PH/PHRED/Apache-DBI-1.11.tar.gz . OK», even when the argument to —mirror doesn’t exists.

when no file is being actually fetched, you’ll get the same errors as above (tar and gzip complaining)

@shiponi it’s normal that even when mirror-only option is used, cpanm copies the file to a working directory. I don’t see what problem you’re trying to describe.

I’ve straced it, and I’m getting this:
execve(«/usr/bin/wget», [«/usr/bin/wget», «—retry-connrefused», «dist/authors/id/P/PH/PHRED/Apach», «-q», «-O», «Apache-DBI-1.11.tar.gz»], [/* 31 vars */]) = 0

The command line was: cpanm -l perl5 —mirror dist -n Apache::DBI

@shiponi please describe the problem more clearly — what you were trying to do, what you expected and what happened.

I was hoping I can use «—save-dists» to create a local repository of perl packages, and then use «—mirror» to use that directory instead of fetching everything using the internet.

Well, that still doesn’t describe what is your problem. If you still have an issue to report, make a new one with the reproducible steps and clearer description.

If all you want is to build a local mirror out of saved distributions, there are several tools on CPAN that does this, among all i can recommend OrePAN.

ok. from the beginning 🙂

cpanm -l perl5 —save-dists dist -n Apache::DBI
that works fine. I get the modules installed under perl5 and the tarball under dist (with full CPAN hierarchy)

I’m then removing the perl5 directory, and try this:
cpanm -l perl5 —mirror dist -n Apache::DBI

I’m getting this (like the original poster):

I haven’t tried, but you probably should use —mirror file://$PWD/dist instead. The mirror URL has to begin with file:// or / to be recognized as a file path.

I’ll try that. Thanks

TLDR: I’m unsure if this is a cpanm bug/issue, but it is a noisy error.

I ran into the same issue, let’s try to get to the cause of it.

To state the error again:

I ran strace on «PERL_CARTON_MIRROR=https://partial.mirror.company.org carton install» that had this behaviour:

After the fail, Crypt::CBC gets installed from the next mirror.

As stated earlier, it’s not a major issue, but since it’s harmless I’d suggest hiding it. E.g. skip the unpack if the file is empty.

I hope this helps, thanks for providing us with this cpanm 👍

I have run into this issue as well and have tried to test various scenarios. I have narrowed it down to it happening when using —mirror with cpanm like so:

The issue arise when a requirement in the cpanfile does not exist at the mirror (which only hold company-internal packages), since we mean to fetch the «publically available» dependencies directly from CPAN. What is strange is that the cpanm-log will show loglines as follows:

While the webserver serving the internal PAN logs as follows:

In the filesystem in the .cpanm/work/ -directory a tar.gz-file is created. This varies between bein zero bytes or a couple of bytes depending on if cpanm was run with —no-lwp , —no-wget or without any such arguments.

Note that cpanm thinks the file was fetched without incident ( OK ) while it clearly was not. Expected behaviour would as I see it be that it would detect that the fetch failed, discard the empty-ish file and try to fetch from CPAN since we did not use the —mirror-only -option.

Is this enough to dig further and does this warrant reopening the issue?

Источник

0 Пользователей и 1 Гость просматривают эту тему.

Comments

Ref #785

This will break builds if the corresponding slugbuilder patch is not
merged first.

Ref #785

This will break builds if the corresponding slugbuilder patch is not
merged first.

Ref #785

This will break builds if the corresponding slugbuilder patch is not
merged first.

Ref #785
Ref #1641

This will break builds if the corresponding slugbuilder patch is not
merged first.

Ref #785
Ref deis/slugbuilder#9

This will break builds if the corresponding slugbuilder patch is not
merged first.

Ref #785
Ref deis/slugbuilder#9

This will break builds if the corresponding slugbuilder patch is not
merged first.

Ref #785
Ref deis/slugbuilder#9

This will break builds if the corresponding slugbuilder patch is not
merged first.