Tarfile extract error could not change modification time

> pip install bokeh==0.12.12
Collecting bokeh==0.12.12
  Using cached bokeh-0.12.12.tar.gz
Exception:
Traceback (most recent call last):
  File "k:usersedappdatalocalprogramspythonpython36-32libtarfile.py", line 2268, in utime
    os.utime(targetpath, (tarinfo.mtime, tarinfo.mtime))
OSError: [WinError 87] The parameter is incorrect

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "k:usersedappdatalocalprogramspythonpython36-32libsite-packagespipbasecommand.py", line 215, in main
    status = self.run(options, args)
  File "k:usersedappdatalocalprogramspythonpython36-32libsite-packagespipcommandsinstall.py", line 324, in run
    requirement_set.prepare_files(finder)
  File "k:usersedappdatalocalprogramspythonpython36-32libsite-packagespipreqreq_set.py", line 380, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "k:usersedappdatalocalprogramspythonpython36-32libsite-packagespipreqreq_set.py", line 620, in _prepare_file
    session=self.session, hashes=hashes)
  File "k:usersedappdatalocalprogramspythonpython36-32libsite-packagespipdownload.py", line 821, in unpack_url
    hashes=hashes
  File "k:usersedappdatalocalprogramspythonpython36-32libsite-packagespipdownload.py", line 663, in unpack_http_url
    unpack_file(from_path, location, content_type, link)
  File "k:usersedappdatalocalprogramspythonpython36-32libsite-packagespiputils__init__.py", line 605, in unpack_file
    untar_file(filename, location)
  File "k:usersedappdatalocalprogramspythonpython36-32libsite-packagespiputils__init__.py", line 581, in untar_file
    tar.utime(member, path)
  File "k:usersedappdatalocalprogramspythonpython36-32libtarfile.py", line 2270, in utime
    raise ExtractError("could not change modification time")
tarfile.ExtractError: could not change modification time

The GNU tape archive command, known as ‘tar’, is used to store many different files together into a single archive file. This makes it easy to perform backups and restores of files and directories in Unix/Linux based operating systems.

The practical examples in this guide will show you how to use the tar command in all sorts of different situations.

How To Use tar – Command Examples

• 1. Create Tar File

  In its most simplest form, the tar command can be used to copy multiple files into a .tar file. The -c option is used to create a new archive file, while the -f option is used to specify the archive file to use (in this case, create).

  [[email protected] ~]# tar -cf archive.tar test1.txt test2.txt
  [[email protected] ~]# ls -la
  -rw-r--r--.  1 root root      10240 Sep  2 12:26 archive.tar
  -rw-r--r--.  1 root root         52 Sep  2 12:24 test1.txt
  -rw-r--r--.  1 root root         18 Sep  2 09:42 test2.txt
  
  

  The original files still exist after being added to the archive, they are not removed by default.

  Note that we do not need the .tar extension, however we use it here to allow us to easily see what the file is.

• 2. List Contents Of Archive

  We can list the contents within a tar file with the -t option. Note that we also need the -f option to specify the tar file to run against.

  [[email protected] ~]# tar -tf archive.tar
  test1.txt
  test2.txt
  

• 3. Extract From Tar File

  We can extract all contents of a tar file with the -x option. In this example, we show the contents of the original test2.txt file, wipe it clean, then extract the contents of archive.tar confirming that the contents of test2.txt have been returned from the copy in the archive.

  [[email protected] ~]# cat test2.txt
  This is test2.txt
  [[email protected] ~]# > test2.txt
  [[email protected] ~]# cat test2.txt
  [[email protected] ~]# tar -xf archive.tar
  [[email protected] ~]# cat test2.txt
  This is test2.txt
  

  This will not remove the .tar file, it will remain in place after extracting files from it.

• 4. Extract Tar File To Specific Directory

  By default tar will extract to the current working directory which may not always be what we want. Rather than first cd to the destination directory, we can use the -C option which will change to the directory specified when performing the extraction for us.

  [[email protected] ~]# mkdir new
  [[email protected] ~]# tar -xf archive.tar -C new/
  [[email protected] ~]# ls -l new/
  total 8
  -rw-r--r--. 1 root root 52 Sep  2 12:24 test1.txt
  -rw-r--r--. 1 root root 18 Sep  2 09:42 test2.txt
  

• 5. Extract Specific Files From Tar File

  Rather than extracting all contents of the tar file which is the default, we can specify a particular file to extract. In this example we first remove the files we extracted from the last example, then specifically extract just the test2.txt file only.

  [[email protected] ~]# rm -f new/*
  [[email protected] ~]# tar -xf archive.tar -C new/ test2.txt
  [[email protected] ~]# ls -l new/
  total 4
  -rw-r--r--. 1 root root 18 Sep  2 09:42 test2.txt
  

  We can also extract multiple files at once this way by adding more onto the end of the command.

• 6. Display Verbose Information

  The -v option for verbose can be added to the tar command which will display useful additional information which may be useful when troubleshooting problems. Verbose output will print the files and directories that are being added to an archive or extracted from an archive.

  [[email protected] ~]# tar -cvf archive.tar test1.txt test2.txt
  test1.txt
  test2.txt
  

  This way we can visually see what has been added into the tar file without the need to list the contents with -t after the fact.

• 7. Delete From Archive

  We can also remove files from within the tar file with the —delete option. In the below example we remove the test2.txt file then list the contents of archive.tar which now only has test1.txt left inside.

  [[email protected] ~]# tar -f archive.tar --delete test2.txt
  [[email protected] ~]# tar -tf archive.tar
  test1.txt
  

• 8. Add More Files To Tar File

  We can add additional files into an existing tar file with the -r option which will append files to the archive. In this example, we add our previously extracted test2.txt file back into the archive.

  [[email protected] ~]# tar -f archive.tar -r new/test2.txt
  [[email protected] ~]# tar -tf archive.tar
  test1.txt
  new/test2.txt
  

  Note that this has also copied the directory path, as shown in the output when we list the contents of the tar file. Now even if we completely delete the new directory, when we extract the contents of archive.tar the directory will be recreated from the archive.

  [[email protected] ~]# rm -rf new/
  [[email protected] ~]# tar -xf archive.tar
  [[email protected] ~]# ls -l new/
  total 4
  -rw-r--r--. 1 root root 18 Sep  2 09:42 test2.txt
  

• 9. Update Files Inside Archive

  We can update the existing files inside a tar file with newer files on the file system with the -u option, it will only append files that are newer than the copy within the archive file.

  In this example we append “new text” to new/test2.txt and then update the archive.tar file with this new file. We then delete the new/test2.txt file, extract it and restore it, then cat the contents to show that the copy from the archive was indeed updated.

  [[email protected] ~]# echo "new text" >> new/test2.txt
  [[email protected] ~]# tar -uf archive.tar new/test2.txt
  [[email protected] ~]# rm -f new/test2.txt
  [[email protected] ~]# tar -xf archive.tar
  [[email protected] ~]# cat new/test2.txt
  This is test2.txt
  new text
  

• 10. Enable Archive Compression

  While we could always compress a tar file after it has been created, the tar command provides options that allow us to compress the archive during creation.

  We can use the -j option to compress with bzip2. This will set the archive extension to .tar.bz2, when I first ran the command I got the below error:

  tar (child): bzip2: Cannot exec: No such file or directory
  tar (child): Error is not recoverable: exiting now
  tar: Child returned status 2
  tar: Error is not recoverable: exiting now
  

  This was because the bzip2 package was not installed on my system, once installed with ‘yum install bzip2’ it worked properly.

  [[email protected] ~]# tar -cjf archive.tar.bz2 test1.txt test2.txt
  

  We can use the -J option to compress with xz. This will set the archive extension to .tar.xz

  [[email protected] ~]# tar -cJf archive.tar.xz test1.txt test2.txt
  

  We can use the -z option to compress with gzip, which seems to be the most common compression method used with tar. This will set the archive extension to .tar.gz

  [[email protected] ~]# tar -czf archive.tar.gz test1.txt test2.txt
  

  We can see that the compressed versions of archive.tar are a lot smaller than the original tar file, despite containing the same contents.

  [[email protected] ~]# ls -la archive.tar*
  -rw-r--r--. 1 root root 10240 Sep  2 12:45 archive.tar
  -rw-r--r--. 1 root root   201 Sep  2 12:49 archive.tar.bz2
  -rw-r--r--. 1 root root   190 Sep  2 12:47 archive.tar.gz
  -rw-r--r--. 1 root root   244 Sep  2 12:47 archive.tar.xz
  

  We have previously investigated the performance differences of these compression methods if you’re interested in seeing which is best for you, compressing these small text files is not a good example to determine which is the best compression method, in general xz will compress better than bzip2, which will compress better than gzip.

  We can extract from compressed archive files without specifying the option used to compress them, tar is smart enough to work this out on it’s own, so the below command is perfectly valid and will extract the content of this compressed archive file.

  [[email protected] ~]# tar -xf archive.tar.gz
  

  It’s important to note that we are not able to add additional files to compressed tar files, they would first need to be decompressed, updated, then manually compressed again with the gzip, bzip2, or xz commands.

  [[email protected] ~]# tar -uf archive.tar.gz new/test2.txt
  tar: Cannot update compressed archives
  tar: Error is not recoverable: exiting now
  

• 11. Create Archive Of Directory

  We don’t only have to add in individual files to a tar file, we can archive an entire directory recursively to archive all contents. In this example we copy all contents of the /etc/ directory to etc.tar.gz, afterwards we list the contents showing that this was successful.

  [[email protected] ~]# tar -czf etc.tar.gz /etc/
  tar: Removing leading `/' from member names
  [[email protected] ~]# ls -l etc.tar.gz
  -rw-r--r--. 1 root root 7092784 Sep  2 12:56 etc.tar.gz
  [[email protected] ~]# tar -tf etc.tar.gz | head
  etc/
  etc/crypttab
  etc/mtab
  etc/resolv.conf
  etc/pki/
  etc/pki/rpm-gpg/
  etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
  etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-Debug-7
  etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-Testing-7
  etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7
  

  By default tar runs with the —recursion option, which will recurse into sub directories and archive the content within as well.

• 12. Disable Directory Recursion

  As shown in the previous example when we archive a directory, sub directories and contents within are also archived by default. We can optionally disable directory recursion with the —no-recursion option, which will avoid descending into other directories automatically.

  This example is the same as the previous, however we can see in the output that there are sub directories listed however no contents within them have been archived as we have only compressed files within the directory specified without recursion.

  [[email protected] ~]# tar -czf etc-no-dirs.tar.gz /etc/* --no-recursion
  tar: Removing leading `/' from member names
  [[email protected] ~]# tar -tf etc-no-dirs.tar.gz | head
  etc/adjtime
  etc/aliases
  etc/aliases.db
  etc/alternatives/
  etc/anacrontab
  etc/asound.conf
  etc/audisp/
  etc/audit/
  etc/avahi/
  etc/bash_completion.d/
  

• 13. Differences Between Archive And File System

  We can use the -d or —diff option to advise us if there is any difference between a file in an archive compared to a file on the file system. In this example we edit the test2.txt file and then perform the comparison, as we can see it reports that the modification time and file size differs. When we run the same against test1.txt which has not been changed, we get no output as there is no difference.

  [[email protected] ~]# echo "change" >> test2.txt
  [[email protected] ~]# cat test2.txt
  This is test2.txt
  change
  [[email protected] ~]# tar -df archive.tar test2.txt
  test2.txt: Mod time differs
  test2.txt: Size differs
  [[email protected] ~]# tar -df archive.tar test1.txt
  [[email protected] ~]#
  

• 14. Keep Newer Files

  When extracting we may not want to replace files on the file system that are newer than the ones stored in the tar file archive (the default action). We can specify —keep-newer-files to ensure that these newer files are not overwritten with older copies from the archive.

  [[email protected] ~]# tar -xf archive.tar test2.txt --keep-newer-files
  tar: Current `test2.txt' is newer or same age
  

  The test2.txt file has not been modified as the file on our file system has been changed after it was added to the archive.

• 15. Remove Files After Archiving

  By default when we add files to an archive file the original files will remain on the file system. We can optionally remove the original files after adding them to the tar file with the —remove-files option. While this will delete the original files from the file system, they will exist within the tar file.

  [[email protected] ~]# tar -cvf archive.tar test1.txt test2.txt --remove-files
  

• 16. Exclude Files By Pattern

  We can exclude adding items into an archive with the —exclude option followed by a specified pattern. In the below example we can see we have two .txt files and a .mp3 file, we do not want to include the mp3 file in our archive so we exclude it.

  [[email protected] new]# ls -l
  total 3336
  -rw-r--r--. 1 root root      52 Sep  2 12:24 test1.txt
  -rw-r--r--. 1 root root      25 Sep  2 14:02 test2.txt
  -rw-r--r--. 1 root root 3407872 Sep  2 14:35 z.mp3
  [[email protected] new]# tar -cvf exclude.tar --exclude=*.mp3 *
  test1.txt
  test2.txt
  

• 17. Confirm Before Adding Files

  By default all specified files are added to an archive automatically without any further confirmation. We can optionally specify the -w option which will prompt us for a yes or no input confirmation for every file. In the below example, we enter no for test1.txt, and then confirm that it was indeed excluded from the tar file.

  [[email protected] new]# tar -cwf confirm.tar *
  add `test1.txt'?n
  add `test2.txt'?y
  add `z.mp3'?y
  
  [[email protected] new]# tar -tf confirm.tar
  test2.txt
  z.mp3
  

  This also works when extracting files from the archive.

  [[email protected] new]# tar -xwf confirm.tar
  extract `test2.txt'?n
  extract `z.mp3'?y
  

• 18. POSIX ACL Support

  POSIX ACL support for files within the archive can be enabled with the —acls option or disabled with the —no-acls option, —no-acls is used by default so if we want ACL support it must be specified.

  In this example we use the setfacl command to set a basic ACL for user1 on test1.txt, we then create a tar file containing this file with —acls. After removing the original test1.txt file, we extract it and run getfacl and confirm that the ACL we added is still there.

  [[email protected] new]# setfacl -m u:user1:rw test1.txt
  [[email protected] new]# getfacl test1.txt
  # file: test1.txt
  # owner: root
  # group: root
  user::rw-
  user:user1:rw-
  group::r--
  mask::rw-
  other::r--
  
  [[email protected] new]# tar -cf acls.tar test1.txt --acls
  [[email protected] new]# rm -f test1.txt
  [[email protected] new]# tar -xf acls.tar
  [[email protected] new]# getfacl test1.txt
  # file: test1.txt
  # owner: root
  # group: root
  user::rw-
  user:user1:rw-
  group::r--
  mask::rw-
  other::r--
  

  As we can see when using —acls, the file is extracted and the custom ACL for user1 still exists. We will now delete the file and export test1.txt from the tar file again without —acls and we will see that the ACL is no longer present on the file.

  [[email protected] new]# rm -f test1.txt
  [[email protected] new]# tar -xf acls.tar
  [[email protected] new]# getfacl test1.txt
  # file: test1.txt
  # owner: root
  # group: root
  user::rw-
  group::rw-
  other::r--
  

• 19. Extended Attribute Support

  Extended Attribute support for files within the archive can be enabled with the —xattrs option or disabled with the —no-xattrs option.

  [[email protected] new]# tar -cf xattrs.tar test1.txt --xattrs
  

• 20. SELinux Support

  SELinux support for files within the archive can be enabled with the —selinux option or disabled with the —no-selinux option.

  [[email protected] new]# tar -cf no-selinux.tar * --no-selinux
  

• 21. Set New User/Group Owner Of Archived Files

  We can set the user and/or group owner of a file when adding it into the archive with —owner=name to force new owner and —group=name to force a new group for the added files.

  [[email protected] new]# tar -cf user-group-change.tar test2.txt --owner=user1 --group=user1
  

  Now once we extract this file the user and group should be set to user1, rather than root.

  [[email protected] new]# tar -xf user-group-change.tar
  [[email protected] new]# ls -l
  total 3364
  -rw-rw-r--. 1 root  root       52 Sep  2 12:24 test1.txt
  -rw-r--r--. 1 user1 user1      25 Sep  2 14:02 test2.txt
  

• 22. Extract Files To Standard Output

  Rather than extracting files from within the tar file to the file system, we can simply send them to stdout instead with the -O option which may be useful if we just want to quickly read the contents of a file without having to extract it first.

  [[email protected] new]# tar -xOf archive.tar test1.txt
  This is test1.txt
  [[email protected] new]# tar -xOf archive.tar test2.txt
  This is test2.txt
  

• 23. Check Archive File

  We can attempt to verify an archive file during creation with the -W option.

  [[email protected] new]# tar -cvWf archive.tar test1.txt test2.txt
  test1.txt
  test2.txt
  Verify test1.txt
  Verify test2.txt
  

Summary

As we have seen the tar command can be used to easily archive various files into a single archive file. When combined with the many options available we can perform powerful tasks allowing us to backup and restore files and directories.