Cloud DevOps Admin Guide

In UNIX files are packed using the Unix Tape ARchive utility (derived from tape archive and commonly referred to as “tarball”), otherwise are compressed and stored using the GNU zip utilities.

The purpose of this post in not do a detailed and exhaustive description about “tar” and “gzip” commands, but present the essential to use it of an easy way. I hope you enjoy the post!

Basically the “tar”(Tape ARchive) command allows pack/unpack, and the gzip(GNU zip) command compress/uncompress files.

tar

The “tar” command allows us group and ungroup( pack and unpack), a set of files and/or folders into a single file.



To pack some folders and files listed above into a single file, we can run the “tar” command with the following parameters. or



Regardless of how the parameters can be referred, below is listed a description of the used parameters:

Option	Meaning	Description
-c	Create	Create a new archive
-v	Verbose	Verbosely list files which are processed
-f	File=ARCHIVE	Use archive file or device

Otherwise the “tar” command can be used to the reverse process: unpack or extract a set of files, directly from the “tar” file:

Option	Meaning	Description
-x	eXtract	Extract files from an archive

Un tar

gzip

This command simply allow to compress a file: any type of file(with the “tar” extension or any other).

To compress a file we can use the following syntax:


The “tar” also allows compress the “tar” result file, by the “z” option as shown below: The “z”option: compress and pack simultaneously, or simply filters the archive through “gzip”.

tar and gzip

 

Oftentimes can be confused to understand the meaning of “tar” and “gzip” commands, because they are used together at the same command line through the pipe operator “|”. Example 1

The first command(before the pipe) is the “tar”, which pack all files in the current directory.

After this, the ”tar” file is compress :”gzip”(after the pipe) and produce the final file(by the redirection operator “>”) : “oracle.tar.gz”.

Example 2

In this example the “oracle.tar.gz” file is unzip and the result is a “tar” file that is placed in the “tar” command to unpack. With the hyphen the data resulting from the “gunzip” command is used as input in the tar command.

 

Useful list

 

Below you can see a list of very useful actions included these commands:

• List the contents of “tar” file

Listing the “tar” archive  contents without any extraction(only lists the contents).

• Extract a single file from “tar” file

This syntax allows to extract a single file:”java/MyLib.java”  from the  ”tar” file.

• Add a file  to an existing  ”tar” file

To add a file to an existing “tar” file, only is necessary to use  the “r” parameter:

• Untar an archive to a different directory

To untar an archive to a different directory, use the following syntax:

tar -zxf oracle.tar.gz -C ora

To save the disk space on my unix server, I have compressed thousands of reports[0-1000].txt files using gunzip program. However, I need to cat .gz file (or open in vim text editor) for reference purposes. I have a log files stores on my server in compressed format using gzip command. How do I display compressed Apache log file without using cat command? How to display the contents of gzip text files in screen without unzipping?

You can easily display compressed files on Linux or Unix without using using the cat command, less, or more command. In this example, show the contents of a text file called resume.txt.gz that has been compressed using gzip and friends. Open the Terminal and then type the following commands.

Syntax

 

On Unix platform, tar command is the primary archiving utility. Understanding various tar command options will help you master the archive file manipulation.
In this article, let us review various tar examples including how to create tar archives (with gzip and bzip compression), extract a single file or directory, view tar archive contents, validate the integrity of tar archives, finding out the difference between tar archive and file system, estimate the size of the tar archives before creating it etc.,

1. Creating an archive using tar command

 

Creating an uncompressed tar archive using option cvf

 

This is the basic command to create a tar archive.

$ tar cvf archive_name.tar dirname/

In the above command:

• c – create a new archive
• v – verbosely list files which are processed.
• f – following is the archive file name

Creating a tar gzipped archive using option cvzf

The above tar cvf option, does not provide any compression. To use a gzip compression on the tar archive, use the z option as shown below.

$ tar cvzf archive_name.tar.gz dirname/

• z – filter the archive through gzip

Note: .tgz is same as .tar.gz
Note: I like to keep the ‘cvf’ (or tvf, or xvf) option unchanged for all archive creation (or view, or extract) and add additional option at the end, which is easier to remember. i.e cvf for archive creation, cvfz for compressed gzip archive creation, cvfj for compressed bzip2 archive creation etc., For this method to work properly, don’t give – in front of the options.

Creating a bzipped tar archive using option cvjf

Create a bzip2 tar archive as shown below:

$ tar cvfj archive_name.tar.bz2 dirname/

• j – filter the archive through bzip2

gzip vs bzip2: bzip2 takes more time to compress and decompress than gzip. bzip2 archival size is less than gzip.
Note: .tbz and .tb2 is same as .tar.bz2

2. Extracting (untar) an archive using tar command

 

Extract a *.tar file using option xvf

 

Extract a tar file using option x as shown below:

$ tar xvf archive_name.tar

• x – extract files from archive

 

Extract a gzipped tar archive ( *.tar.gz ) using option xvzf

 

Use the option z for uncompressing a gzip tar archive.

$ tar xvfz archive_name.tar.gz

 

Extracting a bzipped tar archive ( *.tar.bz2 ) using option xvjf

 

Use the option j for uncompressing a bzip2 tar archive.

 $ tar xvfj archive_name.tar.bz2

Note: In all the above commands v is optional, which lists the file being processed.

3. Listing an archive using tar command

 

View the tar archive file content without extracting using option tvf

You can view the *.tar file content before extracting as shown below.

$ tar tvf archive_name.tar

 

View the *.tar.gz file content without extracting using option tvzf

You can view the *.tar.gz file content before extracting as shown below.

$ tar tvfz archive_name.tar.gz

 

View the *.tar.bz2 file content without extracting using option tvjf

You can view the *.tar.bz2 file content before extracting as shown below.

$ tar tvfj archive_name.tar.bz2

 

 4. Listing out the tar file content with less command

 

When the number of files in an archive is more, you may pipe the output of tar to less. But, you can also use less command directly to view the tar archive output, as explained below.

While opening archive file it shows “ls -l” of the files available in the archive, so you can see the size of file, permissions of it and owner, group too.

$ less autocorrect.tar 
-rwxrwxrwx anthony/anthony 84149 2009-02-02 03:20 autocorrect.dat
-rwxrwxrwx anthony/anthony 443 2009-02-02 03:21 generator.rb
-rwxrwxrwx anthony/anthony 181712 2009-02-02 03:21 autocorrect.vim

For the archived and compressed file also less command shows the output in “ls -l” format.

$ less XML-Parser-2.36.tar.gz
drwxr-xr-x matt/matt 0 2007-11-20 19:58 XML-Parser-2.36/
-rw-r--r-- matt/matt 25252 2007-11-20 19:52 XML-Parser-2.36/Changes
drwxr-xr-x matt/matt 0 2007-11-20 19:58 XML-Parser-2.36/Expat/
-rw-r--r-- matt/matt 3184 2003-07-27 16:37 XML-Parser-2.36/Expat/encoding.h
-rw-r--r-- matt/matt 33917 2007-11-20 19:54 XML-Parser-2.36/Expat/Expat.pm
-rw-r--r-- matt/matt 45555 2007-11-17 01:54 XML-Parser-2.36/Expat/Expat.xs

 

5. Extract a single file from tar, tar.gz, tar.bz2 file

 

To extract a specific file from a tar archive, specify the file name at the end of the tar xvf command as shown below. The following command extracts only a specific file from a large tar file.

$ tar xvf archive_file.tar /path/to/file

Use the relevant option z or j according to the compression method gzip or bzip2 respectively as shown below.

$ tar xvfz archive_file.tar.gz /path/to/file

$ tar xvfj archive_file.tar.bz2 /path/to/file

 

6. Extract a single directory from tar, tar.gz, tar.bz2 file

 

To extract a single directory (along with it’s subdirectory and files) from a tar archive, specify the directory name at the end of the tar xvf command as shown below. The following extracts only a specific directory from a large tar file.

$ tar xvf archive_file.tar /path/to/dir/

To extract multiple directories from a tar archive, specify those individual directory names at the end of the tar xvf command as shown below.

$ tar xvf archive_file.tar /path/to/dir1/ /path/to/dir2/

Use the relevant option z or j according to the compression method gzip or bzip2 respectively as shown below.

$ tar xvfz archive_file.tar.gz /path/to/dir/

$ tar xvfj archive_file.tar.bz2 /path/to/dir/

 

7. Extract group of files from tar, tar.gz, tar.bz2 archives using regular expression

 

You can specify a regex, to extract files matching a specified pattern. For example, following tar command extracts all the files with pl extension.

$ tar xvf archive_file.tar --wildcards '*.pl'

Options explanation:

• –wildcards *.pl – files with pl extension

 

8. Adding a file or directory to an existing archive using option -r

 

You can add additional files to an existing tar archive as shown below. For example, to append a file to *.tar file do the following:

$ tar rvf archive_name.tar newfile

This newfile will be added to the existing archive_name.tar. Adding a directory to the tar is also similar,

$ tar rvf archive_name.tar newdir/

Note: You cannot add file or directory to a compressed archive. If you try to do so, you will get “tar: Cannot update compressed archives” error as shown below.

$ tar rvfz archive_name.tgz newfile
tar: Cannot update compressed archives
Try `tar --help' or `tar --usage' for more information.

 

9. Verify files available in tar using option -W

 

As part of creating a tar file, you can verify the archive file that got created using the option W as shown below.

$ tar cvfW file_name.tar dir/

If you are planning to remove a directory/file from an archive file or from the file system, you might want to verify the archive file before doing it as shown below.

$ tar tvfW file_name.tar
Verify 1/file1
1/file1: Mod time differs
1/file1: Size differs
Verify 1/file2
Verify 1/file3

If an output line starts with Verify, and there is no differs line then the file/directory is Ok. If not, you should investigate the issue.
Note: for a compressed archive file ( *.tar.gz, *.tar.bz2 ) you cannot do the verification.
Finding the difference between an archive and file system can be done even for a compressed archive. It also shows the same output as above excluding the lines with Verify.
Finding the difference between gzip archive file and file system

$ tar dfz file_name.tgz

Finding the difference between bzip2 archive file and file system

$ tar dfj file_name.tar.bz2

 

10. Estimate the tar archive size

 

The following command, estimates the tar file size ( in KB ) before you create the tar file.

$ tar -cf - /directory/to/archive/ | wc -c
20480

The following command, estimates the compressed tar file size ( in KB ) before you create the tar.gz, tar.bz2 files.

$ tar -czf - /directory/to/archive/ | wc -c
508

$ tar -cjf - /directory/to/archive/ | wc -c

428