Linux File Compression

Comprehensive guide to file compression and archiving in Linux. Learn how to use tar, gzip, zip, bzip2, xz, and other compression tools to efficiently store and transfer files.

Overview

File compression in Linux serves several purposes:

  • Space Saving - Reduce disk usage by compressing files
  • Archiving - Combine multiple files into single archives
  • Transfer - Reduce bandwidth usage for file transfers
  • Backup - Create compressed backups of directories
  • Distribution - Package software and data for distribution

Compression Tools Overview

Tool Purpose File Extension Compression Ratio
tar Archive multiple files .tar None (archiving only)
gzip Fast compression .gz, .tar.gz Good
bzip2 Better compression .bz2, .tar.bz2 Better
xz Best compression .xz, .tar.xz Best
zip Cross-platform archives .zip Good
7z High compression .7z Excellent

TAR Archives

TAR (Tape Archive) is the standard archiving tool in Linux. It combines multiple files into a single archive file.

Create uncompressed archive

tar -cf archive.tar directory/ tar -cf backup.tar file1.txt file2.txt file3.txt

Creates a tar archive containing specified files or directories

List archive contents

tar -tf archive.tar tar -tvf archive.tar # Verbose listing with details

Shows the contents of a tar archive without extracting

Extract archive

tar -xf archive.tar tar -xf archive.tar -C /destination/path/

Extracts all files from archive to current or specified directory

Extract specific files

tar -xf archive.tar file1.txt tar -xf archive.tar directory/subdirectory/

Extracts only specified files or directories from archive

GZIP Compression

GZIP provides fast compression and is widely supported. Often combined with tar for compressed archives.

Create compressed tar archive

tar -czf archive.tar.gz directory/ tar -czf backup.tar.gz *.txt

Creates a gzip-compressed tar archive

Extract compressed archive

tar -xzf archive.tar.gz tar -xzf archive.tar.gz -C /destination/

Extracts gzip-compressed tar archive

Compress single file

gzip file.txt # Creates file.txt.gz gzip -k file.txt # Keep original file gzip -9 file.txt # Maximum compression

Compresses individual files with gzip

Decompress file

gunzip file.txt.gz gzip -d file.txt.gz

Decompresses gzip files

BZIP2 Compression

BZIP2 provides better compression ratios than gzip but is slower.

Create bzip2 compressed archive

tar -cjf archive.tar.bz2 directory/ tar -cjf backup.tar.bz2 *.log

Creates a bzip2-compressed tar archive

Extract bzip2 archive

tar -xjf archive.tar.bz2 tar -xjf archive.tar.bz2 -C /destination/

Extracts bzip2-compressed tar archive

Compress single file

bzip2 file.txt # Creates file.txt.bz2 bzip2 -k file.txt # Keep original file bzip2 -9 file.txt # Maximum compression

Compresses individual files with bzip2

Decompress file

bunzip2 file.txt.bz2 bzip2 -d file.txt.bz2

Decompresses bzip2 files

XZ Compression

XZ provides the best compression ratios but requires more CPU time and memory.

Create xz compressed archive

tar -cJf archive.tar.xz directory/ tar -cJf backup.tar.xz *.data

Creates an xz-compressed tar archive

Extract xz archive

tar -xJf archive.tar.xz tar -xJf archive.tar.xz -C /destination/

Extracts xz-compressed tar archive

Compress single file

xz file.txt # Creates file.txt.xz xz -k file.txt # Keep original file xz -9 file.txt # Maximum compression

Compresses individual files with xz

Decompress file

unxz file.txt.xz xz -d file.txt.xz

Decompresses xz files

ZIP Archives

ZIP is cross-platform compatible and stores files with individual compression.

Create zip archive

zip archive.zip file1.txt file2.txt zip -r archive.zip directory/ zip -9 -r archive.zip directory/ # Maximum compression

Creates zip archives with specified files or directories

List zip contents

unzip -l archive.zip zipinfo archive.zip

Shows contents of zip archive without extracting

Extract zip archive

unzip archive.zip unzip archive.zip -d /destination/ unzip -q archive.zip # Quiet extraction

Extracts zip archive to current or specified directory

Extract specific files

unzip archive.zip file1.txt unzip archive.zip "*.txt"

Extracts only specified files from zip archive

7-Zip Archives

7-Zip provides excellent compression ratios and supports many formats.

Create 7z archive

7z a archive.7z directory/ 7z a -mx9 archive.7z directory/ # Maximum compression

Creates 7z archives with specified compression level

List 7z contents

7z l archive.7z

Lists contents of 7z archive

Extract 7z archive

7z x archive.7z 7z x archive.7z -o/destination/

Extracts 7z archive to current or specified directory

Compression Comparison

Format Speed Compression Ratio CPU Usage Best Use Case
gzip Fast Good Low General purpose, web content
bzip2 Medium Better Medium Better compression needed
xz Slow Best High Maximum compression, archives
zip Fast Good Low Cross-platform compatibility
7z Slow Excellent High Maximum compression, multiple formats

Advanced Usage

Exclude files from archive

tar -czf backup.tar.gz --exclude='*.log' --exclude='tmp/' directory/ zip -r archive.zip directory/ -x "*.tmp" "*.log"

Creates archives while excluding specified files or patterns

Progress indication

tar -czf backup.tar.gz directory/ --checkpoint=1000 --checkpoint-action=dot 7z a archive.7z directory/ -bsp1 # Show progress

Shows progress during archive creation

Test archive integrity

tar -tzf archive.tar.gz > /dev/null && echo "Archive OK" unzip -t archive.zip 7z t archive.7z

Verifies archive integrity without extracting

Split large archives

tar -czf - directory/ | split -b 100M - backup.tar.gz. # Rejoin: cat backup.tar.gz.* | tar -xzf - zip -s 100m archive.zip directory/

Creates split archives for size limitations

Best Practices

Choosing Compression Format
  • Use gzip for general purpose and web content
  • Use bzip2 when you need better compression than gzip
  • Use xz for maximum compression and long-term storage
  • Use zip for cross-platform compatibility
  • Use 7z for maximum compression with multiple format support
Performance Tips
  • Use appropriate compression levels (1=fast, 9=best compression)
  • Consider parallel compression tools (pigz, pbzip2, pxz)
  • Test different formats for your specific data types
    • Use solid compression for similar files
  • Exclude unnecessary files (logs, temp files, caches)
Security Considerations
  • Verify archive integrity before extraction
  • Be cautious with archives from untrusted sources
  • Check for directory traversal attacks (../ paths)
  • Use checksums to verify archive authenticity
  • Consider encryption for sensitive data

Common Use Cases

System backup

# Full system backup (excluding system directories) tar -czf system-backup-$(date +%Y%m%d).tar.gz \ --exclude=/proc --exclude=/sys --exclude=/dev \ --exclude=/tmp --exclude=/var/tmp / # Home directory backup tar -czf home-backup-$(date +%Y%m%d).tar.gz /home/

Creating system and user data backups

Log file archiving

# Compress old log files find /var/log -name "*.log" -mtime +30 -exec gzip {} \; # Archive logs by month tar -czf logs-$(date +%Y%m).tar.gz /var/log/*.log

Archiving and compressing log files to save space

Software distribution

# Create source distribution tar -czf myproject-1.0.tar.gz myproject/ \ --exclude='.git' --exclude='*.o' --exclude='build/' # Create cross-platform distribution zip -r myproject-1.0.zip myproject/ -x "*.git*" "*.o" "build/*"

Packaging software for distribution

Database backup

# MySQL backup with compression mysqldump database_name | gzip > backup-$(date +%Y%m%d).sql.gz # PostgreSQL backup with compression pg_dump database_name | xz > backup-$(date +%Y%m%d).sql.xz

Creating compressed database backups

Troubleshooting

Archive corruption

# Test archive integrity tar -tzf archive.tar.gz unzip -t archive.zip 7z t archive.7z # Attempt partial recovery tar -xzf archive.tar.gz --ignore-failed-read

Checking and recovering from corrupted archives

Insufficient disk space

# Check available space df -h # Create archive on different filesystem tar -czf /tmp/backup.tar.gz /home/user/ # Use streaming to avoid temporary files tar -czf - directory/ | ssh user@remote 'cat > backup.tar.gz'

Handling disk space issues during compression

Permission issues

# Preserve permissions and ownership tar -czpf archive.tar.gz directory/ # Extract with original permissions (as root) sudo tar -xzpf archive.tar.gz # Extract without preserving ownership tar -xzf archive.tar.gz --no-same-owner

Handling file permission and ownership issues

See also