Linux I/O Redirection
Control input, output, and error streams using redirection operators to manage data flow in Linux commands.
Understanding I/O Streams
Every Linux process has three standard data streams that can be redirected:
| Stream | File Descriptor | Description | Default |
|---|---|---|---|
stdin |
0 | Standard input | Keyboard |
stdout |
1 | Standard output | Terminal screen |
stderr |
2 | Standard error | Terminal screen |
Redirection Operators
| Operator | Description | Example |
|---|---|---|
> |
Redirect stdout to file (overwrite) | ls > files.txt |
>> |
Redirect stdout to file (append) | echo "text" >> log.txt |
< |
Redirect stdin from file | sort < data.txt |
2> |
Redirect stderr to file | command 2> errors.txt |
2>> |
Redirect stderr to file (append) | command 2>> errors.txt |
&> |
Redirect both stdout and stderr | command &> output.txt |
2>&1 |
Redirect stderr to stdout | command 2>&1 |
<< |
Here document | cat << EOF |
Basic Examples
Output redirection
# Redirect output to file (overwrite)
ls > filelist.txt
# Append output to file
date >> log.txt
# Redirect command output
ps aux > processes.txt
# Save command results
df -h > disk_usage.txt
Redirect standard output to files
Input redirection
# Read input from file
sort < unsorted.txt
# Process file contents
wc -l < data.txt
# Use file as input
grep "pattern" < input.txt
# Combine input and output redirection
sort < input.txt > sorted.txt
Redirect standard input from files
Error redirection
# Redirect errors to file
command 2> errors.txt
# Append errors to file
command 2>> error_log.txt
# Discard errors
command 2> /dev/null
# Separate output and errors
command > output.txt 2> errors.txt
Redirect standard error stream
Advanced Redirection
Combining streams
# Redirect both stdout and stderr to same file
command > output.txt 2>&1
command &> output.txt # Bash shorthand
# Append both streams to file
command >> output.txt 2>&1
command &>> output.txt # Bash shorthand
# Redirect stderr to stdout
command 2>&1 | grep "error"
# Split output and errors
command > success.txt 2> errors.txt
Combine and split different output streams
Here documents and here strings
# Here document
cat << EOF
This is a multi-line
here document that will
be processed by cat
EOF
# Here document with variable expansion
cat << EOF
Current user: $USER
Current date: $(date)
EOF
# Here string (Bash)
grep "pattern" <<< "search this string"
# Here document to file
cat << EOF > config.txt
setting1=value1
setting2=value2
EOF
Use here documents and here strings for inline input
File descriptor manipulation
# Redirect specific file descriptors
command 3> file3.txt 4> file4.txt
# Duplicate file descriptors
exec 3>&1 # Save stdout to fd 3
exec 1> output.txt # Redirect stdout to file
echo "This goes to file"
exec 1>&3 # Restore stdout
echo "This goes to terminal"
# Close file descriptors
exec 3>&- # Close fd 3
# Swap stdout and stderr
command 3>&1 1>&2 2>&3 3>&-
Advanced file descriptor manipulation
Practical Examples
Logging and monitoring
# Create log files with timestamps
echo "$(date): Starting backup" >> backup.log
# Log both success and errors
backup_script.sh >> backup.log 2>&1
# Separate success and error logs
rsync -av /source/ /dest/ > sync_success.log 2> sync_errors.log
# Silent operation with error logging
cron_job.sh > /dev/null 2>> /var/log/cron_errors.log
# Conditional logging
if command > /dev/null 2>&1; then
echo "Success" >> success.log
else
echo "Failed" >> error.log
fi
Implement logging and monitoring with redirection
Data processing
# Process data files
sort < raw_data.txt > sorted_data.txt
# Chain processing steps
cat data.txt | grep "pattern" > filtered.txt
sort < filtered.txt > final.txt
# Create reports
{
echo "System Report - $(date)"
echo "========================"
df -h
echo ""
free -h
} > system_report.txt
# Batch file processing
for file in *.log; do
grep "ERROR" "$file" >> all_errors.txt
done
Process and analyze data using redirection
Configuration and automation
# Generate configuration files
cat << EOF > nginx.conf
server {
listen 80;
server_name example.com;
root /var/www/html;
}
EOF
# Create scripts with here documents
cat << 'SCRIPT' > backup.sh
#!/bin/bash
tar -czf backup-$(date +%Y%m%d).tar.gz /home/user/
SCRIPT
chmod +x backup.sh
# Automated email reports
{
echo "Subject: Daily Report"
echo ""
df -h
uptime
} | sendmail [email protected]
Automate configuration and system tasks
Error Handling and Debugging
Debugging redirection
# Check if redirection worked
command > output.txt
if [ $? -eq 0 ]; then
echo "Command succeeded, check output.txt"
else
echo "Command failed"
fi
# Verify file creation
command > output.txt && echo "File created" || echo "Failed"
# Check file permissions
if [ -w /path/to/output.txt ]; then
command > /path/to/output.txt
else
echo "Cannot write to file"
fi
# Debug with verbose output
set -x
command > output.txt 2>&1
set +x
Debug and verify redirection operations
Safe redirection practices
# Prevent accidental overwrites
set -o noclobber
command > existing_file.txt # This will fail
command >| existing_file.txt # Force overwrite
# Backup before overwriting
[ -f important.txt ] && cp important.txt important.txt.bak
command > important.txt
# Use temporary files
temp_file=$(mktemp)
command > "$temp_file"
if [ $? -eq 0 ]; then
mv "$temp_file" final_output.txt
else
rm "$temp_file"
fi
# Atomic operations
command > output.txt.tmp && mv output.txt.tmp output.txt
Implement safe redirection practices
Performance Considerations
Redirection Performance Tips
- Use
>>for appending to avoid reading entire file - Redirect to
/dev/nullto discard unwanted output - Use
execfor persistent redirections in scripts - Consider buffering when writing large amounts of data
- Be careful with redirection in loops - consider alternatives
Efficient redirection patterns
# Efficient logging in loops
{
for file in *.txt; do
echo "Processing $file"
process_file "$file"
done
} > processing.log 2>&1
# Bulk redirection
exec 3> output.txt
for i in {1..1000}; do
echo "Line $i" >&3
done
exec 3>&-
# Avoid repeated file operations
# Instead of:
# for i in {1..100}; do echo $i >> numbers.txt; done
# Use:
for i in {1..100}; do echo $i; done > numbers.txt
Optimize redirection for better performance
Best Practices
Redirection Best Practices
- Always check if files exist before overwriting important data
- Use meaningful filenames for redirected output
- Include timestamps in log files for better tracking
- Separate stdout and stderr when debugging
- Use
/dev/nullto discard unwanted output - Be careful with permissions when redirecting to system files
Common Pitfalls
- Overwriting important files - Use noclobber or backup first
- Mixing up operators - Remember > overwrites, >> appends
- Wrong order -
2>&1must come after output redirection - Permission issues - Check write permissions on target files
- Disk space - Monitor disk usage when redirecting large outputs