Processing Text with Bash Using Grep, Sed, and Awk

Text processing in Bash using the grep, sed, and awk commands allows developers and system administrators to filter and manipulate input for specific tasks such as searching log files, transforming content, and extracting structured data. You can combine these utilities with loops, conditional statements, and expressions for advanced data processing. This guide walks through each individual tool before demonstrating how to combine them into functional scripts.

The process begins by creating a log.txt file using nano to store multiple lines of text, including timestamped events ranging from successful Apache Web Server startups to database connection errors on port 3306 and disk space warnings. Next, a processing.sh script is created to load the log.txt file into a LOG variable, preparing the data for subsequent manipulation using grep, sed, and awk.

The grep command is utilized to search for specific text patterns within files. By appending grep -i "ERROR" "$LOG" to the script, it scans log.txt for the term ERROR case-insensitively and displays all matching entries, such as database connectivity failures and missing configuration files like www.yml.

Following grep, the awk command analyzes structured data and inspects file patterns. Using awk '{print $1, $3}' "$LOG" extracts and prints the first and third columns of the log file, neatly displaying dates alongside their corresponding log levels such as INFO and ERROR for cleaner data visibility.