The fastest way to query CSV files from the command line

Learn beginner-friendly AWK tricks for filtering and transforming CSV data.

Personally, one of the ways I learned [awk](https://en.wikipedia.org/wiki/AWK) was by solving challenges from Advent of Code. It’s a surprisingly good way to practice text processing, parsing, and building small command-line solutions under pressure. Some of those puzzles get complicated very quickly though, especially when you try solving them with awk alone. So instead of throwing recursive parsing nightmares at you, I prepared something much simpler — practical CSV examples you can actually use in everyday work.

Visual representation of CSV content used for the purpose of this story, styled using HTML and CSS, then captured as a screenshot. The styles were generated by ChatGPT.

CSV files are everywhere: exports from spreadsheets, database dumps, API reports, and logs. When you need a quick way to inspect or transform CSV data from the command line, awk is one of the fastest tools available.

Here’s a practical introduction to querying simple CSV files with awk.

Sample CSV file

Imagine a file named employees.csv:

id,name,department,salary
1,Alice,Engineering,85000
2,Bob,Marketing,62000
3,Charlie,Engineering,91000
4,Diana,Sales,58000
5,Eve,Marketing,67000

Basic CSV querying

By default, awk splits fields using spaces. CSV files use commas, so we set the field separator with -F ','.

Print all rows

awk -F ',' '{ print $0 }' employees.csv

$0 represents the entire line.

Print specific columns

Show only names and salaries

awk -F ',' '{ print $2, $4 }' employees.csv

Output:

name salary
Alice 85000
Bob 62000
Charlie 91000
Diana 58000
Eve 67000

$1 — first column, $2 — second column, $3 — third column and so on.

Filtering rows

Find employees in engineering

awk -F ',' '$3 == "Engineering"' employees.csv

Output:

1,Alice,Engineering,85000
3,Charlie,Engineering,91000

Filter by salary

Employees earning more than 80,000

awk -F ',' '$4 > 80000' employees.csv

Output:

1,Alice,Engineering,85000
3,Charlie,Engineering,91000

Skipping the header row

Most CSV files contain headers. Use NR > 1 to ignore the first line.

awk -F ',' 'NR > 1 && $4 > 80000' employees.csv

NR — current row number

Pretty printing output

You can format results with labels:

awk -F ',' 'NR > 1 {
    print "Employee:", $2, "| Department:", $3
}' employees.csv

Output:

Employee: Alice | Department: Engineering
Employee: Bob | Department: Marketing
Employee: Charlie | Department: Engineering
Employee: Diana | Department: Sales
Employee: Eve | Department: Marketing

Aggregations

In awk, the BEGIN and END blocks let you run code before and after processing the input file. The BEGIN block is commonly used to initialize variables, set separators, or print headers before any rows are read. The END block runs after all lines have been processed, making it perfect for summaries, totals, and final calculations. This structure makes awk especially powerful for quick reporting tasks. For example:

awk -F ',' '
BEGIN {
    print "Processing employee data..."
}
NR > 1 {
    total += $4
}
END {
    print "Total salary:", total
}
' employees.csv

In this example, BEGIN prints a startup message, the main block processes each CSV row, and END outputs the final salary total after the file has been fully read.

Calculate average salary

awk -F ',' 'NR > 1 {
    sum += $4
    count++
}
END {
    print "Average salary:", sum / count
}' employees.csv

Output:

Average salary: 7260

Counting matches

We can declare variables and the use them or even skip declarations. See example below how awk infers what value should be assigned to count:

Count marketing employees

awk -F ',' '$3 == "Marketing" { count++ }
END { print count }' employees.csv

Output:

2

Sorting CSV data

Combine awk with other Unix tools like sort.

Sort by Salary

awk -F ',' 'NR > 1' employees.csv | sort -t ',' -k4 -n

• -t ',' sets the delimiter
• -k4 sorts by column 4
• -n performs numeric sorting

Important limitation

Simple awk CSV processing works well for basic CSV files, but it struggles with:

• quoted commas
• escaped quotes
• multiline fields

Example problematic CSV:

1,"Smith, John",Engineering

For complex CSV parsing, consider dedicated tools or be my guest in the series where I attempt to solve proper parsing limiting myself to Bash only, here’s **the list**.

Why use awk?

awk is great when you need:

• quick terminal-based analysis
• lightweight scripting
• fast filtering and aggregation
• Unix pipeline integration

For small and simple CSV files, it’s often faster than opening a spreadsheet or writing a Python script.

Final example

Find all Engineering employees earning more than 90k:

awk -F ',' 'NR > 1 && $3 == "Engineering" && $4 > 90000 {
    print $2
}' employees.csv

Output:

Charlie

That’s the power of awk: compact, readable, and extremely efficient for command-line data work.

Follow me on Medium | Support on ko-fi.com