Field Reports with awk

awk splits each line into fields $1 to $NF (on whitespace, or on the -F separator) and runs every pattern {action} rule that matches. BEGIN and END rules run before the first line and after the last, and arrays are associative, so n[$9]++ counts lines per status without sorting. A condition on one field plus arithmetic on another replaces a whole grep | cut | sort | uniq chain.

Top 404 paths, bytes per status, and requests per minuteShell

awk '$9 == 404 {print $7}' access.log | sort | uniq -c | sort -rn | head -3   # top 404 paths
awk '{n[$9]++; b[$9] += $10} END {for (s in n) printf "%s %4d %8d\n", s, n[s], b[s]}' access.log
awk '$4 ~ /:13:5[2-5]:/ {print substr($4, 2, 17)}' access.log | uniq -c          # per minute

Output

     36 /about.php
     25 /favicon.ico
     10 /xmlrpc.php
200  396  3762341
403   10     4750
404  111    52392
    135 23/Sep/2026:13:52
    189 23/Sep/2026:13:53
    117 23/Sep/2026:13:54
     55 23/Sep/2026:13:55

/about.php is a broken link to fix and /favicon.ico a file every browser requests; the rest are probes. The 111 misses cost only 52 KB, so the scanner is noise rather than load, and its burst is the 13:53 peak. The last line tests a field with ~ and cuts the timestamp to its minute. for (s in n) has no guaranteed order, so pipe through sort when order matters. Ubuntu 225 's awk is GNU awk 5.3.2 (mawk is also installed); keep scripts to POSIX features and they run under both. A custom LogFormat shifts every field number, so look at a whole line before trusting someone else's log.

MENU

Field Reports with awk