You need to extract one or more fields from each line of output.
Use cut if there are delimiters you can easily pick out, even if they are different for the beginning and end of the field you need:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 |
# Here's an easy one, what users, home directories and shells do # we have on this NetBSD system $ cut -d':' -f1,6,7 /etc/passwd root:/root:/bin/csh toor:/root:/bin/sh daemon:/:/sbin/nologin operator:/usr/guest/operator:/sbin/nologin bin:/:/sbin/nologin games:/usr/games:/sbin/nologin postfix:/var/spool/postfix:/sbin/nologin named:/var/chroot/named:/sbin/nologin ntpd:/var/chroot/ntpd:/sbin/nologin sshd:/var/chroot/sshd:/sbin/nologin smmsp:/nonexistent:/sbin/nologin uucp:/var/spool/uucppublic:/usr/libexec/uucp/uucico nobody:/nonexistent:/sbin/nologin jp:/home/jp:/usr/pkg/bin/bash # What is the most popular shell on the system? $ cut -d':' -f7 /etc/passwd | sort | uniq -c | sort -rn 10 /sbin/nologin 2 /usr/pkg/bin/bash 1 /bin/csh 1 /bin/sh 1 /usr/libexec/uucp/uucico # Now let's see the first two directory levels $ cut -d':' -f6 /etc/passwd | cut -d'/' -f1-3 | sort -u / /home/jp /nonexistent /root /usr/games /usr/guest /var/chroot /var/spool |
Use awk to split on multiples of whitespace, or if you need to rearrange the order of the output fields.
Note the ➝ denotes a tab character in the output. The default is space but you can change that using $OFS:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
# Users, home directories and shells, but swap the last two # and use a tab delimiter $ awk 'BEGIN {FS=":"; OFS="\t"; } { print $1,$7,$6; }' /etc/passwd root ➝ /bin/csh ➝ /root toor ➝ /bin/sh ➝ /root daemon ➝ /sbin/nologin ➝ / operator ➝ /sbin/nologin ➝ /usr/guest/operator bin ➝ /sbin/nologin ➝ / games ➝ /sbin/nologin ➝ /usr/games postfix ➝ /sbin/nologin ➝ /var/spool/postfix named ➝ /sbin/nologin ➝ /var/chroot/named ntpd ➝ /sbin/nologin ➝ /var/chroot/ntpd sshd ➝ /sbin/nologin ➝ /var/chroot/sshd smmsp ➝ /sbin/nologin ➝ /nonexistent uucp ➝ /usr/libexec/uucp/uucico ➝ /var/spool/uucppublic nobody ➝ /sbin/nologin ➝ /nonexistent jp ➝ /usr/pkg/bin/bash ➝ /home/jp # Multiples of whitespace and swapped, first field removed $ grep '^# [1-9]' /etc/hosts | awk '{print $3,$2}' 10.255.255.255 10.0.0.0 172.31.255.255 172.16.0.0 192.168.255.255 192.168.0.0 |
Use grep -o to display just the part that matched your pattern.
This is particularly handy when you can’t express delimiters in a way that lends itself to the above solutions.
For example, say you need to extract all IP addresses from a file, no matter where they are.
Note we use egrep because of the regular expression (regex), but -o should work with whichever GNU grep flavor you use, but it is probably not supported on non-GNU versions.
Check your documentation.
1 2 3 4 5 6 7 8 9 10 |
$ cat has_ipas This is line 1 with 1 IPA: 10.10.10.10 Line 2 has 2; they are 10.10.10.11 and 10.10.10.12. Line three is ftp_server=10.10.10.13:21. $ egrep -o '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' has_ipas 10.10.10.10 10.10.10.11 10.10.10.12 10.10.10.13 |
The possibilities are endless, and we haven’t even scratched the surface here.
This is the very essence of what the Unix toolchain idea is all about.
Take a number of small tools that do one thing well and combine them as needed to solve problems.
Also, the regex we used for IP addresses is naive and could match other things, including invalid addresses.
For a much better pattern, use the Perl Compatible Regular Expressions (PCRE) regex from Mastering Regular Expressions by Jeffrey E. F. Friedl (O’Reilly), if your grep supports -P. Or use Perl.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
$ grep -oP '([01]?\d\d?|2[0-4]\d|25[0-5])\.([01]?\d\d?|2[0-4]\d|25[0-5])\.([01]?\d\ d?|2[0-4]\d|25[0-5])\.([01]?\d\d?|2[0-4]\d|25[0-5])' has_ipas 10.10.10.10 10.10.10.11 10.10.10.12 10.10.10.13 $ perl -ne 'while ( m/([01]?\d\d?|2[0-4]\d|25[0-5])\.([01]?\d\d?|2[0-4]\d|25[0-5])\. ([01]?\d\d?|2[0-4]\d|25[0-5])\.([01]?\d\d?|2[0-4]\d|25[0-5])/g ) { print qq($1.$2.$3. $4\n); }' has_ipas 10.10.10.10 10.10.10.11 10.10.10.12 10.10.10.13 |