Unix command line tricks for Linux, Mac, and Windows: head and tail

This is part of a series on Unix command line tools that are available in on Linux, OS X, and Windows machines. We'll be looking at some of the most useful Unix command line commands, because as the idea of Data Science gets bigger and bigger, it turns out that these old school Unix tools are more valuable than ever for dealing with the different kinds of data files that may show up. See Unix command line tricks for Linux, Mac, and Windows: grep for more on this series.

Today, we'll look at head and tail, which show the beginning and end of files. They may not look like much when we see what they do with small files, but be patient—they can be very useful with the bigger and bigger files that people need to work with these days. We'll start with examples using this small sampledata.csv exported as a CSV file from a spreadsheet program like Excel or LibreOffice Calc:

Employee Number,Family Name,Given Name,Hire Date,Phone Extension
1001,Johnson,Emily,11/13/2016,x0023
1002,Smith,John,03/16/2017,x7225
1003,Baker,Debbie,03/23/2017,x8834
1004,Morales,Kermit,06/09/2017,x2643

The head command shows you the first 10 lines of a file unless you specify another number with the -n option. Here, we ask for the first 3 lines of the file above:

head -n 3 sampledata.csv

Here is the result:

Employee Number,Family Name,Given Name,Hire Date,Phone Extension
1001,Johnson,Emily,11/13/2016,x0023
1002,Smith,John,03/16/2017,x7225

The tail command shows the last 10 lines of a file unless you use -n to specify a different amount. This command asks for the last 3 lines of the file above:

tail -n 3 sampledata.csv

Here is the result:

1002,Smith,John,03/16/2017,x7225
1003,Baker,Debbie,03/23/2017,x8834
1004,Morales,Kermit,06/09/2017,x2643

Again, this may not seem like a big deal, but bear with me. One command line switch for the grep command that I didn't mention when I wrote about grep was -n, which shows the line number of each line where it found the target it was searching for. If I entered the following,

grep -n 7225 sampledata.csv

I will get a list of all the lines that have "7225" in them, with their line numbers. It's only one, at line 3:

 3:1002,Smith,John,03/16/2017,x7225

All these commands send their output to appear on your screen, but you can create a Unix pipeline by sending output of one command to be used as the input of another with the pipe symbol, which is on the same key as your backslash: |.

Let's say that after finding out that "7225" is on line 3, I want to see it with one line before it and one line after it. The following command sends the first 4 lines of the file to the tail command, which displays the last 3 of what it gets; the result will be lines 2 through 4 of the original sampledata.csv input:

head -n 4 sampledata.csv | tail -n 3

Here is the result:

1001,Johnson,Emily,11/13/2016,x0023
1002,Smith,John,03/16/2017,x7225
1003,Baker,Debbie,03/23/2017,x8834

Now imagine that you have a file with 300,000 lines, and grep shows you that something you may be interested in is on line 145,234. You'll know more about whether it's interesting if you can see 4 lines before it and 4 lines after it, but this file is way too big to load into a text editor. So, you tell head to send the first 145,238 lines to the tail command, and you tell tail that you want the last 9 lines of its input:

head -n 145238 my3kfile.txt | tail -n 9

This is a running theme in Unix: commands that seem so simple that they're not very useful can be customized with options such as -n and combined with the pipe symbol.

And, as with all Unix commands, you can learn more about these with the manual (man) command:

man head
man tail