Linux/UNIX Awk Command Tutorial with Examples
Abstract: $ awk '{printrun $ awk '/Hari|Ram/' awk_file
Awk is a scripting language which is used for processing or analyzing text files. Or we can say that awk command is mainly used for grouping of data based on either a column or field , or on a set of columns. Mainly it’s used for reporting data in a useful manner. It also employs Begin and End Blocks to process the data.
AWK Stands for ‘Aho, Weinberger, and Kernighan‘
In this tutorial, we will learn awk command with practical examples.
Syntax of awk# awk ‘pattern {action}’ input-file > output-file
Let’s take a input file with the following data
$ cat awk_file Name,Marks,Max Marks Ram,200,1000 Shyam,500,1000 Ghyansham,1000 Abharam,800,1000 Hari,600,1000 Ram,400,1000
Now, let’s deep dive into practical examples of awk command.
1) Print all the lines from a fileBy default, awk prints all lines of a file , so to print every line of above created file use below command :
$ awk '{print;}' awk_file Name,Marks,Max Marks Ram,200,1000 Shyam,500,1000 Ghyansham,1000 Abharam,800,1000 Hari,600,1000 Ram,400,1000
Note: In awk command ‘{print;}’ is used print all fields along with their values.
2) Print only specific field like 2nd & 3rdIn awk command, we use $ (dollar) symbol followed by field number to prints field values. In below example, we are printing field 2 (i.e Marks) and field 3 (i.e Max Marks)
$ awk -F "," '{print $2, $3;}' awk_file Marks Max Marks 200 1000 500 1000 1000 800 1000 600 1000 400 1000
In the above command we have used the option -F 「,」 which specifies that comma (,) is the field separator in the file.
3) Print the lines which matches the patternI want to print the lines which contains the word 「Hari & Ram」, run
$ awk '/Hari|Ram/' awk_file Ram,200,1000 Hari,600,1000 Ram,400,10004) How do we find unique values in the first column of name
To print unique values from the first column, run below awk command
$ awk -F, '{a[$1];}END{for (i in a)print i;}' awk_file Abharam Hari Name Ghyansham Ram Shyam5) How to find the sum of data entry in a particular column
In awk command, it is also possible to perform some arithmetic operation based on search, syntax is shown below
$ awk -F, ‘$1==」Item1″{x+=$2;}END{print x}’ awk_file
In the below example, we search for Ram and then we add values of 2nd field for Ram word.
$ awk -F, '$1=="Ram"{x+=$2;}END{print x}' awk_file 6006) How to find the total of all numbers in a column
In awk command, we can also calculate the sum of all numbers in a column of a file. In the below example we are calculating the sum of all numbers of 2nd and 3rd column.
$ awk -F"," '{x+=$2}END{print x}' awk_file 3500 $ awk -F"," '{x+=$3}END{print x}' awk_file 50007) How to find the sum of individual group records
For example, if we consider the first column than we can do the summation for the first column based on the items
$ awk -F, '{a[$1]+=$2;}END{for(i in a)print i", "a[i];}' awk_file Abharam, 800 Hari, 600 Name, 0 Ghyansham, 1000 Ram, 600 Shyam, 5008) Find the sum of all entries of specific columns and append it to the end of the file
As we already discuss that awk command can do sum of all numbers of a column, so to append the sum of column 2 and column 3 at the end of file, run
$ awk -F"," '{x+=$2;y+=$3;print}END{print "Total,"x,y}' awk_file Name,Marks,Max Marks Ram,200,1000 Shyam,500,1000 Ghyansham,1000 Abharam,800,1000 Hari,600,1000 Ram,400,1000 Total,3500 50009) How to find the count of entries against every column based on the first column
$ awk -F, '{a[$1]++;}END{for (i in a)print i, a[i];}' awk_file Abharam 1 Hari 1 Name 1 Ghyansham 1 Ram 2 Shyam 110) How to print only the first record of every group
To print only first of every group, run below awk command
$ awk -F, '!a[$1]++' awk_file Name,Marks,Max Marks Ram,200,1000 Shyam,500,1000 Ghyansham,1000 Abharam,800,1000 Hari,600,1000AWK Begin Block
Syntax for BEGIN block is
$ awk ‘BEGIN{awk initializing code}{actual AWK code}’ File-Name
Let us create a datafile with below contents
11) How to populate each column names along with their corresponding data$ awk 'BEGIN{print "Names\ttotal\tPPT\tDoc\txls"}{printf "%-s\t%d\t%d\t%d\t%d\n", $1,$2,$3,$4,$5}' datafile12) How to change the field separator
As we can see space is the field separator in the datafile , in the below example we will change field separator from space to 「|」
$ awk 'BEGIN{OFS="|"}{print $1,$2,$3,$4,$5}' datafile
That’s all from this tutorial, I hope you found it informative. Please do share your feedback and queries in below comment’s section.
Recommended Read: 10 Quick Linux Tail Command with Examples