Pipe Viewer - A Tool to View Progress of Commands in Linux

When you work on some processes, sometimes you need to visualize the progression of file operations. Some commands have a parameter to help you to see the progression but for those which don't have it, you need to use another method. To mitigate this problem, you can use a pipe viewer which indicates the progression of the different processes that you launch.

What is pipe viewer

Pipe viewer (pv) is a CLI tool that allows you to monitor the progress of data through a pipeline by giving information such as percentage completed, ETA, total data transferred, time elapsed, etc. If you have some processes, you can insert the pipe viewer as a normal pipe between the different processes to have a visual indication of the progression.

Options of Pipe viewer tool:

Progress Bar in Python Tkinter with...

To view this video please enable JavaScript, and consider upgrading to a web browser that supports HTML5 video

Progress Bar in Python Tkinter with Source Code | Free Python Projects with Source Code

• -s, --size <size>: Assume the total amount of data to be transferred is SIZE. You can provide a size in bytes or using units (b, kb, mb, gb, tb).
• -N, --name <name>: Prefix the output information with NAME.

1) Installation of Pipe Viewer

The Pipe viewer can be installed through node.js as below:

# npm install -g pv
/usr/bin/pv -> /usr/lib/node_modules/pv/bin/pv.js
+ [email protected]
added 15 packages in 0.953s

Examples of pipe viewer usage

In this part, I will show you some different examples which can help you to use the pv tool

a) Display a dd copy progression

You can display the progression of a file copying when using the dd command

# dd if=ubuntu-gnome-17.04-desktop-amd64.iso | pv | dd of=dest/ubuntu17
 0.00% | 0 ETA | 1.36GB Transferred | 95.71MB/s2964416+0 records in
2964416+0 records out
1517780992 bytes (1.5 GB) copied, 14.7174 s, 103 MB/s
 100.00% | 0 ETA | 1.41GB Transferred | 96.5MB/s2964416+0 records in
2964416+0 records out
1517780992 bytes (1.5 GB) copied, 14.7301 s, 103 MB/s

b) A cat command progression

You can decide to display the progression of the cat command when displaying the content of message log file for example

# cat /var/log/messages | pv
Oct 13 03:36:02 localhost journal: Runtime journal is using 6.1M (max allowed 49.4M, trying to leave 74.1M free of 488.4M available → current limit 49.4M).
Oct 13 03:36:02 localhost kernel: Linux version 4.9.36-x86_64-linode85 ([email protected]) (gcc version 4.9.2 (Debian 4.9.2-10) ) #1 SMP Thu Jul 6 15:31:23 UTC 2017
........
........
100.00% | 0 ETA | 114.83KB Transferred | 114.83KB/s

c) Display the progression while compressing

During some operations of compression, we can need to display the progression. It is possible as below

# tar -cf - /var/log/ | pv -N backlog | tar -C . -x
tar: Removing leading `/' from member names
backlog: 100.00% | 0 ETA | 660KB Transferred | 660KB/s

d) Visualize a live ssh network

You can connect to a host via ssh and displays the live transfer speed. In our case, we will direct all transferred data to /dev/null with a local test

# yes | pv | ssh paul@localhost "cat > /dev/null"
The authenticity of host 'localhost (::1)' can't be established.
ECDSA key fingerprint is SHA256:lqes2c9Obi9lCSepwdT8uovcsmhcboUAsylF69MENDE.
ECDSA key fingerprint is MD5:38:67:a1:25:96:7b:69:02:f6:b8:ff:5d:ce:03:fd:a0.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts.
paul@localhost's password: 
 0.00% | 0 ETA | 800.84MB Transferred | 17.04MB/s

The Unix pipe viewer of Rocco Musolino doesn't display the percentage completed with the progress bar as the "normal" pipe viewer installed through the package. We can see that the pipe viewer is interesting when we face commands which don't allow to visualize the progression of the processes.