Tuesday, 27 May 2008

Process substitution in bash

I like the process substitution syntax in bash. Since I don't expect everyone to already use it at lengths, a small note on it might be welcomed.

Process substitution connects the input or the output of a process list to a pipe whose other end is connected to the rest of your command line. For example, this will list your files in reverse order:

sort --reverse <(ls)

You'd said, I could have used a good old pipe, as in ls | sort --reverse. Fair enough, but sometimes you want more than one input, or output. That's where process substitution comes to the rescue.

I find this particularly useful with diff(1). I like diff. Don't you like diff? (GNU diff, I mean, the one with -u for unified diffs. It don't mean a thing if it ain't got that switch.) I read unified diffs every morning at breakfast. (I swear it's true.)

So let's see, I want to compare the output of two commands. Let's say I want to compare two perl optrees. (A perl optree is the intermediate form in which perl compiles your program before executing them.) To get a good, but concise, overview of a perl optree, dumped on the standard error stream, my favourite command is:
perl -MO=Concise foo.pl

(I think I'll make myself an alias for that one day.)

Let's say my almost similar programs are in files foo.pl and bar.pl. To get a diff of their optrees, the command will then be:
diff -u <(perl -MO=Concise foo.pl 2>&1) <(perl -MO=Concise bar.pl 2>&1)

(Note that I redirected stderr to stdout.) Et voilà.

And if that's too much typing, there's always history substitution:
perl -MO=Concise foo.pl 2>&1
diff -u <(!!) <(!!:s/foo/bar/)


ChrisDolan said...

That is indeed a cool feature. The biggest drawback, inevitably, is the meaningless filenames:

--- /dev/fd/63 2008-05-27 19:52:35.000000000 -0500
+++ /dev/fd/62 2008-05-27 19:52:35.000000000 -0500

Kai Carver said...

This is very neat and useful.

Question: what possible use could there be for the >(cmd) form? I can't even get my brain around what that does.

Let's see, <(cmd) puts the output of cmd into a pseudo file. So... >(cmd) takes the input? of cmd and uh...? I give up.

Loren said...

It is a tragic feature of the Mac version of diff (at least on Tiger -- I don't know about Leopard) that it checks inodes before diff'ing files. Since the 'files' you get this way both appear to diff to have inode 0, it returns no difference on a Mac. (I stumbled across this precisely when trying to show someone the same cool use of process substitution.)