Here we are at the start of Software Tools in Pascal’s third chapter, entitled Files. It’s all very well reading one input and writing one output, but eventually we’re going to want to access “some kind of permanent file system where information can be kept for extended periods on secondary storage”.

In their opening paragraphs Kernighan and Plauger outline the territory - each operating system has its own jargon for describing file system operations, each has its own capabilities and limitations, some operations are easier on this system than that. This is still true, perhaps even more so given that there isn’t a one-to-one mapping of operating systems to filesystems, although happily most of us are able to ignore that in our daily programming lives.

System variations, while a pain, were not Kernighan and Plauger’s biggest obstacle - file system access was just not a thing Pascal concerned itself with. When I was taught Pascal some years after Software Tools in Pascal was written, the language support for file access could best be described as grudging. K&P handle the problem in the same way as they handled console i/o in earlier chapters, presenting a library of file access primitives. The primitives are, unsurprisingly, very similar to those provided in C, and much of the chapter is spent on describing their operation and the operation of file systems in general.

In keeping with their step-by-step approach, Chapter 3’s first program is compare, a file comparison program. Our program needs to read two files, but we can reuse the console output we’re familiar with.

Each program in chapter one was progressively a little more complex than the one before. This is not the case in Chapter Two, where the various programs form more of a loose map of the territory. Here we have a program that replaces many characters with a single character, while over there we have a program that might produce many lines of output for a single line of input, and so on.

This chapter is, without directly saying so, pretty explicitly about the Unix philosophy. While Unix systems (or at least Unix-ish systems) are basically the substrate of most modern computing (yes, even if you’re a Windows, Android, or iOS user) in 1981, when Software Tools in Pascal was published, there were “800 systems …​ in use outside [Bell Labs]”. Kernighan and Pike have to take time to describe what’s now commonplace - combining small programs with pipelines, text as common currency, indeed the whole idea of software tools and of the programmer as tool user.

When I first read Software Tools in Pascal, I’d just started my first substantial job on a Unix system (in fact four Unix systems, all slightly different) and I was only passingly familiar with a handleful of command line tools. The idea that these small programs not only did simple things, but could be combined like some kind of giant tokusatsu robot to do very powerful things was really quite startling. It certainly changed, and continues to influence, how I think about programming and I have no doubt I’m a better programmer for that. I’m not sure I’ll ever touch the heights achieved by my friend Thomas in his masterful He Sells Shell Scripts To Intersect Sets, in which he combines standard Unix tools to do full on computer science, but I’m nearer than I otherwise would be.

Next up, files!

After the relative calm of echo, it all kicks off (or, if you prefer, all comes together) in translit, the finale of Chapter Two. As its name suggests translit transliterates certain characters on their way through. “We would like”, say Kernighan and Plauger, “to have a program translit so that we can write”

“and have all occurrences of x in the standard input be replaced by y on the standard output.”

Ah, so this is why we took that detour into command line arguments. Given what we’ve been through to get here, though, this task is pretty trivial. But hold on, they’re just getting started.

“Multiple translations are also handy:”

“would change all lower case x, y and z to the corresponding upper case letter.”

Ok, that’s slightly more involved but not particularly tricky.

“It would be nice”, they continue breezily, “to have a shorthand for alphabets, so”

“would translate all lower case letters to upper case.”

All right, this still isn’t too difficult. We can deal with the arguments easily enough, and then the transliteration is routine

We don’t even need to write a function - we do all it all in line with a simple lambda. This seems a touch anticlimactic. But Kernighan and Plauger haven’t finished. Not by a long shot. “There are times when we would like to translate a whole class of characters into just one character, and even to squash runs of that translation into just one instance …​ We can specify this squashing operation by giving a second argument that is shorter than the first …​ we could also take [the case when the second argument is not given] as a request to delete all occurrences of characters in the first string …​” Easy there fellows “…​ One final capability is worth adding: sometimes we would like to be able to translate and compress all but a set of characters. For instance”

“would replace strings of non-letters with a dash. And”

“would delete all but lower case letters.”

Wait, wait, wait.

When I write my C++ versions of the programs in Software Tools, I don’t translate from the Pascal. Of course, I do read the code along with the commentary, but then I start by writing unit tests to capture the behaviour. Leaving aside argument expansion, for translit I finished up with five distinct sets of tests -

Obviously the two squashing cases and the two deletions cases are pretty similar, but that still leaves, as far as I was concerned, with three distinct operations. Kernighan and Plauger implement them in one loop

As you read this section of the book, you see the program grow a little bit at a time from an initial half a dozen lines to the code above, along with all the preamble to handle the command line arguments, the inner works of xindex and all the rest of it. By the time you get to it, this piece of code isn’t difficult at all because you know it inside out. Coming to it cold, it’s pretty daunting. There are two different flags, nested loops, an if-ladder nested within another if, and assignment through out parameters. It’s compact and dense and not entirely straightforward.

Partly because I was starting from a different place, partly because I wanted to keep the std::transform model I’d adopted for the preceding filters, and partly because I wanted to try and hoist as many of the conditions up and away from the main body of the code, I finished up with something rather different.

The argument expansion populates the replace and with strings. It’s also sets selectionMode based on the presence of the leading ^ character. Not shown here are the implementations of the in_range and out_of_range functions, which determine whether a particular character is within or not within a given string. The variable matcher then is a function pointer, and I implement the normal or negated search by pointing it to the appropriate function.

The if-ladder then determines which mode translit has been invoked in - deletion, squashing, or simple replacement. In the Pascal implementation, this if-ladder is at the heart of the read loop, traversed for every character. I’ve hoisted it right up and out - we decide here, then never have to worry about it again.

The three modes are then implemented in three separate functions. This turned out pretty well. Two of the cases - replacing and removing - can be implemented using functions provided by the standard library. Replacement uses std::transform, as shown above, while removing uses the catchily named std::remove_copy_if.

std::remove_copy_if copies the elements in the range, except those elements for which predicate returns true. In this case, the predicate is a little lambda wrapped around the matcher function pointer which we set up earlier.

That leaves squashing. The implementation is similar to that of entab. When entab encounters a space in the input, it switches into entabbing mode until it encounters a non-space or hits a tab stop. For squashing, when we encounter one of our from range, we flip into squash mode until we see a character that isn’t in the range. There’s a bit of house keeping to do then, but it’s straightforward enough.