After counting characters and then lines, Kernighan and Plauger take us next to counting words.

You can see the little path Kernighan and Plauger are on quite clearly now. Start with a simple loop, then extend that loop with a simple counter. Next add a simple conditional, and now extend that conditional into a little state machine. Each little step has added something extra, changing the functionality of each program to provide a new and useful result.

My path has to been to look through the list of functions provided in the algorithm header and pick the one that does what’s needed for this task.

Kernighan and Plauger have a straightforward, although entirely reasonably, definition of a word - the maximal sequence of characters not containing a blank, a tab, or a newline. The additional complexity of Kernighan and Plauger’s wordcount over linecount is to keep track of whitespace delimiters between the words. They are, in effect, splitting up the sequence of characters into a sequence of words.

Previously, I blythely said that C++'s istream and ostream provide a number of different iterators. To get down and deal with the raw character stream we use istreambuf_iterator<char>. For formatted input, that is anything that needs a bit of work to process those raw characters in some way, we want some sort of istream_iterator.

Gathering characters up into whitespace delimited words qualifies as a bit of processing work. It’s the kind of thing people do all the time, and consequently is precisely what the istream_iterator<std::string> provides.

I rather loosely described std::distance(InputIt first, InputIt last) as counting the hops between first and last. More formally, it returns the number of iterator increments needed to go from first and last. Incrementing an istream_iterator<std::string> returns the next word, so plugging that into std::distance counts the number of words in the input.

So once you can count characters, you might also want to count the number of lines in some input. Instead of counting every character on our input, we just count the number of newline characters we see.

It’s at this point that Software Tools in Pascal does rather show its age. In their discussion of the code, Kernighan and Plauger say

There follows a paragraph or two about the implementation of their getc and putc primitives, and how they might be implemented on systems which have fixed length records, differing character sets, disk formats, or whatever. The point of these two functions is to insulate their code from the vagaries of the underlying system. Even if the implementations of getc and putc are trivial on a particular system, it is still worth doing.

This kind of insulation is part of what the standard library provides for us. I’ve built this code on three different operating systems, each with a three different file system, using four different compilers, and it’s all just worked. Kernighan and Plauger were not so blessed, and had to build their own low-level library as they went.

Of course our library doesn’t just have low-level stuff. It has mid-level stuff too, like the std::copy and std::distance functions we used previously, and like the std::count we’re using today.

After devoting six pages to bootstrapping copy, Kernighan and Plauger give their next program, charcount, six sentences of setup. Of those six, three of them are an exhortation to just hang on and trust them

In their copy program, they established we could read a character at a time from the standard input. For charcount, it’s straightforward to modify the loop to keep a count of those characters.

Hmm. I was feeling smug about not having a loop in my implementation at all. Perhaps I’ll have to revisit the whole business?

I chose to treat the standard input as a sequence, and consequently was able to lean entirely on a function provided by C++'s standard library. Copying a sequence from one place to another is merely one of a number of operations the standard library provides. Another is std::distance(InputIt first, InputIt last), a function which returns the number of hops you need to advance from first to last.

In this case, passing in our istreambuf_iterators it gives us the count of characters available on standard input. Again, the standard library provides exactly what’s needed.

While distance maybe isn’t the best name, once you’re familiar with it this implementation is about as minimal and clear as it can be. Again, we’ve avoided an explicit loop, nor have we had to come up with any variable names. Even in a program as brief as charcount Kernighan and Plauger’s version has a comprehension overhead. Their variable c is, essentially, redundant. It’s only there so they can call the getc primitive. The variable we really need to be paying attention to is nc. However, because Pascal requires variables to declared ahead of the procedure body, all variables initially appear to be equally important. It’s a small thing here, but the larger the procedure the bigger a barrier it can be.

Rather than have the charcount function write directly to the output, as Kernighan and Plauger did, I chose to return the count from the function and deal with the output higher up. In turn, this makes our test and executable drivers extremely plain.

Wrote a do …​ while loop this week. First time for everything, I guess

Software Tools in Pascal starts off pretty gently. The first program Kernighan and Plauger present is copy, which copies its input to its output. The seemingly trivial nature of this task allows them in introduce how they write and present their code - without getting into detail they mention how they cope with different Pascal implementations on different platforms, the general structure of they adopt, and so on. In particular they introduce primitives - functions that interface to the outside world. The two primitives they introduce first are getc and putc which, respectively, read a character from and write a character to somewhere,

These primitives, of which they build up a fair library over the course of the book, are how they make their Pascal portable across platforms. These days, we take that kind of thing for granted as part our language standard library.

After this preamble they present their program, or at least part of it

They explain

They do then present the whole program with the procedure in context, and talk it through. They conclude this initial section with a reason why this isn’t as trivial a program as it might appear.

The primitives getc and putc come, of course, more or less directly from C’s getc/getchar and putc/putchar. It would be tempting to cast this program directly into C++ by substituting begin and end for a pair of curly brackets and having done with it.

Tempting, but lazy.

Does this actually work? Well, by inspection it looks like it should, but because getchar and putchar are tied to our standard input and output (as we call our interactive terminal or some secondary storage device these days) we’d have to run the program to actually test it. Running a whole program to test it is tedious at best, even for a tiny program like this. Perhaps we can raise the level of abstraction a bit, and make it a little easier to test.

Now we can copy from any istream (which could be standard input, a file, something in memory, almost anything) to any ostream ((which could be the console, another file, somewhere in memory, almost anything). Nice. We can throw a little test wrapper round that, poke some known inputs through it, check the right thing comes out.

This is where I actually started. I wrote the test first, using the splendid Catch test framework.

I wrote the least amount of code I could to get this to compile.

I then ran the tests, which naturally nearly all failed but now I had something to work with.

The function signature - void copy(std::istream& in, std::ostream& os) - is pretty perfect, but what to fill it with? Is a while loop really still the state of the art here?

It is not.

The C++ Standard Library provides a function copy(InputIt first, InputIt last, OutputIt d_first) which copies the elements in the range, defined by [first, last), to another range beginning at d_first.

copy is a generic function. It takes a pair of iterators delimiting some input range and copies what it find to an output iterator. Classically, we always used to describe iterators as pointer-like, ie they pointed to something, you could advance them, and you could compare them. Equally classically, we generally thought about iterators as operating over some known and bounded region - a block of memory, or a container of some sort like a vector or a list. We don’t usually think of IO in these terms - a file is a file, writing to the console sends things to the screen, that kind of thing. However, if we start to think more broadly about iterators as moving over a sequence, and consider our input as source of characters and our output as somewhere to put a sequence of characters, it becomes quite natural to want to iterate over console input and output.

So, if I can connect my istream and ostream up to copy, it’ll do the work for me? Perfect!

Happily, C++'s istream and ostream can provide the iterators we’re after. In fact, there are quite a number to choose from. In this case, I want to pull raw characters from the input and poke them straight down the output, so I want istreambuf_iterator<char> and ostreambuf_iterator<char>.

I like this a lot. There’s no loop, no comparison, no worrying about special end-of-input sentinel values. When you read it there’s not even the slightest mental gymnastics involved in understanding it, because there’s nothing to comprehend. It does exactly what it says - copy this here to that there.