What can you learn from your daily horoscope? For the sake of argument, let's agree that the stars don't actually say anything about your future. Is there anything left over?

Well, yes. We know that people still read and believe horoscopes, even though they don't contain any information. So, horoscopes contain information on what people are willing to believe in the absence of good evidence. People want to believe horoscope-style statements like "You will encounter and overcome an obstacle." Horoscopes contain real information, not about our personal future, but about the nature of human belief.

Another example: the so-called "great books." All we learn from a typical work of fiction is that someone neurotic enough to be an author was able to imagine people behaving in a certain way. If the book is a classic, though, then we can also say that the author's vision is interesting to a great many people — it's kind of like peer review. That doesn't mean it's accurate, but it's demonstratably compelling.

Software Engineering Literature

"Hey, Oran, I don't see what this has to do with programming. Is this another one of your philosophical essays?"

Nope. I'm thinking about software engineering, and it's literature. It seems to me that a lot of what we (collective "we" for the industry) know about software engineering is ad-hoc and informal: consider WardsWiki. There's a lot there that seems familiar, but not a lot of rigorous engineering to prove that the practices will improve productivity in a statistically significant way. The best we can say is it sounds good.

Ok, so WardsWiki is designed to be taken with a grain of salt. It doesn't attempt to be authoritative. But what about something like Scott Meyer's Effective C++? Don't get me wrong, Effective C++ is a fantastic book. But it doesn't provide any statistical evidence for it's suggestions; relying instead on making things logical enough to be obvious. Much of our knowledge about software engineering comes from accounts like this, drawn from the author's experience and speaking to our personal experiences. If this isn't engineering, then what is it? Is it still valuable?

Craftsmanship

Answers: It's craftsmanship, and it is very valuable. My favorite "old school" craftsman in Norm Abram of the New Yankee Workshop.

Norm sails through projects, always choosing the right tool and technique for each join or rabbet. Like Scott Meyers' 55 items, the value of each technique is obviously valuable. That's craftsmanship; do it right, and it's easy.

It's not hard to see that craftsmanship and engineering are different. It's just a little surprising how different they are, once we take a closer look. In some sense, they're both about Quality, but the differences go deep: different activities, different intellectual traditions, different modes of thought, maybe even different parts of the brain.

Look at it this way. Suppose an apprentice blacksmith back in the dark ages (500 - 1000 A.D) and wants to learn how to shoe a horse. Should he study mathematics (Euclid), run a series of experiments on the durability and bending characteristics of different iron mixtures at different temperatures, and come up with the optimal horseshoe, or should he just do whatever the master blacksmith does? If you don't want to starve to death before shoeing a single horse, you just copy the best practices you can find.

The only problem is that his son is going to do the same thing, and his son, and the next thing you know the middle ages are over (500 - 1500 A.D.), and customers are asking about cannons instead of horseshoes, and he can't answer. All the blacksmith knows is horseshoes, and he don't even understand that in any depth. In other words he got stuck in a local maxima while engineers are advancing the state of the art.

Craftsmanship is valuable. Effective C++ improved my C++ code enormously. Most of what I do on a day-to-day basis is craftsmanship, pure and simple. And, quite bluntly, you can't build a good system without a lot of good craftsmanship, any more than you could build a good ship without experienced welders.

The Science of Craftsmanship

Craftsmanship is learned by example and from advice. Both these sources of knowledge are plentiful, vastly outnumbering formal studies on software engineering. To draw upon this knowledge safely, we need to be sure that we always treat it as a source of hypotheses.

A hypothesis is something that one assumes to be true; one acts as if the hypothesis is true, for the time being, while keeping an eye out for errors or logical inconsistencies derived from the the hypothesis Over time, as you accumulate evidence that the hypothesis work, you can trust it more and more, and increasingly use it to decide your course of action.

That sounds like a lot of work, and in truth it does take a certain amount of intellectual sophistication to keep track of the trust level of each hypothesis. However, scientists have been doing this successfully for a number of centuries now, so it's certainly possible.

The benefit is reduced risk. Let me explain: If you accept a framework or methodology entirely, without reservation, then you can save yourself a lot of time and thought that can be spent writing software (or baking cookies, or whatever it is you do.) This is an unbelievably risky strategy, however. There are enough programmers around without out-of-date ideas (and no prospects) to demonstrate the danger.

Don't Learn the Wrong Thing

My "great books" example makes it sound as if one could argue that popular ideas have undergone a kind of informal peer review. That's slightly true, but remember that bad ideas have often seen widespread acceptance throughout history. "Horoscope" ideas in particular (things people want to believe and therefore are willing to believe on little or no evidence) are a problem. Popular acceptance can be useful in deciding what warrants further investigation, but counts for very little in the face of real evidence.

The simple truth is that most ideas aren't very good and don't last very long. Most hypotheses are eventually proven wrong, and a hypothesis can count itself very lucky indeed if it merely proven inaccurate (like Newtonian physics.) Accepting an idea at random - say, by only learning the approach your employer uses - is intellectual russian roulette.

So what can we do? As we work with new tools or in specialized problem domains, we can strive to master the known best practices, but also test them, prove them, and understand them deeply in terms of a wider context. We can collect hard data. We can be skeptical of unproven ideas, while still being willing to try them out. The situation is the same as with horoscopes: as long as we are clear and honest with ourselves about exactly what it is we're learning, we can find knowledge everywhere.