Why developers shouldn’t fear the red light

I hear people say that before each commit, a developer should at least build the whole project and run all the tests. That approach goes against continuous integration. Why do we practice continuous integration? To see a green light all the time, or to find out early when we have a problem? Sometimes I think it's just about seeing the green light, at the expense of using people and machines efficiently.

Every project I've worked on has at some point grown beyond the size where building and running the tests before a commit simply scales. A full build of our HP SOA Systinet takes more than twenty minutes on my computer, and that's before running any tests. If every developer did that on their own machine, they'd either die of boredom or get nothing done while they waited for the build to finish.

It's more efficient to make a commit that might break the build. That's because most commits simply don't break it. Most developers make changes with local effects, and the IDE catches the most common problems those changes can cause. That leaves relatively few problems, and they aren't worth grinding through the whole build on every developer's machine.

Continuous integration servers are generally much more powerful than local developer workstations, and they can use all kinds of optimizations. They can distribute the build, analyze dependencies and build only what changed, use a local cache, batch builds and so on. A developer with limited resources can only dream of doing those things.

A red light is no tragedy! If you work this way, there's one rule to follow. Don't pile up more commits on a broken build. You can enforce that by locking the version control system automatically when the build breaks. Only commits that fix it are allowed. You can recognize those automatically too. The developer marks them in the commit message using a predefined template. That way you avoid a situation where a developer breaks the build, gets run over by a tram and nobody can fix it because the version control system is locked.

There are less invasive ways, of course. With distributed version control, you can have a hierarchy of repositories. A commit can then move automatically up the hierarchy, from the team's repository to the project's, depending on whether the build succeeds at each level. That way you don't have to lock the version control system.