A failure, a reckoning and lessons in agile development

This is the story of a failure, a reckoning and what our team learned while delivering rebranded sign-in and registration pages. I could put it in a series about the difference between saying you work in an agile way and actually doing it. In this case we learned a lot from the failure and took steps that moved the team forward. What follows comes from the notes I managed to take during the retrospective. The story captures several textbook problems, which is why I'm writing about it. Everyone involved contributed to the failure in roughly equal measure. We held a retrospective, calmly identified every problem and agreed on how to fix them, so I can write about them here without anyone feeling singled out.

The road to hell is paved with good intentions

We originally wanted only to rebrand the GoodData sign-in and registration pages. A designer did the visuals and one developer was to map the flows by which users could reach the pages. You might think there's only one sign-in page and one registration page, but the sign-in page appears in different contexts. A few examples are embedded modes, registration confirmation, single sign-on and its failure, and authentication sessions expiring. From the start we wanted to leave the logic of those flows alone. The whole escapade ended up costing us two months of work, made us look less than good to senior management, and strained relations a little with Product Management (PM). Looking back, we made several textbook mistakes in the sincere belief that we were taking the right approach.

Do only what you need to do

When you tell someone you're going to redesign something, stick to the redesign. As part of ours we upgraded to a newer Ember.js, rewrote code to use it, and built and tuned infrastructure for automated browser tests. In other words, under the banner of a redesign, which to most people means changing CSS and HTML at most, we started paying down technical debt. That cost us time we couldn't spend on the feature itself. Mixing features and technical debt is dangerous because the effort is hard to estimate and the feature's delivery becomes dependent on paying down the debt. It's hard to communicate, too. “You see, it would have been ready ages ago, but we still need to tweak the testing framework so we can test X. And we've got to do something about test stability, since the tests don't tell us much right now. When will it be ready? That's hard to say…” So we adopted a rule. We pay down technical debt separately from feature development and keep it separate from new features.

Split the work and deliver in pieces

We divided the work into several logical steps and planned to follow them. But we'd planned to deliver and hand over the whole thing at once! We got real feedback only at the very end of development, after several sprints. We learned, for instance, that we needed to redo validation because it didn't fit the use case. That frayed the whole team's nerves. The story was already late, everyone felt pressure to deliver it, and above all everyone thought we'd reached the finish line. We got another round of development instead.

I think teams usually plan their work so that the result is visible only at the very end. Imagine you need to get a car down a road with a boulder in one lane. The team plans to move the boulder and then let the car through. If you can't move the boulder, you put the main goal at risk, getting the car through. The other approach is to send the car down the opposite lane, even if only temporarily. I think that's the right approach. So we made it a sprint rule to deliver something the customer can try. A minimum product, with the smallest visible change we can deliver to the customer.

The customer is in charge

Another problem was handing over the feature without knowing exactly who its customer was. The team and PM played ping-pong. One part of PM said yes, let's go, and another said they weren't happy and we still needed to change a few things. It strained relations quite a bit, because the developers didn't know who the main point of contact was for changes. Passing the entire feature between people in PM was a big factor. We probably could have avoided the trouble if one person from PM had owned it from beginning to end. So we adopted a rule that during sprint planning we all agree who the customer is and whom we're working for.

One environment

The developers had different pieces in their own branches, which made the customer demo harder. Demos and verification always used a local version from whatever state of the codebase the developer had. At the demo the customer couldn't try the feature as a whole. If a bug appeared somewhere, it wasn't clear whether it was still an issue or whether another developer had already fixed it in another branch. The customer needs one reference environment for handover and reviewing the work. So we agreed to choose the acceptance environment during planning. The difficult merge and bugs that surfaced only after merging into the main development branch were compounded by infrastructure changes, including the Ember.js upgrade. Mainline development might have helped, since integration problems become visible immediately. We'd first have had to sort out process issues associated with meeting Service Organization Controls 2.

Unknown unknowns

We spent quite a bit of time analyzing the existing code to identify the states that lead to the sign-in and registration pages. We drew flows to understand where we'd need to make changes, because we were rebranding, rewriting the affected parts using new technology, and writing new tests. The flows did help, but what we got out of them didn't justify the time we put in. For example, they missed edge cases that produced quite a few bugs during manual testing. The analysis helped us understand what we would need to test, but it did almost nothing to improve our estimates. We agreed the reverse approach would have helped much more. Start fresh and go straight to a minimal product, or build a working prototype, which is faster and reveals problems just as well. You can show someone a prototype and let them try it. Analysis is words and words are wind, as Jon Snow knew, along with everyone who has read the five books of A Song of Ice and Fire. A prototype does more to reveal problems you might never think of during analysis. An unknown unknown is something I can't analyze because I don't know it exists.

Strategy

No plan survives first contact with the enemy, as von Moltke said. You can still have a strategy for situations you didn't anticipate at the start. When problems showed up, we had no process for getting out of them. A strategy could be, “I deliver small pieces and show them to the customer every few days. We adapt to changes as they come up, based on ongoing feedback from the customer. If we find a problem we can't resolve in time, or it means a major change to the brief, we'll take it on in the next sprint as follow-up work.” We had no strategy, because we treated the work as a monolith at every level: planning, development, testing, handover and so on. A strategy for developing the feature could have forced us to break up that monolith at the planning stage.

Conclusion

Here are the basic rules we agreed on in the retrospective.

• We deliver something visible the customer can try in every sprint.
• During planning, the whole development team knows who the customer is or who is playing that role.
• We decide on a development strategy during planning. We'll do X now and Y next sprint.
• We skip detailed planning, because it won't survive first contact with reality.
• We agree on an acceptance environment with the customer during planning.
• We keep infrastructure work separate from new feature development.