Friday, May 1, 2015

On Belief, Honesty, Integrity, and Predictability

"It ain't what you don't know that gets you into trouble; it's what you know for sure that just ain't so." —Mark Twain

I wish I could have attended the Craft Conference 2015 in Budapest, Hungary last week. Not only does Budapest look gorgeous at this time of the year, judging from all the pictures they posted on the conference website and Twitter feed, but it would have been a great opportunity to make a pilgrimage to the birthplace of one of my favorite things to play with, the Rubik's Cube, which celebrates its 35th anniversary of being introduced to the world on July 26 this year.

As it was, I was busy trying to fix some loose tiles in my bathroom instead. My wife had been asking me to take care of these since the winter but until last week I had procrastinated doing anything about it, reassuring her whenever she reminded me that the material on which the tile was laid was some kind of special water-resistant board designed specifically for use in bathrooms and toilets. It can wait until warmer weather, I told her each time, which was about every two weeks or so.

Well, Mark Twain proved to be right again.

Ever since I read Kent Beck's white book about Extreme Programming, I have believed in the promise of Agile methods. And ever since I read Martin Fowler's book on refactoring at about the same time that I read "XP Explained: Embrace Change," I believed that TDD, or test-first programming as it was first known way back then, with its short feedback loops, ruthless refactoring, and collaborative development and design, was the best way to develop software. I still do, as a matter of fact.

And yet, what's most perplexing to me is that to this day and perhaps into the foreseeable near future, there are those who still don't and won't buy into Agile and/or TDD. I just don't understand this. Even notable figures in the industry like DHH and Jim Coplien, who are undoubtably very influential in shaping the opinions of many, have come out against TDD. I won't rehash all that here. You can follow the "Is TDD Dead?" conversations that Martin Fowler moderated and read the InfoQ article that looks at the controversy about TDD in detail. Listening to and reading the reasons behind the dissenting opinions, I was once again reminded of Mark Twain's epigram and left wondering whether or not something I knew (or did I?) to be true really wasn't.

I mention all this because I have been preparing to give a two-day workshop on TDD for some engineers in my group at work. I have a Prezi where I go though some of the ideas on which I base my practice of TDD and kaizen, citing articles and quotes from "Uncle Bob" Martin and others. I'm really more or less trying to channel Uncle Bob in this workshop, promoting the ideas of craftsmanship and professionalism and bringing back pride in the work that we developers produce.

This brings me back to #craftconf. Marty Cagan's keynote was really thought-provoking and I just want to thank the conference organizers and speakers for making the videos of their talks available to those who couldn't attend first-hand. Marty talks about how most of the companies that he works with have really no clue what agility is about, despite claiming to be Agile. He talks about a number of fatal flaws in the development process and rails against how companies are using roadmaps and backlogs to trick developers into believing that they were doing Agile, even though when you really think about it, it really ain't so.

Dan North added even more doubt to my mind by saying that backlog grooming should be outlawed. This was said almost in passing during his talk, "Beyond Features". Dan North, the guy behind BDD and doing TDD the right way. Luckily, his article, "The Perils of Estimation", sheds some light on his statement and dispels some of the doubt that it cast in my head.

Coming back to my bathroom tile issue, when I took out the loose tiles, I was horrified to see some black scum covering the back of the tiles and the board underneath. To make matters even worse, the board, which appeared to be plain old gypsum drywall, was soaking wet and crumbling to the slightest touch. I started to work off the surrounding tiles and soon I had two gaping holes in my shower, exposing the nastiness that had been festering underneath the shiny tile all this while. What's more, one of the holes opens up to an external wall and the insulation was soaked and had more of that black scum, which is likely some kind of mold or fungi, on it. Now I'm looking at some major repairs and a big chunk of change taken out of my rainy day savings. When it rains, it pours.

Just as my belief that the material under the tile would be protection enough against water damage got me in trouble when it turned out not to be so, it seems that some of the beliefs about Agile and TDD that I hold dearly and steadfastly defend may not be so after all either. At least not in every way nor all of the time.

I'm reminded of a recent conversation I had on JavaRanch and how people new to Agile might feel about the enthusiasm of proponents and, admittedly, often obnoxious way it sometimes comes across. Sometimes the nice shiny exteriors can hide the nastiness that lurks inside the walls. I may be guilty of contributing to this kind of disingenuousness in the past but I know I have recently made a conscious effort to empathize more with skeptics and dissenters.

Reflecting on my own beliefs and attitudes, I realize that I have gone off on my own little "rantifestos"—I will take the liberty of co-opting that word, too, Dan— on the Ranch as well, particularly when it comes to software craftsmanship.

Here are a few more things that I had to reflect upon after watching Marty's and Dan's talks:

I thought that backlogs were a good way to get the users and product owners involved and engaged in defining the work to be done and planning how we would proceed in delivering the most value quickly. I thought that grooming backlogs was a good way to get a common understanding between everyone on the delivery team and the product owner as to exactly what needed to be done and approximately how long it would take to complete the work.

I thought that TDD would help developers write better code. I thought that with TDD, the testers could get along better with developers and that they both could work towards the common goal of ensuring that the software had the level of quality needed to go to production without testers expressly focusing on finding where the developers had messed up.

Turns out that Dan North, Marty Cagan, and even DHH before them, made some pretty strong arguments to make me reconsider what I knew so surely to be true.

Yes, TDD can be taken the wrong way and turned into something that developers use to satisfy the strong desire of some managers to see metrics on productivity, quality, and code coverage. This as opposed to just measuring the lead time to delivering a viable product that solved our customer's problem.

Yes, it does seem that the business types have tricked us into accepting their Gantt charts disguised as product roadmaps and backlogs. And yes, we're back to pulling estimates out of our rear ends again and feeling guilty when we don't "meet" those estimates, or rather, "forecasts" which apparently are, in fact, the "new commitments." These behaviors are detrimental to our honesty and integrity as developers and responsible members of a project team.

And yes, after watching the video of Marty Cagan's keynote, I believe that this insidious push for predictability in our projects is jeopardizing developers' integrity, making them less inclined to be honest, and creating a vicious cycle that works against our belief that Agile, or rather true Agility is the best way that we can quiet the business types' incessant badgering about "when can we get this done?"

So as I get ready to pay my $500 deductible and perhaps whatever is above the limit on my home insurance policy endorsement for mold and fungi damage, I must also prepare to warn the developers who attend my workshop next week about the perils of taking TDD the wrong way. I must also prepare to talk to our senior leadership about the perils of taking roadmaps, backlogs, estimates, and metrics the wrong way.

Lastly and most importantly, I must now gird my loins against those inevitable looks from my ever-knowing-what's-going-on wife —you know which looks I'm talking about— and man up and say "You were right, dear, there really was something going on behind those tiles and I should have taken care of it when you told me to."

Mark Twain may have been right but he obviously wasn't talking about wives.