Thursday, March 17, 2016

The Cyclical Nature of Interconnected Things

In Up a Creek Without a Paddle: How Schools are Failing Novice Programmers, I talked about the importance in teaching basic form to novice programmers so they would be better equipped to deal with the programming challenges that they are given.

The question is often asked which of form or function comes first. Modern architects might tell you that form follows function. I would point out that once again, it is important to consider context. I think that form and function in the context of architecture is entirely different in the context of learning and skill acquisition. Comparing these to each other would be like comparing a church with riding a unicycle. When it comes to learning and skill acquisition, I believe that proper form is a prerequisite to good function.

In my last post, I cited the example of my friend and swim coach, Cecil, and how his good form and technique allowed him to be competitive as a swimmer. Cecil always emphasizes good form and technique when he teaches people how to swim and the positive results of his coaching speaks volumes to the effectiveness of his approach. If you follow that link, that's his daughter who is mentioned in the headline.

My experience in the Aikido dojo is no different. Centeredness and good posture are necessary for proper application of techniques. Sensei would often show us how a change in the angles of our feet or an ever so slight turning of our hips could make a big difference in a technique's effectiveness.

My son's viola instructor showed him how to use a bow properly to produce a better quality of sound. The difference that very small adjustments in his bowing technique made to the quality of my son's playing was immediately noticeable. This was the same instructor who told my son, "Practice makes permanent. Only perfect practice makes perfect."

That makes perfect sense to me and again attests to the importance of form before function in the context of skills and learning.

The Importance of Practice and Form

Malcolm Gladwell writes in his book, Outliers, that it takes about 10,000 hours of practice to achieve mastery. The joke about the musician asking for directions to Carnegie Hall and was instead given the stern advice to "Practice, Practice, Practice!" is a humorous affirmation of Gladwell's assertion.

I vaguely recall what one martial artist said about practice and form with regard to wielding a sword. I can only paraphrase from memory since I can't recall exactly who said it or how exactly how it was phrased. The gist of it was, "Practice by making 1000 sword cuts every day. You cannot do this without proper form. Only with proper form can you practice for that long and only with that kind of practice can you survive a day-long battle." 

Expert martial artists are living testaments to the 10,000 hour rule, as are professional athletes and anyone else who is an expert in their chosen field. The Dreyfus brothers will also tell you that mastery is specific to each skill. You can be an expert in one skill even as you have different levels of proficiency in others.

The relationship between form and function is not one way though. Neither form nor function are of much use without the other. It's more of a cyclical relationship really, much like how the chicken lays the egg and the egg produces a chicken which lays the egg and so on and so forth, ad infinitum.

When it comes to form and function, good form enables proper execution in practice, which helps one to improve on form, which leads to better execution and practice, which leads to better form, and so on and so forth.

Virtuous Cycles vs. Vicious Cycles

As the classic question with the chicken and the egg implies, one had to come first. Which one actually did becomes moot once a cycle is established and lasts longer than what memory or recorded history can reveal about its initiation. It doesn't really matter which came first at that point. It matters more that the cycle has been perpetuated.

The question then becomes whether the cycle will evolve or devolve. That is, does the cycle make things progressively better or worse?

A virtuous cycle is one that leads to improvement in the system involved, whereas a vicious cycle, of course, leads to increasing entropy within the system. In either case, acceleration often occurs as the cycle progresses in either direction.

The ultimate and logical conclusion of a vicious cycle would be the destruction or dissolution of the system involved. With a virtuous cycle, I'm not sure if there really is a limit. I suppose the limit to a virtuous cycle is only defined by what the system is able to support. If the system has unlimited capacity to support and engender a virtuous cycle, then I suppose that's how it would attain self-actualization.

As far as learning is concerned, I would think that the strength and speed of a virtuous cycle is dictated to a large degree by the amount and intensity of feedback that it receives from the system involved and its surroundings.

Feedback into a virtuous cycle can cause it to either continue to progress and accelerate or it could make it stall and start to devolve. Therefore, to maintain a virtuous cycle, care must always be taken to give it positive feedback. Any virtuous cycle can easily revert into a vicious cycle if it gets enough negative feedback.

The good thing about this is that it works both ways. A vicious cycle that has not yet reached its ultimate conclusion can still be stopped if given enough positive feedback. With enough positive feedback, a vicious cycle might also be turned into a virtuous one.

Maintaining Equilibrium

The hard part about all this is that it's much easier to get into a vicious cycle than it is to get into a virtuous one. Cycles can, at some point, be self-perpetuating if an equilibrium is achieved and maintained. Equilibrium in a cycle can be either dynamic or static. Vicious cycles, however, can easily be established and maintained simply through neglect and inattention. That is, you can start a vicious cycle simply by doing nothing or not paying attention to the system involved.

As far as learning is concerned, achieving dynamic equilibrium in virtuous cycle is ideal. This would mean that constant progress is being made and learning is happening continuously and at a steady pace. A static equilibrium would indicate that learning has stagnated and that the system is now only functioning off of current knowledge and understanding. A system in static equilibrium is basically on cruise control and maintaining that state depends on how stable the system is in its environment.

A system's environment must be considered because systems seldom exist in a vacuum and input to the system can come from its environment. Even small inputs from a system's environment can have a significant effect on its ability to maintain and sustain static or dynamic equilibrium.

Take for example the case of a team of software developers. George Dinwiddie blogged about the challenges of a new team member. Mark Levison writes more about the same. As far back as 1965, Bruce Tuckman wrote about the now familiar forming-storming-norming-performing  model of group development. Any change to a team, whether it's adding or losing a member will affect its dynamics. Any time a member is added or removed from a team, it becomes an entirely different team, regardless of how similar it was to its previous composition.

Obviously, a team is a very delicate system and one that is susceptible to all kinds of input and feedback that can quickly change the direction of its learning cycle, for better or for worse.

Again with the Aikido Analogies

I guess I'll always related these things back to my practice of Aikido. O'Sensei Morihei Ueshiba was known for shugyo or austere practice. In this article, you'll see a picture of him and his son, the second Doshu of Aikido, Kishomaru Ueshiba, practicing with jo (wooden staff) and misogi (cold water purification) at the foot of a waterfall. As you can see, even at an advanced age, O'Sensei still practiced with much intensity and focus.

O'Sensei was a remarkable man and his story is indeed a reflection of this. It is said that he achieved enlightenment and self-realization after an episode with an expert swordsman who came to challenge him one day in his dojo. The swordsman continuously attacked O'Sensei who, sensing the direction and speed of the sword cut, effortlessly evaded it each time.

This went on for some time until the swordsman because so exhausted that he had to concede defeat. All that time, O'Sensei had only eluded his attacks and yet was still able to prevail. O'Sensei was then said to have gone outside to a well where he sat down and contemplated what just transpired. In that moment of meditation, he perceived his connection to the entire universe. Thus occurred his transformation and enlightenment and the start of his teaching of a higher form of budo, or martial way, in Aikido.

O'Sensei practiced practically every day. His last practice session, according to the Wikipedia page on him, was on March 10, 1969. O'Sensei died suddenly on April 26, 1969. I was only a toddler at that time and completely unaware of the lasting effect his life, work, and teachings would eventually have on me.

I do draw inspiration from him and my other teachers, both on and off the mat, whether it's through direct instruction or vicariously through their books and the examples they set for others, who pass on that knowledge and experience. When it comes to software development, I draw from the experience and inspiration of pioneers like Kent Beck, Martin Fowler, Uncle Bob and other agile software development gurus. I follow examples of their dedication to the practice of the craft of software development and their continuous search for improvement.

I am thankful to all these sensei (one who has gone before) for sharing their knowledge and passion for what they do and I will continue to show my gratitude by following their examples and always trying to give positive feedback to the virtuous cycles that they started.

Wednesday, March 16, 2016

Up the Creek Without a Paddle: How Schools are Failing Novice Programmers

There seems to be no shortage of things to gripe about when it comes to how schools are teaching computer programming these days. Today was no different. In this post, I will talk about how schools are failing to develop the right skills and habits in students as they learn how to program. I will also explain what I think needs to be done about this problem.

With the way I've seen instruction being structured, I think many students who aspire to become professional software developers are getting off to a bad start. Even if they don't intend to pursue a career in software development, they are still not getting much of a chance to learn how to write programs properly given the way teachers are, in my opinion, focusing too much on the "how" rather than the "what" and "why" of programming.


I'm going to use the expression "up a creek without a paddle" to describe the situation many students find themselves in these days in their computer programming courses. If you're unfamiliar with that expression, then perhaps you're more familiar with "stuck between a rock and a hard place." 

Granted, students are not really put in such difficult positions as to navigate fouled up waterways without any implements. They are not made to endure anything remotely close to the harrowing ordeal that Aron Ralston survived only by amputating his own forearm five days after it was crushed by a boulder (rock) against a canyon wall (hard place) while he was out hiking alone in a remote area in Utah. The analogies are in the way students seem to be ill-prepared to meet the programming challenges they are given by instructors.

I think the sense of helplessness these expressions convey and Aron Ralston's true-to-life story are great analogies for how students feel when they are put at a terrible and unfair disadvantage because teachers neglect to give them good basic tools that could help them get through their programming problems in less time and with less confusion and frustration.

Lessons in Teaching taken from a Swim Coach

Speaking of teachers, I have a friend who teaches young children how to swim. Cecil and I are from the same hometown in the Philippines. He used to be a competitive swimmer but the interesting thing about that is that he's not very tall. Height is obviously an advantage when it comes to competitive swimming. I would say Cecil is a just a tad over five feet on a good day. I may be slightly off with those specifics but that's not the point.

The point is that despite his height disadvantage, Cecil was able to stay relatively competitive in his youth by developing a very efficient technique. He would literally glide through the water with very little effort. Cecil was competitive because he had really good swimming form. Of course, having a competition-sized swimming pool at home didn't hurt either. 

Cecil's family, or I should say clan, collectively owns the regionally famous Timoga Cold Spring Pools which are comprised of a dozen or so swimming pools that are fed by clean, cold, crystal clear, and constantly flowing spring water. Each family in their clan owns a portion of the land through which the spring water flows. They practically have money overflowing in their front yard and they have capitalized on it and continue to prosper from it to this day. Obviously, Cecil was able to swim every day so he also developed great endurance to match his excellent form.

When I first met Cecil, he was teaching my then girlfriend and future wife how to swim. For a late learner, she did all right. What struck me about Cecil's teaching method was that he really emphasized good form. He would have my wife and her classmates practice in the shallow end working on lifting their arms above the water just enough to skim the surface with their fingers and then cupping their hands and pulling, then pushing the water under their bodies below the surface. 

He would then have them work on rolling their shoulders just right to make their arm movements more efficient and comfortable. Then he'd have them work on their flutter kick so that, too, would be efficient and require just enough effort to stabilize their bodies and keep them at the right angle in the water so that they created as little drag as possible.

I even improved my swimming technique after trying out what Cecil was teaching, so much so that I was able to keep up with him a little bit while he would swim laps after my wife's swimming classes. Of course, it turned out that he was just maintaining a slower pace for my sake, but still, I did all right for a non-competitive weekend swimmer.

Cecil now coaches for a swim club in Cincinnati, just a couple of hours down Interstate 71 and we still occasionally see each other or call to say "Hi." He stays on the dry side of swim competitions now, still teaching kids the right way to swim and get efficient at it. I hear the kids he coaches in swim club do pretty well at competitions. His two children have carried on their father's legacy, being also somewhat small in stature but big in form and efficiency as well.

Which comes first, Form or Function?

Cecil's teaching method is in sharp contrast with what I have observed from students asking questions on JavaRanch as well as what I've seen with my son, who is taking "Introduction to Computer Programming in Java" at Ohio State University this semester. It's frustrating to see my son's slow progress and the poor habits that he, too, is starting to form in just a few months of studying programming at a college level.

Rather than first teaching students basic form and good coding habits, instructors seem to be asking students to plunge right into the nitty-gritty details of the language, giving them problems to solve without any idea of a good, sensible, and systematic approach for doing so. 

This "sink or swim" approach to teaching, or so it seems to me, does not appear to be limited to students in introductory courses. Apparently, those who go on to study more advanced programming concepts are still sent off without a proverbial paddle into a creek where the current is getting swifter and more challenging to handle.

Take JavaRanch greenhorn and confused student Jason Ram, for example, the original poster in this thread on JavaRanch about calculating Degrees of Separation from Kevin Bacon. Go check out that thread first, so you understand where I'm going with this next.

Jason said that this is his second semester studying programming and that the tips I gave him on basic program organization was more than he had gotten from his instructors in two semesters. It took me less than fifteen minutes to compose the pair of responses I gave him and for him to say that it was more than what he'd gotten in two semesters is quite appalling and disappointing.

Functional decomposition is one of the first basic steps towards managing the complexity of both the problem as well as the solution to that problem. Without properly breaking down the problem and solution into smaller component parts, you end up with a confusing jumble of unreadable code similar to what Jason posted as his working solution. In this case, it was around seventy (70!) lines of code jammed into the static main() method. 

How can this be acceptable for a student who is well into his second programming course? Don't get me wrong, I'm not faulting the student here. It's the teachers who I think are to blame for this, for not teaching the students about code hygiene and basic problem decomposition and organization.

The Fountain of All Evil Programs

This is the never-ending source of the sloppiness and cruft that I have seen in a lot of code written by developers in the real world. Code that was seemingly written to just get it done, without much regard to cleanliness and proper design and organization. I often see code written in Java, an object-oriented language, that can't even pass as good procedural code, written by people who are paid to write enterprise-class programs. It's horrific!

It's like the ever-flowing cold spring water that feeds the dozens of pools in Timoga. Only this water is murky, foul, and unpleasant. Nobody wants to touch it, much less swim in it. And it's the developers who know better who have to go and clean it up.

There seems to be an abundant amount of bad code constantly flowing into our applications and stagnating there, creating a stench that at times becomes so unbearable that organizations have to spend enormous sums of money to clean up. Take a fairly recent poster child for messed up projects, That is a failure on so many different levels, not the least of which is bad code. And we all had to pay for that cleanup, one way or another. 

Pity the fools who had to get their hands dirty with that awful code that was the first few incarnations of This gives a good visualization of just how ugly it was. But then again, some of those people who were brought in to clean up the mess probably made off like bandits afterwards. Again, that's not the point though. Overall, there were very few winners and many losers in this fiasco. Thankfully, enough was done to get it out of the sewer and into a usable state.

You Don't Have to Look Far To Find Sh*t Code

You don't really have to go far up Sh*t Code Creek or explore the recesses of Conundrum Canyon to see how professional programmers can create labyrinths in their programs that take months of laborious effort, patience, and perseverance to escape, in such epic proportions that would put Aron Ralston to shame. Heroic and monumental efforts often have to be made to save poorly written programs so they can survive further use and abuse by increasingly demanding users and adopt to ever-changing business needs.

All it takes for most developers to find themselves eyeballs deep in the muck of bad code is to open up their current project and go to some code that they didn't write. Or even some code that they did write just a few days or hours ago. What's worse is that many programmers don't even realize they're standing in cr*p code and that they probably contributed to the mess. It's all done in a day's work and as long as the lights stay on, everything is smelling roses. 

Well, everything isn't smelling roses. They just don't know how deaf their noses are or have become. Or maybe it's the smell of roses like that at a funeral home, that masks other more unpleasant and morbid smells. Whether it's hubris, delusion, or ignorance, too many developers wallow in muck day in and day out and still don't seem to be bothered too much to do anything about it.

This is why I spend most of my time as a technical leader doing code reviews and teaching basic hygienic coding and refactoring. I have to teach programmers about basic design principles, how to recognize and fix code smells, design smells, test smells. Sure, it's fun seeing lights go on in people's heads but we have a job to do, things to get done. Business is asking us to do things faster. Oh, and by the way, could we also make it better while we're at it? Sure, right on it, boss (with excessive amount of sarcasm dripping).

Forgive Them, For They Know Not What They Do

I guess it's fitting that I write this during the season of Lent, when forgiveness, understanding, reconciliation, and love are things we should be meditating upon and practicing more consciously and deeply.

I stopped seeing programmers who write bad code or do nothing about it as evil a long time ago. I just found that this is not a useful attitude. Sure, I still resent having to clean up the slash and burn mess that itinerant consultants leave for me when they move on to other projects but I no longer want to hack into their social media accounts and deface their profiles. Or find their brand new imported cars at their new places of work and leave a poignant message with my keys. 

As cathartic as that may be for a few moments, besides the fact that it's illegal and potentially life changing for me, it really serves no useful purpose for anyone. The rest of us are still left holding the bag and looking at a huge refactoring and cleanup effort, with more of what's left of our gray hairs and gray matter sitting on death row, waiting to die from frustration and exasperation.

For our Colleagues in Academia

We in the industry need to start reaching out to our colleagues in academia and make them aware that they are creating problems for the rest of us out here in the real world with their outdated and appallingly ineffective teaching methods.

We need them to include instruction on basic coding form, coding hygiene, design principles, and design thinking. We need instructors to teach students about automated testing and encourage them to collaborate with each other instead of stubbornly enforcing strict rules against plagiarism and academic misconduct. Sure, we need to ensure integrity and proper learning and see to it that work is done honestly and by the individual to whom credit is being given. But we also need to balance that with realistic approximations of how these young people will do their work when they eventually join the workforce and get paid to do a job.

New programmers need to be taught that there are higher ideals in software development. They need instructors who will help them understand that software development is more than just writing classes, objects, methods, functions, loops, variables, data structures, algorithms, and just getting something to work. 

Programming is still as much of an art as it is a science. Programming is a creative process. It's a way of organizing your thoughts and telling a clear, coherent story in a language that not only computers can interpret and execute in a way that's congruent to what we intended it to be executed. Programs also need to be written in a way that its human author and other humans who are interested in sharing the story can easily understand and wrap their heads around it. Programmers are authors and it's high time we start living up to that title and teaching new would-be authors the ins and outs of the craft.

We need our colleagues in academia to do this from the very start, when students are still clean slates and have not yet formed any lasting habits, good or bad. Let's choose to do good by them and help cultivate in them the things that will make them better authors of software. 

Let's stop sending them up the creek without paddles or into remote canyons with only a cheap multi-tool and a dull two-inch blade in their pockets. Let's take a cue from my friend, Cecil, and teach them how to swim with proficiency and watch in wonder and joy as they start learning how to cut through the murky waters of software development with effortless efficiency and graceful elegance.

For our Colleagues in the Trenches

We in the industry have our work cut out for us as well. 

We need to get our heads out of our private and public clouds. We need to step out of our nice, shiny, new containers. Let's take a minute and put down all those nice, shiny new toys we're all eager to play around with and look around us long enough to notice that there are kids out there who are literally struggling to just get even a simple class to work properly. They don't even know how to use JUnit to save their lives!

We need to start rehabilitation programs for disadvantaged programmers who came out of school saddled with poor coding habits. Many of them lack the knowledge and understanding of basic frameworks of thinking for tackling complex problems in the ways that we professional developers—and by "professional" I don't just mean getting paid to write software, I refer to those who practice good coding and design and adhere to software craftsmanship principles and values—have learned and refined through years of experience and learning from mistakes. 

Let's not sit idly by and watch these young developers go through the same kind of pain that we had to go through in our day and still often do to this day. They are our colleagues now and hopefully someday we will be happy and proud to call them our peers. But we need to get involved and engaged with them for that to happen. As I said in this post about serendipity and in the one before it about general programming guidelines, we can't just leave it to luck and we can't just send the kids out there ill-prepared to overcome the challenges that they will inevitably have to take on.

Let's help them learn by sharing our knowledge and experience. Let's share and help them experience the joy we feel when we know we've done a good job. Let's help develop the love for craftsmanship and professionalism in our colleagues and soon-to-be peers. In that way we can all change the world for the better. That's a goal worth going up the creek for or cutting off a limb. But let's hope it never comes to that.

It's All A Big Circle of Aiki To Me

I'll close this with a thought about the Aiki arts and how all of the above ties in with the philosophy and teachings I have received through years of study and practice in the dojo. 

As I explained in my Spring cleaning post last night, Aiki is the root of Aikido, which means "the way of love and harmony" or so I've come to take it to mean. There are other, more literal translations of course, but I think I have read and heard enough of the founder's thoughts and have practiced the art long enough to grok that it really refers to the ideas and ideals that the words "love" and "harmony" encompass.

That, in essence, is what I'm calling for. We need to strive for ai nuke or mutual preservation. Competition is good. It motivates us to get better, to hone our skills, to push the edges of our abilities, and to soar to greater heights. But we can't let competition get in the way of progress either. Competition is good when it's beneficial for everyone. It's not so good when there are more losers than there are winners.

Don't get me wrong, I'm not saying that I subscribe to the idea that "everyone is a winner" and that we need to start giving out "participation medals." I personally think that's counterproductive in the long term. Good Aikido instructors will tell their students, "Good! Good! But this is how you can make it better…"  We need to encourage, not coddle. We need to lift up, not prop up.

Many practitioners of Aikido and martial arts with similar philosophies believe that if everyone subscribed to the teachings of O'Sensei Morihei Ueshiba and other like-minded martial artists, we would be well on our way to achieving world peace. That was the founder's ultimate goal in creating Aikido. 

Morihei Ueshiba grew up a warrior in a time and culture where the warrior mentality and spirit of ai uchi (mutual destruction; winning at all costs including, if necessary, sacrificing your own life while taking your enemy's) was still very prevalent and was the accepted and expected ethos.  

O'Sensei saw and experienced war and the horrors that men can inflict on other men. He wanted to change that. Creating Aikido was his way of changing things for the better. He took brutally effective techniques that had been tried and tested on the battlefield and adapted them to be more humane and life-preserving. We need to foster the spirit and way of ai nuke, or mutual preservation. There are now millions of people all over the world who study Aikido and walk that way.

You don't have to become a student of Aikido or study martial arts. Just be aware that this is where we are coming from, those of us who are. That's our goal and that's why we do the things we do the way we do them.

The principles and values of peace, love, and harmony are universal and touch all people and things. You can join us on this path, if you choose to do so. There's plenty of room on this road for those with similarly-aligned ideals and goals, no matter what their flavor or manifestation.

You might have noticed that there are many links in this post. Some of those links lead to related topics that I mention in the new blog description. If you follow them, you might start getting a sense of the kind of interconnectedness I see when I write about the things that I do. It's all a big circle of Aiki to me.

Peace be with you in this season of Lent.

Tuesday, March 15, 2016

A Fresh Coat of Paint for the Jong

Spring is just around the corner so I thought I'd do some early Spring cleaning and freshen up the look of this blog a little bit. You may have noticed that the banner image has been spruced up. I brought back the colored version and with the help of Gimp and a couple of filters, added some effects to reflect dynamism, circularity, and enlightenment through practice. I was really pleased with how it turned out.

I also edited the blog description to give a more accurate sense of what I'll be writing about here. I realize the description is a bit long but I also wanted to give a sense of the variety of topics as well as the eventual connectedness of everything I'm posting. I think the description covers all of the articles I've posted here so far.

Some of the terms up there may not be familiar to everyone so I'll give a rundown of what all these mean in the context of this blog.

Explanation of Terms

Patterns.  This will usually refer to software related concepts like design patterns, coding patterns, test patterns. Occasionally, I may write about learning patterns, organizational patterns, and behavior patterns. Anything else, of course, will just fall under the Merriam-Webster definition, which is "something that happens or is done in a regular or repeated way."

Metaphors. I love metaphors and analogies. People often tell me that I have a knack for making them. I guess I'm all right. I like using them because they help me make connections between different aspects of my life and experience. Metaphors help give me a sense of purpose and destiny, as though things were not just happening randomly. I guess it's just human nature to look for some kind of meaning and direction in things. It's too scary to think that we're just bouncing around in an infinitely large pinball machine that is the universe, completely at the mercy of chance and circumstance.

Agile Software Development. If you're involved in software development in any way and haven't heard about agile software development by now, you've either been living under a rock, in a coma, stranded on a desert island, or abducted by aliens. If you've wandered in here accidentally and are still reading this out of curiosity, agile software development is a way of developing software that values individuals and interactions, working software, customer collaboration, and responding quickly to change over most other considerations. 

You can read more about it in the Manifesto for Agile Software Development. As of this writing, that manifesto is just a tad over fifteen years old.

Craftsmanship. This, I believe, is a natural consequence of the kind of thinking that's involved in developing software with agility. You can also see it as a pre-requisite to agility. Whatever your perspective, software craftsmanship also has its own manifesto which you can find at

I have signed both this manifesto and the one for agility. Software craftsmanship means committing to writing well-crafted software, steadily adding value, being a responsible member of a community of professionals, and developing productive partnerships.

ShokuninClosely related to craftsmanship, shokunin is a Japanese word that roughly translates to "craftsman" or "artisan" but it encompasses more than that. Some of the essence of the word as it is used in its native context is often lost in translation to English, similar to how the Martian word "grok" cannot be expressed adequately in any of our earthly languages by words like "understand", "intuition", and "immersion". I get a sense that "craftsman/artisan" is to shokunin as "understand intuitively" is to grok.

Kaizen (改善). Wikipedia defines this Japanese word as "improvement." In this blog, I'll use kaizen in the same way that the folks who subscribe to lean thinking and methods use it. The idea revolves around groups of people always doing things that will lead to changes for the better.

Zanshin (残心). This is a term used in Japanese martial arts and refers to a state of relaxed awareness and having a mindful connection to things in your surroundings. I guess the ultimate level of zanshin is when you become aware of your connection to the entire universe, as was said to have happened to O'Sensei Morihei Ueshiba, the founder of Aikido. I have been studying Aikido since 2004 and have found many parallels between its philosophy and the other things I write about here.

Genchi Genbutsu (現地現物). Another Japanese term you can read more about in Wikipedia. It means "go and see" and it's popular with the lean folks, who adopted it from the Toyota Production System.

Shugyo.  This term again comes from Japanese martial arts and it refers an "austere training" that serious practitioners undergo to develop their skills to the fullest. You can read more about it here:

Intentional Practice. Related to shugyo, it refers to practice that is done purposely in contrast to experience that is gained incidentally, accidentally, or through actually doing something. Intentional practice is different from on-the-job training. It's more like going to spring camp so you can work on getting better at specific skills without the pressures of being in actual game situations. 

For software developers, intentional practice is exemplified by things like going to code retreats and coding boot camps. I find it kind of ironic that doctors and lawyers call what they do as their "practice." On the other hand, we software developers call a lot of the things we do the same thing. Kind of scary when you think about it.

Aiki (合). This forms the root of the name of the Japanese martial art of Aikido (合気道) which was created by its founder, O'Sensei Morihei Ueshiba. It has roots in Daito Ryu Aiki-jujutsu, which is very effective but more suited for the battlefield. Aikido practitioners aim to be gentler and more humane in applying the same kind of techniques and strive for the higher ideal of mutual preservation (ai nuke) rather than destruction (ai uchi or mutual kill).

Random Connections. Finally, I mention this in the blog description because I find many similarities and relationships between Aiki philosophy and the things I do and the way I think when I'm doing software development. I feel that making these connections deepens my understanding of each of them and this allows me to appreciate them more fully.

And finally,

Jong. Short for muk yan jong, this refers to the wooden dummy that is shown in this blog's title banner. You may have seen the jong being used as a practice aid by Chinese martial artists. Its use in modern times was popularized mostly by practitioners of Wing Chun and Bruce Lee's Jeet Kune Do. I use it here to give a sense of connection between the ideas of intentional practice to sharpen my software development skills, shugyo, and my training in Aikido and related Aiki arts.

What's Still to Come

As I mentioned in a previous post, I have finally started working on a book about test-driven development. I am also eagerly anticipating word from the Agile 2016 conference program selection committee on whether they are going to accept one or both of my session proposals about TDD. If by some stroke of extremely good fortune that happens, then I will be posting more articles related to the TDD book and the presentations. In the meantime, I'll be keeping my fingers crossed until the end of March.

Sunday, March 13, 2016

Pragmatic Thinking & Learning: Serendipity, Revelation, and Validation

I've been thinking about context a lot lately. In my previous blog post, I wrote an entire section about context and the lack of it in traditional introductory computer programming instruction.

Yesterday being a Saturday, I was trying to score some much-needed points with my wife by cleaning up the room above our garage that also serves as my home office. While doing that, I noticed Andy Hunt's book, Pragmatic Thinking & Learning: Refactor Your Wetware on one of my bookshelves. It had been sitting there just collecting dust since I bought it from a discount book store a few years ago. Somehow, I had never gotten around to reading it.

Since I had recently posted an article about learning, I thought now might be a good time to see what the book had to say about the learning process. As I started flipping through it, a realization soon dawned on me. Before I tell you more about that, I need to take you back a few years to set the context for what it was that I realized from skimming over the first two chapters of Pragmatic Thinking & Learning.

It was right around the end of 1999…

A series of fortunate events and the unfortunate one that started it all

I remember reading Kent Beck's Extreme Programming Explained: Embrace Change just a couple of months after its first printing in September 1999. I quickly became enamored with the ideas Kent presented in his book. It all made sense to me when he described the problems he had seen on software development projects and explained how the practices of Extreme Programming (XP) addressed them.

Kent's book resonated with me because I had also experienced many of the problems he described. At the time, I was well into my second year on a multi-year, multi-million dollar project that would eventually shut down in the middle of 2000. I would move on a few months before that but there were others who were not so lucky.

That wasn't my first project ever—I had about eight years of development work experience when I joined the project—but it was the first one of that size and the first one that I was assigned to after arriving in the United States with my young family in late 1997. The people I worked with on the project were great but the project was not. It was, in fact, the worst project I've ever been on in terms of overall experience and outcome, bar none.

The experience of having been on a failure of that magnitude had lasting effects on me. Fortunately, so did having a lifeline of hope for better things as described in Kent's book. It was during that difficult time that I resolved to find a job where I could do Extreme Programming or work in a very similar way. That resolution has made all the difference for me.

An Awakening to Better Things and Better Ways

It was also there in the pages of XPE, as Kent's book would soon be known, that I first read about automated unit testing and "test-first programming," a term that would be replaced in a few years by what we commonly know it as today: test-driven development or TDD. Although the concept was new to me back then, it made a lot of sense and I wanted to try it as soon as I could.

As is often the case on projects that are in trouble, informal discussions about software development practices had, ironically (or maybe naturally), become quite a regular occurrence. During one such discussion, one of my colleagues, Mark Halloran, suggested that I read XPE. He also suggested that I read Martin Fowler's Refactoring: Improving the Design of Existing Code since it was kind of related. So, I bought both books and read them.

Mark and I have continued to stay in touch over the years but he probably doesn't realize how much he changed my life, both on a personal and professional level, with those passing suggestions for further reading. If you're reading this, Mark, thank you. I owe a lot to you.

As I read the first chapter of Fowler's Refactoring and worked my way through the long (50 pages!) first example that introduces the basic ideas of refactoring, something happened to me. It was nothing short of an epiphany. I'm not just saying that for effect either because "something clicked in my head" just doesn't come close to describing it. It was more like the recent "mind blown" commercials where I just sat there, amazed and awe-stricken.

I can't remember exactly what I said back then but I'm sure it was out loud and something along the lines of "Oh my God! I've already been doing a lot of this stuff but I've never put them all together like that!" By "like that!" I would have been referring to the purposeful and methodical way in which Fowler transformed the code and improved its design with a series of small, incremental changes.

To see the code and design literally morph in front of me was, at that moment in my career as a programmer, a revelation of a new and endless stream of possibilities. It was like a veil that I never realized was there before had suddenly been lifted and I could see everything much more clearly. At that moment, I think that I felt that I had really grokked what Fowler was doing and it was mind-blowing.

Things that I know, they know, and then some

That first refactoring example also made me feel validated as a programmer. As I said, I had already been doing a lot of the little things that Fowler showed. Many of the refactoring techniques looked familiar to me. At least the scenarios to which they applied were. And yet, when seen in their entirety, the techniques made me look at the things that I did in a totally different light. Refactoring clarified many things for me and opened my mind to many more possibilities.

It felt good though, to know that I wasn't too far behind on the curve when it came to good programming habits and that I was on the right track with the kind of things that I was already doing. I had just fallen short on scope, method, and purpose.

Whereas I usually made the same kind of little changes in isolation, Fowler's examples showed me how those changes contributed to a bigger picture. Whereas I was making changes by referring to individual notes on best practice floating around in my head, Fowler clearly explained his motivation for making each change, the ramifications on the overall design, and the systematic manner in which he fluidly and seemingly effortlessly moved from one change to the next.

Previously, I made code changes that were based on vague notions of purpose drawn from experience and guided by intuition. I usually went by feeling and my methods were haphazard and relatively clumsy at best. Fowler showed me how I could do much better and apply the changes he described with a clarity of purpose, depth in understanding. and a systematic method of execution.

Where previously all I had were a bunch of random dots of knowledge and experience in my head, Fowler showed me how to connect all those dots so I could see the big picture.

I was thoroughly impressed. The man was obviously a minor programming deity at the very least and I could only aspire to approach that level of skill when I grew up as a programmer.

Common Experiences and Amazing Serendipity

Others who read these two books around the same time must have felt much the same way. In fact, the very first Java Users Group meeting that I attended—and I have to credit Mark Halloran again for taking me to it—was also around that time, in Cincinnati, Ohio.

The featured presentation in that CinJUG meeting was about refactoring and automated testing. It was given by a pair of presenters, one of whom was the late Jim Weirich, who would eventually go on to author the Rake build tool and help create the current RubyGems package manager.

At that time, Jim was working as a consultant for the Compuware Corporation, the same company that I would join just a few months later. Jim's presentation and his enthusiasm about refactoring and automated unit testing was infectious and it strengthened my resolve to learn and get good at these things.

I still think back and wonder at the sheer serendipity of everything that happened to me because, what are the odds, right? Were these all just strange coincidences? Fate? Destiny? Divine intervention? I don't know.

Maybe it was all just pure dumb luck but it seems like too many things fell into place for me just at the right times and right places. But then again, maybe that was my one big jackpot of a lifetime. Maybe that's why I haven't been able to win big in the lottery: I already got my big win at Lady Luck's wheel.

Déjà vu all over and over again

So now we're back to this past Saturday morning with me dusting off Andy Hunt's Pragmatic Thinking & Learning: Refactor Your Wetware and getting distracted from my cleaning duties. The point-making with the wife would just have to wait.

Again, I don't remember ever reading this book since I bought it a couple of years ago. I will often just buy books with the intent of reading them later when I have some free time. I have literally over two hundred books on three bookshelves in my home office and another hundred or so more in the basement. It's normal to see a few of these books strewn around in my home office. I go through a lot of wife points because of this, hence the need to gain some back by cleaning up a bit once in a while.

I won't even tell you about the stacks of magazines that I have around the house and the hundreds more electronic books that litter my hard drives. Let's just say that my intended reading list is quite long and it would be pretty optimistic to think that I'll ever get around to checking off everything on it.

I often tell my wife that I'm just getting all these books for future reference when I need them, not really to read them from cover to cover. She just shakes her head and gives me a knowing look. If you're married, yeah, it's that look.

Anyway, I was trying to see if there was anything in Pragmatic Thinking & Learning that I might be able to reference in my presentations about test-driven development. I have been giving presentations about TDD pretty regularly over the last few years and I'm working on doing more in the near future. It was on page 6 that I stopped and started getting a feeling of deja vu.

Here's what I saw:

Tip #1: Always Consider Context.

Immediately below that, Andy writes, "Put a copy of that up on your wall or your desktop, in your conference room, on your whiteboard, or anywhere you think alone or with others."

Ironically, if you follow Andy's advice, you'll probably have people looking at it and asking you, "What's this thing with 'context' all about? Where are you coming from and where are you going with this?" More on that later.

Anyway, when I read Andy's Tip #1, I thought, "Wow! I just wrote about context a week ago. That's nice." I thought it was a nice coincidence that I had been thinking about context and now find that Andy Hunt had been thinking along the same lines when he wrote the first chapter of his book.

Skimming through some more, I saw these section titles in Chapter 1:

  • Where We're Going
  • Journey from Novice to Expert
  • This is Your Brain
  • Get in Your Right Mind
  • Debug Your Mind
  • Learn Deliberately
  • Gain Experience
  • Manage Focus

Suddenly, I thought I heard the universe faintly calling out to me. "Wow," I thought again, "some of these are the same things I have in my TDD presentation!" The brain, how it works, how it's often wired differently from what it needs to be for TDD, retraining your brain, learning deliberately (I'm borrowing J.B. Rainsberger's term, "intentional practice"). These are all topics in the latest rendition of my presentation on TDD.

Flipping back to page 6, I see a highlight quote above Tip #1 that says, "Everything is interconnected."

You don't say, Andy. I think you and I might have interconnected somehow with these ideas.

Flipping back to page 1 of the chapter, I read "Welcome! Thank you for picking up this book."

WTH is going on here?! Did I actually read this book before and somehow forget that I did?

Quickly, I go over to my computer and bring up tmux and the session where I have the beginnings of my book on TDD. I finally signed up on LeanPub on Friday to start writing this book. I had worked out a draft of the introduction chapter and I had settled on a lead off line that says, "Thank you for your interest in this book. It wasn't easy, but I finally managed to gather years of…" Looking through my earlier commits in git, however, I see that the first few revisions started with "Thanks for picking up this book."

Now I'm looking around to make sure I'm not getting punked somehow. Is there some subliminal message coming out of the TV? Maybe I'm just losing it and having a very senior moment here.

I can't recall ever reading this book before but it seems that a lot of the things that have been on my mind lately are also in there. None of the illustrations look familiar but the topic headings are spot on with the things I've been writing down recently. I really doubt that it's a lapse in memory on my part but then again, I can't be 100% sure.

So I go on to Chapter 2 - Journey from Novice to Expert.

Mind Blown, Again.

Chapter 2 appears to be all about the Dreyfus Model of Skill Acquisition. The similarities continue because I mentioned Shu Ha Ri in my last blog post. I may have subsequently edited it out but it was there when I first posted the article. The Dreyfus Model and Shu Ha Ri are pretty aligned with each other so again, my thoughts seemed to be eerily in synch with Andy's.

Then, on page 30, I found this:

Tip #3: Know what you don't know

Oh. My. God.

That is, word for exactly freaking word, what I have on one of the slides in the presentation on TDD that I'm preparing for the Agile 2016 conference! I don't yet know if my submissions will be accepted—the program selection committee won't inform successful submitters until later this month—but I'm still polishing up my material regardless. What I do know is that I was referring to the cluelessness that many developers have about basic coding and design principles.

In my presentation, I talk about "Awareness" as being a key to success in TDD. Awareness leads to recognition and sensitivity which leads to understanding. Having awareness means that you need to know what you don't know.  I then go on to list some of the important things developers should at least be aware of when they are doing test-driven development: DRY, SOLID, SLAP, GRASP, Law of Demeter, Principle of Least Astonishment/Surprise, code smells, design smells, test smells, simplicity, etc.

If you don't believe that I had this before I saw Andy's Tip #3, I have recorded a number of practice runs of the presentation with Keynote and can show you the files. I swear, I did not have prior knowledge that this was in Andy's book! You know what would be really freaky though? It's if it turns out that Andy was thinking about that infamous Donald Rumsfeld news briefing when he wrote that tip. Because that's what I was thinking about when I added that slide to my presentation.

In his book, Andy's Tip #3 about knowing what you don't know is given in the context of skill distribution and the tendency of practitioners at the lower skill levels to overestimate their own abilities. Andy writes that "metacognitive abilities, or the ability of being self-aware, tends to be possible only at the higher skill levels."

Wow, novices seem to have everything stacked up against them. Not only are they oblivious to what they don't know, they also need to get better to become self-aware. But to get better, they need to be more aware. Now they're in a Catch-22! What's a clueless novice programmer to do then?

Yes, It's All Interconnected…

I think one of the reasons that Pragmatic Learning & Thinking is resonating so much with me is that it reaffirms the point I made in my last blog post regarding context and its importance in teaching programmers how to write good software. 

Without context, it's highly unlikely that novice programmers will recognize the connections between their choices in writing software and the related principles and values of good software development practice. As long as there's a disconnect there, they will continue to make poor decisions and stumble around in the dark. More mistakes will be made as a result. 

In his book, Andy explains how the Dreyfus Model of Skills Acquisition says that novices need context-free rules so that they can still effectively complete tasks. Unlike more experienced developers, novices need recipes to follow since they don't have much experience and intuition on which to base their decisions. 

Ok, fine. The scientists who said that were a lot smarter and probably knew what they were talking about, so I guess I'll have to take their word for it. But then again, maybe I'll do that with a few grains of salt.  

I still think that novices need to have an awareness of context, even if they don't fully comprehend what they see or are able to discern the extent of it and how it relates to the things that they are doing. 

Awareness will eventually lead to recognition and sensitivity. This in turn leads to understanding. Understanding is the basis for learning. 

I submit, therefore, that learning can only start when novices have some measure of awareness of context. Even just a little bit. Having recipes to follow may be fine for them in the short term but this can also make novices too dependent on having context-free rules. You've probably seen them before, the ones who seem to just blindly follow "best practice" and conventions without exception or questioning whether it's really the best thing to do in all cases.

If they're not careful to apply critical thinking to their recipes, novices who are too dependent on context-free rules can become like people hooked on drugs. They are no longer able to function without them. They can exhibit withdrawal symptoms and become confused and disoriented. We need to help wean novice programmers off of their context-free rules and challenge them to start recognizing the interconnection of parts in the larger context.

The Unnoticed Scenery Metaphor: We Can't Assume or Leave it to Luck

We can't just assume that novice programmers will eventually notice how things are interconnected within a context. I think we all know where wrong assumptions can lead us. We have to explicitly point out some of these contexts and interconnections to help novice programmers build up their awareness.

It's like when I'm out on a road trip with my family. My kids are usually sitting in the back with their faces in their phones. My wife and I often complain that they're missing a lot of the nice scenery that we drive past. 

Occasionally, my wife will give them fair warning of something interesting that's near the road up ahead. This gives my kids some time to stop what they're doing and look up to see what it is their mom thinks they'll find so interesting. They may still shrug and go back to their phones again, but at least they saw what it was their mom wanted them to see. At least they are now aware that there was something there, even if it was of somewhat dubious interest to them.

Novice programmers are like that, too. They're often too focused on the immediate details and seldom feel they have time to look up to see what's going on around them and elsewhere in the system. They fail to see the big picture while they're eyeballs deep in their work and just trying to get it done. This is how many opportunities are missed and mistakes are made.

Again, as I said in my last post, we need to help them out more or else they'll just keep flailing about aimlessly, with no definite movement towards growth and learning. We need to break their focus on the details that are often overwhelming them and ask them to look up at the big picture once in a while, so that they can start getting their bearings.

Not everyone will be as lucky as I seem to have been in the past. We can't just leave it to luck or assume that novices will see things when they need to see them. Luck shouldn't have to play a big part in learning and growing. I think Uncle Bob said something to that effect before. We need to make the path to learning more intentional and less serendipitous for novices and everyone else alike. We really need to do this. For everyone. I don't care what the Dreyfus brothers said.

Great(?) Minds Think Alike

I'm averse to using the word "great" to describe myself. It seems too presumptuous and a bit pretentious actually. I don't consider myself a great programmer. I guess I'm pretty good but I know there are many elite programmers out there who could run circles around me. While typing with their keyboards behind their backs. And playing an RPG in the background. While drinking Mountain Dew and… well, you get the point.

However, it is nice to know that once again, after all these years since I read XPE and Refactoring, I can still feel validated in the things that I've been doing to stay as good as I can be and keep improving my skills. 

I do know a lot about basic practices, principles, mindsets, perspectives, and patterns that I have used to keep my programs clean and myself out of trouble throughout the years. I've gotten to a point where many people are actually encouraging me to write a book on the stuff that I know, particularly about TDD, and share the joy when I'm doing TDD that I'm constantly talking about.

So, yeah, I've started on the book. I won't share what I have just yet, but I will eventually, when I'm good and ready. I'll probably have a few sample chapters up on LeanPub for public review and feedback in about a month or so. Hopefully, my submissions to the Agile 2016 conference will get accepted and I'll get to present some of this stuff in Atlanta come July. That should be a good learning experience, too.

Until then, I'll try to post more regularly on this blog to see what some of these ideas of mine look like on virtual paper.

Please stay tuned for more.

Thursday, March 10, 2016

General Programming Guidelines: Things They Should've Taught You in CS-101 but Didn't

As a bartender (moderator) at JavaRanch and as a senior developer and mentor to other developers, I often see programmers committing the same mistakes over and over. Although there's a wide variety of mistakes that can be made, they all have a few things in common, one of which is that they can usually be traced back to a lack of clarity and consistency in thought and practice.

Sure, rookies make mistakes and it's easy to just chalk those off to inexperience and not knowing any better. We've all gone through Shu, the first stage of the Shu-Ha-Ri learning process. The Shu stage is where all beginners start as they fumble around, not really knowing what to do or how to do things. They try different ways that, more often than not, turn out to be the wrong or the not-so-good way.

But I've seem many professional developers with years of experience make the same kind of mistakes. You'd think that after all those years on real-world projects, they'd know better. Obviously, there are many would disappoint you. We have to ask ourselves though, why are there so many programmers out there who don't seem to be learning from all the mistakes of the past?

An Explorer's Metaphor 

I agree with the idea that if you're not making mistakes, then you're not learning and growing. However, schools seem to assume that students will eventually figure things out for themselves. In the professional world of development, it seems like many of the leaders and managers assume that developers will naturally avoid doing things that are known to cause problems. We need to stop making those kinds of assumptions because we all know what happens to you and me when we assume.

Experience may be the best teacher but besides letting them discover things for themselves, new programmers also need to be taught some basic principles. They need to be given sensible guidelines to follow. Guidelines give them a sense of direction, a rope to hold on to so they don't get lost as they muddle their way through the dark. Principles provide a solid anchor around which to tie that rope.

General guidelines help programmers deal with specific obstacles and dangers along the way so they can navigate safely through or around them. Principles provide a place for them to go back to and find some truth and certainty in situations where guidelines fall short or are not strong enough to carry them through chaos and confusion.

When they make mistakes and stumble, programmers can use principles and guidelines to pick themselves up, assess what went wrong, and decide on a more appropriate course of action next time they encounter a similar challenge. They can take any new lessons they learn and lay those on top of the principles they already know, thus strengthening and solidifying their base of knowledge and understanding. This solid base is where intuition comes from and it's what will allow them to cast their ropes in different directions and venture into new areas for further exploration and learning.

Learning how to fly by jumping into the deep end

Yes, I know that subheading is incongruous. Did I mean to write "learning how to swim by jumping into the deep end" instead? No, I meant exactly what I wrote. That's the point. 

The way I see students taught in schools seems to be as incongruous as that subheading. The intent may be there, but the context and focus with which instruction is given leaves students totally out of sorts, IMO. Thus, the students wander into our little online community at the Ranch, like so many dazed and confused cattle desperately seeking to be prodded in the right direction. I see very little indication that clear guidance was provided to them by their instructors.

It's as if they had been led to the edge of a cliff, then suddenly pushed into the void with the expectation that they would know to spread their arms and find out they had wingsuits on that would allow them to glide smoothly and effortlessly to relative safety below.

Unfortunately, that very seldom is the case, if at all. It's more like they were given a pack to strap on their back only to find that when they pulled the ripcord, the pack was filled only with basic camping gear but no parachute. It's like they were given some rope but only enough with which to hang themselves.

That's the point in the incongruity of "learn how to fly by jumping into the deep end." It often seems like teachers focus on giving students the tools they need to do something but provide little to no guidance or support in learning how to use those tools safely or effectively. It's like they went, "Here's the deep end kids, now take your snorkels, jump in, and run to the shallow end! Anyone who doesn't drown gets a passing grade." That's kind of ridiculous, right? It's also patently unfair.

Now, you may say that I'm being unfair to the instructors and I will concede that most of them probably do their honest best, or at least as well as they know how. But I still don't think it's enough. We can do more. We can do better to get new programmers ready to meet the challenges of learning how to develop software.

Lack of Context

You need to stand before you can walk. You need to walk before you can run. There's a logical progression to the process of learning and each step forward builds on the progress already made before. Generally speaking, educators seem to have this down. They start off with simple concepts and slowly build up to more advance concepts. I don't have many problems with this aspect of pedagogy.

It's the context that's almost often very lacking in the delivery of computer programming instruction. Most of us learn how to walk on dry land, not in deep water. Without somewhere to set their feet down so they can get their bearings, all new learners can do is flail about aimlessly while trying to keep their heads above water and not drown. This is where I think the problem lies in the kind of instruction you normally find in a traditional classroom setting. 

There seems to be very little discussion around the practice of programming even as learners are introduced to more and more parts of the programming language they are studying. Focus is strong on language constructs like classes, variables, different ways to control program execution flow, data structures, compilers, errors, etc. Things like functional decomposition, code and design hygiene, basic layering and organization, and development process are deferred as "advanced" topics. These things are arguably as important, if not more, when you're developing software. Yet, I find that most new programmers are not even aware that there are principles and concepts that they need to consider or at least know about, if not sooner, then definitely later if they are going to become good software developers.

Some people may argue that topics like design principles, clean code, maintainability, and extensibility are too advanced for beginners and that introducing them too soon would not be very useful and would only confuse them. To some extent, I agree, but to totally defer all these until later is also irresponsible and I think even borders on being condescending and disingenuous.

Awareness leads to recognition and sensitivity which in turn leads to understanding. Without even an inkling of the things that make up the "big picture" of software development, new learners are deprived of a way to see how the little things they learn about at the start fit into a bigger context as they get further along in their studies.

We need to give learners more credit or at least set the bar a little bit higher and challenge them to think about some of the deeper, more abstract aspects of the practice of programming. Many new learners are just that, new. They're inexperienced, not stupid. I believe they're fully capable of comprehending even just the basic notions of some of these arguably "advanced" topics. I could be wrong but I'm willing to give them a chance to prove me so.

Lack of Preparation

Since they lack a basic understanding of the practice of programming, I find that many students are ill-prepared to handle the rigorous way of thinking that is needed to solve computer programming problems. The teaching process seems to go something like this: "Here's how you write a main method. Here's how you test for a condition. Here's how you set up a loop. Now go and write a program that generates a simple payroll report."

That's kind of like saying "Here are some pistons, some rods, some cams, a cylinder block, a crankshaft, spark plugs, etc. Now go build me an engine." Or like showing a teenager how the controls of a car work, then asking them to go straight out and drive on the road. That's a recipe for disaster, right? There's a lot more to building an engine than just putting together some mechanical parts. There's more to driving on the road than just knowing how to work the controls of a car. Likewise, there's a lot more to programming than just knowing how basic language constructs work.

Before they can drive a car out on the open road, new drivers must demonstrate that they have a good understanding of the rules of the road. These rules tell them how to operate a car safely in the context of their environment and in relation to other drivers who are also using the road. Even informal rules of thumb like the 3-second rule for keeping a safe distance is important to teach to new drivers. Especially new drivers. It is essential that all drivers follow the rules of the road consistently so that everyone can stay safe and roads stay clear and free of accidents and other hazards that would stop or impede the flow of traffic.

Likewise, before asking programmers to go out and solve problems with their newly acquired knowledge of basic language constructs, somebody should prepare them by giving them a basic set of rules to follow so that they can stay safe and so that they don't do things that will cause problems for themselves and/or other people who may come in contact with the code that they write.

If we don't give new programmers a framework around which they can organize their thoughts and clarify and evolve their ideas, we're not doing right by them. It would be like moving to a new place, buying your six-year-old a backpack, some comfortable shoes and clothes, pens, paper, a lunchbox, and then telling that child to go and find their new school for themselves. That's just not right. 

I'm often dismayed to see students asking for help on the Ranch who are halfway through their introductory programming course but are still writing Java programs that use static methods exclusively. They're still very confused and trying to figure out why the hundreds of lines of convoluted code that they've written in their main method doesn't work.

I just have to wonder what all these teachers are thinking. Or do they even care? Do they really expect that these kids will just figure it out for themselves? None of the course materials I have seen referenced tell me otherwise. 

I mean, c'mon, we have to do better than that for the new guys. We need to prepare them better for the challenges that they are going to face. We experienced guys all know what those are. It's not only for their sake that we should prepare them but also for the sake of whoever has to come in contact with code that these new guys write, now and in the future.

Programming is a way of thinking

Programming is about thinking and organizing our thoughts in a clear and coherent way so that our ideas come together as a solution to a given problem. I say "our" because programming nowadays usually involves more than just one person, so it's the aggregate of the ideas of all the people involved that need to come together to form a coherent solution. It's our problem, our ideas, our solution.

Most of the non-trivial problems that we try to solve with the help of computers are difficult, complex, and complicated. Naturally, the solutions we create are also often difficult to come by and just as, if not even more, complex and complicated. Nobody ever said that programming was easy. Anybody who does is lying or trying to sell you something. Same difference.

Regardless of how difficult, complex, and complicated the problems we try to tackle may be, the process of thinking that allows us to come to one or more solutions to those problems invariably follow some common patterns. I call these patterns in the same sense as I do when talking about design patterns. They are generally applicable to a wide variety of specific contexts and their applicability and appropriateness for use is also very dependent on context.

Much of these thought processes revolve around simplification. To solve a difficult, complex, and complicated problem, it's almost always necessary to have a systematic way of breaking it down into smaller, more manageable chunks, then organizing and coordinating the solutions to these small individual problems into something that addresses the original problem coherently and holistically.

The things that help me write better programs

Below you'll find some guidelines that I offered one confused student on JavaRanch this morning. They are the same guidelines that I use every day when I write code and design computer programs.

If you're a new programmer, read these guidelines. If you're an experience programmer, read them, too. You just might remember something. Or you might even learn something new. You never know.

If you don't have a lot of experience, don't be intimidated by words like "design", "parameterize", "extract", "duplication" and other words that maybe aren't part of your current daily vocabulary. Give yourself some credit. They are not too advanced for you. In fact, you have already encountered the concepts behind these words, albeit perhaps unknowingly, from the very first moment that you started thinking about writing your first computer program. You need to know these words and what they really stand for anyway, so you might as well start getting comfortable with them.

Read the guidelines below and try to always keep them in mind. Every time you sit down to write a single line of code, think about these guidelines. They're just like the rules of the road when you're driving: You should always have them in the back of your mind, even if you don't have any immediate need for them. When a situation arises where one or more of these guidelines do apply, at least they're there for reference to guide your decisions on what to do next.

Somewhere in there are some principles, too. I won't point them out but the fact that you're reading through this stuff will already make you aware of them, even if you don't know that's what they are. That should be good enough for now.

You can read my original post on JavaRanch but I will reiterate and refine it here. These are not my original ideas in any way. They have been stated more succinctly and eloquently by other, far smarter and better programmers that myself. I only offer these to you with a few more details and elaborations that I have found helpful over the years. I hope you find them helpful, too.

General Guidelines for Programmers

While you are programming, as often as you can, stop and check to...
  1. Make sure you're using names that make sense in a given context

    Ask yourself, "Is this name clear and unambiguous? Does it have the right semantics for how it's used and what it's used for? Does it fit with the ideas that I'm trying to reflect in the code? Does it imply any assumptions, mislead, or misinform the reader as to its purpose? Does it help to tell the story of what this program is doing? " If necessary, find a better name and rename that thing for clarity of intent.
  2. Make sure you're not repeating yourself unnecessarily

    Ask yourself, "Is there code elsewhere in the program that is very similar or exactly the same as what I just wrote here? Are there sections in the code that use the same values, formulas, calculations, conditions, as I am here? If I change something here, will I need to make the same kind of changes in other parts of my program?" If necessary, identify the pattern of repetition and put it in one place that can be accessed by everything that will need it. Extract and parameterize to eliminate duplication.
  3. Make sure you're telling a clear, coherent story with your code.

    Keep ideas that are high-level and abstract separated from low-level and concrete implementation details. Don't mix generalities with details. Prefer to tell the general story first and use good names to reveal your intent. Hide details but be sure to put them behind a good name that clearly and accurately represents what they do as a whole. Maintain single levels of abstraction.
  4. Make sure that you're following the 4 Rules of Simple Design.

    If you're already following the previous three guidelines, then you're already doing that for the most part.

Remember, the best way to eat an elephant is to take one bite at a time. Keep things small and manageable. Don't be a glutton. Go easy. When in doubt, slow down.

Most importantly, test your code. 

If there's one fundamental truth in computer programming, this is it: Testing can only show the presence, not the absence of bugs. A really smart computer scientist once said that. He was right. 

Therefore, use tests wisely and appropriately. Test your program frequently, as you're writing your code. If you're not writing a test for every few lines of production code, you're not writing enough tests. You're human, you're going to make mistakes. Lots of them. Tests will tell you when you do make them.  Listen to your tests. They'll help you stay out of trouble. They'll also help you get out of trouble. It's going to happen. You're going to miss a few bugs. Tests got your back though.

Nothing slows you down more than not being sure if your program works, or if you've broken anything, or why that thing that used to work isn't working anymore. So learn how to write automated tests. Again, you're only human. You forget things. You're inconsistent. You're not that fast. At least not as consistent or as fast as a computer when it comes to doing boring, repetitive, and tedious things like running dozens or hundreds of tests. No seriously, even a small, non-trivial program should have more than just three or four tests.

Related to testing is experimentation. Don't be afraid to try things, even if you know it's the wrong thing. If you can't think of a better name at the moment, just write something down, even if it's a totally goofball, silly name. Often, it's in seeing the wrong thing right there in front of you in the code that you find inspiration and realize what the right thing should be. If you don't know what the right thing is yet, at least you can definitely throw away the wrong things after you try them on for size.

Programming involves a lot of thought but it's not useful to sit on ideas and dwell on them. This is not theoretical physics. Thought experiments are not very useful in our daily work. Leave that up to the computer scientists and Turing Award winners, unless of course you're one of them. Write a test to formulate a hypothesis of how your program should behave. Then run that test as an experiment to see what happens. When the test fails, write an implementation to make it pass.

After you've run your experiment, go back and read your code. Read your code out loud. Discuss the ideas that are reflected in your code with your development partners. Make sure your understanding aligns with their understanding and that all your understandings align with what the code is saying and doing. If not, go back and apply the general guidelines to clean up your code and make it accurately reflect your understanding of how the software behaves. Check out what Ward Cunningham said about this stuff. Spoiler alert: Debt is supposed to be a good thing.

Learn how to use JUnit or something like it. Tools like this will save your life. Literally. Seriously.

Lastly, have fun. Programming is about learning. If you don't think that learning is fun, you probably won't be a good programmer. If that really is the case, find something better to do with your time. For everyone's sake.

Programming isn't for everyone but if you decide that it's really your thing, and that you want to get good at it, then you really should have fun doing it. Otherwise, what's the point?