Allow me to tell a bit of a story.

Back in the late 90s/early 2000s I was a regular visitor and sometime contributor to a website where XP and Agile were first being brought to the attention of the wider community outside the original team that developed the practices. At the time I was steeped in Structured, Top-Down Programming and Getting the Design Right before starting on coding. The practices of XP looked to me, in my inexperience, like cowboy coding. I believed that good programming required a capital-M Methodology and lots of UML with Rational Unified Process. I had previously worked in academia where there was no programming process and everything was slapdash. XP and agile seemed to be a similar “we don’t need no rules here” way of writing code.

By today’s standards of 3-5 years of experience, I was Senior, so I really thought I knew what I was doing. I couldn’t conceive of how software could get done if it wasn’t planned out in advance. Software development was supposed to look like this diagram:

From Winston Walker Royce’s 1970 paper Managing the Development of Large Software Systems

I called XP a “pseudo-methodology”. It looked like chaos.

James Gleick wrote a book titled Chaos, published in 1987. I don’t recall when I read it but certainly by the the late 90s, before or alongside reading about XP.

Let’s stop here and talk a bit about that word, “chaos”, because it’s important, and I think a lot of where I’m going hinges on it. If you look in the dictionary you’ll see definitions of chaos like “complete disorder and confusion”, or “a state of utter confusion“. Gleick definitely did not write an entire book about utter confusion. The meaning of chaos in his book is an alternate definition at the dictionary citation previously linked: “the inherent unpredictability in the behavior of a complex natural system”. A very small change in conditions at one point in time could result in very large and unpredictable changes in behavior later.

You could not remove a single gram of sand from its place, without thereby, although perhaps imperceptibly to you, altering something throughout all parts of the immeasurable whole

The Vocation of Man, Johann Gottlieb Fichte, W. Smith tr.

A related concept is: emergence. Emergence, or more fully, emergent behavior, is like a flock of birds. Watching birds move in a flock might lead to the idea that there is a leader, but there is not. Because each individual acts according to the same rules, in the same environmental conditions, we see flocking. Schools of fish are similar. This is decentralized behavior.

I’ll have to go into chaos, emergent behavior, and complex systems in another post, later.

But sometime before March of 2001 I had an epiphany. I won’t go into the whole story but my realization was the when the work gets to the coding and testing phases all but the most trivial projects will discover things that hadn’t been accounted for in the requirements and analysis phases. It also became clear to me that in anything but the smallest projects, in the time between requirements and coding, something will change, invalidating at least some of the decisions made in the requirements phase. What I didn’t know before then was that Royce, in his paper that laid out what would be called the waterfall model of software development, including the graphic previously shown, rejected the concept. He wrote, “I believe in this concept, but the implementation described above is risky and invites failure.”

required design changes are likely to be so disruptive that the software requirements upon which the design is based and which provides the rationale for everything are violated. … In effect the development process has returned to the origin and one can expect up to a 100-percent overrun in schedule and/or costs.

Winston W. Royce, 1970

What I think was happening when XP came along was that programmers realized that the systems they were working on had become too complex to reason about. The systems were, in Gleick’s terms, chaotic. The concepts of chaos, emergence, and decentralized control informed XP: The practices of XP are simple, and out of the simple practices arise complex behaviors. Simplicity, coupled with communications and feedback, make XP a programming discipline expressly, and perhaps uniquely, developed and suited for writing complex software of the sort that came about at the turn of the 21st century, in the kinds of organizations that became dominant in the software profession.

You might say I became an XP convert. I did have a tendency to get personally invested in my world view, and at my next job I became the Change Agent that was going to bring XP/agile to my employer. And you know what? I made a lot of mistakes and misunderstood a lot of things, but it sort of worked. My co-workers started thinking about things differently. They stopped being quite so annoyed when, halfway through the project, the businesspeople decided they wanted something kind of different from what they had said up front. Programmers and non-programmers started to look at the work-in-progress together and decide if things were on track, if the requirements were right, if everyone had come to a shared understanding of what the requirements really meant.

Mostly, I started to accept that users and stakeholders only sort of know what they want, that communicating complex ideas between people is hard, and that realizing ideas in software is harder still. The only way to get software that works as desired is to build incrementally, share progress, and iterate.

It’s been 20 years since the Agile Manifesto, and I have met two or three of the original signatories. In the interim, I’ve come to agree with observers who say agile, as done today, feels like chaos. Agile, done poorly, is chaos. And by that I mean “a state of utter confusion”.

Three Months in the Grass Roof Hut

Staying Focused, Motivated, and Engaged

Though the hut is small, it includes the entire world.
In ten feet square, an old man illumines forms and their nature.

Shitou, Translated by Taigen Dan Leighton and Kazuaki Tanahashi, from Inside the Grass Hut, Ben Connelly.

A big worry I had in taking a sabbatical was that I’d just end up a couch potato all day for months and end up with nothing to show for it. This is especially true during the pandemic lockdowns. After three months I can say I’ve done OK. Remember this schedule from my earlier post when I started my sabbatical? Here it is again.

Daily Schedule

8:15Feed Yuli, Make Coffee
8:30Daily inspirational reading with coffee
9:30Study and Reading
25-min intervals
Alternate with Yuli playtime, reading for book group
NoonFeed Yuli
14:00Unstructured Time
nap, exercise, chores, shower, Yuli playtime, journal writing
Note: All times are approximate

Let me be clear: I don’t follow the the schedule rigorously or consistently. It’s not a Procrustean bed, it’s a guide. Having a schedule is important. Even before sabbatical, before the pandemic-enforced working from home, I had learned that outside the office world, professionals always lean a schedule. Beethoven had a routine. To a significant level having a set of rules and rituals is one of the keys to being productive.

One of the best outcomes of setting the schedule is that I spend a lot less time doom-scrolling social media. When I find I’ve gotten distracted I can stop, look at my schedule, and know what I should go back to doing. I also know how much time I’ll devote to something on a day-to-day basis, which helps me when I’m not especially motivated to focus on what’s scheduled, because I know that I have a definite stopping time and don’t have to spend any more time on it than that.

In addition to the daily schedule, I have a rotation of areas of focus for each day of the week. On any given day I have a guide for which of my interests gets my focus (or at least my primary focus) for the day.

Weekly Rotation

  • Sunday: Spirituality
  • Monday: Photography
  • Tuesday: Writing
  • Wednesday: Productivity
  • Thursday: General Programming
  • Friday: Identity and Access Management (a sub-field in software)
  • Saturday: wildcard, catch up or off

Now, after three months of this, I’m starting to discover areas I may refine or evolve this routine. First, I need to be more explicit about when, exactly, am I writing or producing vs reading, organizing, and planning. Second, only working on a topic every seven days is seeming less than ideal.

When Does the Writing Happen?

I’ve done a lot of soothing the burnout, reading, and pondering ideas in the past three months. I’ve managed to regain my ability to focus for long periods, and I’m very happy with that. As part of my reading and pondering, I of course take notes and that involves writing, but that kind of writing is only the raw material for producing published pieces like this blog post.

Now… Where Was I?

When I pick up my topic for the day, I find it hard to pick up where I left off a week ago. That is to say, I lose the momentum too much with a seven-down interval. Putting something aside after working on it a bit is a necessity, as intentional down time. Creativity requires what Joe Kraus calls “gap” time, but seven days seems to be too long.

Dude, Where’s My Vacation?

And the big one. My sabbatical plans never included unending weeks of this. I was going to do it for a while until the situation with the pandemic was better, then I was going to go on the road, do some hiking, photography, visit people, see places. And where are we now?

Source: Trends in Number of COVID-19 Cases and Deaths in the US Reported to CDC, by State/Territory

Was I too optimistic in April when I looked towards summer? For the past few days I’ve been in a bit of a funk over this. It was bad enough in Spring of 2020 when it became clear that Americans weren’t going to take it seriously, and by Fall of 2020 when the inevitable surge came, but now it’s disheartening in a way that’s hard for me to express. It’s definitely impeding my recovery from burnout.

It looks like I started this post over a month ago. I’m sure that I’ve been sitting on it trying to figure out how to make it longer. I think it’s long enough now.