Getting Digital Transformation Wrong, Software Development Edition

Just figuring out how to piece it all together.

“Digital transformation,” we’re all supposed to be doing that right? Most conversations I get involved now-a-days amount to “how do we do that?” and “how’s it going for other organizations?” When it comes to the custom written software part of whatever “digital transformation” is, here’s a few common “anti-patterns” I’ve been noticing:

1. Products, not projects.

Management has to change from a project-driven mindset to a product-driven one. Instead of annual budgets and plans, they’ll need to move to the weekly/monthly loop of shipping code, observing it, improving it, etc. I suspect this idea of “uncertainty” in long-term plans will seem baffling to most. Of course, the “uncertainty” is always there in long-term planning: you have the uncertainty of ever being successful — just because you have a plan doesn’t mean…well…anything when it comes to success in software.

2. Accept that you have no idea what you’re doing, and build a system that helps you figure it out.

A “waterfall” approach (plan, code, deliver, done) like is very fragile when it comes to unexpected events and changes (e.g., executive and market whim). This is why classically, development teams do not like scope and requirements documents changed. Changing and adding features “in the middle” of projects is very damaging as it screws up the order of work, budgets, and innumerable other things. It’s like if half way through assembling an Ikea bookshelf you decided you wanted a desk instead.

Companies get in a whiplash vortex where they’re constantly late and over budget because they keep changing the plan (bookshelf-cum-desk). They never get anything done because they’re using the wrong process. This is where the idea of “small batches,” mixed with “code must always be shippable,” comes into play to reduce risk. If you do small batches, you can change your mind every week…in a less painful way than in a waterfall mentality. (Pro-tip: it’s actually not a good idea to change your idea much, but, instead, experiment and use this feedback loop to explore new ideas and features. “Pivoting” is fine early one when you’re trying to discover your product/market fit, but once you do, you don’t want to change you mind too much.)

3. “Good cooking takes time. If you are made to wait, it is to serve you better, and to please you.”

Most big companies — rather, people managing those companies — like to rush things and spend zero resources on them, where resources means “time, money, and corporate attention.” Software is hard work that must be studied and understood. That’s why we pay software people a lot. Management often sets “the game” incorrectly. The first thing they do is rush the work and create really short project timelines for too much stuff. “Creating a sense of urgency” can be more damaging than helpful.

Instead or lighting fires under everyone’s collective asses, management needs to start small, painfully small.

Companies also often don’t want to spend any incremental budget, and their IT Service Management practices treat computational resources are sparse, static, and expensive (Mainframe, Unix, Windows) rather than infinite, fungible and cheap (cloud, Linux, etc.): you have to wait months to get a server. Once you layer in audit, compliance, and security (other resources that are treated as sparse and have endless downside/risk if something goes wrong [you get hacked!] and little upside to speeding it up), it delays things even further.

In short, management has framed the whole process of “doing software” in a way that’s slow and expensive. Thus, by (un)intention, doing software is slow and expensive.

(The the next item is a consequence too much resource control: the unwillingness to train and spend time on “the factory.”)

4. “You’re doing it wrong.”

Many of the productive practices in software — like using version control, doing CI and CD, writing tests before code, A/B testing, strict automation etc. — seems like a lot of unnecessary work until you’ve experienced the benefits, mostly in the medium and the long term. It’s always easier to just SCP code onto a server, or demo your application on someone’s laptop instead of doing the proper CI/CD pipeline and automation modeling on a real production environment. I’m still shocked at this, but most large organizations follow very little, if any good software development practices:

From : “Town Hall: Agile in the Enterprise,” Mike West, Nathan Wilson, Thomas Murphy, Dec 2015, Gartner AADI US conference.

Fixing this is tedious, “hand-to-hand” battle: OK, so, start writing tests first… fully automate your builds and get a proper environment that you can deploy at will to, etc. That’s likely a good 3–6 months of calendar time right there (for training, screwing up, then re-training, and also building all your own platforms if you don’t just use off the shelf workflows) before you can start really getting into the flow of shipping code.

5. Innovation takes a lot more time than you think.

Corporate managers and product owners don’t understand how much work is involved in innovation. In my experience, “management” thinks that their dictates and “requirements” are clear and straightforward. They think they’re dealing with the “known knowns” quadrant of Rumsfeldian knowledge, but at best they’re dealing with “known unknowns,” and likely deal with “unknown unknowns”…about their own wants and desires!

For example, as a requester of software, you often have no idea what you actually want until you see it several times. This means you need to set-up a process to continually check-in on progress (e.g., demos ever 1–2 weeks with the option to change direction) and, more importantly, be involved: management must “spend” their attention on the software development process. And at the same time, you have to resist the urge to manage by table flipping and instead be happy “learning by failing.”

6. Technical debt and legacy are rarely addressed, let alone quantified.

Most any large organization will have a pile of legacy applications and processes. Most of this will be riddled by technical debt, if only parts of the system that (a.) no one really understands how they works, and, (b.) there really aren’t automated tests. This means — building on Michael Feather’s Legacy Code Dilemma — that it’s too high risk to touch/change your legacy code, let alone do it quickly. Beyond just modifying those systems, this can include integrating with systems as well, usually highly customized ERP systems that any business needs to work with to actually do work. While it’s possible to “refactor” code, I’m becoming more and more of the opinion that at best you quarantine it (or do some “strangler pattern” mumbo jumbo if you’re really good), or just put it all on an a piece off ice, light it on fire, and shove it off into the ocean.

7. Fix the Meatware.

As I’m ever going on about, many of these are meatware problems: technology is often just fine, it’s the people that need fixing. The confounding thing with this issue — how do you transform large organizations to [A|a]gile? — is that all actors in this system (except, sometimes, “top management”) knows exactly what to do and what’s wrong. There’s some sort of complex game theory going on that ruins the easiness of fixing the problem, and everyone ends up becoming expert at telling you why nothing can improve instead of figuring out how to do better. I think that’s something of what Andrew’s always on about with his prisoner’s dilemma and Nash equilibrium who walk into a stone cutter bar.

For a more recent, detailed study on this topic, check out my book Monolithic Transformation.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.