We’re getting exactly the government IT we asked for

If there’s one complaint that I hear consistently in my studies of IT in large organizations, it’s that government IT, as traditionally practiced, is fucked. Compared to the private sector, the amount of paperwork, the role of contractors, and the seeming separation between doing a good job and working software drives all sorts of angst and failure.

Mark Schwartz’s book on figuring out “business value” in IT is turning out to be pretty amazing and refreshing, especially on the topic of government IT. He’s put together one of the better “these aren’t the Droids you’re looking for” responses to ROI for IT.

You know that answer: you just want to figure out the business case, ROI, or whatever numbers driven thing, and all the DevOps-heads are like “doo, doo, doo-doo – driving through a tunnel, can’t hear you!” and then they pelt you with Goldratt and Deming books, blended in with some O’Reilly books and The Phoenix Project. “Also, you’re argument is invalid, because reasons.”

A Zen-like calm comes over them, they close their eyes and breath in, and then start repeating a mantra like some cowl-bedecked character in a Lovecraft story: “survival is not mandatory. Survival is not mandatory. Survival is not mandatory!”

Real helpful, that lot. I kid, I jest. The point of their maniacally confusing non-answers is, accurately, that your base assumptions about most everything are wrong, so before we can even approach something as precise as ROI, we need to really re-think what you’re doing. (And also, you do a lot of dumb shit, so let’s work on that.)

But you know, no one wants to hear they’re broken in the first therapy session. So you have to throw out some beguiling, mind-altering, Lemarchand’s boxes to change the state of things and make sure they come to the next appointment.

Works as Designed

Anyhow, back to Schwartz’s book. I’ll hopefully write a longer book review over at The New Stack when I’m done with it, but this one passage is an excellent representation of what motivates the book pelters and also a good unmasking of why things are the way we they are…because we asked for them to be so:

The US government is based on a system of “checks and balances”—in other words, a system of distrust. The great freedom enjoyed by the press, especially in reporting on the actions of the government, is another indication of the public’s lack of trust in the government. As a result, you find that the government places a high value on transparency. While companies can keep secrets, government is accountable to the public and must disclose its actions and decisions. There is a business need for continued demonstrations of trustworthiness, or we might as well say a business value assigned to demonstrating trustworthiness. You find that the government is always in the public eye—the press is always reporting on government actions, and the public is quick to outrage. Government agencies, therefore, place a business value on “optics”—how something appears to the observant public. In an oversight environment that is quick to assign blame, government is highly risk averse (i.e., it places high business value on things that mitigate risk).

And then summarized as:

…the compliance requirements are not an obstacle, but rather an expression of a deeper business need that the team must still address.

Which is to say: you wanted this, and so I am giving it to you.

The Agile Bureaucracy

The word “bureaucracy” is something like the word “legacy.” You only describe something as legacy software when you don’t like the software. Otherwise, you just call it your software. Similarly, as Schwartz outlines, agile (and all software processes) are insanely bureaucratic, full of rules, norms, and other “governance.” We just happen to like all those rules, so we don’t think of them as bureaucracy. As he writes:

While disavowing rules, the Agile community is actually full of them. This is understandable, because rules are a way of bringing what is considered best practices into everyday processes. What would happen if we made exceptions to our rules—for instance, if we entertained the request: “John wants to head out for a beer now, instead of fixing the problem that he just introduced into the build?” If we applied the rules capriciously or based on our feelings, they would lose some of their effectiveness, right? That is precisely what we mean by sine ira et studio in bureaucracy. Mike Cohn, for example, tells us that “improving technical practices is not optional.”15 The phrase not optional sounds like another way of saying that the rule is to be applied “without anger or bias.” Mary Poppendieck, coauthor of the canonical works on Lean software development, uses curiously similar language in her introduction to Greg Smith and Ahmed Sidky’s book on adopting Agile practices: “The technical practices that Agile brings to the table—short iterations, test-first development, continuous integration—are not optional.” I’ve already mentioned Schwaber and Sutherland’s dictum that “the Development Team isn’t allowed to act on what anyone else [other than the product owner] says.”17 Please don’t hate me for this, Mike, Mary, Ken, and Jeff, but that is the voice of the command-and-control bureaucrat. “Not optional,” “not allowed,” – I don’t know about you, but these phrases make me think of No Parking and Curb Your Dog signs.

These are the kind of thought-trains that only ever evoke “well, of course my intention wasn’t the awful!” from the other side. It’s like with the ITIL and the NRA, gun-nut people: their goal wasn’t to put in place a thought-technology that harmed people, far from it.

Gentled nestled in his wry tone and style (which you can image I love), you can feel some hidden hair pulling about the unintended consequences of Agile confidence and decrees. I mean, the dude is the CIO of a massive government agency, so he be throwing process optimism against brick walls daily and too late into the night.

Learning bureaucracies

The cure, as ever, is to not only to be smart and introspective, but to make evolution and change part of your bureaucracy:

Rules become set in stone and can’t change with circumstances. Rigidity discourages innovation. Rules themselves come to seem arbitrary and capricious. Their original purpose gets lost and the rules become goals rather than instruments. Bureaucracies can become demoralizing for their employees.

So, you know, make sure you allow for change. It’s probably good to have some rules and governance around that too.

Think your organization is too caustic for “agile”? That actually makes it perfect

Many organizations object to the idea of doing anything remotely similar to “agile” based on the lack of communications between different groups and an overall caustic corporate culture. While more cynical than most agile adherents would like, the functional practices in agile may actually be a perfect fit for such corrupted cultures. As Grady Beaubouef ingeniously explains it:

The customer should perform a realistic assessment of the relationship between their Business and the internal IT organization. If the relationship does not foster effective collaboration or there is a known alignment challenge, then this presents a significant project risk. An iterative approach should be considered in this situation – especially when it comes to software developments for packaged software enhancements and customizations.

I finished up Beaubouef’s book on the plan today. It’s somewhat tedious to get through – the topic of planning and project managing big COTS installs isn’t exactly thrilling – it’s chock full of useful thinking.

Moving beyond the endless debate on bi-modal IT

I get all ants-in-pants about this whole bi-modal discussion because I feel like it’s a lot of energy spent talking about the wrong things.

This came up recently when I was asked about “MVP”, in a way that basically was saying “our stuff is dangerous [oil drilling], so ‘minimal’ sounds like it’d be less safe.” I tried to focus them on the “V” and figure out what “viable” was for their situation. The goal was to re-enforce that the point of all this mode 2/small batch/DevOps/PDCA/cloud native/OODA nonsense is to keep iterating to get to the right code.

Part of the continual consternation around bi-modal IT – sad/awesome mode – is misalignment around that “viability” and scoping on “enterprise” projects. This is just one seam-line around the splits of the discussion being unhelpful

Bi-strawperson

The awesome mode people are like:

You should divide the work into small chunks that you release to production as soon as possible – DevOps, Agile, MVP, CI/CD – POW! You have no idea what or how you should implement these features so you need to iteratively do it cf. projectcartoon.com

And the sad mode folks are like:

Yes, but we have to implement all this stuff all at once and can’t do it in small slices. Plus, mainframes and ITIL.

Despite often coming off as a sad mode apologist, I don’t even know what the sad mode people are thinking. There’s this process hugger syndrome that, well on both sides really, creates strawpeople. The goal of both methods is putting out software that makes users more productive, including having it actually work, and not overpaying for the whole thing.

The Enemy is finding any activity that doesn’t support those goals and eliminated it as much as possible. In this, there was some good scrabbling from the happy mode people laughing at ITSM think early on, but at this point, the sad people have gotten the message, have been reminded of their original goal, and are now trying to adapt. In fact, I think there’s a lot the “sad mode” people could bring to the table.

To play some lexical hopscotch, I don’t think there is a “mode 1.” I think there’s just people doing a less than awesome job and hiding behind a process-curtain. Sure, it may to be their choice and, thus, not their fault. “Shitty jobs are being done,” if you prefer the blamelesss-veil of passive voice.

Fix your shit

When I hear objections to fixing this situation, I try to b nice and helpful. After all, I’m usually there as part of an elaborate process to get money from these folks in exchange for helping them. When they go all Eeyore on me, I have to reframe the room’s thinking a little bit without getting too tough love-y.

“When I put these lithium batteries in this gas car, it doesn’t seem to work. So electric cars are stupid, right?

You want to walk people to asking “how do we plan out the transition from The Old Way That Worked At Some Point to The New Way That Sucks Less?” They might object with a sort of “we don’t need to change” or the even more snaggly “change is too hard” counter-point.

I’m not sure there are systems that can just be frozen in place and resist the need to change. One day, in the future, any system (even the IRS’!) will likely need to change and if you don’t already have it setup to change easily (awesome mode), you’re going to be in a world of hurt.

The fact that we discuss how hard it is to apply awesome mode to legacy IT is evidence that that moment will come sooner than you think.

(Insert, you know, “where’s my mobile app, Nowakowski?” anecdote of flat-footedness here.)

ITIL end in tears(tm)

The royal books of process, ITIL, are another frequent strawperson that frothy mouthed agents of change like to light up. Few things are more frustrating than a library of books that cost £100 each. There’s a whole lot in there, and the argument that the vendors screw it all up is certainly appetizing. Something like ITIL, though, even poorly implemented falls under the “at least it’s an ethos” category.

Climbing the Value Chain

I’m no IT Skeptic or Charles T. Betz, but I did work at BMC once: as with “bi-modal,” and I really don’t want to go re-read my ITIL books (I only have the v2 version, can someone spare a few £100’s to read v3/4?), but I’m pretty sure you could “do DevOps” in a ITIL context. You’d just have to take out the time consuming implementation of it (service desks, silo’d orgs, etc.).

Most of ITIL could probably be done with the metaphoric (or literal!) post-it notes, retrospectives, and automated audit-log stuff that you’d see in DevOps. For certain, it might be a bunch of process gold-plating, but I’m pretty sure there’s no unmovable peas under all those layers of books that would upset any slumbering DevOps princes and princesses too bad.

Indeed, my recollection of ITIL is that it merely specifies that people should talk with each other and avoid doing dumb shit, while trying to improve and make sure they know the purpose/goals of and “service” that’s deployed. They just made a lot of flow charts and check lists to go with it. (And, yeah: vendors! #AmIrightohwaitglasshouse.)

Instead of increasing the volume, help spray away the shit

That gets us back to the people. The meatware is what’s rotting. Most people know they’re sad, and in their objections to happiness, you can find the handholds to start helping:

Yes, awesome mode people, that sounds wonderful, just wonderful. But, I have 5,000 applications here at REALLYSADMODECOGLOBAL, Inc. – I have resources to fix 50 of them this year. YOUR MOVE, CREEP!

Which is to say, awesome mode is awesome: now how do we get started in applying it at large orginizations that are several fathoms under the seas of sad?

The answer can’t be “all the applications,” because then we’ll just end up with 5,000 different awesome modes (OK, maybe more like 503?) – like, do we all use Jenkins, or CircleCI, or Travis? PCF, Docker, BlueMix, OpenShift, AWS, Heroku, that thing Bob in IT wrote in his spare time, etc.

Thus far, I haven’t seen a lot of commentary on planning out and staging the application of mode 2. Gartner, of course, has advice here. But it’d be great to see more from the awesome mode folks. There’s got to be something more helpful than just “AWESOME ALL THE THINGS!”

Thanks to Bridget for helping draw all this blood out while I was talking with her about the bi-modal pice she contributed to.

Doing ROI for agile, DevOps, et. al. is difficult

I wrote up a rant-y style summary of thinking about ROI when it comes to planning out Agile, DevOps, cloud native, and otherwise “new ways of doing things in IT” schemes.

A little excerpt:

Doing “agile,” however, isn’t like dropping in a new, faster and cheaper component into your engine. Many people I encounter in conference rooms think about software development like those scenes from 80s submarine movies. Inevitably, in a submarine movie, something breaks and the officer team has to swipe all the tea cups off the officer’s mess table and unfurl a giant schematic. Looking over the dark blue curls of a thick Eastern European cigarette, the head engineer gestures with his hand, then slams a grimy finger onto the schematics and says “vee must replace the manifold reducer in the reactor.”

Solving your digital transformation problems is not like swapping “agile” into the reactor. It’s not a component based improvement like virtualization was. Instead, you’re looking at process change (or “culture,” as the DevOps people like to say), a “thought technology.” I think at best what you can do is try to calculate the before and after savings that the new process will bring. Usually this is trackable in things like time spent, tickets opened, number of staff needed, etc. You’re focusing on removing costs, not making money. As my friend Ed put it when we discussed how to talk about DevOps with the finance department:

In other words, if I’m going to build a continuous integration platform, I would imagine you could build out a good scaffolding for that and call it three or four months. In the process of doing that, I should be requiring less help desk tickets get created so my overtime for my support staff should be going down. If I’m virtualizing the servers, I’ll be using less server space and hard drive space, and therefore that should compress down. I should be able to point to cost being stripped out on the back end and say this is maybe not 100% directly related to this process, but it’s at least correlated with it.
In this instance, it’s difficult to prove that you’ll achieve good ROI ahead of time, but you can at least try to predict changes informed by the savings other people have had. And, once again, you’re left to making a leap of faith that qualitative anecdotes from other people will apply to you.

Read the rest over in my Medium feed.

Maybe “bi-modal” is about temperament, not just enabling “sad IT”

More for the “I don’t think bi-modal is that far from what all us methodology hipster talk about” file here:

Q. Have you seen organizations that have officially reorganized around a Bimodal approach? What kind of job titles/job descriptions are you seeing?

It’s much less an organizational structure change, and much more about culture, process and method. But yes, it is not unusual in the short to mid-term to differentiate teams. Mode 2 usually starts life as being quite small, and often needs “protection”, to prevent the organizational anti-bodies killing it off. It is better to go narrow and deep and focus the capability on a particular part of the enterprise, or on particular domains like mobile for example. Small IT organizations (<50) don’t have the option to create any kind of split of course).

I'd still wager that most people who complain about bi-modal haven't read up on it too much (it's hard to do, and there's bat-shit expensive paywalls around it). Also, I think Gartner has been filling out the nuance of the original, clear idea: softening it and making it more practical, which makes it more ambiguous.

My theory here – based on conversations with people transforming large organizations – is that the focus here should be on the temperament of the people, not the process. For any given project, the proven process of small batches applies, but you’ll often want “cow-pokes” (risk takers who hate long term commitment) for new projects and “city-folk (enjoy stability) for post-1.0 projects.

There’s a chance for judgement about who the cool kids are vs. the old folks there, but I’d suggest finding that distinction is all about the perceiver. In my career, I’ve found people who are perfectly happy being in one of those two boxes. Problems arise (I think Apple’s negligence for apps their no longer interested in is a good bad result to ponder here), but I feel like it fits the staffing and prioritization needs of most large organizations.

But, I don’t call it “theory” for nuthin’.

What is “waterfall”?

“Waterfall” comes up a lot in my line of work: talking with orginizations about how to do software better. People like myself take the definition of “waterfall” for granted and usually don’t spend too much time talking about it, let alone when it might be a good idea of how to do it well.

Someone asked me the simple question “what is waterfall?” as a follow-up to one of my talks recently. Here’s what I typed back:

Continue reading What is “waterfall”?

What I’m looking forward to at SpringOne Platform

The biggest cloud native conference is coming up at the first week of August, SpringOne Platform. To plan out my time I took at look at the sessions. Here’s what I’m looking forward to and what I think you, dear readers, will find interesting as well. Doing a list like this, of course, ends up excluding some awesome sessions, so be sure to check out the talk list yourself as well.

Also, if you’re interested and haven’t registered yet, be sure to use the code pivotal-cote-300 to get $300 off.

Dealing with legacy

Almost every conversation I have with large organizations involves a discussion about dealing with legacy software. While you may not be running JFK era IT, you probably have to deal with legacy. Here’s some sessions on that topic:

Cloud Native Coding

Moving to The New, New Thing requires different ways of architecting and coding your software. Here’s some sessions that go over those new ways:

Case Studies

While cooked up demos of Pet Stores and Breweries are education, I’m most interested in hearing tales of what’s actually happened out in the world. Here are some of the case studies that look interesting:

The Usual Chuckle-heads

And, to highlight talks from my team:

(And, remember: if you want to come, you can get $300 if use the code pivotal-cote-300 when you register.)

Agile Methodology In-depth Review, Government Edition

I’m giving a 90 minute overview of agile for an agency later this month. Here’s my slides, so far. As ever, “government” is just an extra layer of sprinkles on-top of advice for all large organizations.

There’s some extensive (for me) talking points in the slide notes if you’re into that kind of thing.

Getting collaboration right in Agile & DevOps – Press Pass

I don’t do press passes as much as I did when I was an analyst, but here’s one from a recent email interview for a ProjectsAtWork story:

Q: What’s your favorite tip to improve collaboration when an organization moves to agile and DevOps?

A: I think the core DevOps thing with collaboration is getting people to trust each other. Most corporate cultures are not built on people trusting each other and feeling comfortable: they’re based in competitive, zero sum structures or command and control management at best.

Organizations that are looking to DevOps for help are likely trying to innovate new software and services and so they have to shift to a mode of operating that encourages collaboration and creativity. Realizing that is a critical step: we want to create and run new software, so we need to understand and become a software producing organization.

In contrast, if you operate differently if you’re just driving down costs each quarter and not creating much with IT. We’d counter-argue that if you’re a large organization and you’re not worrying about software then you’ll be creamed by your competition who is becoming a software organization.

If forced to pick one tip to increase collaboration I would say: do it by starting to work. How you do this is to pick a series of small projects and slowly expand the size of the projects. These projects should be low profile, but have direct customer/revenue impact so that they’re real. It’s important for these projects to be actual applications that people use, not just infrastructure and back-end stuff. It will help the team understand the new way of operating and at the same time help build up momentum and success for company wide transformation later down the road.

As a basic tactic, Andrew Shafer has a fun, effective tactic about having each people on the team wrote fantasy press releases about each other to start to build trust.

(See the full piece by Will Kelly over on the site.)