Beware of High Expectations in Enterprise Tech

Here are three charts.

First, while everyone is freaking out about AI, it’s had little impact thus far:

Second, since 2020, usage of cloud seems to have leveled off at a 50/50 split between on-premises and public cloud:

But don’t worry, everyone is planning to move more to public cloud in the next three years. At least that’s been the case for the past seven years:

There are some rules of thumb for the impact of new enterprise technology here:

1. It will probably take much longer than you think.

2. The business value you achieve will probably be less than you think.

3. There’s not as much urgency to “transform” as us vendors would like you to believe.

The AI craze is an especially interesting case for me. It is clearly a big deal. I never used Docker, but the first usage of ChatGPT is what it must have been like to use Docker. You don’t believe what’s happening, it’s so effortless, and you can immediately start getting value from it.

For enterprises, however, things are more complicated. Just as you wouldn’t want your developers doing whatever they wanted with Docker - especially just downloading images of the Internet and, like, using them to run your paycheck processing, missile guidance, or just where to ship this box of celery systems - you don’t want ChatGPT just running around willynilly inside your enterprise. Even more: it can’t. You can’t just upload everything to ChatGPT and have it do wonderful things. Believe me: I’ve tried!

You can imagine all sorts of stupendous improvements to how a company operates if they unleash AI on everything. But, it will be less stupendous that we think, and take much longer. First, you’ll need to make sure all the governance and (as we’d say in the consumer world) privacy concerns are taken care of. This isn’t just about annoying lawyers and security people being annoying, it’s often to comply with regulations and laws. And it’s often a good idea!

Then, you have to figure out a way to actually load all that data into THE AI MACHINE. Access to all that corporate data across old ERP systems, CRM systems, Salesforce, email, Sharepoint, Google Docs, etc., etc. is already fraught and difficult. We’ve tried “data lakes” and…well? One of the main issues here is paying for all of that.

If you’ve ever tried to do what seems like simply pulling information from Salesforce or other ERP systems, you know what I mean. You rarely can just do what should be simple. First, you need access. And you’re going to have a get budget for that. Second, you need to actually understand the format the data is in. Third, you need the skills to actually find, clean, transform, and build reports out of that data. What this means is that you can’t do it alone: you always need to know “a guy” who can do it on the side for you. (Sidenote: if you’re working in a big company, always be looking out for this “guy.” If you see some really good charts or data in a presentation, ask who put that together and take that person to lunch so that you can ask them for help in the future.)

ChatGPT like systems could make this all much, much better. But it will take time, and it might not just, like happen as fast or as broadly as you think. Compare that to the public cloud usage in the above charts. We all know that using public cloud makes totally sense for any given apps. We all know that there are very few technical reasons to do it. We all know that the costs are, at worst, net-neutral. And yet…it’s going to take us three years to get there, apparently.

Why so slow?

This emerging technology adoption ass-dragging is driven by numerous things. The price to use new technologies and switch over to use them is often too high to easily deal with. If you’re serious about building an app platform on-top of Kubernetes, you’re going to pay a lot more money than you think. For a the smallest enterprises (let’s say, 5,000 to 10,000 people or more), you’re talking about millions of dollars a year. That can be either in staff if you want to DIY, buying vendor/public cloud services, etc. Just paying for the operating systems - a “free,” open source operating systems! - is going to cost millions. I have no idea how RHEL pricing works at this scale, but if you look at Salesforce’s numbers (moving 200,000 servers to RHEL), they feel like what big banks, large retailers, large government agencies, and other enterprises would face. That’s got to be, again, millions of dollars a year.

These costs are often worth it. You need computers and I’ve heard software is eating the world. However, when you introduce a brand new technology with such a steep price, you’re talking 12 to 18 months to get going. Enterprises work on annual budgeting cycles. If they’re using a regular calendar year for their fiscal year, you start “socializing” your budget needs around October, figuring out what you need, making deals with peers, sniffing out what your budget approvers actually value and will approve, getting your PowerPoints all prettied up with arts and crafts. Then you pitch it towards the end of the year. The budget is approved, then everyone disappears for two to three weeks for the December holidays. You come back in January, sort of putter around for a couple of weeks, and then you start finish talking with your vendors and public cloud people (who, hopefully, you started talking with back in October), finally selecting one to go with. Then you need to install the thing. But first you’ll need to talk with he networking and storage admins. The compliance and security people are going to want to see things. Also, you’ll need to update some training in the cloud center of excellence. And before you know it, a quarter has passed. You’ll have to spend a week or so preparing your quarterly review - more arts and crafts - to report on the status of it.

And so forth. (Also, I forget dealing with procurement. It’s hard enough to get an enterprise to pay for a $100/year expense, let alone $1.4 million.)

Then October rolls around again. Us vendors are like, “hey, how’s your consumption of this stuff: I gotta maintain the ARR, do some LTV farming…could you introduce me to that other group? I they’d like some steak dinners….”

Then, all of the sudden, those goofballs in the bond department screwed up their inflation estimates, and next thing you know another bank is buying you. “We’ve got some other things we need to handle now,” you tell your steak dinner friend.

You’ll get the stuff going eventually. You’ll get benefits from the new technology for sure - just ask us vendors to carpet bomb you with case studies! This shit is for real. It’s just that it’s for real once you get it all setup and learn how to do it.

It’ll take a lot longer than you think.

The revelation of reality

The Gartner hype cycle describes another phenomena here: inflated expectations. You’ve heard that AI (or public cloud, or cloud native, or CRM systems) is stupendous, revolutionary. There’s case studies from early adopters. But you haven’t actually used it yet. The hundreds (thousands, even) of people in your organization haven’t used it yet. You customers have not experienced its effects yet, and so forth.

Until you actually get your hands on it, at scale (not just some early PoC’s) you won’t know how stupendous it is in your organization working within your constraints and taking advantage of your capabilities. Getting to that point takes at least a year, often more to get more organization wide usage.

You can certainly get a better and better sense as you go along. You have to build in bureaucratic humbleness that is willing to learn and observe itself in action to get that sense. You have to be careful with your first impressions. What the hype cycle is telling you is that they’ll be inflated.

ChatGPT is a good example here. Once you use it day in and day out for months, you adjust your astonishment at it. It is amazing! But it’s now where near the level of earth shattering that the anti-AI people make it out to be.

Listen, it sounds like I’m being a wet-sandwich here. But it’s the opposite. The hype cycle has another mind-trick in it. After the peak of inflated expectations, everyone gets disappointed and you go into the trough of disillusionment. You get upset! You get letdown! (And when it comes to public cloud, all the on-premses vendors smell the delicious disappointment at migrating to the new and swarm - steak dinners galore!)

If you know that expectations are false, that it will take a long time to get the stupendous returns, you won’t get disillusioned. You’ll be smarter, reality will be revealed.

The rotting core

The other thing you have to keep on eye on is spending too much attention on the stupendous new thing. Meanwhile, you have the existing business to run, the existing systems to operate and optimize. I suspect that most of the benefits you can achieve from IT come just from doing what you’re currently doing better. For example, if you look at how you do software development, you probably call it “agile,” but most people are probably not following the actual practices.

A cool trick is to ask these agile teams how frequently they ship software. If it’s not at least monthly, you can probably do some improvements. And the improvement is probably just putting in some automation, talking with auditors to show them that that automation means they can trust you systems more and, thus, have less meetings about each release. Are you actually doing pair programming? What are your test coverage numbers? Are you actually standing up in your stand-up meetings?

I’ve studied the problem of rotting, neglected systems in large enterprises for the past two years and it is a big, big problem. One of my co-workers said at his previous job, at a big bank, they were spending near 50% of their time just on fire-fighting, upkeep, and dealing with compliance. As he said, time spent on that is time stolen from making the business better. And, also as he said, there’s well known, proven practices and technologies that could improve that 50%…you’d just have to actually decide to do it.

The issue with this rotting core is that (a) it’s boring compared to, like, AI, and, (b) it’s really hard to make a business case for addressing rot in your IT systems and software. It’s not going generate direct revenue, so unless your organization thinks in terms of running more efficiently (speeding the workflows your software does - for example, speeding up approving insurance applications or doing case management) and values agility, the business case is going to be difficult. Corporate strategy is a zero-sum game: those who win take from those who lost. There’s only so much budget and time that a company has. So if you’re pitching “just do what we do now, but better” versus “implement AI to revolutionize our business!”…it’s going to be tough.

The software factory

Most corporate strategy think doesn’t value well maintained software. The people making strategy decisions in enterprises seem to of software as one-off, install it and forget it projects. You have to think about your overall IT estate as a factory. Just as a you would spend a lot of time maintaining and optimizing your factory, you have to do the same with your IT “factory.”

I suspect that many organizations could get huge improvements just by getting better at what they’re currently doing.

That’s seemed to be the case business wise in the past decade. For example, banking startups were supposed to destroy traditional banking. Instead, existing banks just got better…without really changing how their business functions and what they do. And, as the public cloud charts show above. Well. That hasn’t rocked the world (but it will over the next three years!)

Two stupendous beguilement traps you should navigate right now

There’s two things going on right now that you can immediately apply this to.

Kubernetes

First, make sure you get realistic expectations about Kubernetes. The overall ecosystem has hyped the shit out of it, myself included many times. But if you look at actual adoption, it’s incredibly small. There’s probably about 10% of enterprise applications running in containers now, less in Kubernetes. There simply has not been enough use of kubernetes in enterprises to know if it’s a good or bad idea, let alone how much of a good or bad idea it is.

Don’t let Kubernetes migrations become a distraction from just improving how you currently work.

Instead, you need to setup a system - a culture, if you’re a DevOps fan - of learning and experimenting to figure out if Kubernetes will work for you. There’s a huge knowledge and experience vacuum when it comes to Kubernetes, and the strategic engineering move now to is fill that vacuum and then start making big decisions.

Platform engineering

Second, make sure you understand what platform engineering is before you launch big plans to transform how you do everything. Platform engineering is just DevOps under a new name. (At least that’s what I think this week.) At most, platform engineering just adds in internal developer collaboration tools (in the form of IDPs). Platform engineering might include building and running your platform-as-a-service…eventually.

But if you think through all the things in the umbrella of platform engineering, it’s too much. In a large enterprise (or even a small one), one group doesn’t have enough time, enough cognitive load to manage the tools developers use, the runtime environment they use, the middleware they use, and also do all that magic DevSecOps (pardon me, I mean PlatformEngineeringSecOps), and…all the other stuff. If you have a mature, proven platform like the Cloud Foundry ones, sure. There’s all sorts of people building platforms like that on-top of Kubernetes right now - so you can start with one of those instead of building your own. (You could also just short-circuit all of that and hand over the platform building and running to the public cloud - the dream!)

But people in enterprise IT are obsessed with building custom, bespoke platforms. I mean, builders like to build.

Most every enterprise I’ve ever talked with has numerous reasons why they’re different, why they can’t just use an off-the-shelf platform, and, thus, why they need to customize it. For the most part, their reasons are false. And, in most cases, they’re very common reasons that all of their peers and other enterprises have. And, as such, those reasons have been taken care of in the off-the-shelf platforms. Rather than change the unique constraints that drive the need to build their own platforms, these organizations work keep those constraints and are forced to build around it. It’d be like if you had to build a custom bed around an old radiator instead of just removing the radiator.

With the DIY platform mentality, you’re building a complex system, and you’re going to need a complex meatware system to manage it. “Complex systems build complex systems.” You’re probably already in a flywheel of doom made up of two complex systems - your software and your meatware - that’s making things more and more complex as the quarters tick by.

In that kind of system, you’re always going to need separate developer tools groups and separate platform operations groups. You can only break down silos if you put, as we used to say, an opinionated platform in place that standardized and centralizes what you do.

Have you tried actually standing?

So if you’re launching a Kubernetes and/or platform engineering, uh, “strategy” or “program,” figure all that out. Don’t have inflated expectations. It’s probably better to just, like, call in the DevOps Engineering team and the application development team, let’s see, the infrastructure management team, the security team…hell, get all the silos in one room, and ask them “so, tell me some daily tasks that we could be doing better. Are y’all actually standing up in your stand-up meetings?”

IncrementalOps

You’ve got to keep an eye on the new stupendous stuff. You should always be doing several projects to try it out. Not just “pilot” it or do a “PoC,” but to actually do real work and make sure you’re following a process to learn from it. If you think AI is going to be a big deal, spin up three projects that are directly connected to how the business runs and check in every month to learn what works and doesn’t work. You can’t do a business case for that. You just have to do it.

But, at the same time, spend a lot of your time just improving how you’re currently working. Remember, five years ago you did all of that transformation work, right? Maybe it’s time to go back and make sure you’re still following the practices and doing the things. That you haven’t just reverted to the old habits.

You know, instead of sitting, try actually standing up in your next stand-up meeting.

If you’re into the above kind of stuff, you should subscribe to my newsletter. It’s full of topics like these a few times a week, plus links I’ve found and other fun oddities to look at.

Subscribe!

Upcoming

Talks I’ll be giving, places I’ll be, things I’ll be doing, etc.

July 19th Improving FinTech with cloud native think, speaking. July 19th Stop Tech Debt and Start Using Faster, More Secure Paths to Production. August 21st to 24th SpringOne & VMware Explore US, in Las Vegas. Sep 6th to 7th DevOpsDays Des Moines, speaking. Sep 13th, stackconf, Berlin. Sep 14th to 15th SREday, London, speaking Sep 18th to 19th SHIFT in Zadar, speaking. Oct 3rd Enterprise DevOps Techron, Utrecht, speaking.

I’ve got what I hope will be a podcast-y type version of a webinar series I just wrapped up. I don’t know: something like a after show special where we talk about the series?

It’ll be live next week, on July 19th at 3pm Amsterdam time (check you local listings).

We did a good job in the series talking about how cloud native apps and thinking gives banks and other financial organizations new capabilities, how it changes, for the better, how you do security and compliance, and how to start thinking about modernizing all those legacy applications.

Get a reminder to watch it

You can watch it in LinkedIn, or YouTube if you prefer that. Come with some questions! We didn’t get any during the webinar series.

Logoff

I think I accidentally worked out the outline for my keynote at DevOpsDays Des Moines!