Just when you thought (perhaps, hoped) we finally had an understanding of what “platform engineering” is, Gartner and Forrester came out with a Magic Quadrant and a Wave that re-confuses the category.  Or, if you like, helps bring more clarity to the category! Perhaps just a sub-set of it. Let’s find out.

The PDFs reduce it down to CI/CD…mostly…with some optional metrics and IDP seasoning thrown in to varying degrees by each firm.

Or maybe they’re just describing a subset of platform engineering. Or nothing about platform engineering. You see: re-confusination.

I’m projecting the a framing onto these two PDFs that kind of aren’t totally there. I don’t think they actually care too much about staking out a position on defining platform engineering. I think each analyst shop has other PDFs that do that.

Let’s, then, look at my pleasent confusion and use it as a sort of critique of the whole, you know, deal with platform engineering as of  [checks calendar] June 15th, 2023, [checks watch] 9:52am Amsterdam time.

(Oh, and before we start: I work at VMware. We work a lot in this space and some of our products are included in each report. So, like - I don’t know -: figure out if you care about that bias. Moving on.)

PDFs define markets

Both reports are defining a newish category that’s related to most everything about custom software except for the actual programming, but also not about the infrastructure that apps run on or how the apps are managed in production (except for one, weird part, that we’ll get to). The Gartner one is more about defining a new category, the Forrester category already exists in their neck of the woods.

A Magic Quadrant and to a lesser, but still import extent, Forrester Wave feed into both (1) how vendors (and when I say “vendor,” I mean public cloud providers too - I can’t go around using a phrase like “vendor and public cloud providers” all the time - I need space for all these parentheticals) build their products, and, (2) how buyers (“enterprises”) think about their strategies, architectures, and building/buying decisions.

That is: these PDFs are important.

The next 12 months of “cloud native” strategy planning, enterprise sales, PoCs, etc. will include these two PDFs  (and many others! Along with endless blog posts and videos - maybe even screenshots of thought leaders in Twitter and Mastodon [maybe Bluesky - I think that community I much less interested in the enterprise app stack world than, say, well…you know]) as people try to figure out making Kubernetes more usable by app developers, or just hidden from them. That’s why Red Hat and GitLab licensed the PDFs!

The Stacks

These reports describe a stack that comprises a huge part of what most notions of “platform engineering” seem to be. Let’s look at them.

The Names

Gartner calls this the “DevOps Platform” (queue the grey beards who threw a shit-fit about “bi-modal IT” seven years ago to see a fun show! But will it be in Mastodon or Bluesky this time? I mean, no one reads blogs anymore, right?).

Forrester calls it an “Integrated Software Delivery Platforms” (does an “ISDP” include an IDP? idk - but, imho, idt).

What’s in the stacks

Each describes a stack of what DevOps tooling has come to mean in practice: the CI/CD pipeline and all the associated goop. (Please, don’t bimodal-IT me - I know, I know: CALMS [I still maintain it should have been CLAMS, why so serious and all that], DORA, “do the DevOps,” burn out, cool hair colors, systems thinking, sociotechnical systems, blameless postmortems, occasionally observability, jokes about pagers, culture…and all that.).

Said goop in enterprises mostly means compliance and security controls. That’s right: “CIS, NIST, FedRAMP, PCI-DSS, HIPAA, CSA, etc., etc. and so forth”.

Gartner treats platform engineer as a subset of its DevOps Platform category, listing platform engineering as one of the “multiple use cases” for the DevOps Platform category. They define platform engineering as “provid[ing] self-service, internal developer platforms [IDPs] to scale DevOps and software engineering practices.”

Let’s look at the definitions of the stacks.

Gartner’s DevOps Platform

Gartner defines it’s DevOps Platform category as:

Gartner defines DevOps platforms as those that provide fully integrated capabilities to enable continuous delivery of software using Agile and DevOps practices. These capabilities run the gamut of the software development life cycle (SDLC) and include product planning, version control, continuous integration, test automation, continuous deployment, release orchestration, automating security and compliance policies, monitoring, and observability. DevOps platforms support team collaboration, secure software development and measurement of software delivery metrics.

CI/CD, developer/project websites (you know, Backstage), and metrics.

Things get a little crazy in Gartner’s definition when they throw in monitoring and observability. So, like, Datadog, Aria, and Honeycomb and them? No, them’s not in there. However, those two phrases aren’t actually mentioned much in the PDF: monitoring just three times outside of the initial definition, and observability just twice. So, I’d sort of cast the whole monitoring and observability off as fun flourish.

Here’s a good overview of the why/what from a outcomes perspective:

DevOps platforms support multiple use cases, including, but not limited to:

• Agile software delivery — operationalize Agile development practices

• Mobile app delivery — build/test/deliver native mobile and mobile web applications

• Edge computing scenarios — support for secure delivery/update for IoT/edge devices

• Regulatory requirements — support for compliance, auditing, traceability and governance

• Cloud-native application delivery — build and deliver cloud-native applications across hybrid and multicloud environments

• Platform engineering — provide self-service, internal developer platforms to scale DevOps and software engineering practices

Gartner’s write-up of GitLab gives you a good taste of what they mean by DevOps Platform as well:

GitLab is a Leader in this Magic Quadrant. Its DevOps platform is a single product that includes capabilities for planning, source code management, continuous integration, deployment automation, observability, application security testing, software supply chain security, compliance reporting, value stream analytics and incident management.

That’s actually pretty good, except for the inclusion of “observability” (see above).

Forrester’s Integrated Software Delivery Platform

Forrester’s ISDP definition from 2022:

Integrated software delivery platforms enable an automated software delivery process from code build to code release, linking together discrete automation capabilities into a unified and open platform that provides APIs and other integration options. They enable users to customize the platform to suit their unique needs by combining capabilities of the platform with custom or third-party tools.

These platforms differ from standalone, best-of-breed tools in that they provide an out-of-the-box integration across a core segment of the software delivery process. They differ from legacy, end-to-end software development lifecycle (SDLC) tools by offering open platforms that anticipate their users augmenting the solution with third-party tools such as security scanning or impact analytics, enabling users to create a modern tool chain where new capabilities can more easily be added or replace existing capabilities as needed.

Forrester’s ISDP rating criteria is helpful too. In addition to the soft criteria (“vision,” “strategy,” “pricing flexibility and transparency” and the like), the tech capabilities criteria are:

1. Platform capabilities (but, not, like managing Kubernetes…I think?).

2. CI/CD capabilities (this is the most important criteria for them, with an evaluation weighting of 25% versus either 5% or 10% for all the others, aside from “platform capabilities,” which is 20%)

3. Development, security, and operations (mmm…getting a little broad).

4. Extensibility (can mix and match, modify how it works, etc.)

5. Governance and compliance (“enterprise goop”).

6. Infrastructure management (see “platform capabilities parenthetical above).

7. Product support (i.e., not just some project on GitHub that your developers found and are now running in production).

8. Support for test automation.

Metrics Sidebar

Both of these PDFs are interested in metrics, but Gartner much more so. These are primarily the DORA Four-a, maybe some golden SRE ones here and there, and whatever else.

This is understandable given that the four metrics are the benchmark for DevOps goodness, ROI business cases, and, generally, showing that you’ve done a good job on your digital transformation imperatives, pillars, strategy, OKRs, MBOs and KPIs when it comes time for annual reviews and budgeting. Plus, you know, actually running the business.

But, I don’t know.

Metrics are important, but they’re extremely difficult to collect. For one, even five apps, sure. You can kind of just get those word-of-mouth. But once you get to the low hundreds, and especially few thousands of apps across an enterprise, collecting the metrics is difficult and making the metrics consistent enough to compare them apples-to-apples is downright impossible. (Maybe Planview/Tasktop would beg to differ - they’ve heavily invested in enterprise metrics goop for 16 years.)

So much so, that you start to wonder how the respondents to the DevOps surveys figured out what to say. (I was not trained to science the shit out of anything [feel free to ask me about the flaws in most philosophy, minus that eye-rolling period between, say, 300 AD up until Nietzsche finally rescued philosophy from absolutely boring-ass proofs of why God exists] so I don’t know what I’m talking about here with respect to surveys.)

Don’t get me wrong: metrics are important, but it’s asking a lot of this category to support them. “DevOps Metrics” deserves to be its own category (holy shit, the bi-modal shit-fitters would just 🤯 over that one, right?). In theory, if you had everyone in your BigCo actually using the same build pipeline, following the same enterprise architecture and governance, put in place something like Planview/TaskTop, and got apples-to-apples-ized prod data…yeah? An that might be just the aspiration here.

Kubernetes Korner

There’s an interesting feels about Kubernetes in each. Forrester only mentions Kubernetes three times (support is in the roadmap for one vendor, and one vendor [disclaimer: my work] is dinged because “platform support seemed limited to Kubernetes workloads”). Gartner, however, is all over the Kubes.

I think both would agree that their platform do not include managing Kubernetes. Even though the platform they evaluate may do exactly that, those capabilities are another part of those stacks.

This is the most important thing when contemplating how these PDFs fit into platform engineering. Sometime soon the platform engineering community needs to decide if managing Kubernetes is part of their scope, or not. I think the answer should be: no, we don’t want nuthin’ to do with that.

Single-source versus Best of Breed

Forrester hits on what’s the, like, existential dilemma of these two categories, I think: build versus buy. Both reports discuss this, but Forrester is much more interested in this idea (rightly so). Based on their conversations with IT leaders:

while these ISDP tools hold great potential for a complete end-to-end toolchain, many individual capabilities fell short of enterprise needs, forcing these leaders to source those capabilities from separate best-of-breed vendors.

Our reference customer survey backs this up by indicating that enterprises are using (on average) only 53% of the capabilities they are paying for, with the remaining capabilities either being delivered by a separate vendor or simply not used. This means that ISDP vendors are not only competing with each other but also competing with the best-of-breeds. ISDP vendors recognize these challenges and are responding with a number of innovative solutions that will make them more attractive than a traditional DIY toolchain but open and extensible enough that end users can add the tools they want without sacrificing capability. The challenge is striking the right balance to create overall value.

This is what’s going on with all of this: do you buy one, integrated stack from a vendor, or assemble it yourself from parts?

There’s probably (I’d hope!) a companion piece from another part of Gartner, the former Burton Group, uh…group (apologies, I forget what they’re called just now: GTP?) that goes over how you’d build your own platforms and how to figure out if that’s a good idea or not. The parts of Forrester and Gartner that make the PDFs we’re looking at are interested in rating things you can buy, not things you can build.

Now, I mostly hate to believe this, but I think you have to go with the idea that people are going to build their own stacks. And by “people,” I mean application developers. They’re going to first assemble stuff from free (read: open source) components or already paid for (read: they don’t need to open a ticket to get access, let alone figure out how procurement works) cloud services. The app developers will get bored after a year (less, even), and frustrated when they have to deal with the above mentioned enterprise goop. Then they’ll hand it off to an infrastructure group (the one who stood up Kubernetes and figured out reining in YOLO public cloud usage across the enterprise). To which the infrastructure people will be like, and I quote, “whaaaaaaat da fuck?…,” let out a big sigh, mutter something like “FML,” and then call up all us vendors (maybe some outsourcers if there’s an existing, multi-year contract in place), and be like “got a mop for this?” And us vendors reply “Golly-Wow! That’s a new one!” and tag that account as “highly qualified.” The infrastructure people get good lunches for the next three months. (Pro-tip: look-up when your vendors quarters end, especially Q4.)

…I mean: right?

Related to that: there’s an implied notion as well in the vendor selection and discussion: if you choose one of the public clouds (AWS, Azure, Google), does their big ol’ portfolio of cloud stuff include these capabilities, and integrate together enough to be good?

A welcome addition to to definitional stew - it’s tasty!

Much like those bespoke platforms application developers build, I think of the platform engineering category as a ball of tentacles that’s flailing about, slowly collating into a solid core.

And, I haven’t even started to look through this year’s PlatformCon videos to smell out the new tentacles yet!

Listen, I like these PDFs.  If you asked me if I like analyst PDFs I’d be like, “I used to be an analyst and do M&A. I like PDFs. That’s what I do. That’s what I live for.”

They define a good grouping of tools that most every shop doing custom software will be doing, or already does - aside from that observablity and monitoring part: ¯_(ツ)_/. The ideas in the reports are valuable to contemplate when trying to sort out what platform engineering is. I’m pretty sure those ideas are part of DevOps too - one of the reports is even called the DevOps Platform. I’d say they’re defining a subset of “platform engineering.”

Here’s how the CNCF defines a (cloud native) platform, which I think is pretty damn good, the best we have so far (that pastel palette is questionable…but that’s just, like, my opinion, man):

You can see that what’s in these PDFs fits mostly on the right side, of the middle part, of the burger. Gartner is more into container registries than Forrester is - I suspect any platform definition will need to include image/container registries going forward.

It’s clear that there’s a lot of definitional work ahead of “platform engineering,” probably for the next year or two. I think it’s inescapably tied to Kubernetes. We used to know what a platform was and we have great, proven ones. Now that we’ve emerging out of the Container Wars with Kubernetes as the agreed on winner, it’s time to climb back up the stack and help application developers.

These two PDFs loosen that tie to Kubernetes which is important to pay attention to when it comes to defining platform engineering. And, I agree with them! Platform engineering probably needs to have little care about Kubernetes, just like the culture-faction of DevOps has little care about Kubernetes as it has with whatever other infrastructure has been in the rotating cloud door since ~2007.

My prediction is that “platform engineering” is going to come to mean “DevOps tools and culture/practices + IDPs + being able to deploy to Kubernetes.” Managing and care and feeding of Kubernetes will rest with either the public cloud providers or Infrastructure groups. Let’s hope at least. We’ll see!

(And, if you can’t get enough of this, we discussed the Gartner MQ more last week in our Software Defined Talk episode.)

Upcoming

Talks I’ll be giving, things I’ll be doing, places I’ll be going.

June 21st Cloud Foundry Day, Heidelberg, speaking. June 21st Making digital transformation stick in government agencies, online. June 22nd to 23rd DevOpsDays Amsterdam June 28th, July 4th, July 11th Cloud Native for Financial Services talk series.August 21st to 24th SpringOne & VMware Explore US, in Las Vegas. Sep 6th to 7th  DevOpsDays Des Moines, speaking. Sep 18th to 19th SHIFT in Zadar.

Logoff

I’ve been neglecting the newsletter of late, but here we are! I’ve got many links stored up, and some little notes. In the meantime, and while it’s also getting some neglect, you can follow my blog for links, pictures, and day notes. As noted in the Upcoming section, I’ll be at Cloud Foundry Day next week. It’ll be fun to see that group. I’ll re-highlight this in a newsletter tomorrow - or whenever the next one is - but check out this interview with Cloud Foundry Ram.