"We can’t hire the right people” is the eternal complaint of CIOs everywhere. They’re beset by all this FUD’ing out about Facebook coming and disrupting the toothpaste industry, Google accidentally dis-intermediating car insurance, Elon Musk launching a bank1, or Amazon doing anything new.
I call this freaking out the “macro-economic headwinds” part of any pitch, freakout, or “TED-style” talk. The CIO’s reply is often: if only they could just hire the right people to sling the code, install Kubernetes the hard way, write good ChatGPT prompts, or whatever, they could turn those headwinds into tailwinds.
We’re trying our best, the executives clamor. But, the skills! Our people just don’t have the skills! So, like, sorry: can’t improve.
CIO goes: ¯\_(ツ)_/¯
There is, of course, this:
And there has been this for a long time as well.
The Skills Problem came up a lot during the VMware Explore executive summit I was at earlier this week. Instead of just being mopey about it, though, there was a lot of talk about how to solve it. Most of it sprung from the notion of: instead of focusing on hiring new people, focus on training your current staff.
Solving the "skills" problem is possible, people were saying. It just takes some planning and operating changes. One plan was very prescriptive: (1) put in place all you can eat self-directed training services like Pluralsight and O'Reilly; (2) plan out (in person) classroom-like training for new technologies and practices that you’re planning on using; and then (3) activity encouraging staff to always be learning. You can “enforce” these with annual review metrics, sure, but if you just make it part of expectations and allocate time with it, most people in IT will be happy to learn. Some enjoy stacking up certificates, which is fine if it works for you.
If "skills" means that you either (1) can't afford to hire people who have the skills in the new tools and ways of working you want to do, or, (2) that your existing people are not learning how to use new technologies, then solving #2 is likely easier and cheaper. It just has to become a high priority: training and education.
Fortune 100 CTO: "We can't copy Netflix because it has all those superstar engineers, we don't have the people."
Adrian Cockcroft, Netflix: We hired them from you, and got out of their way..."
A good way to discover how high a priority "skills" is in your organization is to track how much time executives spend on planning, meeting, and budgeting on that topic. You can also track people's use of the self-directed learning platforms. As ever with management, the managers are the ones responsible for the overall system of the "organization," so if skills are a problem, that's something they have in their power to solve by changing the organization, how it works and allocates resources (time and money).
The second related topic that I found interesting was making people happy at work, and happy with their work. Since knowledge work (most IT work) requires a lot of self-motivation to learn and adapt (see above skills problems!) and to problem-solve with open-ended, unknown problems, you need people who are in a good state of mind, not grumpy and unmotivated. As one discussion participant put it "a happy developer is a productive developer," which applies to any role, of course.
A lot of the important work in studying developer productivity over the past few years has been on this topic: how do you create the environment for thriving IT staff. There's the SPACE work from Microsoft and others, and some work from this year that I just discovered at Monktoberfest from the Developer Success Lab. A lot of this work is about metrics and measuring, but there's also a lot to learn about creating those happy employees who will then thrive.
If skills are a problem, that's great! Unlike all those macro-economic headwinds that executives have no control over, they can actually do something about that by putting in place operational changes. As ever: you just need to do it.
I’m Barcelona, you can always get an Uber in five minutes if you don’t mind waiting fifteen minutes first.
I propose we call all official conference events in the evening “soirees.”
AV guy at #VMwareExplore “I presume this is another talk about computers? You should try doing one sometime on wildlife.”
“an activity that is restorative, intentional, relaxing.” Here.
Sometimes, being the person who knows how to print something out at their hotel is the most valuable thing you can do that day.
Big Blue Can Still Catch The AI Wave If It Hurries - I am not a chip person, so I don’t understand any of this. Sounds good for IBM if they commercialize it and make some hoopla out of it?
OpenAI introduces GPT-4 Turbo: Larger memory, lower cost, new knowledge - Updates to ChatGPT (and API updates too). A much longer memory for conversations (the “context window”) is what I care about most, that is: “The model is 2.75 times cheaper than GPT-4 and has a context window equivalent to 300 pages of text per prompt”
Forrester has some succinct coverage of OpenAI the announcements, and, of course, some prescriptive advice.
Gartner Says AI Ambition and AI-Ready Scenarios Must Be a Top Priority for CIOs for Next 12–24 Months - This either means people are barely starting, or that you don’t need an “AI vision statement”: “A Gartner survey in June 2023 of 606 CIOs and technology leaders found that only 9% of organizations have an AI vision statement in place, and more than one-third of respondents had no plans to create an AI vision statement.”
The 2023 State of Kubernetes in Production - Embrace the Kontainer Khaos, just get some good management tools: “If you’re trying to standardize and have the same stack in the same environment for all your applications – that’s not realistic for today’s IT environment. It’s never been realistic,” Newman said. “The idea instead has to be how do you manage diversity, rather than how do you limit it.” // A survey on Kubernetes.
I’m at VMware Explore EU this week. There’s a lot of VMware announcements. Here’s some coverage:
VMware makes data services push with help for DBAs and devs - Soon, it’ll be a metal to UI private cloud stack.
While it waited for Broadcom, VMware set out to do to data services what it did to storage, The Register.
FinOps for Kubernetes in Multi-cloud, Multi-cluster Environments with Tanzu Mission Control - FinOps in VMware Tanzu land: “see the fully loaded cost of their clusters in the Tanzu Mission Control GUI. The cost will be based upon information added to Tanzu CloudHealth related to their custom pricing with AWS cloud and Azure platforms. The widget will display both single cluster costs and the total cost of a cluster group to support teams in making informed decisions for their Kubernetes environments.”
VMware Explore - VMware focuses on cloud journeys and AI implementations.
We went to a small Italian place in Barcelona called Teta de Monaja last night: it was great! I feel like a mother and son were running the place. At least, it had that vibe. And the food was tasty and affordable.
Actually, as pointed out by many, this is probably not a problem: would you trust your money to to that business?
Here’s the final installment of my talks with Here’s the third little interview I did with Torsten Volk at EMA research. We talk about things that slow developers and other staff down. He’s done a great, very thorough look at Kubernetes usage and the state of things. You can get it for free thanks to my work.
“To ‘help gamers keep the crunch to themselves,’ Doritos is debuting what it calls ‘Doritos Silent.’ Gamers download Doritos Crunch Cancellation software and when the technology is turned on, the software detects the crunching sounds and silences it while keeping the gamer’s voice intact.” Tyler.
“I’m local, but I shot the bingo finals. Almost got an award for that!”
“Andreessen-style gibberish.” Here.
Oh, you know: It’s the usual bullshit.
I saw someone working in QBR slides in the plane. Planned versus won, targeted funnel versus actual sales. There was a lot of customers slide work - some embedded Excel sheets which they would go edit. These metrics are all, largely the same. It’s hard to know what any company would get out of customization them more than a set of three or five slides. I’m sure Salesforce just spits out oodles of them. These are the kinds that slides that, if you need to make custom ones, something is wrong with your sales ops stack, or worse, you might be trying to arts and crafts around the cold numbers. And, yet, to suggest they everyone should stop it with these custom slides, charts, and analysis and just use, like, 20 universal ones, is clearly bonkers.
Hilariously, the defaults in PowerPoint and Excel are probably enough. Simply mandating that you don’t use custom slides, Smart Art, and charts would probably fix a lot of white-collar waste.
“FOMO continues to outweigh FOF (fear of fraud).” Here.
"That’s turbo-amazing!
We are watching Adventure Time again, this time for the three year old. It’s, of course, wonderful.
“And check out these classic stylings! They don’t make them like this anymore!”
Subscribe to read my polemic: Forks Have Ruled Too Long - Spoons are the Best Cutlery.
“I don’t know how to make coffee. I’ve tried. I cannot master it. I’ve really tried.” Here.
“How’d you sleep last night?” “Terrible! I couldn’t control that stupid heat - it’s cold, it’s hot!” American problems during the grand tour.
Shift-down - Seroter’s term is taking off!
More promo of the CNCF platform maturity model.
You Keep Using That Word - He’s not a fan: “given that the industry is not known for creative, timely or even useful category descriptors, the next best step might be to gradually phase out or minimize the usage of the term edge in favor of terminology that actually communicates something that can be commonly understood.”
I mean, these things are pretty annoying when they’re used by negligent speeders - Amsterdam considering license plate requirement, minimum age for fatbikes - “Last week, the police checked over 50 fatbikes at various places in the city and found that a third had an illegal throttle.”
G.I. Joe PSA Compilation, Remastered - still amazing if you’re into complete absurd art.
Dell GA’s APEX Cloud Platform for Red Hat OpenShift - Doing this kind of “it’s all integrated private cloud box” always seems like a good idea for Dell. But I never know if it’s a success. Also, does it actually work to talk about developers or in this context, or should the focus be all on the infrastructure people that are being asked to support (cloud native) developers? It feels like if you go to an application developer with this kind of pitch, unless you’re Bryan Cantrell or dhh, you’ll be hardcore ignored. Also: hopefully Dell gets dhh to keynote Dell World. The thought leadership for hardware vendors doing this probably needs to be explaining how application developers work and what they need to the infrastructure people buying and running the gear. I’m not sure talking to “developers” gets you much of anywhere, unless you can do it Cantrill/dhh style.
Talks I’ll be giving, places I’ll be, things I’ll be doing, etc.
Nov 6th to 9th VMware Explore in Barcelona, speaking (twice, and at a booth). Nov 15th DeveloperWeek Enterprise, speaking.
I’m at VMware Explore in Barcelona this week. That means I’m not at KubeCon, but there’s plenty of my team-mates and co-workers there. Go say high to the Carvel people and the folks at the VMware booth - tell ‘em I sent you. Also, Matt Ray will be there for his FinOps stuff, OpenCost?
I have three talks at Explore this week: a VMware booth/theater one at 2:30pm tomorrow, a panel on years of running developer platforms on 12:45pm on Wednesday, and then a talk about platform marketing (getting developers to use your stuff, and product managing it) at 2pm on Wednesday. The way they do listings for talks is a little weird, but you can find the talks listed here.
Here’s the third little interview I did with Torsten Volk at EMA research. We talked about managing Kubernetes, especially if you’re a large organization that has lots of everything. You know: MULTI-CLOUD, Y’ALL! He’s done a great, very thorough look at Kubernetes usage and the state of things. You can get it for free thanks to my work.
Watch it if you’re into this kind of thing.
“If you’re a baker, making bread, you’re a baker. If you make the best bread in the world, you’re not an artist, but if you bake the bread in the gallery, you’re an artist. So the context makes the difference.” Marina Abramović from here.
Related: The more modern art I see, the more I think: “I’m pretty sure all these people were totes shit-faced and horny as fuck all the time.”
The Zurich airport is super chill.
“‘malicious compliance,’ meaning you have completed something that was requested of you, simply to illustrate the stupidity of the request.” April Dunford
We act like doing public transit is impossible. As Europe proves, what we really mean is that we don’t want to change, pay the costs, wait the time. Most things in “American Exceptionalism” are like this: we just aren’t imaginative enough, nor have enough faith that it’s possible to change. The Exceptionalism also rises because of how well isolated America is, geographically and culturally. We simply don’t have an aggregate first hand experience outside of our own. I mean, you know, that sounds like some weak, “oh, I see you went to college, fancy-ass,” thinking. But even in our own massive country, we get locked into the thinking of our geography. It’s like knowing only one religion from the whole history of humanity, and never being exposed to the hundreds of alternates across the world and across all of human history. If that one religion was caveperson animism, the idea that you could instead worship a human-like figure or an inedible Nothing, or just think the whole idea of religion was more akin to fairy tales…all of that would seem stupid, impossible, and wrong. But! Here we are, most of us not worshiping drawings on cave walls.
“I like orderly confusion very much. But this is neither orderly, nor properly confused.” Dieter Rams.
“Hot yak stew is brought, and ale, and you are soon feeling very comfortable as Grokkung tells you what his warriors have learnt recently.”
“embracing momentary boredom” - I read this while I was bored waiting for my Uber to come…har har har…
Talks I’ll be giving, places I’ll be, things I’ll be doing, etc.
Nov 6th to 9th VMware Explore in Barcelona, speaking (twice, and at a booth). Nov 15th DeveloperWeek Enterprise, speaking.
Discount code for KubeCon US - while I won’t be at KubeCon US this year, my work has a discount code you can use to get 20% off your tickets. The code is: KCNA23VMWEO20.
This is all I have for you today:
Here’s the second little interview I did with Torsten Volk at EMA research. We talked about security concerns with Kubernetes. He’s done a great, very thorough look at Kubernetes usage and the state of things. You can get it for free thanks to my work.
Torsten says that last he counted, there were over 140 distress, services, and different ways of getting Kubernetes. As I say, this is probably too many. But how many should there be? I don’t think we actually give a number, but we think through it a bit.
Watch it if you’re into this kind of thing.
Yesterday, I referenced some 2022 Gartner estimates on containerized applications. By coincidence, I came across an updated estimate from them:
By 2028, more than 95% of global organizations will be running containerized applications in production, which is a significant increase from fewer than 50% in 2023.
By 2028, 25% of all enterprise applications will run in containers, which is an increase from fewer than 15% in 2023.
[And, later in the report, summarizing the state of things in March, 2023:] Gartner estimates that close to 50% of enterprises are running containerized workloads in production and nearly 15%1 of total workloads run on containers today.
This is from March, 2023. So, going into CY2024, I’m guessing it goes up 2% to 5%, depending on what the growth curve looks like. Also, notice that the 25% figure has been extended by a year. The 2022 version of this report had it 25% in 2027.
Here is a crude chart using dates from the 2022 and 2023 report. This kind of chart is dicey to use, so consider it for entertainment purposes only:
You can get a copy of the report linked here. Also, if you’re into these kinds of charts, here’s my most recent collection of Kubernetes and platform engineering related charts (I hesitate to call it “data,” it’s just charts), including the updates above.
“Costservability” Brian.
“Optimist: The glass is ½ full. Pessimist: The glass is ½ empty. Excel: The glass is January 2nd.” Here.
I’m not really into the idea that devrel people don’t “sell.” (1) If you’re from a public cloud company, you’re going to use your work’s stuff for demos. (2) You don’t need to sell products, you just need to sell the idea of way of working that aligns with your company’s market definition. Just go watch the greats of Kubernetes, Spring, AWS, Microsoft, etc. They call this “thought leadership.” (3) If you’re not selling, you probably need to improve your “non-vendor vendor talk” skills, which are surprisingly not hard to improve if you get over the false idea that it’s “icky.” I mean, have you seen how much those new MacBooks cost?
Related: “a bullshitter who delivers” Benedict Evans on Elon Musk.
I'm hoping to have a DEEP heart-to-heart about the use of powdered cheese in an American cuisine classic: Kraft Mac and Cheese.
I’m pretty sure that the next plot point in this developer productivity metrics nerd-fight is someone saying “this is a meaningless phrase made up by people to sell things.” Reverse the FUD-flow!
Q: A couple years later, what has “DevSecOps” ended up meaning? A, from @kerfuffle@mastodon.online: “It still just means "make sure the concerns of each discipline are properly anticipated by the others so they don't become apparent too late and impede delivery", but that isn't reflected in job descriptions.
In practice, #DevOps is an Ops engineer who uses infrastructure as code, #DevSecOps is someone who sets up a #CVE scanner in the delivery pipeline, and only few folks think about the silobreaking mentality of mutual understanding that it all was supposed to entail.”
Announcing the Platform Engineering Maturity Model, CNCF - I want to dig into this further. But, I did an early review of it and it’s great - a very useful tool if you’re putting a platform in place (and, I’d argue, if you’re doing some kind of “Kubernetes strategy”).
OpenCost Expands Its Horizon: Introducing Multi-Cloud Cost Monitoring! - “open source cost monitoring for cloud native environments”
IBM to scrap 401(k) matching, offer alternative benefit - I’m no US accountant, but this both weird and against the norms of white collar employment. The 401(k) is one of the backbones of how the middle class system works in the US: best not to fuck with it. Insert “workers of the world,” etc.
The World’s Most Popular Painter Sent His Followers After Me Because He Didn’t Like a Review of His Work. Here’s What I Learned - Case study of parasocial relationships: “Artists' personal stories have long been part of how art is marketed, from Vincent van Gogh to Frida Kahlo, but in those cases, the artists' paintings attracted interest first, and the biography became part of its legend as its fame grew. Today, personal biography and narrative are more important than ever in the gallery–most art comes equipped with some kind of story. But social media gives an added twist: Hordes of people can feel as if they have a relationship with a painter like Devon Rodriguez without ever having had any direct experience of his painting at all.”
The CFP for cfgmgmtcamp 2024 is open, here. This is a good, under-the-radar conference in Ghent, Belgium, especially for Kubernetes, SRE, DevOps-y kind of talks. I’ve been many times (and finally spoke last year) and it’s always more than worth my time to go.
All the greats often make it. I usually end up spending a couple hours sitting in the cafeteria catching up with someone.
I haven't checked, but you can usually also submit a multi-hour workshop. One year, for example, Tasty Meats Paul (and someone else?) did one along the lines of "getting Kubernetes up and running." It's usually timed to be the week after FOSDEM to attract that crowd and speakers.
If you’re on the “infrastructure stuff” circuit, you should check out speaking there. And, definitely attending if you can.
Here's the talks from last year to get a sense of topics.
Talks I’ll be giving, places I’ll be, things I’ll be doing, etc.
Nov 6th to 9th VMware Explore in Barcelona, speaking (twice, and at a booth). Nov 15th DeveloperWeek Enterprise, speaking.
Discount code for KubeCon US - while I won’t be at KubeCon US this year, my work has a discount code you can use to get 20% off your tickets. The code is: KCNA23VMWEO20.
It’s getting colder here and we got a big pile of wood for the fireplace. I used to know how to light a fire, but now I have no idea.
Meanwhile, if you’re wondering how I’m doing, this best represents my combined current mood about things:
I mean, I guess “fewer than 15%” means 13% or 14%, right?
…
Here’s a little interview I did with Torsten Volk at EMA research. We talked about security concerns with Kubernetes. He’s done a great, very thorough look at Kubernetes usage and the state of things. You can get it for free thanks to my work.
As ever with security and a new technology, there’s a lot of uncertainty and finishing off the security features as a new technology is used more and more in the mainstream. And this is exactly what you see with Kubernetes now, especially in “the enterprise.” Despite the “everyone’s doing it” feel, when you look at estimates for how many applications are running in Kubernetes, it’s sizable and growing, but far from “everything”:
[B]y 2027, 25 percent of all enterprise applications will run in containers, an increase from fewer than 10 percent in 2021.
From: Gartner, CTOs’ Guide to Containers and Kubernetes — Answering the Top 10 FAQs, Arun Chandrasekaran, WataruKatsurashima, 31 May 2022.
In my mind, this means that something around 15% of enterprise applications are running in containers. Are those containers running in Kubernetes? That’s harder to tell - there are many container-based platforms from the Container War days, including the Cloud Foundry ones, probably some Docker-based ones, etc., etc.
But, I’d assume that a lot of those containerized applications, especially the growth that moves the estimate up to 25% will be running in Kubernetes. You know: yeah, probably.
As more and more enterprise applications are created and migrated to Kubernetes, things will quickly be discovered, proactively fixed, and things will be peachy. Such is how technology goes.
The other things driving security in the Kubernetes world is that it’s a stack designed to be DIY’ed. Instead of one, integrated and unified stack that’s polished off, you can assemble together all the parts. You know, because you’re probably a victim of the “we’re special and so we have to tinker with the stack” anti-pattern. The number of combinations of that final stack, then, are massive, and each time you customize the stack, you open yourself up to screwing up the security. So it goes with building your own stacks! Hopefully you’ve done your Wardly map analysis, or whatever.
Being the one interviewed is great - just like a being on a panel, you pretty much just show up and talk. Easy and fun.
Watch the interview, it’s only 10 minute long!
A consultant would clearly say that one problem you have is refusing to accept outside help. Here.
“Yet sometimes, I suffer from what you might call achievement fatigue: I question my motives, I ask myself what the hell I’m doing or supposed to be doing, and sometimes, I cut myself some slack.” Achievement Fatigue.
‘the “pressure to presence.”’ Here.
I get all my news from The Register.
OpenAI’s ChatGPT can look at uploaded files in the latest beta updates - I’m thinking that, soon, ChatGPT will be a better way that create images than Midjourney. Midjourney’s prompts and refinement method is way too complex and uncontrolled. It’s very difficult to figure out how to say things like “add a skateboard to the second image.”
PowerPoint gone crazy in the US Military (2010) - From 2010. Yes, and: it’d be cool if there was something better - memos?
TikTok Star Devon Rodriguez Is Now the Most Famous Artist in the World. But What About His Work? - “The case of Devon Rodriguez is more evidence for the shift in emphasis from consuming art as content, to consuming artists as content.” And: “In 2023, Rodriguez is essentially in a race to develop an audience with a more-than-superficial interest in his actual painting faster than his social-media presence is drained of goodwill through over-exploitation.” // Also, a bunch of fans apparently got angry at this. I mean, of course: the critic said the art wasn’t good enough to be art. This is what the Internet can do for us.
Leaving Twitter, Benedict Evans - “This is often the real challenge to tech incumbents: once the network effects are locked in, it’s very hard to get people to switch to something that’s roughly the same but 10% better - they switch to something that solves one underlying need in an entirely new way.” // I’m starting to think that the whole innovator’s dilemma is a much, much rarer occurrence/success than we assume. Instead, there are often deep-pocketed incumbents who simply make a better technology/product and/or are more successful at driving market adoption (sales and marketing that leads to purchases). Plus, sometimes, the dominant companies just get lazy.
Talks I’ll be giving, places I’ll be, things I’ll be doing, etc.
Nov 6th to 9th VMware Explore in Barcelona, speaking (twice, and at a booth). Nov 15th DeveloperWeek Enterprise, speaking.
Discount code for KubeCon US - while I won’t be at KubeCon US this year, my work has a discount code you can use to get 20% off your tickets. The code is: KCNA23VMWEO20.
We had a great week off in Spain (Sitges, to be exact) last week. I’m not often able to completely relax and disengage from life on vacations, but this time it worked. We had a place right across the street from the beach, and we didn’t worry about spending our time wisely. So, we wisely spent our time just going to the beach each day.
I have travel for the next three weeks, including tonight. Work travel has been a huge part of my life since 2006, and I’m not really sure what my role and purpose would be otherwise. But, it’s starting to seem like something I should figure out. Got any ideas?
Suggested playlist for this episode.
I’ve got a mostly newish talk coming up next week about people’s fear of change in organizations, on Oct 24 2023 at 11:00am Amsterdam time.
It’s narrowed down, of course, for software and ops stuff. You know: all that digital transformation, cloud native stuff I’m always on and on about. It’s the last in the series I’m going with my co-workers Bryan Ross. He’s great to do these with because he’s been an actual practitioner so he bring, like, real content to the talks.
Check it out, it’ll be free to watch, of course. And if you can’t make the time, register for it anyhow and you’ll get an email when the recording is up.
“My favorite thing someone says to one of my friends is, ‘Why isn’t she famous?’ I love when they say that because that means they think maybe I’m good enough to be famous. To me, famous looks like a lot of work.” Toni Price
“too cool for the music business and too sensitive to chew their own steak.” Here.
“Feelings are real and life’s hard. You learn to suck it up. Expressing feelings is important. So is MANAGING them. If adulthood is ONE THING it’s that.” Ibid.
Cont.: I do not know how to operate the “expressing feelings” part. It is a weird, scary notion for me. I don’t understand the psychological, uh, “mechanics” of that: how “being heard” results some discharge of the toxic mind-goblins that dance around your head. And yet, everyone does it (boy, do they!), and it seems to work fine. To me, this “being heard” is complaining and, I think, my aversion to it is that I instantly react by thinking I’ve now been assigned a task that must get done - more shit for me to worry about and do, that is not even my “shit.” But I think that is wrong: you have to separate our complaining…wait, I mean…“being heard” from someone asking you for help. The problem I have there is that I’m not calibrated to know the difference. And people get really upset if you ask them “what, if anything, do you want me to do about it, or are you just complaining, I’m sorry, looking to be heard?” So, following the golden rule, I don’t “express feeling,” as I don’t want to subject people to this. But I think most people don’t suffer from the mis-calibration or ability to figure it out. Yeah?
Viz. I don’t like to believe this, but complaining is often just releasing pressure, nothing to be taken too seriously.
Don’t think of it as increasing developer producing, think of it as getting to a good baseline by removing waste.
“in the canvas of life, incompetence is my brush.” Here.
Hacking you bureaucracy.
Johnny Webinar.
“Pigeonhole.”
Sameboatitious.
“Let me, let you, let me go.”
Shift left without shifting burdens left.
Necessity is the parent of unflappablity.
“I like Swiss cheese, unless I’m with four or more people.” Deep cut.
“I used to be a programmer, but now I’m a professional. ‘Hello, World’ application developer.”
“The commercial internet was still in its infancy. We could judge it based only on its potential, rather than its results.” // This is the thinking I’ve been missing from my obsession with the diffusion of innovation curve. Early in the cycle, we judge the technology by the optimistic potential it has, not by the yet to be known actual value of it. Coupled with the hype cycle, you get the notion that the initial hopes and dreams (potential) is way overhyped, and resets itself to the trough of disillusionment.
More like the “trough of reality,” amiright?
New web show idea: The Webinars Family.
Software Defined Talk Podcast, Episode 437: The Let it Ride Lifestyle - This week, we discuss Amazon embracing Microsoft Office 365, offer some SBF hot takes, and review the lessons Docker learned when building an open-source business. Plus, we share thoughts on the new Apple Pencil, USB-C, and some Tim Cook fan fiction.
Tanzu Talk Podcast, Keep it small to make big changes, with Betty Junod - You always hear that making a series of small changes is better than BIG BANG change. How does this really play out though? Doesn't that seem a little...naive when it comes to large organizations doing the BIG and MISSION CRITICAL work? In this episode, Coté [that’s me!] talks with Betty Junod on this topic. She draws from many sources like her first hand experience as an executive and working with the DevOps and cloud native community. They also discuss making culture changes in organizations and how recent work to figure out how to determine "business value." You can also watch the video version if you prefer that king of thing.
Three More Things - Good suggestion here to think of video as a form of writing. Indeed, all those commentaries on “TV” are written after all, they’re just reading them on, uh, “tape.” And: “I don’t know what it means (yet?) to think by means of video editing.” And this, which explains the more important thing to ponder about AI stuff: “Generative models suggest writing doesn’t involve thinking at all, and to me that feels like existential invalidation.”
Running and depression: Headlines suggest exercise is just as effective as meds. It’s not. - “the evidence also seems to show that as a practical intervention, exercise has limited applicability to real people in the real world. And there’s currently no good reason to believe that it will be as effective as medication.” // Generalizing, there are a category of cures for problems that I file under “if I could do that already, I wouldn’t have a problem.” In this case, if you could manage to start and stick to exercising three times a week, your life and mental state would probably already be better. The same goes for “digital transformation” guidance: if you could just start following all the agile and DevOps practices, then you’d already be in in good shape. “If lived, you’d home already.” Or “if it’s in stock, we’ve got it.” And: “the secret to wealth is to become a millionaire in your twenties.”
What Should We Measure? - “Slack Time. Speed of Processes. Condition of Most-Edited Files. Escaped Defects. Net Promoters.”
Acquisitions - “Why can’t people just be content with their yearly revenue? Why does everything have to be bigger—and clearly, not better? If the revenue curve doesn’t go up each year, blind panic seems to help us gravitate towards that poisonous philosophy: why not try an acquisition or two to give sales a healthy boost?”
There’s always money in the infrastructure stand: “IDC recently published our first GenAI forecast … half of the opportunity over the next 18-24 months will be in infrastructure… IDC Forecasts Spending on GenAI Solutions Will Reach $143 Billion in 2027 with a Five-Year Compound Annual Growth Rate of 73.3%,” from Matt Eastwood.
VMware Explore North America 2023: Customer Spotlights - “Despite having over 75 developers, OneMagnify efficiently manages their current Tanzu Application Service environment with less than one full-time operator, thanks to comprehensive automation built into VMware Tanzu.”
“One of the biggest dangers with being someone who considers themselves ‘results driven’…” - “What Buckingham and Goodall discovered was evidence of the old adage, “employees don’t leave companies, they leave bad bosses”. Team culture, not company culture, they argue, has the largest and lasting impact on how we do our work.” And: “One of the biggest dangers with being someone who considers themselves ‘results driven’ leader is that you can suck the intrinsic motivation out of people’s work, thus robbing them of purpose. This typically happens when leaders become obsessive about hitting targets that have been set, even when they make no sense or have become abstracted from their purpose or intended outcomes.” And this intriguing one: “The problem is that the organisation’s leadership have ambitions greater than the resources they are willing to expend on those ambitions. As the management consultant cliché goes, they have champagne tastes and mineral water budgets.”
Nvidia staff are still working from home - More of a round up of wfh tech company policies.
What non-standard items do you always travel with? - Travel gear, mostly for the work trip, but some good ones to have on hand for car trips with kids.
iPhone 15 Pro Max Camera Review: Depth and Reach - Check out those photo from an airplane window.
Would generative AI have made me a better software architect? Probably. - Yup.
Talks I’ll be giving, places I’ll be, things I’ll be doing, etc.
Oct 10th, 17th, 24th talk series: Building a Path to Production: A Guide for Managers and Leaders in Platform Engineering Nov 6th to 9th VMware Explore in Barcelona, speaking (twice!). Nov 15th DeveloperWeek Enterprise, speaking.
Discount code for KubeCon US - while I won’t be at KubeCon US this year, my work has a discount code you can use to get 20% off your tickets. The code is: KCNA23VMWEO20.
I’m off to a week’s vacation in the south of Spain. We’ll see how the newsletter’ing goes.
If you’re interested enough in this newsletter to have scrolled this far down, perhaps you could give me some Halloween cheer by recommending it to other people. You know: I love seeing the numbers go up!
If you thought yesterday’s edition was way too long deep into some stinky, weeds, today’s episode is for you!
It works well enough, I think, given how cool they look. The scaling is obviously not perfect, but it’s fine. I just did some basic prompts, with the only difference being a stream or not and “flat” or isomoprhic. With a combination of using seeds (for consistency and style) and figuring out really good prompts, I think it’d work well.
As ever, DALL-E seems to default to some cheesy cartoon style, while Midjourney defaults to MCU epicness. I’m sure this could be tuned, but I dislike DALL-E’s style for sure.
Another good test would be upload classic Dungeons and Dragon maps as seeds/inspiration and see what happens. Also, zooming out would be interesting. Here’s a 2x zoom out I did:
The idea here would be that you could make very large maps that you could then zoom back into for actual battle maps. The fully zoomed out one could be a full, big map hex. The problem with the above, I think is that you’ll end up zooming out of the picture. Witness, using the last one above:
Finally, here is something interesting. In Midjourney, the prompts tend to be descriptive of the picture. In DALL-E you can just write something and say “make a picture of that.” So, I took a newer description of my D&D setting (here’s an older version) and asked DALL-E to make a map for it:
That’s promising! It even got real text on there, which Midjourney doesn’t do. With these maps, it would be especially good at places you were just making up on the fly. I have a very specific idea of what this map should look like because I’ve - uh, you know: hobby! - thought about it a lot. But if I just wanted to make something up with no preconceived idea, these would all be great.
So, you could image using the ChatDM to help come up with adventures and, along the way, come up with maps and stuff. I’ve used Midjourney in the past to make pictures of custom magic items which was great.
“Jiffle is now live for Explore.”
My friends, The Capitalists.
“It’s ok - we can filter you out.”
“DevOps World in New Jersey”
Apple introduces new Apple Pencil, bringing more value and choice to the lineup - Much cheaper then the previous ones, but lacks a couple of features as the expensive one.
The Developer Platform - Pretty good overview of all the platform stuff, including marketing and product-think.
I don’t know what it is, but this whole thing just seems so cool to me:
I mean, I love the actual song. The part at the end where he’s all like “we’ll discuss it further” kind of jolts you out of the song, but, whatever.
But I think it’s also some vague nostalgia I have for 70s and 80s culture. There’s a tired Gen-X thing where you’re like “hey, man, back in my day, before the Internet, we all stewed in the culture artifacts since the 1940s. Those people were still alive - so many World War Two movies and Bugs Bunny pretending to be Groucho Marx.” At some point, my parents must have been really into Yacht Rock style music - or maybe it was just on the radio all the time and I heard it. That song came out as year before I was born, so it’s not like I was watching it climb the charts: it must have been radio play in the 80s and my parents playings it. (My dad listened to a lot of Heart and Stevie Nicks on the 45 minute commute to Georgetown when he picked me up in Austin for my weekends with him - we ate sausage wrapped in tortillas on a stick and he, in true 80s style, drank a tall boy in a brown paper bag as we inched through traffic.)
By modern standards, those guys look hella grubby and, if it weren’t for the sun glasses, you’d assume they were taxi drivers, dock workers/teamsters (maybe that blue collar thing was the goal!) - especially Michael McDonald’s little gut just chilled there, down to the belly button. It’s like the ancestors of the Benji Hughes look.
This is an example of a weird use of “developers are the new kingmakers”:
There is a simple rule: developers rule. You run the global economy, right? Not politicians, not CEOs - developers are the ones running this global economy. What aspect of your life is not getting more digital? Everything, sports, entertainment, social experience or health, everything is becoming more digital. It’s a foundational aspect of all economy and human experience. We’re just seeing everything become a computer, replacing industries like oil that defined geopolitics for five decades. It’s now silicon and the technology supply chains that it enables.
Here, you have a computer hardware (chips) company saying that developers are the kingmakers and, I guess, the continual power-source of the kings. I didn’t watch the keynote, I’m just taking the idea in that text and commenting on it. Maybe Intel is on about something else. But I want to look at this idea of “developers as kingmakers,” apart from whatever is going on at Intel.
I’ve recently been uncomfortable with how much people latch onto the idea of developers as king-makers. This is despite having worked at the firm, RedMonk, that was the main source of that idea back in the 2000s. Stephen’s book has several case studies of developers playing huge roles in technologies “winning.” If you work in any business that works with developers, then: YES!
BUT. For what feels like over a decade, the importance of developers has started to get way overblown and misused to over simplify, like, how things actually work.
Related to the above quote, let’s look at the hardware case. This is an area I have first hand experience with from when I did corporate strategy at Dell, covering software and cloud. Dell was super interested in the importance of developers. I did many, many reports, slides, big meetings, and studies about developers. Dell is still very interested in developer! They’ve obviously funded it an incredible amount more than when I was there.
Here is where it starts with a hardware company: Hardware without software is useless, it just sits there.
So, someone - more likely, some software company - has to write and provide/sell software to run on that hardware.
How is software made? Well, people program it. These people are…developers. But they’re not the only ones involved.
Yes, and, there are also product managers; testers; platform engineers/DevOps engineers, and ops/infrastructure people who either run it or run the labs and equipment that everyone uses; designers; general managers of business units that trickle down strategy from the corporate layer; the corporate layer that is making all sorts of choices about which technology to use and not use (see Adobe, Apple, and Facebook examples below); then there’s legal and regulatory requirements.
Also, now-a-days, you have to do a lot to manage culture and overall worker happiness. If you read through the past several years of developer productivity metric studies, you’ll realize that management’s ongoing “programming” of corporate culture is a critical part of the success of software as well.
On the buyer/user side: there are the people who select which software to buy; the people who procure it; the people who install and integrate it; the people who administer it ongoing; and then the business side at the buyers who determine how it’s used and for how long. Also, all sorts of cost/benefit analysis that’s done: is it worth while to upgrade our old stuff, or, like, do anything new?
With respect to hardware, there are strategic decisions like running on your own hardware (private cloud, edge), or in public cloud. There’s also just people who decide they like AMD versus Intel versus whatever. You have security people who will decide which hardware and software to get based on the reputation of the vendors - real and perceived. You have computer makers and hyper-scalers (Dell, Apple, Facebook), who are building hardware and deciding which chips and other things go in there, or just licensing(?) ARM and making their own. Maybe they don’t like Intel’s pricing, so for two years they’ll go with AMD just to stick it to Intel. Maybe they choose Nvidia because, like, it’s the best technology. Perhaps there’s licensing and patent barriers that force people into choices.
(I don’t think there’s much choice in storage - they’re all pretty much the same right, a commodity. I’m pretty sure developers don’t king-make hard drives? Sure, they gave SSD an early boost, but, like, SSD is just better, regardless of who’s deciding to buy it. You could argue that for the SSD market to exist, you needed the early developer market to buy the expensive SSDs before there was enough scale to build more and lower the prices…but…really? I don’t have the numbers for this, but I feel like if you took some contemporary estimates of the developer population, reduced it to those who could decide to buy SSDs, and figured out the spend, it’d be pretty small.)
Then there’s also marketing, especially brand marketing, lower level marketing - the people who put together giant keynotes and events. You’ve got the people who write documentation, manuals, training, staff StackExchange, and garden not only developer communities, but also the communities for all roles above.
You have people who manage channels and partners (at Apple, someone has to manage the relationship with all those phone stores and carriers to get favorable deals and make it all possible - if you could only buy iPhones directly from Apple…how would that have worked out? And see the partner/channel hijinks in RIA below!).
And so. And so forth.
Are the needs of developers taken into account for all of this? Sure. The hardware has to work with the software. Your developers need to understand it, sometimes even like it all. If the organization wants to use Kubernetes, your hardware needs to work well with Kubernetes and container-based architectures. These can be make or break considerations.
But so are all the other considerations and roles above. Often, developers have to fit their skills, desires, and work into the constraints that this whole system drives. It’s certainly not the case that there’s some kind of “great developer of history” theory of IT.
Developers are required, but so are all the other people, roles, and strategic considerations above.
Sure, you can have “indie” developers that collapse all these roles into one or two skin-bound entities, but all of those non-developer functions need to happen.
And, further-sure, if you’re talking about products and services that you sell to developers, then developers have a key, often king-making role…just as any customer for a product or service does.
There are many technologies that have won - or lost - because of non-developers.
This must especially be true lower down the stack, that is, with hardware. I feel like decisions about which software to support, the amount of brand management, marketing, design of the hardware, pricing, and getting your supply chain right have much more influence over the success of hardware companies than developers.
In other words, you should make good hardware that is sold at a reasonable price that solves the problem the buyer has. (I should put that in an airport business book!)
Here is a case that I lived through: Rich Internet Applications. In the 2000s, using HTML (web pages) for the kinds of applications we use now was magic. When GMail came out, few had ever seen an application that worked in the browser. It was hard to make rich, GUI-like applications for the web.
Meanwhile, browser plugins had been getting around this problem by not using HTML. Cheif among them was Flash, from Macromedia, acquired by Adobe. As Flash evolved past delivering delightful cartoons with limited user interaction, it allowed you to have desktop like applications (GUIs) in the browser. I’ve used many of them, especially enterprise software systems.
People - users - didn’t always like them, but then some people did like them (and some people nowadays have nostalgia eyes for them). As ever with software, it wasn’t the technology (Flash), it was how well you did the screens, “UX,” whatever. I
n the mid-2000s, there was a long-game/war between major software vendors to win the emerging smart phone market, which, actually meant winning the new PC platform. Microsoft wanted a version of Windows on smart phones. Adobe wanted a version of Flash on there. Sun wanted Java on there. There was also Blackberry! Winning the smart phone market was incredibly important because…well…look at the value of Apple today!
Apple shocked everyone (Well, I think? Me at least) when Apple launched the iPhone. Adobe tried to get Apple interested in Flash. They developed a whole other framework called Air that was, like, Flash++ (that might have been after Apple’s rejection of Flash, I forget). They made partnerships with other phone manufactures, streaming services (I think Netflix streaming used Flash originally, for a long time, right?), and so forth. But, at some point, Steve Jobs wrote a memo where he was, in summary, all like “Flash sucks.”
And that was it for Adobe! Did a bunch of developers petition Steve Jobs to do that? I don’t know the history, but I’m guessing not. I’m guessing Apple: (1) strategically was all “why would we let someone else own part of our fully vertical stack, are you insane?”, and, (2) it always seemed like there was a super-weird relationship between the two.
Also, people argued that Flash was just not good enough for the task. It was technically inadequate You could say “developers decided that,” but they would have decided that because it was the bad option. Imagine the case where, instead, developers chose Flash even though it was the worse option. They would have king-made Flash, but made the wrong choice. We wouldn’t be celebrating that.
Oh, and also: maybe it was just licensing. Maybe Apple and the hand-set carriers didn’t want to pay Adobe to include Flash in their stack. Flash could have been awesome - developers could have been clamoring for it! - but if the handset providers (with respect to Adobe, in strategy terms, the partners/channels) refused to pay Adobe and shipped something else. Well. Then you have Android (right?). As mentioned above, for Apple, it was an obvious strategic choice to make their own platform, regardless of what developers wanted.
Adobe tried to find traction for Flash and AIR for many more years, but eventually it just sort of died off.
Here is a sort-of contrast the Flash story. Meanwhile! Facebook was trying to figure out how to do mobile apps. At one point Mark Zuckerberg got really into HMTL5 as their app framework. The idea was that you should use the open web for apps. Strategically, this is similar to Apple rejecting Flash: using the open web meant Facebook had very little ties and dependencies to any company. This is, you know, very different than how Facebook and the entire web acts now - a closed off web with the gates made of network effects and user preference.
(You can also see, in recent, history why it’s was bad for Facebook to have dependencies on Apple. Apple changed how tracking for advertising was done a year or so back and this caused all sorts of problems for Facebook and others who’d been using those capabilities to track and target ads. I think Facebook figured it out, but, like, bummer on their cashflow!)
Eventually, Facebook was like, nope, we’re making native iPhone and Android apps. They still have HTML ones as well.
Here, you can see some developers contributing to this, but like, a handful of developers at Facebook, probably - not, you know, the millions of developers out there in the world. Also, I’m pretty sure it was a strategic choice.
First, I could be missing a major point in the smart phone cases above. Once Apple put its own app platform in place, it needed developers to actually write apps for it. And, like: yes! But, let’s go back to that laundry list of what goes into making software. Apple needed the software producing industry to make apps that ran on the iPhone…not just developers.
The container wars are another thing I don’t have sorted out in this “developers are just part of it all” thinking. Developers can play a large role in the success of technology. I’ve seen this first hand recently with the rise of Docker and then Kubernetes. But…to me, this is something more of a perception and business strategy “war.”
The actual usage of containers in production is incredibly low compared to how much we talk about “cloud native” and Kubernetes.
By 2025, 15% of enterprise applications will run in a container environment, up from 5% in 2022, hampered by application backlog, technical debt, skills availability and cultural change.
–From a recent “Gartner Forecast Analysis: Container Management (Software and Cloud Services), Worldwide.”
Also, the market (how much money people pay for) for container management is incredibly low as well. And then there’s the bizarre thing that the Kubernetes people keep saying that Kubernetes is not for developers: you know, platform for building platforms and all that.
I don’t have this whole container wars thing figured out, but I suspect there’s a lot more going on then just developer preference.
The cost of Kubernetes (free to download and install the hard-way); the brand-glow of Google (and then Red Hat jumping in and acting as a kingmaker and channel for it); the missteps of competitors (usually understandable with out reverse Halo Effect vision, just like Adobe and Facebook above); the price of alternatives (Pivotal Cloud Foundry is/was expensive compared to free - OpenStack sort of lost the plot); Docker’s strategy waffling and flameout; HashiCorp somehow not getting a foothold in platforms despite its foothold in other areas; and then all the existing VM-based “platforms” from the public cloud vendors to VMware (where I work), Microsoft, etc.
Also, there’s endless castle intrigue and rumors about people doing plain old bickering, acting on grudges, and all that petty bullshit that drives history. I wasn’t first hand to any of that, so I have no idea - but I’d love to know!
Again, we see an incredibly complex system where the part of one role - developers - is important, but not any less important than other roles?
(Oh, and then, how about OpenStack as a case from the other direction: what made that not work out? Further, was it the developers in telco that made and continue to make OpenStack successful in telco? Or maybe OpenStack is just fine?)
Perhaps you could say that developers can be one of the early gate-keepers of a technology. They can decide if the technology gets past the early parts of the diffusion of innovations curve. But, whether early on or along the curve (especially the fat part in the middle), so does every other role in the big system.
Pricing and procurement can be a king-maker: if the software is expensive and hard to buy, you’ll have problems…well, not exactly…you could hire an enterprise salesforce and sales engineers that can support this long process, do the marketing and product management needed to justify a high price, and then increase the prices to support the whole mostly self-inflicted hassle of enterprise sales. That all works perfectly in the ERP: witness Oracle, Salesforce, SAP, etc.
(Another side case that’d be fun to ponder. When I was at RedMonk, SAP cared a great deal about developers. Did all that effort matter? Was it make-or-break for SAP’s continued success? SAP was also heavily aligned with Adobe’s RIA stuff above.)
To say that “developers rule” is like saying that the cheese makers rule with when it comes to cheese enchiladas, or even queso. The cheese is important, but the decision process and hands-on work that leads up to putting a chip in queso is so complex, involves so many people, and, most importantly, user/eater preferences, that to say the cheese maker rules is weird. Maybe the proper mixture of spices, getting a ripe avocado, deciding on putting chilli con carne or not (and is it ground beef or brisket?) - I like a lot of pico de gallo myself. But, then, procurement wise, sometimes you don’t have the time and money so you just melt some Velveda and you’re perfectly satisfied - connivence is the king maker. (Is the developer the queso eater here, or the maker? I’ve lot track.)
Similarly, there are so many things, people, decisions, tricks of fate, grudges (I’ve heard!), etc. involved in making, buying, and using software that to assign an outsized amount of importance to developers is…weird.
Many of the people and roles involved in the lifecycle of process can have a pivotal role in it, developers can for sure! But people making strategic choices also can have make or break control (Steve Jobs and Mark Zuckerberg above, or the strategic choices they make at Concur to not update their software which are, I assume based on their software’s UX, very much in favor of controlling costs and driving profit since they have mental lock-in in their market - from what I can tell).
A designer who comes up with the best way to do a workflow for a user can have make or break effect - they could use “dark patterns” to drive revenue that keeps the company alive, allowing the developers, and everyone else, to keep doing their job. Someone in billing could just decide that you have to call (on the phone!) to cancel your subscription.
At the time, the point Stephan was making was incredibly important because most people didn’t think about developers as having much of any role. Sure, there was the mythical idea of a garage of app developers, but from what I recall, caring about developers wasn’t a big enough part of the conversation.
It was weird. You had that whole “developers! developers! developers!” thing. And Linux and the broader open source world was driven by developers (as cataloged by Stephan).
But, still, the idea of thinking about developers strategically was only done by a select few, like Microsoft. Now, most everyone considers the developer. And this is fantastic! YES TO THAT. We do, after all, need software.
I just think, you know, let’s focus on all the parts, not overly on one, developers.
So, going back to the original quote a the top, I think what he means is “software.” Software is, like, pretty important now-a-days.
And, sure, developers program software, but there are so many other factors and roles involved in the life-cycle of software that also play a critical role (see above!) that, you know, developers, just like the cheese maker, are only one part of making a good bowl of queso.
Speaking of developers…
Next week, on October 17th, a whole passel of my team mates and I are hosting an online SpringOne Tour. It’s free to attend, of course. If you can’t make it to one of our in-person events, check this one out. There’s over 20 talks you can choose from, including mine on platforms.
If you’re a programmer - especially a Java programmer - or doing anything with cloud native apps, there’s something in it for you. Register for free, and check it out on October 17th.
I got access to the ChatGPT conversation feature this weekend. It is pretty amazing, really. I’ve used it for my D&D messing around, and a little for other things: I’m thinking through a method of collaborative storytelling that could make it a better ChatDM. And, with the conversation thing, you could do it while walking the dog, washing the dishes, etc. I still don’t think it could do D&D combat, but I might be able to finally figure out how to get it to be fun.
I think it was John Willis who told me that a long time ago.
What is Technical Debt? - “In practice, my observations are that most development teams, even in small companies, are too far separated from these types of financial models to reason about technical debt in terms of revenue, but absolutely do feel the pain of technical debt in their day-to-day work (even if they can’t quantify it in money).”
Six Months Ago NPR Left Twitter. The Effects Have Been Negligible - “Six months later, we can see that the effects of leaving Twitter have been negligible. A memo circulated to NPR staff says traffic has dropped by only a single percentage point as a result of leaving Twitter, now officially renamed X, though traffic from the platform was small already and accounted for just under two percent of traffic before the posting stopped.” // I haven’t done a deep analysis, but I feel like I’ve seen the same thing with my stuff.
Call it a bro-celet. It’s a male bracelet and once you see it, you’ll see it everywhere - And here I thought these were some kind of meditation, Buddhist token…
Next week, on October 17th, a whole passel of my team mates and I are hosting an online SpringOne Tour. It’s free to attend, of course. If you can’t make it to one of our in-person events, check this one out. There’s over 20 talks you can choose from, including mine on platforms.
If you’re a programmer - especially a Java programmer - or doing anything with cloud native apps, there’s something in it for you. Register for free, and check it out on October 17th.
“revealed management practices”
Software implements strategy for breakfast.
“She leans 5’8”."
Understand what you’re measuring, or you’ll just get measurements.
An acceptable level of awful.
“Nothing gets done, however, unless there are people responsible and held accountable for the measurable improvement. This is another area in which strategic plans sometimes fail.” Here.
Talks I’ll be giving, places I’ll be, things I’ll be doing, etc.
Oct 17th SpringOne Tour Online (free!), speaking. Oct 10th, 17th, 24th talk series: Building a Path to Production: A Guide for Managers and Leaders in Platform Engineering Nov 6th to 9th VMware Explore in Barcelona, speaking (twice!). Nov 15th DeveloperWeek Enterprise, speaking.
Discount code for KubeCon US - while I won’t be at KubeCon US this year, my work has a discount code you can use to get 20% off your tickets. The code is: KCNA23VMWEO20.
I did a quick practice of some new slides as a livestream today. If you want to see the actual talk, check out the webinar Bryan and I are giving on Tuesday: free to watch!
Meanwhile:
Suggested outro.
No links today, but this:
Once you suggest tracking an individual software developer’s performance, you get into big trouble with the thought leaders. This is, you know, pretty much a correct a response. McKinsey decided to have a go recently, giving us all a chance to think about “developer productivity” again. In recent years, I’ve mostly thought about developer productivity in terms of build and deploy automation - I know, just mind-blowing thrill rides, right! That’s cloud native for you! - but this new round has been more higher up the stack. Though, come to think of it, I don’t know if that scope was actually specified.
Anyhow, I think these are what’s on the current stone tablets of developer metrics:
You should measure the developer team, not the individual developer.
Management should only measure business outcomes, not development activities.
Metrics will be gamed by developers and misused by management, likely, with bad results for the business.
I’m sure this is, as Dan North likes to put it, “reductionist,”1 but, sure…?
For this week’s podcast recording (it’ll be here once published this Friday), I went back and read the McKinsey piece on developer productivity.
It seems…fine?
Like, so much so that I was wondering if the McKinsey people went back and edited after the big stink-up about it.
With that confusion in my brain, I re-read three things three times:
Gergely Orosz and Kent Beck’s two part rebuttal (part one, part two).
Dan North’s review notes on the McKinsey piece.
I think I get what the rebutters are so upset. Let’s see!
The authors at McKinsey are looking for a way to measure developer productivity that the, like, “C-Suite” (“management,” as I’ll put it) can use to make decisions about software development in their organization. A good goal!
They start with using DORA and SPACE, and adding in four new metrics:
The new metrics are the - what would call that? - purplish-blue ones2 - from the article (I just included the quick definitions:
Developer Velocity Index benchmark. The Developer Velocity Index (DVI) is a survey that measures an enterprise’s technology, working practices, and organizational enablement and benchmarks them against peers.
Contribution analysis. Assessing contributions by individuals to a team’s backlog (starting with data from backlog management tools such as Jira, and normalizing data using a proprietary algorithm to account for nuances) can help surface trends that inhibit the optimization of that team’s capacity.
Talent capability score. Based on industry standard capability maps, this score is a summary of the individual knowledge, skills, and abilities of a specific organization.
Inner/outer loop time spent. To identify specific areas for improvement, it’s helpful to think of the activities involved in software development as being arranged in two loops. An inner loop comprises activities directly related to creating the product: coding, building, and unit testing. An outer loop comprises other tasks developers must do to push their code to production: integration, integration testing, releasing, and deployment.
For the forth, I think what they should have meant was “developer toil,” but that inner loop/outer loop framing is so tempting.
The reaction to this was not good. Here’s my summary of the sentiment based on the two rebuttals I read:
You should measure the developer team, not the individual developer.
Management should only measure business outcomes, not development activities.
Metrics will be gamed by developers and misused by management, likely, with bad results for the business.
Let’s look at each!
This one is the foundation of the counter-arguments. Software development done well is very much a “team sport.” You know, there’s no “I” in “team” and all that.
It may be hard to detect from the outside, but software is not just the aggregate of individuals writing code. It’s like this: I don’t know what to tell you if you don’t just know that - you must not have ever been a developer and experienced it first hand?
For our metrics discussion, the consequence of this is that it’s very difficult to measure an individual’s contribution to the team.3 Rather…it’s a nuanced task. If you do allow people to specialize in certain parts of the code base and/or be “hero” trouble shooters then, like, it is easy to identify who’s important and valuable. Structuring the work so that it is individual based generally causes problems. Generally, allowing people to specialize like this is highly frowned upon.
Dan North brings up a more qualatative way to judge indivcidual performance:
If you are going to assess individuals in a team, then use peer feedback to understand who the real contributors are. I ask questions like “Who do you most want to be in this team, and why?”, or “What advice would you give to X to help them to grow?”, or “What do you want to most learn from Y?” An astute manager will quickly see patterns and trends in this peer feedback, both positive and negative, which are then actionable.
In this method, you are still rating individuals, just on how they help the team and their peer’s assessment of their skills. This feels right. If you’ve been on an application development team, I mean, you know that some individuals contribute more (“are more valuable” if you can stomach that phrasing) than others. You know who’s slacking off. You know who’s padding their estimates. You know who’s stopped learning. You know the bad performers, and the good ones. As a peer, you can also see through the “they’re not a failure, system has failed them” thinking. You know this because you’ll have tried to help them many times. You’ll have tried to change the system such that you can many times. They’ve taken advantage of the five free therapy sessions, and all that stuff. (Why just five? Does any HR department think that, like, any mental issues that are dampening my productivity can be solved in five therapy sessions? You’re going to need at least one just to introduce yourself and set context. Then you’re down to four hours. And, if you’ve been to therapy, you know you spend, like, half the time just meandering about as you and the therapist try to find something to talk about and how to talk about it. And then, even if you solve your problems in those four sessions, you need frequent reinforcement of the tactics you deployed to fit is. Five sessions is better than zero sessions, but if you’re concerned about the mental well-being of your staff damaging productivity, they’re going to need more. Which, I guess, you can, like, get from good health insurance if they provide it. Anyhow. Uh. I was talking about how peers in an application development team will know if someone is a low performer and can’t be helped further, good system or bad system…) And, like, you’re doing your job despite all the things…what’s their deal?
So, if anything, when it comes to individual performance metrics, base it on their peers. And, definitely, absolutely, don’t let management ever see those metrics, as we’ll cover below.
As with a sales person, management should really just care about the business outcomes that application development teams produce.
“Business outcomes” are things like revenue (sales), cost savings, keeping the application up and running (performance, loosely put), and, though not really considered by the business much, overall application agility (how quickly and easily can you add new features or modify existing ones to change/help how the business runs). There’s always lots of other business outcomes, but you get the idea.
The McKinsey piece adds four metrics (above) that are based on the daily activities of individuals, and, worse from the perspective of the thought lords, there’s one that’s some kind of skills assessment. I didn’t seek out the definitions of these four new metrics much, so maybe they have tons of business value and team laced into them. I mean: they could!
To be fair, the McKinsey piece is not suggesting that these are the only metrics to track. They throw in DORA and SPACE into their overall metrics rating. Yay lots of metrics! Fill the dashboard!
Here’s all the metrics again. The new McKinsey ones are the “opportunity-focused metrics” ones:
There’s only a few “business outcome” metrics in there: customer satisfaction, reliability, lead time for changes, etc. Notably, none of the metrics are “made money for the enterprise,” or the like for non-profit/government organizations.
This is, really, the whole problem with “developer metrics.” All you really need to track is “did this software help the business?” That is, “business outcomes,” a phrase only just a little better than “business value.”
Gergely provides an example of how his team at Uber tracked the team’s business outcomes, which is awesome! The problem is that most organizations don’t track to these business outcomes, and I suspect it’s because (a) they just think of application developers as a factory4 that delivers apps to spec (you know, “waterfall”), and, (b) it’s hard to do.
If you can track the business outcomes of application development teams, that’s the only metric you need to track. Even if the team has “low performers,” who cares if the money’s good?
And, sure: measuring the business outcome of the team is what you should be doing. So, like, do that. It’s very difficult in most organizations, and not even considered a serious idea in not-tech organizations, as Gergely pointed out last year.
Even more wicked here, when you measure developer activities, figuring out the right activities to measure is difficult. So many things that a team (let alone an indivdual) does in development are unseen and un-trackable, as Dan North points out:
most of programming is not typing code. Most of programming is learning the business domain; understanding the problem; assessing possible solutions; validating assumptions through feedback and experimentation; identifying, assessing and meeting cross-functional needs such as compliance, security, availability, resilience, accessibility, usability, which can render a product anything from annoying to illegal; ensuring consistency at scale; anticipating likely vectors of change without over-engineering; assessing and choosing suitable technologies; identifying and leveraging pre-existing solutions, whether internally or from third parties. I am only scratching the surface here.
This is another one of those things that you only appreciate, and believe, if you’ve been a developer.
Given all of that - you should measure the business outcomes of the teams, you should measure the team not the individual - bad things will happen if you track individual performance. First, people will game the system and max out the activity based tracking. As the rebutters point out, if you value commits/PRs, developers will just make a bunch of tiny commits. And so on.
I find this gaming thing only half of the counter-argument. There’s whatever named notion that once someone knows how they’re measured, they’ll game the system. The second part that’s left off is that, yeah, sure, if management is dumb. We all know that metrics will be gamed, so you try to redesign the metrics ongoing and use them skeptically. There’s a spiral into cat-and-mouse games here, I guess.
Kent Beck points out an example at Facebook where that didn’t work out - and I think the outcome was they stopped doing it? Hopefully. On the other hands: that’s probably the least of Facebook’s problems, if they, really, even have any problems based on the shit-tons of cash they generate. Still. One should strive to be excellent, not just well paid. (At least, that’s what we should tell our management chain.)
But, like, does this mean we shouldn’t use the DORA metrics because people will “game” them? No, it just means use good metrics and adapt. Use metrics to get smarter about your qualitative (a fancy word for, I guess, “subjective,” rather, “your gut feel”) assessments.
This brings up the second part, here: you can’t trust management to use these metrics well…unless they understand how software is actually done. Gergely and Kent give a good, simple overview of exactly that.
So, the worry with the McKinsey metrics is that management will use them without understanding how application development is actually done. This means they’ll misuse them, either to fire people, not promote them, misallocate budget (giving too much to some teams and not enough to others), etc….all because someone didn’t break up their PRs/commits into small enough chunks. Hahahah - jokes! (But not too far off.)
As the rebutters all point out, what’d really be useful is to get management to understand how application development is done (above). With that understanding, you’d realize the above flaws and come up with some better metrics…or indicators and health checks.
The DORA metrics are trying to predict business outcomes without actually knowing the business outcomes (revenue, etc.). They’re saying “if you do well at these four metrics, then it’s possible for the business to do a good job…so…hopefully they do that.” Hey, hey! SILOS. Local optimization! FAT BOY SCOUT! But, you know, that’s probably fine.5
So, I think it goes like this:
If you could tie the application development teams work to actual business outcomes, then you’re all set. And if you can’t, you should figure that out.
Else, if you can’t do that, you should use the DORA metrics to measure your own local optimization.
Else, if you can’t do that, you should only rate individuals based on their peer reviews.
And, if you can’t do that, you’re working in an unenlightened, possible even toxic work environment.
And, if you can’t get a new job (or are adept at working in that system and maximizing your take with reasonable long-term stability), whatever you do, if you have the data on individual performance, first, destroy it and stop tracking it, and, second, do not let management outside of development see it.
And, if you have no idea what’s going on, must keep your commits small and your lines of code voluminous and you’ll probably be fine.
Next week, on October 17th, a whole passel of my team mates and I are hosting an online SpringOne Tour. It’s free to attend, of course. If you can’t make it to one of our in-person events, check this one out. There’s over 20 talks you can choose from, including mine on platforms.
If you’re a programmer - especially a Java programmer - or doing anything with cloud native apps, there’s something in it for you. Register for free, and check it out on October 17th.
In his write-up about developer productivityDan North has a discussion about the inner loop/outer loop model. Here’s McKinsey’s diagram:
As he points out, it seems to be in conflict with the “shift left” mindset. Without typing too much about it: yes?
In our cloud, DevOps, cloud native world, I don’t think we’ve every figure out a good answer to the question “what are things application developers should be doing?” These would be things the application developers should be automating, or letting other people do.
Unless you’re dhh, your application developers should not be racking and stacking servers. They probably shouldn’t be writing their own container management systems, nor working with kubernetes directly instead of layering a buffet of CNCF landscape on-top of it. They should probably not being creating your security risk models? If you’re of the Heroku/Cloud Foundry mindset, they shouldn’t be deploying their applications to production.
I’m blowing up the scope of inner loop/outer loop, sure. But defining what application developer “toil” is versus is not is both very clear (building clouds) and not very clear (doing compliance audits - not a great example?).
All models are flawed, just some better flawed than others. For the McKinsey piece, if you just replaced the inner loop/outer loop talk with “toil,” I think you’d be cool. And, of course, you’d need a footnote that said “one application developer’s toil is another application developer’s competitive advantage.” '
What we want to get at is something more like: there’s some stuff that’s a waste of time for application developers to do, and they should not do those things.
For example, Dan North points out that your path to production and infrastructure stuff (“A fast, automated release pipeline is a key prerequisite for frequent releases, and skill in defining, provisioning and evolving the infrastructure to support this is a differentiator”) is incredibly valuable.
Like, big yes (my monthly paycheck depends on that infrastructure being incredibly valuable!): but that’s probably the work of a team other than your application developers. That seems to be the case at the tech company darlings that have separate developer tools and platform groups.
Talks I’ll be giving, places I’ll be, things I’ll be doing, etc.
Oct 17th SpringOne Tour Online (free!), speaking. Oct 10th, 17th, 24th talk series: Building a Path to Production: A Guide for Managers and Leaders in Platform Engineering Nov 6th to 9th VMware Explore in Barcelona, speaking (twice!). Nov 15th DeveloperWeek Enterprise, speaking.
Discount code for KubeCon US - while I won’t be at KubeCon US this year, my work has a discount code you can use to get 20% off your tickets. The code is: KCNA23VMWEO20.
Related to developer productivity, charts/surveys that silo handoff problems are always fun. Here’s one from Cat Hicks and crew:
Schedule (“timeline”) is always a problem and misalignment. But, schedule is a problem in all walks of life. It’s not only developers that are bad at time estimates, it’s everyone.
Budget and cost is not shown (maybe “lack of technology resources” is a proxy?): management always wants to pay less (more for the shareholder and, thus, themselves). Again, as in life, so in business: "I'm glad I paid more than I had to," said no one ever.
Getting the requirements right is difficult, but a lean design approach can help: the business should take part in this and use the agile advantages of software. If you do software right, you can experiment and develop how the business functions: you don't have to get it right the first time. E.g., the commercial kitchen pivoting to measuring mayonnaise. This idea of "pivoting" is key and needs to go up to the business. It's a pun, right: Pivotal Software.
This last point is getting us closer to the business output dream. The enlightened stage of understanding how software is done comes when you get that you don’t have to be perfect each release. And, in fact, that is not good. If you’re not getting your “requirements” wrong frequently, you’re not trying hard enough to come up with new things and improve existing one.
I don’t know sportsball, but my understanding is that they don’t get the ball in the goal every single time they try. And this is considered a-OK, just fine. The hockey puck quipper also said: "you miss 100% of the shots you don't take." So, what if I told you with you software you could take, virtually, unlimited shots? Yeah. SOFTWARE.
“Might-could be fixin’ be real peachy-like.”
Sometimes you don’t squeeze to get the juice.
“You know me: I don’t wear velcro shoes.”
Per above, I have the second installment of the the path to production talk series coming up next week. There’s something scurrying around in this developer productivity talk that gets real close to what I want to discuss: what does it mean to think of and use your software “factory”6 strategically?
If we know the wrong metrics to monitor, and we know the right ones, that should be some kind of trailing(?) indicator of how to correctly think about software. We’ll see! Also, we were going to cover some OGSM thinking, which I’ve looked at many times and am always left bemused and puzzled.7
Also, yet again, as ever, ThoughtWorks has written the definitive piece on all of this before us Pivotal/Tanzu people can get around to typing it up. No, really, it’s great stuff.
You should register for it and watch it - all for free, you know.
Finally, a rare Garbage Chair of Amsterdam, Duivendrecht edition:
🎶 Suggested mid-week outro.
I’m pretty sure when people use the word “reductionist,” it’s a synonym for “stupid,” or, at the very least, “wrong.”
My color dropper thing says it’s #1c51fe. Pretty nice to look at, actually.
This is especially true if you use the practice of rotating pairing (which most people don’t, let alone pair programming) where you have two people work on the same thing, and have them switch at least two times a day, moving to a different part of the code base. Pair programming has many advantages: you avoid people specializing in one part of the code base, which also avoids people being ignorant of other parts of the code base (thus, doing some bus number risk management as well). You train people on the job, both senior to junior, and junior to senior. And so on!
For future management consulting blog writers, “factory” is another word the thought leaders get all upset about. The US military can call what they do a factory - I guess you don’t want to argue with that lot - but you can’t apply that word to software developers. The thinking goes that a factory stamps the same thing out over and over again, there’s no creativity. Sure. What that misses, though, is that the developers are the ones using the factory, all of their tooling, the platform they use, the processes they follow - those are generally more static, like a factory. If you consider the developers themselves as “the factory,” then the term is shit (who wants to be reduced to a factory?). I think the work “factory” is great for all that stuff below the application developers. But, then again, the platform engineers (or DevOps engineers…or whatever) will probably be upset at being called a factory. Real org-chart geometry diction quagmire there.
I suppose this might make the product managers out there sad. But, again, hey, if you have product managers on your team, then you’re probably a-OK: your product manager should have an ongoing idea of the business value you’re actually creating with the software.
Yeah, yeah. See footnote above about “factory.”
Usually, when I read these kinds of “operationalize your business strategy real-good-like” frameworks I think “yeah, if I had the capability, knowledge, and corporate will to do that kind of modeling…I wouldn’t have a problem in the first place.” But, I haven’t really given the OGSM thing time. And, you know, what do I know: I just make slides.