The problem with the “DevOps tools” category, and the RightScale Cloud Survey

There’s always some good year/year stuff in this survey. I’ll have to check it out soon.

Meanwhile, some coverage from Serdar Yegulalp:

“DevOps tools”:

This category is going to be increasingly weird to cover. Here, what they really mean are “containers and container orchestration tools…oh, and configuration management tools.”

Arguably, you’d put something like Cloud Foundry in there if you have a gumbo of three different software categories, each used by people who want to follow DevOps practices and get the benefits of improving software. We haven’t put public numbers recently, but back in September of 2015 Pivotal Cloud Foundry had $100m in annual bookings, and the growth has continued.

Here’s how 451 characterized our customer base and foot-print out there:

While Pivotal Labs’ early success was among startups, PCF was designed for organizations needing to increase velocity and change their application delivery process, which appeals to large enterprises. About 90% of Pivotal’s revenue comes from enterprise customers today. The company reports that its number of paying customers is in the hundreds, with average deal sizes in the high six-figure range. It also says most of its customers are using PCF on top of VMware’s IaaS offering. AWS is currently in second place among PCF deployments with Azure as a close third that is gaining momentum among the Fortune 100.

All of that means: Pivotal Cloud Foundry has many customers running in production (with the resulting revenue) and, I’d argue, much market share. If you throw in IBM and MicroFocus/HPE’s Cloud Foundry distro revenue, plus OSS use (like at 18F), you’ve got a large chunk of Cloud Foundry usage out there, which should translate into a large chunk of “doing software better (with DevOps)” marketshare for all of Cloud Foundry.

¯\_(ツ)_/¯

Like I said, coverage will be weird for awhile as people sort it out. Kind of like “cloud” was in 2009, and pretty much every “PaaS” category now, though it’s getting better. I’m looking forward to what they sort out in the new(ish?) DevOps practice at IDC.

More coverage of the survey:

Private, public, hybrid:

One key trend RightScale’s tracked over 2014, 2015, and 2016 has been the use of private cloud through OpenStack, VMware, and related products. This time around, while private cloud adoption hasn’t fallen off a cliff, it’s edged down: 72 percent of respondents this year run a private cloud, versus 77 percent last year and 63 percent the prior year. Hybrid cloud has shown a similar trajectory of 67 percent in 2017, 71 percent in 2016, and 58 percent in 2015.

After margin of error, pretty slow, but a good enough pace I guess.

Make no mistake: AWS is cloud king, and it’ll safely remain that way for a long time. But Microsoft Azure has made remarkable amount of progress in the last year. In 2016, only 20 percent of respondents were running apps there; this year, it’s 34 percent. With enterprise users specifically, the growth was even more dramatic, from 26 percent to 43 percent.

And, Oracle and DigitalOcean:

Azure’s growth isn’t coming at the expense of business from other cloud providers. Oracle Cloud and DigitalOcean, the two decliners on the list, didn’t have enough business to lose in the first place to constitute Azure’s jump: respectively, 4 and 5 percent in 2016, and 3 and 2 percent this year.

Now, of course all this hinges on what exactly they mean by cloud, what the sample base is (just RightScale customers?), etc. But, anything that’s multi-year is useful.

See also Barb’s brief coverage, e.g.: “Azure, meanwhile, saw a bigger jump with 43% of respondents on Azure now, compared to 26% last year.”

Source: Docker’s tops for devops, AWS is the cloud king, InfoWorld

Gary Gruver interview on scaling DevOps

I always like his focus in speeding up the release cycle as a forcing function for putting continuous integration in place, both leading to improving how an organization’s software:

I try not to get too caught up in the names. As long as the changes are helping you improve your software development and delivery processes then who cares what they are called. To me it is more important to understand the inefficiencies you are trying to address and then identify the practice that will help the most. In a lot of respects DevOps is just the agile principle of releasing code on a more frequent basis that got left behind when agile scaled to the Enterprise. Releasing code in large organizations with tightly coupled architectures is hard. It requires coordinating different code, environment definitions, and deployment processes across lots of different teams. These are improvements that small agile teams in large organizations were not well equipped to address. Therefore, this basic agile principle of releasing code to the customer on a frequent basis got dropped in most Enterprise agile implementations. These agile teams tended to focus on problems they could solve like getting signoff by the product owner in a dedicated environment that was isolated from the complexity of the larger organization.

And:

You can hide a lot of inefficiencies with dedicated environments and branches but once you move to everyone working on a common trunk and more frequent releases those problems will have to be address. When you are building and releasing the Enterprise systems at a low frequency your teams can brute force their way through similar problems every release. Increasing the frequency will require people to address inefficiencies that have existed in your organization for years.

On how organization size changes your managerial tactics:

If it is a small team environment, then DevOps is more about giving them the resources they need, removing barriers, and empowering the team because they can own the system end to end. If it is a large complex environment, it is more about designing and optimizing a large complex deployment pipeline. This is not they type of challenges that small empowered team can or will address. It takes a more structured approach with people looking across the entire deployment pipeline and optimizing the system.

The rest of the interview is good stuff. Also, I reviewed his book back in November; the book is excellent.

Link

Not actually a DevOps talk

I get asked to talk on DevOps a lot. Here’s my current (late 2016 and 2017) presentation, going over the why’s, the how’s, the technologies, and the meatware that supports including some best and worst practices based on what Pivotal customers do. See the slides.

Also, here’s a more blatantly pro-Pivotal (and longer) version that you might have seen, esp. if the talk title was something like “Digital Transformation in the Streets.”

Much of it draws a lot on my cloud native journey booklets as well.

Book Review: two DevOps books

Check out my review of the DevOps Handbook and Start and Scaling DevOps in the Enterprise over on The New Stack.

Unsurprisingly, I liked both of them, esp. the second:

What I find so helpful, and even thrilling, about Gruver’s book, is that it’s exacting in its instructions and walks through several what-if scenarios for addressing common problems that come up when applying agile and DevOps at scale. Plus, it’s the perfect size for a book of this type: about 90 pages that’ll take you about 90 minutes to read.

IT/Business alignment: why it’s different this time

Re-reading Nick’s piece on “digital transformation,” I like how he explains what’s new and different from past waves of IT innovation (lik ERP and econmerce), e.g.:

“Going digital results in an explosion in the amount of data you have. New channels of engagement between customers and organizations have resulted in new sources of information coming into the organization at speeds not seen before. In the past, customer interaction was mostly one-way – from the organization to the customer. Now it is about customer-directed, on-demand two-way engagement anywhere on any device. Customers want to communicate on their terms in their preferred channels. That causes organizations to have to transform the way they handle such information, since having a large call center may not be enough – or even that relevant in the future, given that so much communication will come via social media, in messages or increasingly via video. Add to that the explosion in information from Internet of Things (IoT) devices, and it’s pretty clear that the days of management by gut-feel and hunch are over, and data-driven decision-making is the only way to go.

And also, some numbers:

  • “Less than 25% of organizations that participated in a recent 451 Research survey (451 Research VoCUL, April 2016) said they had a well-defined formal digital transformation strategy. So we’re in the early stages of digital transformation, and there’s lots of work to be done.”
  • “Erik Brynjolfsson, Lorin Hitt and Heekyung Hellen Kim from MIT and University of Pennsylvania found that companies with data-driven decision environments have 5% higher productivity, 6% higher profit and up to 50% higher market value than other businesses.”
  • “Our research shows about 65% of IT decision-makers using agile methods and about 40% adopting DevOps today (VotE Software-Defined Infrastructure Q4 2015).”

Source: Digital transformation: the what, the why and the how

6 DevOps columns: government, compliance, ROI, management

Source: Two studies, first with 1,137 respondents, second with 1,097, involved in their company’s IT buying decisions participated in the Jan 2014 and July 2014 survey, including 470 and 445 whose company currently use public cloud. “Corporate Cloud Computing Trends,” 451 ChangeWave, Feb, 2014 & “Corporate Cloud Computing Trends,” 451 ChangeWave, Aug 2014.

Last year I wrote several columns for FierceDevOps. Nancy Gohring was the editor there and graciously asked me to do so (she’s moved over to being an analyst at 451 and is doing awesome work over there). The FierceEmpire has shifted their stuff around and now it’s either impossible or impossibly tedious to find those pieces, so I moved them over to Medium. I’ve got to get my URLs to be my overly self-referential self, after all!

Here they are:

  1. Software Defined Businesses need Software Defined IT Departments
  2. Here’s how we can help push DevOps into the mainstream
  3. There’s no easy way to model DevOps ROI
  4. Management’s role in DevOps: orchestrating the why
  5. Barriers to DevOps in government
  6. Addressing the DevOps compliance problem

Slow down cowboy, start with just integrate your code regularly and fixing the bugs you find

From Gary Gruver, one of the better “how to do agile and DevOps stuff in large organizations” authors:

For these organizations implementing DevOps principles (the ability to release code to the customer on a more frequent basis while maintaining or improving stability and quality) is more about creating a well-designed deployment pipeline that builds up a more stable enterprise systems on a regular basis so it is much easier to release the code on a more frequent basis.  This is done by creating a deployment pipeline that integrates the code across the enterprise system on a much more frequent basis with automated testing to ensure that new functionality is not breaking existing code and the code quality is kept much closer to release quality.

From my perspective this approach to DevOps is rather different from the more unicorn type approach described in this article.  It though does address the biggest opportunity for improvement that does exist in more large traditional organizations which is coordinating the work across teams.  In these cases the working code in the deployment pipeline is the forcing function used to coordinate work and ensure alignment across the organization.  If the code from different teams won’t work together or it won’t work in production, the organization is forced to fix those issues immediately before too much code is written that will not work together in a production environment.  Addressing these issues early and often in a deployment pipeline is one of the most important things large traditional organizations can and should be doing to improve the effectiveness of their development and deployment processes.

Source: DevOps killing outsourcing? Another point of view – DevOps.com

38% of enterprises now using DevOps, Gartner

A little survey bit from a recent Gartner report on security in DevOps:

At the time of the original DevOpsSec research in 2012, DevOps was relatively new. However, recent Gartner research indicates that 38% of enterprises are now using DevOps, and 50% will be actively using it by the end of 2016. In the same survey, security and audit tools represented the single highest-rated category of tools in terms of importance to an effective and effcient DevOps implementation, and 82% of respondents indicated that they had to deal with one or more regulations in their DevOps initiatives.

Details of the study from the footnotes:

Gartner Enterprise DevOps Survey Study: This research was conducted via an online survey from 9 May to 13 May 2016 among Gartner Research Circle Members — a Gartner-managed panel composed of IT and business leaders.
Objectives: To learn how organizations are adopting DevOps as a means to accelerate enablement; to go faster while improving quality; additionally, to inform on topics, including starting a DevOps approach, pitfalls to avoid, scaling efforts, integrating information security, pursuing this in a regulated environment and quantifying bene ts.
In all, 252 IT and business leaders participated, with 95 members qualified by indicating they are already using DevOps.

Moving beyond the endless debate on bi-modal IT

I get all ants-in-pants about this whole bi-modal discussion because I feel like it’s a lot of energy spent talking about the wrong things.

This came up recently when I was asked about “MVP”, in a way that basically was saying “our stuff is dangerous [oil drilling], so ‘minimal’ sounds like it’d be less safe.” I tried to focus them on the “V” and figure out what “viable” was for their situation. The goal was to re-enforce that the point of all this mode 2/small batch/DevOps/PDCA/cloud native/OODA nonsense is to keep iterating to get to the right code.

Part of the continual consternation around bi-modal IT – sad/awesome mode – is misalignment around that “viability” and scoping on “enterprise” projects. This is just one seam-line around the splits of the discussion being unhelpful

Bi-strawperson

The awesome mode people are like:

You should divide the work into small chunks that you release to production as soon as possible – DevOps, Agile, MVP, CI/CD – POW! You have no idea what or how you should implement these features so you need to iteratively do it cf. projectcartoon.com

And the sad mode folks are like:

Yes, but we have to implement all this stuff all at once and can’t do it in small slices. Plus, mainframes and ITIL.

Despite often coming off as a sad mode apologist, I don’t even know what the sad mode people are thinking. There’s this process hugger syndrome that, well on both sides really, creates strawpeople. The goal of both methods is putting out software that makes users more productive, including having it actually work, and not overpaying for the whole thing.

The Enemy is finding any activity that doesn’t support those goals and eliminated it as much as possible. In this, there was some good scrabbling from the happy mode people laughing at ITSM think early on, but at this point, the sad people have gotten the message, have been reminded of their original goal, and are now trying to adapt. In fact, I think there’s a lot the “sad mode” people could bring to the table.

To play some lexical hopscotch, I don’t think there is a “mode 1.” I think there’s just people doing a less than awesome job and hiding behind a process-curtain. Sure, it may to be their choice and, thus, not their fault. “Shitty jobs are being done,” if you prefer the blamelesss-veil of passive voice.

Fix your shit

When I hear objections to fixing this situation, I try to b nice and helpful. After all, I’m usually there as part of an elaborate process to get money from these folks in exchange for helping them. When they go all Eeyore on me, I have to reframe the room’s thinking a little bit without getting too tough love-y.

“When I put these lithium batteries in this gas car, it doesn’t seem to work. So electric cars are stupid, right?

You want to walk people to asking “how do we plan out the transition from The Old Way That Worked At Some Point to The New Way That Sucks Less?” They might object with a sort of “we don’t need to change” or the even more snaggly “change is too hard” counter-point.

I’m not sure there are systems that can just be frozen in place and resist the need to change. One day, in the future, any system (even the IRS’!) will likely need to change and if you don’t already have it setup to change easily (awesome mode), you’re going to be in a world of hurt.

The fact that we discuss how hard it is to apply awesome mode to legacy IT is evidence that that moment will come sooner than you think.

(Insert, you know, “where’s my mobile app, Nowakowski?” anecdote of flat-footedness here.)

ITIL end in tears(tm)

The royal books of process, ITIL, are another frequent strawperson that frothy mouthed agents of change like to light up. Few things are more frustrating than a library of books that cost £100 each. There’s a whole lot in there, and the argument that the vendors screw it all up is certainly appetizing. Something like ITIL, though, even poorly implemented falls under the “at least it’s an ethos” category.

Climbing the Value Chain

I’m no IT Skeptic or Charles T. Betz, but I did work at BMC once: as with “bi-modal,” and I really don’t want to go re-read my ITIL books (I only have the v2 version, can someone spare a few £100’s to read v3/4?), but I’m pretty sure you could “do DevOps” in a ITIL context. You’d just have to take out the time consuming implementation of it (service desks, silo’d orgs, etc.).

Most of ITIL could probably be done with the metaphoric (or literal!) post-it notes, retrospectives, and automated audit-log stuff that you’d see in DevOps. For certain, it might be a bunch of process gold-plating, but I’m pretty sure there’s no unmovable peas under all those layers of books that would upset any slumbering DevOps princes and princesses too bad.

Indeed, my recollection of ITIL is that it merely specifies that people should talk with each other and avoid doing dumb shit, while trying to improve and make sure they know the purpose/goals of and “service” that’s deployed. They just made a lot of flow charts and check lists to go with it. (And, yeah: vendors! #AmIrightohwaitglasshouse.)

Instead of increasing the volume, help spray away the shit

That gets us back to the people. The meatware is what’s rotting. Most people know they’re sad, and in their objections to happiness, you can find the handholds to start helping:

Yes, awesome mode people, that sounds wonderful, just wonderful. But, I have 5,000 applications here at REALLYSADMODECOGLOBAL, Inc. – I have resources to fix 50 of them this year. YOUR MOVE, CREEP!

Which is to say, awesome mode is awesome: now how do we get started in applying it at large orginizations that are several fathoms under the seas of sad?

The answer can’t be “all the applications,” because then we’ll just end up with 5,000 different awesome modes (OK, maybe more like 503?) – like, do we all use Jenkins, or CircleCI, or Travis? PCF, Docker, BlueMix, OpenShift, AWS, Heroku, that thing Bob in IT wrote in his spare time, etc.

Thus far, I haven’t seen a lot of commentary on planning out and staging the application of mode 2. Gartner, of course, has advice here. But it’d be great to see more from the awesome mode folks. There’s got to be something more helpful than just “AWESOME ALL THE THINGS!”

Thanks to Bridget for helping draw all this blood out while I was talking with her about the bi-modal pice she contributed to.