New DevOps Column at The Register

I started a new column at The Register, on the topic of DevOps. I used the first column to layout the case that DevOps is a thing, and baseline for how much adoption there currently is (enough, but not a lot – a “glass almost half full” type of situation). I was surprised by how many comments it kicked up!

Next up, I’ll try to pick key concepts and explain them, along with best and worst practices for adoption of those concepts. Or whatever else pops up to fill 800 words. Tell me if you have any ideas!

(You may recall had a brief column at The Register back when I was at 451 Research.)

Compliance & Audit for Cloud Natives

Questions around audit and compliance always come up in discussions about improving software, and certainly when it comes to introducing things like continuous delivery, DevOps, and esp. something as big and different as Pivotal Cloud Foundry. To that end, I wrote up a way to approach those issues, along with a few tips for dealing with compliance and audit for my FierceDevOps column last month.

The onerous steps auditors want you to do were usually put in there for good reason, but, as I put it:

Unfortunately, the way that three-ring binder wielding ninjas and IT staff battle it out over these and other compliance check-lists often loses sight of the original, good intentions. Instead, it infects everyone with a bad case of table-flipping madness. Thanks to cloud technologies and the empathy over table-flipping approaches in DevOps, we’ve been finding ways to get over compliance hurdles and even, in some cases, make compliance projects easier and better.

There’s a summary on the Pivotal blog, and/or you can check out the full piece.

(Binders picture from tookapic)

Getting collaboration right in Agile & DevOps – Press Pass

I don’t do press passes as much as I did when I was an analyst, but here’s one from a recent email interview for a ProjectsAtWork story:

Q: What’s your favorite tip to improve collaboration when an organization moves to agile and DevOps?

A: I think the core DevOps thing with collaboration is getting people to trust each other. Most corporate cultures are not built on people trusting each other and feeling comfortable: they’re based in competitive, zero sum structures or command and control management at best.

Organizations that are looking to DevOps for help are likely trying to innovate new software and services and so they have to shift to a mode of operating that encourages collaboration and creativity. Realizing that is a critical step: we want to create and run new software, so we need to understand and become a software producing organization.

In contrast, if you operate differently if you’re just driving down costs each quarter and not creating much with IT. We’d counter-argue that if you’re a large organization and you’re not worrying about software then you’ll be creamed by your competition who is becoming a software organization.

If forced to pick one tip to increase collaboration I would say: do it by starting to work. How you do this is to pick a series of small projects and slowly expand the size of the projects. These projects should be low profile, but have direct customer/revenue impact so that they’re real. It’s important for these projects to be actual applications that people use, not just infrastructure and back-end stuff. It will help the team understand the new way of operating and at the same time help build up momentum and success for company wide transformation later down the road.

As a basic tactic, Andrew Shafer has a fun, effective tactic about having each people on the team wrote fantasy press releases about each other to start to build trust.

(See the full piece by Will Kelly over on the site.)

DevOps ROI

Recently, for my column over at FierceDevOps, I opined about doing ROI for DevOps. This topic comes up a lot as I note in my Pivotal post on the topic.

Here’s a summary/excerpt of the three ways of thinking through it:

1.) Bottoms-up ROI: We know everything and have put it in this spreadsheet

…if you have a good handle on the costs during some period of time where you were doing DevOps, and the gain that resulted from that period of time, you could come up with a bottoms-up ROI analysis. I think it’ll be somewhat dicey since it’s so hard to attribute costs and gains directly to DevOps but hey, it’s better than either telling people they’re asking the wrong question or its mute cousin: nothing.

2.) Were the efforts to change worth it? That is, DevOps is all cost

…if you want to run the numbers on something like “they tell me it will take three months and $50,000 in training and consultants to ‘do the DevOps'” this might satisfy your ROI craving. Again, you’ll need to have a pre-existing ROI at hand to simply plug your DevOps costs into.

3.) Pain avoidance and remediation

In the “DevOps is all cost” ROI scenario, we avoided ascribing gain to changing to DevOps. Again, while this is overly simplified, the deliverables of DevOps are to provide a continuous delivery process for your product and ensure that your product has excellent uptime. How could you account for the gain of those two desirables? You could create a way of assigning value to the knowledge you gain from weekly iterations about how to improve your product. You could also calculate the savings from avoided downtime.

Check it out, and tell me how you’ve been answering that question.

Enterprise DevOps interview with iThome Weekly

A little while back I did an email interview with Ray Wang from iThome Weekly, in Taiwan. It’s a little piece about DevOps getting more and more into the enterprise. To read the Google, robot translation, it looks like I did some things “single-handedly,” where in fact I was one of many hands.

As always, here’s the original email exchange we had:

Q. You mention about software-defined business in your article, can you tell more details about what is software-defined business? Why CEO have to think about the software-defined business? Why DevOps is so important for software defined business?

“Software defined businesses” are companies that are using custom written software to dramatically change and enhance how they run their business. Uber is a good example. Instead of just being a taxi or car service, they use software they wrote to change how their business runs: calling and paying for a taxi on your sell phone is much different than hailing a cab and paying in cash. Insurance and banking companies that are moving more and more of their daily business and interaction with customers to run over mobile apps and other custom written applications are another good example; we see this happening at Pivotal customers lie Allstate, Humana, and banks that use Pivotal Cloud Foundry.

Q: What’s your definition of DevOps? Does DevOps equal to Continuous Delivery?
Which definition of DevOps you don’t like most and why?

In general, I think of DevOps as the process and “culture” you wrap around continuous delivery to get the full effect of CD. I tend to speak about them interchangeably at this point; I suppose you don’t need DevOps to get the full benefits of continuous delivery, but they seem to go together well (you could always have just a jelly or just a peanut butter sandwich, but they seem to show up together a lot). CD is always looking to automate as much as possible, delivery to production frequently, and use the feedback loop this rapid cycle gives you (you an observe what your users does each week or day instead of each six months) which are many of the things DevOps seeks to enable as well.

It’s easy to get caught up in DevOps conversations that spend all of them time talking about “culture” and the need to change. I’m interested in that, but I always want to hear about actual, tactical things companies can do to get the benefits of DevOps. We all know how businesses use IT needs to change to be better, and that it’s hard to do so. I want to make sure the overall community is giving advice that’s helpful and, dare I say it, “actionable.”

Q: Should companies have to implement agile development before implement DevOps? Why?

It certainly helps to know what agile development is as a school of thought and to have done some form of agile to trust that way of thinking. If you’ve never done agile before, it just becomes part of trying to do DevOps. It’s certainly hard to think of being successful at DevOps without also doing agile software development.

Q: If CIO want to tell his CEO boss about what’s the value for non-technology companies , what’s your suggestion?

Time to market is the main, measurable, benefit. What that means, to me, is that software is being given to customers more frequently. New features and fixes come out weekly instead of once every 6 to 12 months. The business (the CEO) has to figure out what to do with time to market. If you can put a new features in production each week, how will that help the business? In the consumer space (where much of this mentality comes from) you can add more and more features to out-compete competitors. In the business space, the actual business has to change and evolve at a fast pace to fully take advantage of time to market. All that said, I don’t think any CEO is satisfied with the pace of change in IT. They’d all like it to move not just “faster,” but to get more meaningful features in production more often. Humana provides an interesting example: because they had been honing their software delivery process they were able to launch an Apple Watch app in just five weeks. That timeline is pretty amazing for most enterprise IT projects, let along being in the App Store on the first day of the Apple Watch’s release.

Q: Gartner say 2016 will be the year of DevOps. Do you agree with that? why or why not?

Sure, but I think you’ll see the next 3 or so years be the year of DevOps. I don’t think there’s any one year in particular that will stand out. I don’t think there was “a year of Agile Software Development” in the 2000s, it just took over slowly. What important is for companies to understand what DevOps can help give them – faster time to market for their software-driven products and services – and figure out what they’d do with that new ability. “Doing DevOps” is not easy, so you really need to value the end result or you’ll likely loose interest in the transformation process and let it unhelpfully fizzle out.

And, check out the recording of the DevOpsDays Austin talk referenced in the article if you’re interested.

What does IT need to start doing to become a software defined business?

I was asked to talk to do an internal, “brown-bag” style talk at a company this week. I chose to do a slightly more technical-oriented version of the talk I tend to give, commentary and pointers on moving your orginization over to relying on more and more custom written software to run your business. Here, I give a brief business context and then throw out three areas to start focusing on if you’re interested in cloud, DevOps, and all this nonsense. The recommendations are to look into contious delivery and DevOps, figure out cloud-native applications (and microservices), and then plan out your cloud platform strategy.

As ever, I’m trying to make actionable a few things that are often fuzzy. I don’t acomplish that too well – a seperate hour long talk on each topic would be better – but I at least hope to explaing the thing, say why it’s useful, and give some pointers for further study.

I put the slides up previously, but because people often ask me for the talk track, I thought I’d record a rehersal run and post it here. So, check out the video if you’re into this kind of thing.

DevOps Success Metrics, Nationwide

From an IBM article:

Since adopting [DevOps], Nationwide has achieved a 90 percent on-time software delivery rate (up from 60 percent previously) and a 70 percent increase in users’ system availability. But what’s truly impressive is that, in concert with the increased pace of software delivery, Nationwide reduced critical defects 80 percent and high-level defects by 86 percent, which means the company is delivering higher-quality software faster. Nationwide says it can now “invest more into playing offense” to help the company drive innovation and build new offerings.

See the full-court case study as well.

The Coming Donkey Apocalypse, DevOpsDays Austin recording

My talk from DevOpsDays Austin is up. Check it out if you’ve been curious about the talk track to the mute slides.

As a reminder, there’s a prose version of the talk available as well in one of my FierceDevOps columns, and here’s the slides.

Check out my love affair with “uh” in the begining, I think it clears up a bit at the end.

Moving up the stack

Hashimoto suggested that prior to Terraform, there could be an incredible responsibility placed on an operations team when managing production stack, as they had to deeply understand the current cloud platform(s), determine the current infrastructure state, and calculate the resulting state transitions. Hashimoto argued that some operators or DevOps engineers may want to ‘move up the stack’ and leverage tools such as Terraform to achieve their goals, in much the same way as many developers have moved from assembly language to third-generation programming languages.

Software Defined Businesses need Software Defined IT Departments

(I originally wrote this April 2015 for FierceDevOps, a site which has made it either impossible or impossibly tedious to find these articles. Hence, it’s now here.)

Quick tip: if you’re in a room full managers and executives from non-technology companies and one of them asks, “what kind of company do you think we are?”…no matter what type of company they are, the answer is always “a technology company.” That’s the trope us in the technology industry have successfully deployed into the market in recent years. And, indeed, rather than this tip being backhanded mocking, it’s praise. These companies are taking advantage of the opportunity to use software and connected devices in novel ways to establish competitive advantage in their businesses. They’re angling to win customer cash by having better software and technology than their competitors.

What does it look like “on the ground,” though when it comes to “being a technology company”? I’d argue that the traditional ways we think about structuring the IT department is different than how technology companies structure themselves. To massively simplify it, traditional IT departments are oriented around working on projects, where-as technology companies are oriented around working on products.

The project-oriented IT department

Project oriented thinking takes in requests from an outside entity and works on solving an immediate, well understood problem. There’s often a definitive end to the project: the delivery of the new “service.” Project oriented thinking is good for creating an initial version of an application, installing and upgrading existing packaged software, setting up new offices, on-boarding employees, and other things that have a definitive completion date and well known tasks.

When organizing around this type of work, you setup a functional organization that can be assembled to implement the specific problem. Here, by “functional organization,” I mean groups of people who are defined by their expertise in something: networking, server administrator, software development, project management, audit and compliance, security, and so on. These people are typically shared across various projects as needed and usually are responsible for just what they know about. (For a very different take on how to use functional organizations, see Horace Dediu’s discussion of how Apple organizes itself.)

On-top of this, you take a request-driven approach to change management, which defines when to launch a new project or make “small” changes to an existing one (like adding a user). In the 2000s, we fancied up this concept by calling it “service management.”

Once the project is up and running, there may be something called “maintenance mode” which sees IT making sure, for example, that the ERP application has enough disk space available, that new users are added to the application, and that extra capacity is added when needed.

This mind-set is very handy when you’re dealing with keeping a bunch of products from tech vendors up and running. It’s even good if you have custom written applications that are not changed frequently. What’s also great, is that because each project is well defined at the start and has a definitive end, you can measure success and financial metrics easily: how many requests did we handle (tickets closed) this month? did we deliver on time? did we deliver on-budget? is the project profitable (thus, did we pay too much for that software or get a good deal?)

However, two things have been changing this state of affairs, pushing IT to be more exploratory in nature. As a consequence, the structure of the IT department will need to change as well to maximize ITs value to the overall business.

Cloud changes the organization into a software eating the world type situation

I often joke that it’s been impossible to see a keynote in recent years without seeing the horsemen of the digital apocalypse. These are the cliche topics that seem to come up in every keynote. Two of these lay the groundwork for why the structure of the IT department needs to change:

  1. Software is eating the world — Cloud technologies and practices have made huge improvements in productivity and costs when it comes to creating and running custom written applications. It’s easier to write and run software now, and the rise in “always on” devices (all those super-computers in our pockets that are on the Internet 24/7) creates a massive foot-print for computation: an endless buffet for software.
  2. Change or die! — with this huge buffet of opportunity, there’s a rallying call for companies to invent new business models that rely heavily on software. This means that most every business has the opportunity to use custom written software to change the nature of their business. Think of the opportunity for taxi companies to use software to change how they operate, or for the hotel industry to come up with a brand new business model to sell empty capacity…and you’re thinking of Uber and AirBnB. The “or die” part is a rhetorical trick to position this imperative as dire. And, indeed, studies have shown that remaining on-top has become harder in recent decades. Change is needed to survive.

These two alone create a pull for more custom written software in businesses. It’s fast and cheaper to create software, and competition is relying on that to create new business models that challenge incumbents or, rather, those businesses that are not evolving how they run their business with software. Again: think of all those taxi services versus Uber.

IT — SaaS = what?

There’s a third “horseman” in the broader industry that’s driving the need to change how IT departments are structured: the rise of SaaS. Before the advent of SaaS across application categories, software had to be run and managed in-house (or handed off to outsources to run): each company needed its own team of people to manage each instance of the application.

Source: Source: Two studies, first with 1,137 respondents, second with 1,097, involved in their company’s IT buying decisions participated in the Jan 2014 and July 2014 survey, including 470 and 445 whose company currently use public cloud. “Corporate Cloud Computing Trends,” 451 ChangeWave, Feb, 2014 & “Corporate Cloud Computing Trends,” 451 ChangeWave, Aug 2014.

As SaaS use grows more and more, that staffing need changes. How many IT staff members are needed to keep Google Apps or Microsoft’s Office 365 up and running? How many IT staff do you need to manage the storage for Salesforce or Successfactors? Indeed, I would argue that companies use more and more SaaS instead of on-premises packaged software, the staffing needs change dramatically: they lessen. You can look at this in a cost-cutting way, as in “let’s reduce the budget!” Hopefully you can look at it in a growth way instead: we’ve freed up the budget to focus on something more valuable to the business. In most cases, that thing will writing custom software. That is: developers.

The product oriented IT department

This is where the shift to thinking like a product organization is vital. First of all, if you feel the need to develop more custom software — as you should! — you’ll need to hire and train more software developers, product managers, QA staff, and related folks. You’ll also want to cultivate an environment where new ideas can be explored, user-tested in production, and then quickly refined in a loop that spans mere weeks if not one week. You’ll need to become a continuous delivery and learning organization. Jonathan Murray has called this type of organization a “software factory” and has explained how he implemented the change while at Warner Music. More recently, books like Lean Enterprise have explained how this type of thinking can be applied outside of “startup culture,” whose concerns tend to be more around achieving a high valuation to get the company acquired or IPO rather than building and maintaining sustainable business models.

Setting up an organization like this requires not only developers, but creating the actual “factory” that they operate in. I think of this factory as a “platform” and the folks responsible for standing up and caring for that platform are a new type of operations staff. They’re in charge of, really, providing the “cloud” that developers effortlessly deploy and run their applications in.

This new type of IT staff has to think about how they add in as many self-service and highly elastic services in their “cloud” as possible. They too are creating a “product,” one that’s targeted at the internal developer teams and which must continually have new features added to it.

Meanwhile, your developers will be arranged into product-centric teams, hopefully working more closely with line of business managers and staff who are helping craft and grow new applications. No doubt they’ll need operations skill on the team: staff who know how to properly architect and operationalize cloud-native applications.

This is where the now classic DevOps mentality comes in: in order to properly focus on a product, the team must be responsible for all parts of that product’s life, from development through production and back. With a proper cloud platform in place and the operations team to support it, these goals are more achievable than if the product team has to start from bare metal, or work with IT through a ticket system.

To be pragmatic, you probably can’t dedicate all people fully to a product and will need to share them. This carries large risks, however, namely, making sure you properly prioritize an individual’s time and realizing that they’ll have a harder time keeping up with fewer products rather than more. Quality and ability to deliver on time will likely decrease. It may seem like an impossible goal, but often in order to stay competitive — to survive — large, seemingly impossible changes are needed

Coté Memo #000: No jokes here.

Tech & Work World

“What kind of company do you think we are?”

Here’s some excerpts from a FierceDevOps column I submitted yesterday.

Quick tip: if you’re in a room full managers and executives from non-technology companies and one of them asks, “what kind of company do you think we are?”…no matter what type of company they are, the answer is always “a technology company.” That’s the trope us in the technology industry have successfully deployed into the market in recent years. And, indeed, rather than this tip being backhanded mocking, it’s praise. These companies are taking advantage of the opportunity to use software and connected devices in novel ways to establish competitive advantage in their businesses. They’re angling to win customer cash by having better software and technology than their competitors.

And, later revisiting my old IT – SaaS = what? trope…

There’s another “horseman” in the broader industry that’s driving the need to change how IT departments are structured: the rise of SaaS. Before the advent of SaaS across application categories, software had to be run and managed in-house (or handed off to outsources to run): each company needed its own team of people to manage each instance of the application.

451 Research ChangeWave Cloud Usage

Source: 451 Research/ChangeWave

As SaaS use grows more and more, that staffing need changes. How many IT staff members are needed to keep Google Apps or Microsoft’s Office 365 up and running? How many IT staff do you need to manage the storage for Salesforce or Successfactors? Indeed, I would argue that companies use more and more SaaS instead of on-premises packaged software, the staffing needs change dramatically: they lessen. You can look at this in a cost-cutting way, as in “let’s reduce the budget!” Hopefully you can look at it in a growth way instead: we’ve freed up the budget to focus on something more valuable to the business. In most cases, that thing will writing custom software. That is: developers.

Quick Hits

tumblr April Fools

Fun & IRL

#WorkingFromHome

I’ve been collecting some little aphorisms and such on working from home when you have young kids. I find it extremely challenging, and rewarding at the same time. I’m curious how other people cope. Part of the issue is that, with a 1.5 and 5 year old, about once every 30 minutes someone is crying or wants attention. There’s just no letting up. If you’re the parent working, you have to just ignore it, which is weird.

Here’s something I wrote up recently:

The end of the day is the worst. Your family asks you every five minutes when you’ll be done; they start wanting to play with you. At the same time, you’re desperately trying to find time to get done. Each time they interact with you, it slows you down.

The answer, of course, is the same as always: you have to control access to you in a way that;s not assholey. Lock a door, go to a distant room. You have to hide.

Five year olds aren;t up to speed on the cost of context switching and haven’t read the maker/manager essay.

Anyhow, I think there’s a good presentation in collecting enough tips and, more helpful, counseling to make it work.

Sponsors

Meta-data

What are normal people doing with continuous delivery?

//player.vimeo.com/video/120746550?wmode=opaque&api=1

My latest Pivotal blog post is up, it re-caps a presentation I did recently covering what “the market” is doing with continuous delivery.

There’s a lot of opportunity, the glass is half full. See the slides over in my previous post on this talk.

Also, check out the recording of the full talk (it has some bonus material on containers recent role in CD) from HeavyBit, embedded above.

3 easy reasons why you’ll move your business to the cloud

Why are frequent deployments important? If you can deploy hundreds of times per day, you can recover from mistakes almost instantly. If you can recover from mistakes almost instantly, you can take on more risk. If you can take on more risk, you can try wild experiments—the results might turn into your next competitive advantage.

From Matt Stine’s upcoming book, Cloud-Native Application Architectures.

Coté Memo #053: there’s a lot of earth for software to eat, day 1 of #CAWorld

Follow-up

  • It’s been awhile. The family and I were on vacation for a bit in Paris, and then I was at the OpenStack Summit.

Tech & Work World

Quick Hits

Las Vegas Conference: CA World

The longer you stay away from a conference in las Vegas, the weirder it is to be here for one. Or, maybe, it’s just that it’s always weird. I’m here for CA World now, which is an odd, interesting conference: attendance and sponsorship seems sparse but they have some good DevOps messaging.

I was last at CA World in 2010, where the message was a resounding “cloud or die!” There’s a similar message around “the application economy,” which is pretty well aligned to what I’m always going on about: developers are important, you business people should be hiring them to write applications. Coming from CA, a company never really known for having much to do with developers, this is a odd message, but it’s the one you’re stuck with in an IT – SaaS = what? world.

CA has good messaging and solutionaring around DevOps. They have release management (from Nolio), mock testing (“service virtualization” they call it), and monitoring of course, APM notably. They have lots of parts, and can put them all on a slide well. They’re also very clear about their approach being solution oriented: not DevOps products, but just ways of combining their tools together.

After a discussion with a fellow attendee, the question in my mind is: what’s the unique thing about CA that’d make you go to them for all of this? Their answer would likely be breadth of tools and integrations. The usual for a large company.

For as much effort as they put into DevOps – by my count, the most out of their class of companies – they don’t come up in the DevOps world much. One theory is that DevOps has, thus far, been product and commercialization resistant. Another might be that the message simply hasn’t gotten out. Another could be that the “DevOps market” is so small that nothing would register.

There’s a lot riding on mainstream “enterprises” committing to the idea of writing more and more custom software: that whole “software is eating the world” but. It feels true to us techies, but there’s a question of the rate of change and if it’s net-new growth. I’m also unsure how a company structures itself to take advantage of more enterprises needing to develop software.

As a round-about example, I’ve been looking into Docker and cloud orchestration software. There’s so many projects built around those problems: there’s even too many! In a software eats the world world, there’s almost too many options, making it hard for a company to cement in competitive advantage (the reason customers come to them and pay a premium over competitors – the reason companies make profits).

It’s just day one of two I’ll be here. Perhaps all the answers are tomorrow.

Fun & IRL

  • With kids on vacation, you watch a lot of TV. That Regular Show is pretty good.

Sponsors

Meta-data

Video for #InnoIT – Enabling Innovation in IT – Dell World Social Think Tank

The video for the think tank I mentioned last week is up. They say they’ll slice it into smaller chunks as well, but if you’re interested in a discussion of sorting out how “The IT Department” can do more than keep the lights on, here’s 60+ minutes on it!