2017 Cloud Foundry Application Runtime Survey – Highlights

There’s a new survey out from the Cloud Foundry Foundation, looking at the users of Cloud Foundry. Here’s some highlights and notes:

  • Another ClearPath joint, n=735.
  • It’s important to keep in mind that this is covers all distress of Cloud Foundry, including open source (no vendor involved).
  • “The percentage of user respondents who require over three months
    per app drops from 51 percent to 18 percent after deploying Cloud Foundry Application Runtime”
  • “…while the percentage of user respondents who require less than a week climbs from 16 percent to 46 percent.”
  • “Nearly half (49 percent) of Cloud Foundry Application Runtime users are large enterprises ($1+ billion annual revenue).”
  • This chart is hard to read, but it shows a reduction in time to deploy across various time periods:
    before-after-release.
  • Uptake is early, but there are definitely mature users: “A plurality of Cloud Foundry Application Runtime users (61 percent) describe their deployments as somewhere in the early stages—trial, PoC, evaluation, or a partial integration into specific business units. Meanwhile, 39 percent have deployed Cloud Foundry Application Runtime more broadly across their company, from total integration in specific business groups to company-wide deployment.”
  • “Comcast, for example has more than 1500 developers using Cloud Foundry Application Runtime daily. Home Depot reports more than 2500 developers.”
  • “Comcast has seen between 50 percent and 75 percent improvement in productivity.”
  • “Half of Cloud Foundry Application Runtime users are currently using containers, such as Docker or rkt, with another 35 percent evaluating or deploying containers.”
  • Container management – there’s a wide variety of tools that people use for container orchestration, including DIY (14%). There’s a lot of interest in having CF do it: “Nearly three-quarters (71 percent) of Cloud Foundry Application Runtime users currently using or evaluating containers are interested in adding container orchestration and management to their Cloud Foundry Application Runtime environment.” Hence, validating the Cloud Foundry Container Runtime.
  • Of course, the surveyed are already CF users, so they’re biased/driven by what they know.
  • Almost half of respondents say that getting started with CF. But people end up liking it: “An overwhelming majority of users (83 percent) would recommend Cloud Foundry Application Runtime to a colleague, including 60 percent who would do so strongly.”
  • “As more companies roll out Cloud Foundry Application Runtime more broadly, the footprint continues to grow. Currently, 46 percent of users have more than 10 apps deployed on Cloud Foundry Application Runtime, including 18 percent with over 100 (and eight percent with over 500).” 4% have over 1,000 apps.
  • CF’s uses: “The primary use is for microservices (54 percent), followed by websites (38 percent), internal business applications (31 percent), Software-as-a-Service (SaaS) (27 percent) and legacy software (eight percent).”
  • Validating multi-cloud: “60 percent say this is very important, and another 30 percent describe it as somewhat important.” Meanwhile, 53% are using more than one type of IaaS.

The news from Docker-land, plus, the money being fought over – Notebook

With DockerCon this week, there’s no end of Docker quotables and items. Here’s my collection

General momentum

Once landed in an account, Docker usage grows their CEO says:

There has also been expansion within customers, with organizations that start with Docker expanding their usage on average by five times within six months

Way back in 2015, the (now annual?) DataDog study of Docker usage among their customers said that 2/3 of companies that try Docker adopt it. Which is all to say: once it gets in, it spreads.

Moby

A toolkit for putting together docker stacks:

In essence, Moby is the build system that creates Docker Community Edition, which is akin to Fedora, and Docker Enterprise is derived from Moby and is akin to Red Hat Enterprise Linux. Link

People got all freaked out. I’d even say “freaked the fuck out.” Competitors, of course, gloated, if only in silence. Criticism of handling the announcement aside (ideally, you wouldn’t like to kick up a stink), I feel like it was more like a tempest in a teapot.

Docker momentum/penetration and types of applications/workloads

Global 2000 customers have somewhere on the order of thousands to tens of thousands of applications, and across these major firms, less than 5 percent of the applications have been containerized so far. While somewhere between 5 percent and 10 percent of the applications that are being containerized are net-new, microservices-style applications that everyone is talking about all the time, the other 90 percent to 95 percent are just lifting and shifting legacy applications from bare metal or virtual machines to containers. Link

VMware threat…or just legacy gobbling?

Docker bounces back and forth between “replacement for VMware” and “a different thing, so don’t worry about VMware.” In this round of Docker news, there’s been some strong pull towards the “replacement for VMware” camp. To be fair, it’s more like doing both:

In general, says Johnston, customers who move from bare metal or VMs to Docker containers can provision, scale, and deploy applications up to 75 percent faster, and those moving from bare metal to containers can save 50 percent on compute and those who are moving from VMs will save around 25 percent. Link

This might also come from the obvious move to start gobbling up legacy (more accurately “existing”) applications. Here, Docker had two customer reference:

Northern Trust, a leading international financial services company, experienced  deployment times that were 4X faster and noted a 2X improvement in infrastructure utilization

And, Microsoft IT:

Microsoft is not only a partner in this program; their IT organization is also a beta customer.  Microsoft IT increased app density 4X with zero impact to performance and were able to reduce their infrastructure costs by a third.

There was also a story of Visa using Docker:

Kocherlakota said Visa is aiming to move as many workloads at it can to the container model to help improve overall efficiency.

See more on this legacy migration stuff and the program with Avanade, Cisco, HP, and Microsoft from Docker’s Scott Johnson.

Major vendors

Other tech companies are often cautious about working with Docker. They’re not really certain about how it helps or threatens their position in the IT stack and, therefore, their ability to sell higher profit margin products and services. No one wants to become the x86 manufacturer of the cloud (read: low margin, commodity).

I’ve noticed this cautiousness slightly melting as more and more vendors are at least putting their stuff in Docker images and, on the public cloud front, supporting the use of Docker. My company, Pivotal, ingests Docker images.

A brief whack at why Microsoft cares, from Christopher Tozzi:

Although there remains work to do to get Docker on Windows ready for prime time, the platform will be important in helping Windows Server stay as nimble as Linux environments in hosting the workloads of the future…. Microsoft’s interest in Docker may seem strange. Microsoft already offers traditional virtual machine products, most notably Hyper-V. In some respects, Docker containers compete with virtual machine platforms…. But that’s not necessarily the case. Depending on how they’re used, containers can complement virtual machines, rather than replace them. If you use virtual machines to host the environment in which Docker runs, your Docker environment becomes more scalable and portable than it would be if it ran on bare metal. That’s likely the type of use case Microsoft envisions for containers on Windows.

More from Nick Martin on Microsoft and Docker.

Oracle bundling middleware in Docker containers:

Oracle becomes the latest enterprise IT vendor to jump on the Docker container bandwagon as it seeks to expand its reach in the public cloud market. Among the container-based application, middleware and development tools made available on the container platform are Oracle’s MySQL database and its WebLogic server. Those tools are in addition to the more than 100 images of Oracle products already available on Docker Hub, its cloud-based image registry.

So, what’s going on here? Staking a claim on The New Stack

I’m often asked to explain all the various cloud stacks, to help Pivotal buyers sort out what CaaS, PaaS, cloud-native, and “cloud strategy” means. They’re trying to figure out their planning for building out new IT, for “doing DevOps.” It’s a mess out there w/r/t to figuring all this out if you’re not a vendor or analyst who’s steeped in this shoggoth every day.

In all the Docker, container, and cloud-native wars, the revenue battle for vendors is mostly about two things:

  1. The pool of money in simply migrating the VMware workload to a new, more efficient layer, hence the ongoing attention to “the VMware threat” that Docker poses). I’m not sure how big this market is because, as a disruptive shift (cf. Linux vs. UNIX vs. Windows vs. z) part of it is reducing the overall spend through lower prices and more efficient usage. But, the existing virtualization market is best described as “fucking huge.”
  2. Fighting over who “owns” (and therefore collects the most profit from) the stack that companies are using to build and run their software. By my estimate, this is something like around a $20-25bn market in the future. You can see a Spanish Civil War like precursor going on in the Java application server market; it’s spreading to a “World War” with respect to all custom software stacks.

On that second point, here’s my latest attempt to describe how things are shaking out category/definition wise:

Of all the SPI cloud categories, PaaS is the most problematic place as all us vendors hate the PaaS term and are trying to re-define what it means. I would break PaaS into two categories currently: (1.) container orchestration, and, (2.) cloud platform.

Container orchestration takes an IaaS and manages the installation and configuration of container images on your new cloud. By “images” here, I mean that you’ve chosen to put your software (probably custom written software, not packaged software) into containers (or the delegated way we do it with buildpacks in CF), specified how all the different nodes are wired together with all the ACLs and configuration, and then given it over to the orchestration software to deploy those containers, set the configuration, and do the ongoing health-checks/remediation.

Ideally, the orchestration platform should also have “day 2” tools to help you monitor and manager (“fix”) problems that happen in production. I assume things like kubernetes, the Docker/Moby constellation of things, Mesosphere, etc. fit here.

People are obsessed with container orchestration now and it’s pretty much all anyone talks about. I think all this is what’s becoming known as “CaaS” – Containers as a Service.

(On this next section, I’m extremely monetarily biased, of course:) A cloud platform either has or depends on an orchestration layer, but adds in integrated middle-ware, ALM tools (from basics like “cf push”, and an overall programming and deployment model with all the tools and enforcements. Heroku is the classic example here in public cloud, and now Cloud Foundry (CF) has taken over this model in public and private cloud, the second (it seems) where most of the usage and money is, at least in the enterprise space. I’d argue, that CF is the enterprise market-leader (by revenue at least, but increasingly penetration in the F500 – while Pivotal has impressive numbers, throw in the other CF distros and it’s even larger, no doubt); at the very least, “the highest growth and in enterprise production usage.” That all depends how you slice it, and of course my slicing favors me.

A cloud platform “pulls together” everything into a fully working “cloud” that deploy and provisions the servers, builds/maintains/deploys the containers, takes care of your networking configuration and concerns (inc. firewalls, etc.), and configs/manages all the middleware needed (e.g. “I want a database” means you just ask for it, instead of having to configure it and make container images of it and specify how it all works together).

The end goal of a cloud platform is the original end-goal of a PaaS: developers don’t have to “setup” any of the infrastructure or, really, middleware (databases, queues, etc.) that they use: they just write the “business logic” of their applications.

All this standardization is technically “restrictive” (developers can’t just install anything they download off the Internet, it has to be integrated into the platform). This is why we often call this model “opinionated,” but it follows the same contract/promises model that Google SREs follow: we promise we can support your applications in production if you use only the things we support, otherwise it’s all on you.

However, the benefit of such opinions is a huge jump in productivity as we see at all our customers: one Pivotal customer manages 1,000+ applications (all angles toward very frequent, DevOps-style releases for fast feedback loops and all that small batch stuff) with just 4 PCF operations staff, etc.

Our DIY white paper makes the case that snow-flaking this all out is a bad idea. At the very least, if you build your own platform, you should try to just have one used organization wide.

In comparing CaaS and cloud platform, the key distinction to me is that a cloud platform bundles and integrates together all your middleware and “services” frameworks. For example, if you want to do microservices with all the bulk-heads and such, that functionality should be built into the cloud platform – you should have to go read-up how to set most of that up. PCF, of course, has Spring Cloud and more for that. All of the systems management tools (thing used in production to detect and fix problems) should also be built in, or the cloud platform should be instrumented so deeply that third party tools can do the managing as well.

Now, these two categories are likely to converge, and then the discussion will just be which cloud platforms are more featureful and better. It’ll be like battling Java application servers.

I haven’t made one of my own “burger” stacks of all this in a long time, but I think (again, highly biased) the ones we use for PCF are pretty good:

More

In case you don’t know, working at Pivotal, I obviously have a stake in how all this turns out, so I’m biased on multiple angles of the above whether I want to be or not. 

Microsoft buys Deis, deeper into Kubernetes & $1.1bn container market – Notebook

A round-up of the news and some context around Microsoft burrowing down further into Kubernetes-land by acquiring Deis:

The deal & market

  • Microsoft: “Deis gives developers the means to vastly improve application agility, efficiency and reliability through their Kubernetes container management technologies…. We expect Deis’ technology to make it even easier for customers to work with our existing container portfolio including Linux and Windows Server Containers, Hyper-V Containers and Azure Container Service, no matter what tools they choose to use.”
  • Deis: “We look forward to making Azure the best place to run containerized workloads.”
  • Deis is/was part of EngineYard, right? – Notable that EngineYard (on April 10th, 2017, day of announcement) doesn’t mention it on their blog, or press release list. And that Deis and Microsoft don’t really either. See 451’s Jay Lyman’s coverage of that deal in 2015.
  • No deal-size was disclosed, of course, but Deis was small and I’m guessing it didn’t fit into EngineYard’s overall strategy, or what (little?) cash they got was a nice to have versus synergies of keeping Deis.
  • Containers are rising in usage, as 451’s Donnie said: “Our latest data says production use of containers has doubled from 10.2% to 22.5% of orgs between Q1 and Q3 2015. Amazing.”
  • 451’s January 2016 container market TAMs and forecast:
Screenshot 2017-04-10 13.56.56

The technology: not so much PaaS anymore, but Kubernetes management

Deis stack

Microsoft likes Kubernetes

  • Seems like Microsoft has gone all k8-crazy. So this is adding k8 support and some cloud-native services/middleware (package mgmt, routing, etc.) to Azure?
  • Back in July of 2016, Microsoft hired a k8 big-wheel (and other, “small wheels,” I’d assume), so they’re obviously into the thing…or at least the thinking behind the think. This leave, once again, Amazon as the last major cloud hold-out on k8.
  • That said, I think Microsoft’s new thing is to like everything that layers on-top, below, or around them. As long as you’re in every deal, you make a lot of money even if you’re not all of every deal. It’s pretty hard, now, of course, to compete with the big clouds.
  • Or, put another way: “Satya is like the Pope Francis of software,” says Alex Polvi, founder and CEO of CoreOS, a company that plays in the same area as Deis. “He took this old institution and made it cool again.”

Misc.

How JPMC is making IT more innovative with PaaS, public and private

wocintech (microsoft) - 154

A good, pretty long overview of JPMorgan Chase’s plans for doing cloud with a PaaS focus. Some highlights.

More than just private-IaaS and DIY-platforms:

Like most large U.S. banks, JPMorgan Chase has had some version of a private cloud for years, with virtualized servers, storage and networks that can be shared in a flexible way throughout the organization.

The bank is upgrading its private cloud to “platform as a service” — in other words, the cloud service will manage the infrastructure (servers, storage, and networks), so that developers don’t have to worry about that stuff.

On the multi-/hybrid-cloud thing:

By the second half of 2017, the bank plans to run proprietary applications on the public cloud. At the same time, it’s building a new, modern internal cloud, code-named Gaia.

While “hybrid-cloud” has been tedious vendor-marketing-drivel over the past ten years, pretty much all of the large organizations I work with at Pivotal have exactly this approach. Public, private, whatever: we want to do it all.

Shifting their emphasis innovation:

“We aren’t looking to decrease the amount of money the firm is spending on technology. We’re looking to change the mix between run-the-bank costs versus innovation investment,” he said. “We’ve got to continue to be really aggressive in reducing the run-the bank costs and do it in a very thoughtful way to maintain the existing technology base in the most efficient way possible.” …Dollars saved by using lower-cost cloud infrastructure and platforms will be reinvested in technology, he said.

On appreciating the scale of “large organizations” that drive their very real challenges with adopting new ways of running IT:

The bank has 43,000 employees in IT; almost 19,000 are developers.

Good luck having the “we have no process by design” process with that setup.

On security, there’s a nice, almost syllogistic re-framing of “cloud security here”:

For years, banks have worried about using the public cloud out of security concerns and fears of what their regulators will say. Ever since the 2013 Target data breach, in which hackers stole card information from 40 million customers by breaking into the computers of an air conditioning company Target used, regulators have strongly urged banks to carefully vet and monitor all third parties, with a specific focus on security.

“We’re spending a significant amount of time to ensure that any applications we choose to run on a public cloud will have the same level of security and controls as those run internally,” Deasy said.

Most notable corporate security breeches over the year have involved on-premises IT (like the HVAC example above). The point is not to make sure that “cloud is as secure as [all that on-prem IT that’s been the source of most security problems in the past], but to make sure that all IT has a rigorous approach to security. “Cloud” isn’t the security problem, doing a shitty job at security is the security problem.

Source: Unexpected Champion of Public Clouds: JPMorgan CIO Dana Deasy, Penny Crosman, American Banker

SUSE to Acquire HPE’s OpenStack, Cloud Foundry Portfolio, Boost Kubernetes Investment, TheNewStack

“We see PaaS as a strategic component of our software-defined infrastructure and application platform strategy,” stated SUSE President of Strategy, Alliances and Marketing Michael Miller, in a note to The New Stack, “and Cloud Foundry as the open source project and technology that brings together the best innovation and industry collaboration. We want to leverage that innovation for the benefit of our customers, and we have a vision for the convergence of CaaS technologies [in SUSE’s case, Containers as a service] like Docker and Kubernetes and PaaS technologies like Cloud Foundry that we think will address the real-world needs of our customers and partners. We will now work with the Cloud Foundry community to develop that vision.”

http://thenewstack.io/suse-add-hpes-openstack-cloud-foundry-portfolio-boost-kubernetes-investment/

Top container challenges: management, monitoring, storage

I keep wanting to write up the recent Cloud Foundry Foundation container survey, mixing it in with other recent container surveys. Yup, “keep wanting to.” Meanwhile, Abby wrote-up a brief overview over in O’Reilly land.

Source: 3 facts about container adoption you don’t know – O’Reilly Media

025: .NET and Beyond 12 Factors with Kevin Hoffman (Pivotal Conversations)

We’ve seen a goodly spate of news in the container space recently which we cover in the episode. In the second half, we talk with Kevin Hoffman about the .NET world, Steel Toe, and his book, Beyond the Twelve-Factor App. A recent survey from the Cloud Foundry Foundation is widening the framing around container management, adding in the use of Platform-as-a-Service into the usual container orchestration mix. The survey also shows some interesting results around adoption, e.g., managing containers in production ends up being more difficult than people predict during evaluations. Also since our last episode, DockerCon brought a bevy of announcements in the container ecosystem which we cover briefly. And highly relevant to our guest, Kevin Hoffman, .NET Core 1.0 was officially released, as open source. In the second half we talk about the recent history of .NET and how it’s being used to create microservices. We also talk about the three extra “factors” Kevin’s book adds to the 12 factor app and typical experiences when migrating to 12 factor apps.

Full show notes: http://pivotal.io/podcast Feeds, archives, etc: https://soundcloud.com/pivotalconversations

Full show notes: pivotal.io/podcast

Download the episode, check it out in iTunes, subscribe to RSS, or check it out in SoundCloud.

Link: IDC: Federal government seeing cloud spending push

“In addition, the government plans to increase PaaS spending from $227.1 million in FY15 to $231.3 million [in FY16].”

We’re still in a phase where categorization causes weird slices of spend like this, but there you have it. More figures on “cloud” spending in the piece.

Source: IDC: Federal government seeing cloud spending push