Cloud-native at Comcast, working with Pivotal – Highlights

I’m doing a podcast with Comcast in a few weeks, so I’ve been going over all their public talks on their cloud-native efforts. They’ve been working with Pivotal since around 2014 and are one of the more impressive customer cases with over a 1,000 applications now on Pivotal Cloud Foundry.
Here are some highlights from the talks I’ve been watching. As always, things I put in square brackets are my own comments, the rest are quotes or summaries of what people said:

August, 2016 – Empowering Devops with Cloud Foundry – Sergey Matochkin, Neville George; Comcast

  • Sergey Matochkin.
  • Slides.
  • (17:00) Every deployment to production took at least 6 weeks, but most commonly around 2 months end-to-end. Which also means you need to plan capacity much in advance.
  • We started to use virtualization and containerization “well, well before Docker existed… it was some success, we had some improvements, but those improvements were marginal.”
  • Traditionally, it’d take at least 4-6 months to setup your dev/test infrastructure. But, luckily, virtualization came along.
  • (9:20) Business drivers… Comcast phone service, set-top boxes get DVRs, VoD, etc. All of these require apps on the backend, so the portfolio of apps starts to grow, and with they way they were before it meant they had to build a new datacenter every six months. Virtualization helped here, of course.
  • Also, virtualization allowed us to put a service layer [think “platform”] on-top of the infrastructure.
  • It’d take 4-6 weeks for testing environment, but now it takes 10-15 minutes in a self-service portal.
  • Demo of using Pivotal Cloud Foundry for much of the automation needed to deploy and scale an application.
  • (~32:00) We used to have things like “order servers” and “make load-balancer changes” and somewhere in the bottom of the backlog was “write some code and do some testing.” [That is, they were focusing on items with low business value, below “the value line,” rather than customer features.]
  • “What Cloud Foundry essentially helped us with was to get all those unnecessary user stories out of our backlog so we can focus on the writing code, on testing, and deploying rather than managing infrastructure.”
  • (33:45) momentum/proof-points:
  • momemtum
  • 9 PCF instances; 900+ developers; 2,000+ active apps “most of which are in “the critical path of our customer experience”; 4,100 application instances; 2,000 requests per second.
  • Lots of Slack/ChatOps usage for monitoring and such.

August 3rd, 2016 – Transforming the monolith at 20M tph – Nick Beenham, Comcast

  • Slides.
  • Existing state:
    • 250m transaction per day.
    • Would take 3 months to get a server useful, from moment of purchasing to using.
    • “Over a 100 services run by development teams.”
    • In functional, silo roles.
  • (3:45) “We knew we had that large, rigid infrastructure. [Pivotal] Cloud Foundry and it’s adoption really enables us to change that to gain the agility, to gain the elasticity at scale.
  • Taking away roles to reduce finger-pointing and all the negative stuff, and unified team, of course.
  • (7:35) Anecdote of Nick going from “ops guy” to writing code and liking coding.
  • (12:18) ESP router that was a small router written in Go to translate SOAP requests as part of a strangler pattern. Decades old SOA layer that they wanted to modernize. But they couldn’t strip it out, would take so long. So, were going to duck-type as SOA, but do REST and micro services underneath. Strangler pattern, etc. This is what the ESP router does marshals and unmarshalls between microservices and SOAP stuff. But new things need to be done in new style.
  • Also, “de-mingling data,” moving off Oracle RAC/GoldenGate for multi-site. Some simpler CRUD services to front the data.
  • (~15:00) Used to take a week+ to deploy the entire stack, but with Pivotal Cloud Foundry it takes minutes. It gives us a great deal of velocity that we’ve never had before. “Sometimes we’ll deploy multiple times an hour.”
  • (17:00) From 1,000’s of lines of bash to deploy out to various WebLogic clusters, which has for the most part moved to Cloud Foundry.
  • Improving production updates: bringing new node up and shutting old node down slowly; canary updates, with a CI test suite, then switching over to a production install.

August 1st, 2016 – James Taylor – The Power of Partnership & Building a Cloud Native Tier-1 Platform

  • @jctbmwi8
  • “Sparrow, Service Activation Platform.”
  • “Helping someone put a smile on their face is one of the greatest gifts we can give each other.”
  • Their VP provides the feedback loop of things to focus on. Right now: reducing technical debt, reducing incidents, increasing velocity, experimentation.
  • (~6:30) “You can’t move forward – innovate – if you don’t have time to try new things.”
  • (~18:35) “If you’re spending time configuring a Docker container, that’s time you’re not spending coding or solving a problem.”
  • (13:51): “At the end of the day, [business] value is what puts money in everyone’s pocket. If our company, Comcast, can’t create something of value, no one’s gonna pay for us…if we can’t create value. So it’s important for us to understand ‘how can you create value?’”
  • (~22:02, starting epic rant!) “Who is our customer and what value do we bring to our customers…”
  • If you’re spending money on support, that’s cutting into your margins. A call coming in costs $8 right off the bat, then more as it takes longer. So you want to figure out preventing customer support problems… which points to understanding your customers more.
  • [A good overview of thinking about “value” in the context of a specific application, their customer activation center, Sparrow.] “If you have a [support] call rate of 30%, you’re probably cutting out all the value… So we try to figure out, how do we prevent calls?” [Very similar to IRS cloud-native story.]
  • “We’ve been holding technical workshops”: Internal training things every month with Pivotal people, leveraging Pivotal knowledge. With our development teams every month: webinar, or on-site visit.
  • Sparrow: 5 junior Java developers… we built it from scratch in parallel while existing teams maintained the platform… we then had to integrate the processes together… figure out decomposing the monolith platforms, etc….then we had to just cut off stuff when it was too much of a hassle.

August 17th, 2016 – Greg Otto SpringOne Platform keynote

  • Slides.
  • X1 boxes – a new release about once a month.
  • Processing 10’s of millions of transactions on this new platform daily on Pivotal Cloud Foundry/new platform.
  • “About a 75% lift in velocity as well as time to market, and the business is really feeling it.”
  • Developer reactions:
  • comcast what customers are saying.png
  • Momentum Stats:
  • comcast key state from otto.png
    • 40 apps to 900 apps, 2015 to 2016
    • 300 AIs to 4,100 AIs, 2015 to 2016
  • All with “zero outbound marketing from my team, this all word of mouth from all those happy developers.”

June 9th, 2016 – Greg Otto CF Summit keynote

  • “Late last year in 2015” – live in production [on Pivotal Cloud Foundry] with business critical systems from our back-office systems on our Cloud Foundry environment.
  • We put Pivotal Cloud Foundry directly in the customer critical path.
  • Applications doing 30,000 event a second on Cloud Foundry.
  • Started in 2014, met with Pivotal.
  • Had sort of thrown all the people into the Pivotal Cloud Foundry pool, they had to do a lot of research and such.
  • But, people were really interested in the ease of working with the platform [the productivity improvements].
  • Successful prototype app 30 days after platform.
  • Idea to feature, before after: “several weeks, at least”/“2-3 days”
  • Time-line and summary:
  • comcast otto summary.png

June, 2016 – Open source at Comcast story

  • Write-up.
  • “If Comcast has a problem to solve, there are three possible approaches: solve it themselves by making an investment in teams and resources; solve it through a commercial vendor that could build a product for them; or work with the open source community.”
  • OpenStack: “In addition to Linux, Comcast is a heavy user of OpenStack. They use a KVM hypervisor, and then a lot of data center orchestration is done through OpenStack for the coordination of storage and networking resources with compute and memory resources. Muehl said that Comcast has roughly a petabyte of memory and around a million virtual CPU cores that they are running under the OpenStack umbrella. As an operator, Comcast does a lot of things around operations, and they use Ansible to deploy and manage OpenStack at scale.”
  • Cloud Foundry: “They also use Cloud Foundry, but according to Muehl that work is in the very early stages at Comcast.”

May 2015 – Running Cloud Foundry at Comcast talk

  • Neville George, Sam Guerrero, Tim Leong, Sergey Matochkin
  • They wanted to make custom URLs.
  • Used Puppet for stuff.
  • (~8:30) Their requirements for a platform:
  • comcast platform requirements.png
  • A lot of emphasis on self-service and the micro services benefits of operating independently, product management wise.
  • They use OpenStack, Docker, and [Pivotal] Cloud Foundry.
  • Pre-provisioning resources for a pool of containers that are ready to go, etc.
  • (~27) a couple applications in production today… we’ll be ramping up quickly.
  • (Either this video or the 2016 one, a few minutes from the end) Q, training mode. A, Sergey: “I can’t say we have a really good training model…. We do brown-bags to have people aware. We focus on 12 factor application model… on overall microservices model, not just to shape application, but also data. Developers need to understand how they [do] applications for PaaS instead of traditional.

Reactions to Cloudera’s IPO, prospects – Notebook

There’s lots of opinions on Cloudera’s IPO today. Here’s some that I’ve collected in my notebook.

Not valued high enough?

Despite the share-price being up 20% at close, some negative commentary focuses on their valuation dropping from Intel’s funding round, e.g., from Brenon at 451:

The chipmaker paid up for the privilege, putting a ‘quadra unicorn’ valuation of $4.1bn on Cloudera. Altogether, Cloudera raised more than $1bn from private market investors, making the $225m raised from public market investors seem almost like lunch money.

And then there’s the small matter of valuation. In its debut, Cloudera is only worth about half of what Intel thought it was worth when it made its bet.

https://twitter.com/alex/status/857992394595119104

The counter-point goes a little something like this (as pointed out by Derrick Harris):

“Much has been made of the huge valuation of that Intel-led round, but that’s all misguided noise,” according to IPO Candy, a website founded by Kris Tuttle, the director of research at Soundview Technology Group. “Intel didn’t make the investment for a financial return so the valuation isn’t relevant.”

Back in 2014, Intel was still smarting from missing the shift to mobile computing and Big Data was a favorite as the next big thing. The Santa Clara chip giant’s bet was placed chasing a strategic return, not so much banking a direct return on investment.

You know, all of this is a little bit of ¯_(ツ)_/¯. As I recall, Facebook’s IPO was all wiggly-woggly. If Cloudera makes a lot of money, gets bought for a lot of money, etc., no one will care to remember, just like with Facebook. Success is the best deodorant.

Their business, finances

Also from 451, earlier this month, a profile of their business:

Cloudera is nearly one-third bigger than Hortonworks, recording $261m in sales in its most recent fiscal year compared with $184m for Hortonworks. Both are growing at roughly 50%.

Since 2008, the company has grown steadily. As of January 31, it reports more than 1,000 customers. However, Cloudera is currently emphasizing and banking its success on what it calls the Global 8,000, which are the largest enterprises worldwide. The company notes that its number of Global 8,000 customers increased from 255 as of January 31, 2015, to 381 as of January 31, 2016, and 495 as of January 31. For the year ended January 31, the Global 8,000 represented 73% of Cloudera’s total revenue, while a further 10% of total sales came from the public sector. The company reports 1,470 fulltime employees as of January 31, a slight increase from its headcount of 1,140 the prior year.

More from Katie Roof at TechCrunch:

Cloudera’s market cap is now about $2.3 billion, significantly less than the $4.1 billion valuation Intel gave in 2014. This increasingly common phenomenon is now nicknamed a “down round IPO.”

In an interview with TechCrunch, CEO Tom Riley insisted that this was not a problem for the company because of the “growth prospects ahead of us.” If it performs well in the stock market, it could ultimately achieve the $4 billion-plus value. Square, which went public in 2015 at half its private market valuation, has since seen its share prices more than double.

(Side-note: comparisons of companies, Square and Cloudera, that have nothing to do with each other except being “tech” – and Square is payment processing, not “pure tech,” at that! – drive me a bit crazy, as listeners know.)

And a quick revenue/spend write-up from her:

Cloudera’s revenue is growing, totaling $261 million for the fiscal year that ended in January. The company brought in $166 million at the same time last year.

Losses were $186.32 million, down from $203 million in the same period the year before.

And, according to Jonathan Vanian: “Cloudera spent $203 million on sales and marketing in its latest fiscal year, up 26% from the previous year.”

TAM

I don’t really follow this space well enough anymore to quickly figure out the TAM: I suspect Cloudera operates in several data and BI related ones.

Cloudera isn’t only Hadoop, but 451 put the Hadoop market at $1.3b in 2016, growing to $4.4b in 2020, with a CAGR of 38.3% between 2015 & 2020.

If you throw data warehousing, BI, analytics, and an injection of the mega-databases TAM together, you get a really big TAM, anyhow. Keep in mind though that one of the traps of (definitionally orthodox) disruptors in this space is lowering the TAM of their respective markets, a la Red Hat in operating systems. I don’t get the sense that Cloudera is on that game plan, but others in the market might be.

Buyers’ plans & needs

With respect to what people would do with Cloudera and others in this space (including Pivotal), here’s a good ranking of the information infrastructure priorities Gartner recently found in enterprises:

info plans survey

Also of public/private cloud interest from the summary of that survey: “Based on survey responses, plans for on-premises deployments for production uses of data will drop from today’s 45% to 14% in 2018.”

Looking forward

People in the tech industry care a great deal about IPO’s like this. We’re all curious what The Market’s read on valuation of enterprise IT business models is for our own benefit, and just a general sense of the health of the sector. There’s also usually people you know at the company, so “yay” for people you know.

One day isn’t long enough to tell anything, though, cf., in a completely different space, that Facebook debut weirdness. People got all excited about Cisco buying AppDynamics because that seemed to show some “healthy” signs that money valued this kind of software/SaaS.

At any rate, people still seem to love the Big Data and such. From Cloudera’s CEO, Tom Reilly: “We’re competing with IBM and Watson, so our customers seeing the strength of our finances allows us to do more.” Think of all the free marketing!

And, Mike Olson (original CEO) adds:

The ensuing years have been remarkable. Our company has grown with the market. The original technology has morphed almost beyond recognition, adding real-time, SQL, streaming, machine learning capabilities and more. That’s driven adoption among some of the very biggest enterprises on the planet. They’re running a huge variety of applications, solving a wide variety of critical business problems.

Our early bet has proven correct: Data is changing the world. In applications like fraud detection and prevention, securing networks against cyberattacks and optimizing fleet performance in logistics and trucking, we’re delivering value. We’re helping to address big social challenges, improving patient outcomes in healthcare and helping law enforcement find and shut down human trafficking networks.

Against that background, an IPO takes on a more appropriate scale. We started Cloudera because we believe that data makes things that are impossible today, possible tomorrow. There’s more data coming, and there are plenty of impossible things to work on. Our journey is only well begun.

I admittedly don’t know Cloudera’s business model too well, but my sense is that they align well with the “have something to sell” model that many open source companies in the enterprise space forget to put in place.

Scaling DevOps in large organizations – My April Register column

apololypse_now_smell_of_napalm_in_the_morning

My The Register column this month is on scaling DevOps/cloud-native teams to the entire organization. It’s easy to build one team that does software in a new and exciting way, but how can you move to two teams, five teams, and then 100’s? It goes over the amalgamation of a few case studies and plenty of over-the-top gonzo analogies, per usual.

Check it out, and check out past ones if you’re curious for more.

Ode to Airports

An airport is a time pause. It’s an excuse to not stress or try. You’re trapped in the system and will eventually get there. You can’t leave or you’ll have to re-humiliate yourself through security. Airports are even powerful enough to make you cancel meetings if your flight is late, canceled…or you pretend it is. Your wedding could be delayed because of the airport and no one would really fault you.

Everyone is transiting, coming and going, and while the entry fee might exclude the very poor (and the super rich fly their own), you see everyone.

At a major hub, you’ll see people from all over: the guy with the “Ragin’ Cajun” hat, domestic and international grandmas, the harried big city lawyer, the dad-jeans set, and the local staff. People dress in all manners of business-business or super casual for comfort.

The mix of experienced and novice travels creates a crackly dynamic, paired with either overly friendly or direct gate agents. While some can escape to airline lounges, even those environments are little different than the actual terminal: you just get much friendly staff and free drinks and peanuts.

Airports can be calming if you look at them as escapes and the sort of delightful, enforced boredom that I understand meditation to be.

They can be toxic if you stress out about delays, lines, other people, overhead bin space, and how flight delays effect your plans outside the airport. And they can be distracting like an opium den if you let their peaceful hum shut-out your real life.

Don’t ruin your time at the airport. If you let it, it’ll make sure you get back out right where you wanted to go.

Are tech H1-B visas actually that big of a deal? How we even evaluate the question?


25720976280_daf8a3d827_k_d

Over the decades, the number of H-1B workers allowed into the US each year has grown. With the 1998 update, the visa cap lifted to 115,000. In 2000, the limit was boosted again, this time up to 195,000. That year, the law was also tweaked so that renewals no longer counted toward the cap. In 2004, the cap was reset to 65,000, but an exemption was added for 20,000 students graduating from US institutions with master’s degrees. Exemptions were also added for workers affiliated with academic institutions, which can include schools and teaching hospitals. According to Ron Hira, a professor of Public Policy at Howard University who has studied the H-1B issue and testified about it before the Senate, the actual number of visas handed out each year has been around 135,000 over the last five years. Link

There’s a good rant on the relative importance of all of this in last week’s Political Gabfest. While us “on-shore” workers in the tech industry may see that 135,000 as a threat to our cashflow, it’s a drop in the bucket of employment in America. As Adam Davidson argues well, therefore, worrying about H1-B visas should be pretty low on the list of how to setup up more people with good jobs:

The question of H-1B visas has rhetorical importance far beyond its actual economic relevance. The unemployment rate for computer and mathematical occupations is, currently, 2.1 per cent. This is what economists consider full employment, meaning that pretty much everyone who wants a job has a job or is in a brief hiatus between positions. The number of jobs in those fields is growing fast—by about twelve per cent a year—and the number of qualified workers is not growing enough to catch up. In short, the plight of computer professionals is on few people’s list of urgent concerns…. According to the Bureau of Labor Statistics, ten thousand computer professionals start a new job every working day. In this context, the eighty-five thousand foreigners given H-1B visas each year represent little more than statistical noise.

He goes off an a political jag after this saying that the H1-B discussion is a proxy for “fear of brown people,” which certainly has appeal to leftist people like myself. There’s a business question here, too, though: are H1-B visas a good idea and why? Are they ethical and effective?

What types of jobs?

Also, some interesting analysis of the types of jobs H1-B visas are used for. Mostly for jobs at outsourcing firms:

But it’s how H1-B visas are being used by applicants that’s really changed. Data from the 2016 batch of H-1B petitions show that the top 10 sponsors of H-1B visa workers in the US are all corporations with large outsourcing businesses: Indian companies like Infosys, Tata, and Wipro, which pioneered the business, and US-based firms like IBM, Accenture, and Cognizant, which saw the success of the Indian contractors and began offing their own competing outsourcing programs. Those 10 firms have more workers currently employed through the program than the next 90 companies combined, a group that includes all of America’s largest tech companies and banks.

So, the discussion about H1-B visas in tech ii, by bulk, about the 60,000+ jobs in IT outsourcing. This is in addition to the estimated 1.7m off-shore jobs in outsourcing already.

In theory, most of these are “lower value” jobs where you’re more operating IT (help-desks, managing the daily operations of enterprise applications) rather than creating it (like programming). Anecdotally, there’s still programming running around in there, esp. when it comes to modernizing applications. The going theory is that you can’t just slot in workers on higher-value IT work like writing custom software.

How do you think about all this?

There’s an odd ethical vs. business-sense argument scurrying about as well that I’ve never seen addressed. One, you’d seem to be happy that the H1-B visa worker was getting work. By nature of accepting the job and up-rooting themselves, it must be good for them: or, at least, better than other alternatives. Also, if it’s actually cheaper to get the same services/output from an H1-B visa worker, why would you pay more for “native” worker? On the other hand, it’s equally confusing to figure out what companies “owe” workers that they’re firing in favor of the H1-B visa workers.

Tech companies like to skirt all that by talking about “we have to hire from a global pool,” which is fine if you’re hiring for an individual with unique skills. However, the divide between outsourcing firms and tech companies suggests that the bulk of H1-B visa hires in tech are not for all the super unique, AI experts that may not live on-shore. Then again, it’s insulting to even think that: why do I value one set of people over another in any context?

Businesses say they’re not satisfied

However we figure out talking about this, it’s clear from surveys that companies are dodgy on the value of outsourcing. As I summarizing some HfS work recently:

Outsourcers too often do exactly what the contract (from five to ten years ago) says instead of helping you innovate and keep the business growing. Itʼs little wonder that in a recent study, more than 75% of senior executives said they want to replace their legacy outsourcers because those providers are so unwilling to change to new models.


If we take Adam Davidson’s perspective, it’s not really even a problem worth thinking about (versus all the other hair-on-fire issue we have). However, when it comes to outsourcing (which I’ve shifted to because so many H1-B visa workers end up at outsourcing firms), it’s clear that we could be doing much better.

Spanning goes private, what might happen next?

Long ago, Spanning Sync was the only viable way to synchronize your GMail calendar and contacts with the (then) OS X iCal and Address Book. It was great! I also know one of the original founders, Charlie Wood, and we’d talk from time to time about the growing company. At some point, it became a Google Apps (now “G Suit”) back-up service that had a clever value prop: cloud storage, sure, but it’s not redundant you know, you gotta do the basics.

Anyhow, I always kept a close eye on the company. It was a little odd to see EMC buy them back in 2014: as VMware demonstrated with their dropbox competition products years ago, Apple is pretty goofy here, and even Google has demonstrated over the year, large software companies are pretty and at long-term plays for individual software; Microsoft is of course an exception with Office and sort of proves the rule.

We’ll see what Insight Venture Partners does with them. I’m guessing if you just left Spanning alone, more or less, it’d turn into a cash machine at some point. That said, I don’t think Dropbox and Box are exactly profitable. Here’s Box’s last four financial years:

…but it seems like a back-up service could controls costs better and do a lot less marketing: Box and Dropbox have been acquiring companies and re-positioning themselves as they go from more than just to cloud storage to something like “sort of Office, but not really, but maybe – or like Trello… er… let’s acquire another company and go to a conference where we have wooden floors and free espresso in the booth and think about this at next year’s company retreat in Italy.” (I KID! I KID!)

Spanning Momentum

Here’s some Spanning momentum from one of the write-ups:

Spanning has seen 70 percent year-over-year revenue growth and more than 7,000 customers, according to a press release. It restored around 18 million items for customers in 2016, and expects to continue growth with its global data center expansion, and distribution agreements with major channel partners.

A wet-finger-in-the-wind business case

It’s hard to quickly find pricing for Spanning on their page (smells like enterprise software!), but a few searches, particularly from Spiceworks, says it’s like $35 a month.

There’s certainly discounts on some of those customers, but let’s say the revenue would be a max of $2,940,000 annual to something like $1.5m on the low-end if you do all sorts of discounting on clusters of users.

Now, 70% y/y growth is pretty impressive, but not too insane for a relativly new offering. Let’s say they do that two more years and then it goes down to like 30 or 40% for any length of out years we care about.

Then, let’s just take a swag at storage costs. Who knows if they use S3, but let’s assume they can get down to similar pricing, we’ll take S3’s mid-tier: $0.0125/GB/month. My work Google Drive says it’s 22 GB, but I save a lot more stuff than most people do. Let’s just go with 20GB as an average. Then let’s assume you at least duplicate it, so you’re paying for 40 GB a month (across two cloud zones), which is $6/year. (Let’s ignore networking transfer charges – adding that in is left as a exercise for the reader!)

Then you need all the meat-sacks. You could probably get by with 6 to 12 product staff (programmers, product manager – you probably outsource design at this point as needed).

You need the CEO, HR, CFO, and probably 1-2 people to work for them (6 people max); you could probably cut out HR depending on how Insight likes to run HR (outsourced or pooled across companies). Maybe the CFO, but probably not.

I’m no enterprise SaaS business expert, but I’m guessing it’s marketing and sales heavy, so:

Then you need probably 2-3 people in marketing (if you were slick, you could outsource a lot of this, esp. for something as easy to understand as “backup”): 5-10 face-to-face enterprise hustlers, and let’s say a team of 5 “inside/web” sales people who send all those annoying “Re: catching-up. I see you read out white paper on BACKUP. Would you like to talk more? Are you the right person at your organization?” emails. So, max 18.

That’s around 36 people, which seems really low to me. But, if you were, I don’t know, a private equity firm, you’d probably think that was OK, if not a little heavy for a company that basically just copies files from one place to the other (yes, I’m being MBA-fatuous).

Without getting a spreadsheet to do some clustering, doing salary cost across such a diverse set is hard. Many of them are in Austin (I assume, still), so let’s just of with $150,00 all-in per head (I’m sure the admin staff and your “strategic account” sales people get paid well plus extra comp, and the more senior tech staff get paid more). So, that’s something like $5,400,000 in people expenses. Then there’s going to conferences, probably a large ad budget, that nice office they have in downtown Austin (which I think is an EMC office, so they’ll get the boot?) which means buying a lot of organic beef-jerky and craft beer etc., then there’s flying those 5-10 enterprise hustlers around and their $70-100 a day per diems, plus wining and dining. Let’s just trow in another million and go to $6.5m.

So, with some mumbo jumbo business casing (I grow revenue by 70% for two years, then level it off to 30% for the last two years; I grow staff up to 60 people max), you have something like this:

Screenshot 2017-04-23 09.16.10

Those storage costs look insanely off. And from their press release, they claim to have actual data-centers (probably co-lo’d racks that are, at best, caged for compliance reasons, far from “having data centers”), which sounds like building your own, which might actually be as cheap, or slightly higher.

Who knows. Cloud storage is insanely cheap, so maybe that figure isn’t so bonkers. Of course, you need networking transfer chargers, etc. So, double, even quadruple the cost if you care too: still “nothing,” relative to the other numbers.

With this kind of Sunday morning, armchair analysis, there’s no end of flaws. Like I should have found the comparable costs, growth, the TAM, and staffing for Box, BackBlaze, etc., and even made sure I actually understand Spanning’s business model, but: ¯_(ツ)_/¯

Over years, that’s a pretty small gap to close to be profitably, and there’s a lot of things to play with in the spreadsheet (can we fire most all the sales and marketing people and go pure channel, hiring up a biz dev team of 2-3 people to get 5 or so key channel partners?).

It’s probably even easier to bundle up the company for sale to another large company after a few years. Someone like Microsoft or Salesforce might even want them to add that functionality to their own products, or any company that’s concerned about filling in it’s “enterprise SaaS” strategy gaps.

I’ve always like Spanning (RIP Sync). I hope it works out well!

The news from Docker-land, plus, the money being fought over – Notebook

With DockerCon this week, there’s no end of Docker quotables and items. Here’s my collection

General momentum

Once landed in an account, Docker usage grows their CEO says:

There has also been expansion within customers, with organizations that start with Docker expanding their usage on average by five times within six months

Way back in 2015, the (now annual?) DataDog study of Docker usage among their customers said that 2/3 of companies that try Docker adopt it. Which is all to say: once it gets in, it spreads.

Moby

A toolkit for putting together docker stacks:

In essence, Moby is the build system that creates Docker Community Edition, which is akin to Fedora, and Docker Enterprise is derived from Moby and is akin to Red Hat Enterprise Linux. Link

People got all freaked out. I’d even say “freaked the fuck out.” Competitors, of course, gloated, if only in silence. Criticism of handling the announcement aside (ideally, you wouldn’t like to kick up a stink), I feel like it was more like a tempest in a teapot.

Docker momentum/penetration and types of applications/workloads

Global 2000 customers have somewhere on the order of thousands to tens of thousands of applications, and across these major firms, less than 5 percent of the applications have been containerized so far. While somewhere between 5 percent and 10 percent of the applications that are being containerized are net-new, microservices-style applications that everyone is talking about all the time, the other 90 percent to 95 percent are just lifting and shifting legacy applications from bare metal or virtual machines to containers. Link

VMware threat…or just legacy gobbling?

Docker bounces back and forth between “replacement for VMware” and “a different thing, so don’t worry about VMware.” In this round of Docker news, there’s been some strong pull towards the “replacement for VMware” camp. To be fair, it’s more like doing both:

In general, says Johnston, customers who move from bare metal or VMs to Docker containers can provision, scale, and deploy applications up to 75 percent faster, and those moving from bare metal to containers can save 50 percent on compute and those who are moving from VMs will save around 25 percent. Link

This might also come from the obvious move to start gobbling up legacy (more accurately “existing”) applications. Here, Docker had two customer reference:

Northern Trust, a leading international financial services company, experienced  deployment times that were 4X faster and noted a 2X improvement in infrastructure utilization

And, Microsoft IT:

Microsoft is not only a partner in this program; their IT organization is also a beta customer.  Microsoft IT increased app density 4X with zero impact to performance and were able to reduce their infrastructure costs by a third.

There was also a story of Visa using Docker:

Kocherlakota said Visa is aiming to move as many workloads at it can to the container model to help improve overall efficiency.

See more on this legacy migration stuff and the program with Avanade, Cisco, HP, and Microsoft from Docker’s Scott Johnson.

Major vendors

Other tech companies are often cautious about working with Docker. They’re not really certain about how it helps or threatens their position in the IT stack and, therefore, their ability to sell higher profit margin products and services. No one wants to become the x86 manufacturer of the cloud (read: low margin, commodity).

I’ve noticed this cautiousness slightly melting as more and more vendors are at least putting their stuff in Docker images and, on the public cloud front, supporting the use of Docker. My company, Pivotal, ingests Docker images.

A brief whack at why Microsoft cares, from Christopher Tozzi:

Although there remains work to do to get Docker on Windows ready for prime time, the platform will be important in helping Windows Server stay as nimble as Linux environments in hosting the workloads of the future…. Microsoft’s interest in Docker may seem strange. Microsoft already offers traditional virtual machine products, most notably Hyper-V. In some respects, Docker containers compete with virtual machine platforms…. But that’s not necessarily the case. Depending on how they’re used, containers can complement virtual machines, rather than replace them. If you use virtual machines to host the environment in which Docker runs, your Docker environment becomes more scalable and portable than it would be if it ran on bare metal. That’s likely the type of use case Microsoft envisions for containers on Windows.

More from Nick Martin on Microsoft and Docker.

Oracle bundling middleware in Docker containers:

Oracle becomes the latest enterprise IT vendor to jump on the Docker container bandwagon as it seeks to expand its reach in the public cloud market. Among the container-based application, middleware and development tools made available on the container platform are Oracle’s MySQL database and its WebLogic server. Those tools are in addition to the more than 100 images of Oracle products already available on Docker Hub, its cloud-based image registry.

So, what’s going on here? Staking a claim on The New Stack

I’m often asked to explain all the various cloud stacks, to help Pivotal buyers sort out what CaaS, PaaS, cloud-native, and “cloud strategy” means. They’re trying to figure out their planning for building out new IT, for “doing DevOps.” It’s a mess out there w/r/t to figuring all this out if you’re not a vendor or analyst who’s steeped in this shoggoth every day.

In all the Docker, container, and cloud-native wars, the revenue battle for vendors is mostly about two things:

  1. The pool of money in simply migrating the VMware workload to a new, more efficient layer, hence the ongoing attention to “the VMware threat” that Docker poses). I’m not sure how big this market is because, as a disruptive shift (cf. Linux vs. UNIX vs. Windows vs. z) part of it is reducing the overall spend through lower prices and more efficient usage. But, the existing virtualization market is best described as “fucking huge.”
  2. Fighting over who “owns” (and therefore collects the most profit from) the stack that companies are using to build and run their software. By my estimate, this is something like around a $20-25bn market in the future. You can see a Spanish Civil War like precursor going on in the Java application server market; it’s spreading to a “World War” with respect to all custom software stacks.

On that second point, here’s my latest attempt to describe how things are shaking out category/definition wise:

Of all the SPI cloud categories, PaaS is the most problematic place as all us vendors hate the PaaS term and are trying to re-define what it means. I would break PaaS into two categories currently: (1.) container orchestration, and, (2.) cloud platform.

Container orchestration takes an IaaS and manages the installation and configuration of container images on your new cloud. By “images” here, I mean that you’ve chosen to put your software (probably custom written software, not packaged software) into containers (or the delegated way we do it with buildpacks in CF), specified how all the different nodes are wired together with all the ACLs and configuration, and then given it over to the orchestration software to deploy those containers, set the configuration, and do the ongoing health-checks/remediation.

Ideally, the orchestration platform should also have “day 2” tools to help you monitor and manager (“fix”) problems that happen in production. I assume things like kubernetes, the Docker/Moby constellation of things, Mesosphere, etc. fit here.

People are obsessed with container orchestration now and it’s pretty much all anyone talks about. I think all this is what’s becoming known as “CaaS” – Containers as a Service.

(On this next section, I’m extremely monetarily biased, of course:) A cloud platform either has or depends on an orchestration layer, but adds in integrated middle-ware, ALM tools (from basics like “cf push”, and an overall programming and deployment model with all the tools and enforcements. Heroku is the classic example here in public cloud, and now Cloud Foundry (CF) has taken over this model in public and private cloud, the second (it seems) where most of the usage and money is, at least in the enterprise space. I’d argue, that CF is the enterprise market-leader (by revenue at least, but increasingly penetration in the F500 – while Pivotal has impressive numbers, throw in the other CF distros and it’s even larger, no doubt); at the very least, “the highest growth and in enterprise production usage.” That all depends how you slice it, and of course my slicing favors me.

A cloud platform “pulls together” everything into a fully working “cloud” that deploy and provisions the servers, builds/maintains/deploys the containers, takes care of your networking configuration and concerns (inc. firewalls, etc.), and configs/manages all the middleware needed (e.g. “I want a database” means you just ask for it, instead of having to configure it and make container images of it and specify how it all works together).

The end goal of a cloud platform is the original end-goal of a PaaS: developers don’t have to “setup” any of the infrastructure or, really, middleware (databases, queues, etc.) that they use: they just write the “business logic” of their applications.

All this standardization is technically “restrictive” (developers can’t just install anything they download off the Internet, it has to be integrated into the platform). This is why we often call this model “opinionated,” but it follows the same contract/promises model that Google SREs follow: we promise we can support your applications in production if you use only the things we support, otherwise it’s all on you.

However, the benefit of such opinions is a huge jump in productivity as we see at all our customers: one Pivotal customer manages 1,000+ applications (all angles toward very frequent, DevOps-style releases for fast feedback loops and all that small batch stuff) with just 4 PCF operations staff, etc.

Our DIY white paper makes the case that snow-flaking this all out is a bad idea. At the very least, if you build your own platform, you should try to just have one used organization wide.

In comparing CaaS and cloud platform, the key distinction to me is that a cloud platform bundles and integrates together all your middleware and “services” frameworks. For example, if you want to do microservices with all the bulk-heads and such, that functionality should be built into the cloud platform – you should have to go read-up how to set most of that up. PCF, of course, has Spring Cloud and more for that. All of the systems management tools (thing used in production to detect and fix problems) should also be built in, or the cloud platform should be instrumented so deeply that third party tools can do the managing as well.

Now, these two categories are likely to converge, and then the discussion will just be which cloud platforms are more featureful and better. It’ll be like battling Java application servers.

I haven’t made one of my own “burger” stacks of all this in a long time, but I think (again, highly biased) the ones we use for PCF are pretty good:

More

In case you don’t know, working at Pivotal, I obviously have a stake in how all this turns out, so I’m biased on multiple angles of the above whether I want to be or not. 

Picking off the slow-movers: $15bn for tech PE now sloshing around at Silverlake, more to come

Silver Lake plans to announce on Tuesday that it has closed its fifth buyout fund at $15 billion, one of the biggest ever dedicated to technology deals. That exceeds the $12.5 billion fund-raising target that the firm had previously aimed for and brings the firm’s total assets and committed capital to about $39 billion.

They seem to get good returns:

Silver Lake’s fourth fund, with $10.5 billion under management, currently boasts returns of nearly 31 percent, according to the data provider PitchBook.

Meanwhile, as Dan Primack mentioned, you can expect $100bn from SoftBank.

What this means is that more older, lower growth software companies will be taken private. More than likely, their day-to-day operations will be optimized to get their cash-flow fixed up and increase profits. These companies can then act as cash machines and find some exit after the PE owners “fix” management and operations problems at the company.

That usually means consolidation, which results in firing people, but also fixing stubborn “frozen middle” problems that have preventing each product line from evolving and getting a better ongoing product/market for, meaning: being something that customers want to use and keep buying. There can also just be a lot of “bloat” in older product lines, esp. when it comes to effective product management, marketing, and developers following old, slow, but comfortable processes.

And, sometimes, as you see at IBM, you just have to shut down old business in favor of building new ones. This means a top-line revenue hit, which means slowing or killing quarterly growth. As IBM has been demonstrating for 20 quarters, when you’re public, ain’t nobody got time for that. In theory, when private, you can choose that option.

As Brenon at 451 has noted, going private deals like these are growing much more than “corporate” acquisitions (like when Microsoft, Cisco, IBM, etc. buy a company to integrate into their product portfolio rather than optimize the company as discussed here). It doesn’t always work, but that “nearly 30%” return indicates that it works more than enough.

Link

With no competition, government websites often have no incentive to be good

In contrast to agile, private-sector companies, the public sector does not face any pressure from competition. When it comes time to renew your license, there is only one place for you to do that: and, unfortunately for Americans, that’s the DMV. With no competitive forces, government agencies do not have to innovate or take bold risks when it comes to digital.

And, as ever, being smart about using updated tools and new methods yield huge productivity results:

While running technology for Obama’s WhiteHouse.gov, open-source solutions enabled our team to deliver projects on budget and up to 75% faster than alternative proprietary-software options. More than anything, open-source technology allows governments to utilize a large ecosystem of developers, which enhances innovation and collaboration while driving down the cost to taxpayers.

While open source has different cost dynamic, I’d suggest that simply switching to new software to get the latest features and mindset that the software imbues gives you a boost. Open source, when picked well, will come with that community and an ongoing focus on updates: older software that has long been abandoned by the community and vendors will stall out and become stale, open or not.

With most large organizations, and especially government, simply doing something will give you a huge boost in all your KPIs in the short term. Picking a thriving, vibrant stack is critical for long term success. Otherwise, five or ten years from now, whether using open or closed source, you’ll end up in the same spot, dead in the water and sucking.

Link