My recap of the agile leadership track at SpringOne 2020

We did a recap/favorites for talks at SpringOne Platform this week. Here’s my section.

In addition to praise for Cora and Maria’s talk on service meshes, I called out these talks:

  • Air France-KLM talk.
  • TD Ameritrade on ROI
  • Jana Werner, Tesco Bank – “Tesco Bank has embarked on a digital transformation journey, and at the heart of it lies a shift of culture and the adoption of modern product development practices. What could go wrong? Everything! Culture, leadership, bureaucracy, route to production, you name it. Yet, with the help of VMware Pivotal Labs, we delivered an amazing product during a time of great need for our customers: a digital gift card allowing volunteers to shop for self-isolating and vulnerable customers, while creating our very first cross-functional Product Team, now scaling out rapidly with enthusiastic people and full exec sponsorship. If you’d like to learn what it takes, what not to do, and how to fast-track your digital transformation, don’t miss this talk.”
  • BT’s talks

Perhaps I had other mentions too, watch the video to see!

Source: VMwareTanzu – Twitch

BT SpringOne Talk: developer first and a willingness to learn and change as needed – Notebook

BT SpringOne Talk: developer first and a willingness to learn and change as needed

A keynote given by Rajesh Premchandran, BT

BT wants to get better at how their do business, through software. Their strategies are, of course, operational excellence, but also getting software that improves the customer’s experience. This means they need to simplify how business is done, which they did by transforming the way they do software: IT underpins this corporate transformation, Rajesh says.

His approach to telling this story is to talk about how BT questioned some initial assumptions. On some of their initial plans, they pivoted (adapted) when they discovered new needs and realitis. On others, they persisted when they really needed the change. For example, at first, they thought that having a PaaS would be good enough for all development and app needs.

A pivot example: they discovered that more fine grained control was needed for some apps (esp. legacy ones), so they added in kuberntes. Here. PCF/TAS and TKG.

A persist example: security had to move from security by static IP. Because they were moving to AWS and microservices, things needed to be more dynamic. There was no way to achieve their tech-stack transformation otherwise. So, they persisted.

The third area he describes is their approach to removing ops toil from application developer’s agenda. As ever, they want apps developers to focus on…apps! And have time to explore and innovate making the apps better to deliver on customer service goals. (You see several example from Comcast on this with their new TV apps and improving customer service with ChatBots, and even in-home wifi coverage calibration, etc.) He also comments on a discovery, or validation of a predictable state, in my rewording: developers aren’t good at production ops and don’t really realize all the new tasks they’ll need to do – responisbilities they have! – in a DevOps model. So, they had pivot to training them and putting in place tech stuff to make it work better for them.

He has some fantastic framing: "instead of letting [developers] grapple with operating model choices, we adopted a more human-centric approach to educating teams."

He touches briefly on some portfolio modernization strategy. The benefits of doing, I think, were that they spent their time wisely, modernizing apps that were feasible and valuable to modernize instead of all of them. I could use more detail here, but I think the point is made – I mean it’s a five or six minute talk, so it’s fine. For a lot more, check out the ever excellent Rohit explaining this kind of portfolio app modernization analysis.

There’s another, long BT talk that I haven’t fully dissected yet.

Raw(er) notes

  • BT business – Fixed line, mobile TV, and networking.
  • Goals:
    • Lead in converged activity by simplifying business.
    • Build scalable platforms for growth.
    • Creating leaner business models.
    • IT underpins this corporate transformation.
  • "The challenges arise from our ways of working, process, account policies, security postures, and even we inventory software assests."
  • Three challenges and how they worked with them:
    1. Strategy.
      • A PaaS, PCF/TAS with Spring – time to deploy from 2 months to 2 minutes. Also, microservices.
      • But, people wanted kubernetes for finer grain control, and needs for non/fuzzy cloud-native apps. Used TKG for this.
      • Listening to devteam
    2. Security.
      • Moved from static IP to static IP, public IP addresses.
      • AWS networking needs, i.e., elastic IPs and REST endpoints.
    3. People.
      • "They also were not fully aware of the [new] shared responsibilities of managing infrastructure as code, or how their roles changed when adopting the cloud."
      • E.g., with a more DevOps approach, "their responsibilities did not end with a cf push."
      • Enterprise architects looked through portfolio to bucket apps into legacy, strategic, SaaS, PaaS, and IaaS to plan out the way of working, priorities, app modernization tasks.
      • Drives concensous on "application treatment" and how to educate teams on their roles and new skills.
      • Also, this eliminates scope overlap in our budgets and app modernization plans, allowing the PaaS team to focus on executation, [instead of work that wasn’t applicable to the various apps strategies and bucketing].

InfoWorld interview

Also, an excellent InfoWorld interview with Rajesh where he talks about several of these points, even more stridently.

Some highlights:

  • Rajesh: "What really happens is you’ve got to tease out what is containerizable"
  • "To overcome this challenge, BT has established a platform team, dedicated to helping application teams identify these containerizable elements and find the best environment in which to host them, be it in the public cloud or on a platform-as-a-service (PaaS)."
  • Rajesh: "You have to handhold them, otherwise they will take the biggest unit they can handle, put that into a Docker container and then lo and behold you’ve got the same monster on better infrastructure — that doesn’t solve your business problem."
  • "This is a constant tussle," he admits, "where people want Kubernetes by default, but I think you’ve got to be a bit more prescriptive in the beginning to developers and then as you see them maturing, have them make those independent choices."
  • "the next task is to scale it out via documentation and word of mouth buzz, both for in-house engineers and with external partners."
  • Rajesh: "If you look at how standards are democratized, you’ve got enterprise architecture that looks at the overall architecture standards and policies, then you have solution architects, who are a level down and are specific to a product, and finally we have distinguished engineers — we call them expert engineers — who are also key influencers for other developers, who look up to them."

🗂 Link: Waste Management dumps legacy processes, drives digital change | CIO

We saw a spike of over 70% points for our new monthly bill-pay option. In the past, we’ve said that monthly billing is not convenient for us, but our customers told us that’s what they want. When we gave it to them, they rewarded us with an auto bill-pay rate that spiked, which is important because autopay is a leading indicator of how long a customer will stay with us. We saw a 40% jump in ecommerce revenue almost overnight. We are now levering those learnings in our commercial markets.

Source: Waste Management dumps legacy processes, drives digital change | CIO

🗂 Link: Silicon Valley software techniques modernize 75-year-old plant

Raytheon Systems Engineer Sam Sauers and her team spearheaded one of the latest DevOps transformations on the program, introducing Silicon Valley-like processes like paired programming and pipeline development to help the Air Soldier team rapidly develop the technology.

“We’re using commercial software best practices, including Agile and DevOps, to get new capabilities in days instead of years,” said Sauers. “We’ve also been implementing user-centered design: getting ahead of the users and figuring out the next thing they’re going to need. We then develop toward that rather than getting something out there and getting feedback that it wasn’t what they wanted.”

Source: Silicon Valley software techniques modernize 75-year-old plant

Link: How Booking.com A/B Tests Ten Novenonagintillion Versions of its Site

“According to research by Evercore Group L.L.C., Booking.com’s “testing drives conversions across the whole platform at 2–3 times the industry average.” That means massive increases to their revenue and bottom line.”

How Booking.com A/B Tests Ten Novenonagintillion Versions of its Site
https://blog.usejournal.com/how-booking-com-a-b-tests-ten-novenonagintillion-versions-of-its-site-25fc3a9e875b
via Instapaper

Source: How Booking.com A/B Tests Ten Novenonagintillion Versions of its Site

Link: AT&T’s ‘Public-Cloud First’ Proclamation a Stake in the Ground

For AT&T to now start the process of adopting the public cloud for what are admittedly “non-network applications” is a big move. It shows that even the most stodgy industry verticals are on board with moving to the public cloud. This will provide a significant new revenue stream for those cloud providers but at the same time allow for greater scale that could drive down pricing models.

Source: AT&T’s ‘Public-Cloud First’ Proclamation a Stake in the Ground

Link: Camera scanning vehicles result in big jump in parking fines

Before Amsterdam started using scanning vehicles in 2013, the Dutch capital issued 18.5 million euros in parking fines. Last year Amsterdam issued nearly 30 million euros in parking fines, an increase of 61 percent. Before using scanning vehicles, Rotterdam issued 177,895 parking fines in 2014. In 2016 that increased by 86 percent to 330,326, before dropping by 6 percent to 310,684 in 2017. AD did not receive more recent figures from Rotterdam.

Delft saw a 26 percent increase in paring fines, Utrecht saw an increase of 17.5 percent. In The Hague, the parking fines increased by half. Tilburg told the newspaper that its issued fines tripled since it started using scanning vehicles.

Source: Camera scanning vehicles result in big jump in parking fines

Link: Comic Relief switched from multi-cloud to serverless with AWS and saw a 93% cost reduction

As a team, going serverless has given us a lot more velocity, we can rapidly release, we can test the same infrastructure we’re deploying in production, in a pull request environment, in a staging environment, we can rapidly retest ideas- and every developer can do that because we’re using Lambda to load test, so the power it gives you as a developer and engineering team is pretty amazing.

Source: Comic Relief switched from multi-cloud to serverless with AWS and saw a 93% cost reduction

Link: Users forge ahead with Cloud Foundry-Kubernetes integration

“There are a million solutions out there to your technical problems, but what we wanted was to solve the people and process problems,”

And:

“It depends on Pivotal. If they add a common pattern in the future for deployment with Istio and Envoy through a cluster and platform-agnostic service mesh, then, yes, we will combine them,” said another senior engineer at the carrier.

Source: Users forge ahead with Cloud Foundry-Kubernetes integration

Link: Standard Bank contracts with AWS for mass migration to the cloud

The bank has selected AWS as its preferred cloud provider with the intention of porting its production workloads, including its customer facing platforms and strategic core banking applications to the cloud.

From what I can tell talking with banks, they’re over that 2010 thing of “public cloud isn’t secure enough.” Now it’s a scramble to move their shit up there.

Source: Standard Bank contracts with AWS for mass migration to the cloud

Why Wells Fargo Wants to ‘Repave’ Its Platform Every Day

Wells Fargo, explains how the company is combating advanced persistent threats, as well as an onslaught of CVEs, by repaving its entire platform multiple times per week — with a goal of doing so every day by the end of 2019.

That is, they rebuild production three times a week, probably now more.

Source: Why Wells Fargo Wants to ‘Repave’ Its Platform Every Day

Link: How Air France – KLM designed its PaaS Cloud Foundry

“it was not a question of replacing our experts but of increasing the skills of the group’s internal teams,” says Thierry Morcq.

(Translated with Google Translate.)
Original source: How Air France – KLM designed its PaaS Cloud Foundry

Link: HSBC chief architect: Why machine learning is accelerating cloud adoption

If you build it, you own it, big data ed.:

“We got some value out of that but to be honest we found it hard to keep on top of, just hard to build skills at the pace required to integrate new technologies,” Knott says.

“No matter how hard we ran there is always something new coming in that we wanted to get access to, but we couldn’t get there quite fast enough to have really finished deploying what we were deploying previously.

“So it was hard to manage, hard to keep on top of, and also hard to scale. We had reasonable success but we were having these challenges.””

Original source: HSBC chief architect: Why machine learning is accelerating cloud adoption

Link: Mission capability delivered at startup speed

More coverage of the US Air Force going all in on digital XP, lean design, and cloud native to dramatically – almost unbelievably so – modernize their software.

The mission capabilities these war fighters received in 120 days or less span deliberate targeting, mission reporting, advanced target production, refueling operations and many more, saving over $6.4 million and 1,100 man-hours per month within the Air Force Central Command.

More:

Embracing agile software development processes, the AOC leverages a highly disciplined approach to software engineering called Extreme Programming (XP), to gain the benefits of test-driven development, pair programming, continuous integration and continuous delivery (CI/CD).

Original source: Mission capability delivered at startup speed

Link: Cloud Foundry Cult

Owen covers CF Summit Basel:

“The users we spoke with didn’t just see it as a PaaS – it was the underlying philosophy of application delivery and management upon which future developments would be based. The Foundation claims Cloud Foundry saves, on average, 10 weeks of development time and $100,000 per app development cycle. In fact, in its own survey, 92% of users cite cross-platform flexibility as important. If these panelists are gaining such benefits, it’s easy to understand why they are so enamored with it.”
Original source: Cloud Foundry Cult

Link: PCF and New Relic at West

“Thomas said he plans to migrate hundreds of existing applications to Pivotal Cloud Foundry, which will also serve as the standard platform-of-excellence for all new applications. New Relic will ensure reliability, availability, and performance of those workloads as well as enable West’s ops team to monitor Pivotal Cloud Foundry itself.”
Original source: PCF and New Relic at West

Link: The Air Force Will Treat Computer Coding Like a Foreign Language

For example, when Defense Innovation Unit went to air operations centers in Middle East, the defense tech expert envisioned software changes that would optimize the way that airmen tracked refueling tankers. Teaming up with a commercial firm in Boston called Pivotal Labs, the new software is saving about $200,000 in fuel every month.

It’s usually reported at $200,000 a day.

Mostly, though, lots of AI talk:

During Operation Inherent Resolve, he led 1,800 Air Component Commander analysts to create the first-ever visualization of millions of data points from sources including unmanned aerial vehicles and open-source streams. That reduced 1.5 hours of daily target development to five minutes. He also provided intelligence support to the fight against ISIS that increased deliberate (i.e. pre-planned) target development by 68 percent.
Original source: The Air Force Will Treat Computer Coding Like a Foreign Language

A series of small projects, building momentum to scale

This post is an early draft of a chapter in my book,  Monolithic Transformation.

Not actually a picture of what’s described here, but it looks cool.

Every journey begins with a single step, they say. What they don’t tell you is that you need to pick your first step wisely. And there’s also step two, and three, and then all of the n + 1 steps. Picking your initial project is important because you’ll be learning the ropes of a new way of developing and running software, and hopefully, of running your business.

When it comes to scaling change, choosing your first project wisely is also important for internal marketing and momentum purposes. The smell of success is the best deodorant, so you want your initial project to be successful. And…if it’s not, you quietly sweep it under the rug so no one notices. Few things will ruin the introduction of a new way of operating into a large organization than initial failure. Following Larman’s Law, the organization will do anything it can — consciously and unconsciously — to stop change. One sign of weakness early, and your cloud journey will be threatened by status quo zombies.

In contrast, let’s look at how the series of small projects strategy played out in the US Air Force.

The USAF had been working for at least 5 years to modernize the 43 applications used in Central Air Operations Command, going through several hundreds of millions of dollars. These applications managed the US’s and allie’s daily air missions throughout Iraq, Syria, Afghanistan, and nearby countries. No small task of import. The applications were in sort need of modernizing, and some weren’t even really applications: the tanker refueling scheduling team used a combination of Excel spreadsheets and a whiteboard to plan the daily jet refueling missions.

Realizing that they’re standard 5 to 12 years cycle to create new applications wasn’t going to cut it, the US Air Force decided to try something new: a truly agile, small batch approach. Within 120 days, a suitable version of the tanker refueling application was in production. The tanker team continued to release new features on a weekly, even daily basis. The project was considered a wild success: the time to make the tanker schedule was reduced from 8 hours to 2, from 8 airmen to 1, and the USAF ended up saving over $200,000 a day in fuel that no longer needed to be flown around as backup for error in the schedule.

Number of USAF CAOC transformed applications over time, starting with 0 and ending with an estimated 18. Sources from several USAF presentations and write-ups.

The success of this initial project, delivered in April of 2017, called JIGSAW, proved that a new approach would work, and work well. This allowed the group driving change at the USAF to start another project, and then another one, eventually getting to 13 projects in May of 2018 (5 in production and 8 in development. The team estimates that by January of 2018 they’ll have 15 to 18 applications in production.

The team’s initial success, though just a small part of the overall 43 applications, gave them the momentum to starting scale change to the rest of the organization and more applications.

Project picking peccadilloes

Picking the right projects to start with is key. They should be material to the business, but low risk. They should be small enough that you can quickly show success in the order of months and also technically feasible for using cloud technologies. These shouldn’t be science projects or automation of low value office activities — no augmented reality experiments or conference room schedulers (unless those are core to your business). On the other hand, you don’t want to do something too big, like migrate the .com site. Christopher Tretina recounts Comcast’s initial cloud-native ambitions in this way:

We started out with a very grandiose vision… And it didn’t take us too long to realize we had bitten off a little more than we could chew. So around mid-year, last year, we pivoted and really tried to hone in and focus on what were just the main services we wanted to deploy that’ll get us the most benefit.

Your initial projects should also enable you to test out the entire software life cycle — all the way from conception to coding to deployment to running in production. Learning is a key goal of these initial projects and you’ll only do that by going through the full cycle.

The Home Depot’s Anthony McCulley describes the applications his company chose in the first six or so months of its cloud-native roll-out. “They were real apps. I would just say that they were just, sort of, scoped in such a way that if there was something wrong, it wouldn’t impact an entire business line.” In The Home Depot’s case, the applications were projects like managing (and charging for!) late tool rental returns and running the in store, custom paint desk.

A special case for initial projects is picking a microservice to deploy. Usually, such a service is a critical backend service for another application. A service that’s taken forever to actually deliver, or has been unchanged and ancient for years is an impactful choice. This is not as perfect a use case as a full-on, human-facing project, but it will allow you to test out cloud-native principals and rack up a success to build momentum. The microservice could be something like a fraud detection or address canonicalization service. This is one approach to migrating legacy applications in reverse order, a strangler from within!

Picking projects by portfolio pondering

There are several ways to select your initial projects. Many Pivotal customers use a method perfected over the past 25 years by Pivotal Labs called discovery. In the abstract, it follows the usual BCG matrix approach, flavored with some Eisenhower matrix. This method builds in intentional scrappiness to do a portfolio analysis with the limited time you can secure from all of the stakeholders. The goal is to get a ranked list of projects based on your organization’s priorities and the easiness of the projects.

First, gather all of the relevant stakeholders. This should include a mix of people from the business and IT sides, as well as the actual team that will be doing the initial projects. A discovery session is typically led by a facilitator, preferably someone familiar with coaxing a room through this process.

The facilitator typically hands out stacks of sticky notes and markers, asking everyone to write down projects that they think are valuable. What “valuable” means will depend on each stakeholder. We’d hope that the more business minded of them would have a list of corporate initiatives and goals in their heads (or a more formal one they brought to the meeting). One approach used in Lean methodology is to ask management this question: “If we could do one thing better, what would it be?” Start from there, maybe with some five whys spelunking.

Once the stakeholders have written down projects on their sticky notes, the discovery process facilitator draws or tapes up a 2×2 matrix that looks like the following:

Participants then put up their sticky notes in the quadrant, forcing themselves not to weasel out and put the notes on the lines. Once everyone finishes, you get a good sense of projects that all stakeholders think are important, sorted by the criteria I mentioned, primarily that they’re material to the business (important) and low risk (easy). If all of the notes are clustered in one quadrant (usually, in the upper right, of course), the facilitator will redo the 2×2 lines to just that quadrant, forcing the decision of narrowing down to just projects to do now. The process might repeat itself over several rounds. To enforce project ranking, you might also use techniques like dot voting which will force the participants to really think about how they would prioritize the projects given limited resources.

At the end, you should have a list of projects, ranked by the consensus of the stakeholders in the room.

Planning out the initial project

You may want to refine your list even more, but to get moving, pick the top project and start breaking down what to do next. How you proceed to do this is highly dependent on how your product teams breaks down tasks into stories, iterations, and releases. More than likely, following the general idea of a small batch process you’ll

  1. Create an understanding of the user(s) and the challenges they’re trying to solve with your software through personas and approaches like scenarios or Jobs to be Done.
  2. Come up with several theories for how those problems could be solved.
  3. Distill the work to code and test your theories into stories.
  4. Add in more stories for non-functional requirements (like setting up build processes, CI/CD pipelines, testing automation, etc.).
  5. Arrange stories into iteration-sized chunks without planning too far ahead (least you’re not able to adapt your work to the user experience and productivity findings from each iteration)

Crafting your hockey stick

Starting small ensures steady learning and helps contain the risk of a fail-fast approach. But as you learn the cloud-native approach better and build up a series of successful projects, you should expect to ramp up quickly. This chart shows The Home Depot’s ramp up in the first year:

Chart shows the number of application instances, which is not 1:1 to applications. The end-point represents about 130 applications, composed of about 900 services. Source: “Cloud-Native at Home Depot, With Tony McCulley,” Pivotal Conversations #45.

The chart measures application instances in Pivotal Cloud Foundry, which does not map exactly to a single application. As of December 2016, The Home Depot had roughly 130 applications deployed in Pivotal Cloud Foundry. What’s important is the general shape and acceleration of the curve. By starting small, with real applications, The Home Depot became learned the new process and at the same time delivered meaningful results that helped them scale their transformation.

See also another post of mine on this topic.

This post is an early draft of a chapter in my book,  Monolithic Transformation.

Link: How Companies are Saving Millions with Pivotal Cloud Foundry

“A recent Forrester study commissioned by Pivotal which analyzed the benefits PCF customers see when adopting the platform found that developers gain 50 percent more coding hours a week. How? The automation and self-service features of Pivotal Cloud Foundry decrease manual and mundane deployment tasks. Wait times for environment setup and code to be prompted to production are also significantly reduced… That 50 percent gain in coding hours led to more releases per year, speeding up release schedules from once every two months to once a week — and sometimes even daily. Forrester estimates this increase in productivity equates to more than $31 million over three years, while the reduction in DevOps time allocated to provisioning, patching and scaling across multiple clouds at almost $6 million.”

Also, Rackspace has a managed Pivotal Cloud Foundry service.
Original source: How Companies are Saving Millions with Pivotal Cloud Foundry

Link: Air Force looks to rapidly develop software with Project Kessel Run

More coverage of the USAF modernizing their approach to software. Here, what some of the apps are: “Kessel Run has been able to push five applications to the classified network, Kroger said…. The project is currently working on a number of things, including how the Air Force plans air tasking orders, a document which tasks units to fly their aircraft, Kroger said. It’s also working on building a tool that automates mission reports, which have to be written for every mission that flies, Kroger said.”

Original source: Air Force looks to rapidly develop software with Project Kessel Run