Update: this book is now available, for free even!
After transforming management, comes changing theactual teams working on your software. Most organizations think about software as projects: something that has a finite set of requirements, schedule and budget, and a finite end date, the date it’s delivered. There’s some consideration put into updating the software, managing its full life. This project mentality can seem to make a lot of sense, esp. if you’re outsourcing much of your software.
Mark Schwartz calls this project approach the “contractor control” model. His theory is that this model is based on management’s mis-trust of the people doing the work, whether they’re contractors or internal staff. As he outlines in A Seat at the Table, this drives a heavy emphasis on status meetings, compliance, and lock-step road-maps:
How could you control a contractor? You asked for an estimate and pressured the contractor to deliver at or close to that estimate. Or you agreed on a fixed price. How could you control IT? Same model, but with the twist that the IT staff were your own employees who were paid a fixed salary — a bit awkward. Since their cost was fixed (at least in the short and medium terms), your biggest worry was that they would waste time on frivolous activities. How could you know that they weren’t? Simple: you insisted that they deliver on schedule, and kept the pressure on them to do so. Some of IT’s work was transactional: user support, device provisioning, updates, and maintenance. In those areas, costs and lead times could be benchmarked and monitored. But a good deal of IT’s work involved delivering capabilities — developing and integrating applications, rolling out ERP systems, installing collaboration tools, and so on. For IT to demonstrate that it was performing that type of work responsibly and for the business to verify that it was doing so, the scope of each task had to be defined precisely, bounded, and agreed upon in advance. The work had to be organized into projects, which are units of work with a defined set of deliverables, a beginning, and an end. You could establish control by making sure the project was completed within the bounds of its estimated cost and schedule. How perfect the Waterfall model is for this purpose! How perfectly it aligned with the business’s need to know that IT was under control.
The results are predictable: frustrated people, bad software, and slow failure like two drunks in a bar-brawl.
Sobering up starts with two words: product and team. Doing software well requires thinking about it as a product, not an ongoing project. This is how software vendors think, of course: the software they create it what they sell. A product is interested in accomplishing its user’s (and customer’s) goals, ongoing, until it’s no longer needed. A product evolves to match the user’s evolving needs. The focus is on continually making the best software possible.
Product-centric organizations move from the functional organization, contractor control model to an agile, product-centric model.
Focusing with a product team
Typically our developers, or business analysts, or designer, work on five projects at the same time, and if you are good you work on ten. That is tremendously inefficient. So the first task was to get them out of their traditional environment and put them into the garage, concentrating on one project. That is mandatory that the team members work co-located — all disciplines from market management to analysts, designers, and developers. — Dr Andreas Nolte, CIO, Allianz
One of the most important aspects of a product team is focusing on just one application. The contractor-control model typically drives an organization to be functional based: developers, QA, DBAs, networking engineers, support staff, and on and on. After all these years, I’m not sure why this is: it must have seemed like a better use of scarce resources, people and compute, at the time.
Organizations that are improving their software are finding that this “silo” approach doesn’t, however, optimize much. There are too many handoffs between each group, the lack of trust across silos drives both too much paperwork and a “not my problem” mind-set, all resulting in poor software.
A small batch approach to discovering, and then perfecting, your software requires a lot of trial and error experimentation. You’re operating in an incredibly chaotic, information-poor environment, where decisions need to be made constantly to keep moving ahead. In complex systems like that, speed and quick access to information are important to make decisions. You should always be exploring and looking for the best fit for your software to address the customer problem you’re looking to solve.
When you’re executing on a weekly, if not daily, deployment feedback loop, you don’t have a lot of time to synchronize across teams. You also don’t have time to continually “go up the chain” to ask permission to try something new.
Thus far, product teams seem to be the best staffing approach to maintaining focus on that overall outcome. Because of the huge degree of automation in cloud technology, organizations are now able to free up resources from the bottom of the stack to focus almost all of their resources on the more valuable, application, layer on top, the actual applications being used to run the business.
Although the exact implementation of a balanced team can vary, in general, it’s one team of people per application with responsibility for and authority over the software you’re creating and delivering. This team is dedicated full time to the application for which they have ownership.
An innovative, product-centric approach is what’s needed in an exploratory process like software creation, where people don’t know what they want and are often (at first) completely wrong about what they want. Teams in this setting should be more closely attached to the software being written rather than parachuted in as needed. The team’s understanding of the problem being solved, approaches tried in the past, and the overall tribal knowledge will be invaluable in figuring out the right product to build. Integrated teams are not only important for product management continuity, but also for ensuring that the product is resilient in production. It’s vital to keep these teams small and they should have all the skills needed for the full life cycle of the product, including development, testing, design, and operations.
Staffing wise, this typically means teams of roughly 6–12 developers, at least one designer, a product manager, and often some part time roles like QA and operations support. In larger organizations, there will likely be shared resources that may start to look dangerously like traditional teams in silos (e.g.,security testers or domain experts). Ideally, you truly want every role and person on the same team, but that’s not always possible.
Next, let’s look at the types of roles that are typically on these teams.
Product team roles
The composition of product teams will change over time as each team gels, learning the necessary cloud native operations skills and mastering the domain knowledge needed to make good design choices.
As detailed below, the core team is composed of developers, designers, and a product owner. There are also some supporting roles that come and go as needed — testers, architects, DBAs, data scientists, and other specialists.
These are programmers or software developers. Through the practice of pairing, knowledge is quickly spread among developers, ensuring that there are no empires built and addressing the risks of a low bus factor. Developers are encouraged to rotate through various roles from front to back-end to get good exposure to all parts of a project. By using a cloud platform, like Pivotal Cloud Foundry, developers can also package and deploy code on their own through continuous integration and continuous delivery (CI/CD) tools.
As developers are the ones implementing your applications, they are closely involved with estimating how long stories in your backlog and prioritizing them for releases. Similarly, developers can help product managers and designers walk through the technical trade-offs involved in deciding how to implement and design features.
Developers are not expected to be experts in all operations concerns. Instead, they rely on the self-service and automation capabilities of cloud platforms for the most common operations needs. This means they don’t need to wait for operations staff to perform configuration management tasks to deploy applications. There will, of course, be operations knowledge that developers need to learn, especially when it comes to designing highly networked, distributed applications. Initially, prescriptive platform patterns help here, as well as embedded operations staff. In addition to relying on the cloud platform to automate and enforce (now) routine operations tasks, over time, developers often gain enough operations knowledge to work without dedicated operations support.
The number of developers on each team is variable, but so far, following the two pizza team rule of thumb, we typically see anywhere from one to three pairs, that is two to six developers, and sometimes more.
Product Owner/Product Manager
This role defines and guides application requirements. It is also one of the roles that varies in responsibilities the most across products. At its core, this role is the owner of the software under development, though it’s more accurate to think of product managers as the one setting the vision week-to-week and pointing the team in the right direction. In that respect, product managers help prioritize, plan, and deliver software that meets requirements or, stories as they’re commonly called. Someone has to be the final word on what happens in the team. The amount of control versus consensus-driven management is the main point of variability in this role, plus the topic areas in which the product owner has knowledge.
It’s best to approach the product owner role as a breadth-first role: these individuals have to understand the business, the customer, and the technical capabilities. This broad knowledge helps them make sure they’re making the right prioritization decisions.
In organizations that are transitioning to cloud native, this role also serves as the barrier between the all-too-fragile new teams and the existing, legacy teams. The product owner becomes the gatekeeper that keeps all the helpful interest and requests at bay so that the teams can focus on their work.
One of the major lessons of contemporary software is that design matters a tremendous amount more than previously believed. While nice looking UIs are, well, nice to have, design in software is so much more than looks. The designer takes responsibility to deeply understand the needs and challenges that users have, and how to create solutions to overcome these challenges. You might think of designers as the empathizers in chief.
As discussed, the small batch mentality of learning and improving software afforded by cloud platforms like Pivotal Cloud Foundry gives designers the ability to design more rapidly and with more data-driven precision than ever.
The designer focuses on identifying the feature set for the application and translating that to a user experience for the development team. As some put it, design is how it works, not (just) how it looks. Activities may include completing the information architecture, user flows, wireframes, visual design, and high-fidelity mock-ups and style guides. Most important, designers have to get out of the building and not only see what actual users are doing with the software, but get to know those users and their needs intimately.
In addition the core roles above there are many other roles that can, and do, exist in IT organizations. These are roles such as DBAs, security operations, network operations, and storage operations. In general, as with any tool, you should use what you need when you need it. Any given role must reorient itself to enabling the core teams rather than governing them. As the DevOps community has discussed at length for nearly 10 years, the more you divide up your staffing by function, the further you move from a small, integrated team, and achieving your goal of consistently and regularly building quality software will become harder.
Below are four common other roles on product teams. Many of them are “part time,” and some are used to help bootstrap the product team.
Until business capabilities teams in a cloud native environment have learned the necessary skills to operate applications on their own, they will need operations support. This support will come in the form of understanding (and co-learning!) how the cloud platform works, as well as gaining assistance troubleshooting applications in production. Early on, you should plan to have heavy operations involvement to help collaborate with developers and share knowledge, mostly around getting the best from the cloud platform in place. As with development, using rotating pairing will help quickly spread knowledge. You may need to assign operations staff to teams at the beginning, making them designated operations staff instead of dedicated, as explained in Effective DevOps.
In some organizations, the operations role never leaves the team, which is perfectly normal. Indeed, the desired end-state is that application teams have all of the development and operations skills and knowledge needed to be successful. Most organizations find that product teams need operations support early on as the cloud platform is built out and developers learn how to use the self-service functionality of the platform to manage infrastructure. These operations staff frequently rotate out once developers are better skilled.
As a side note, it’s common for operations staff to freak out at this point, thinking they’re being eliminated. While it’s true that margin-berserked management could choose to look at operations staff as waste, it’s more likely that following Jevon’s Paradox, operations staff will be needed even more as the amount of applications and services multiply.
Although the product manager and overall team are charged with testing their software, some organizations either want, or need, additional testing. Often this is exploratory testing where a third party (the tester[s]) are trying to systematically find the edge cases and other bugs the development team didn’t uncover.
Some Pivotal customers have reported that they’ve been able to dramatically reduce their QA staffing and thus, overall IT spend. While this may not always be the case, if you find yourself with a lot of QA staff, it’s worth questioning the need for separate testers. Much routine QA is now automated (and can be done by the team through automated CI/CD pipelines), but you may want exploratory, manual testing in addition to what the team is already doing to verify that the software does as promised, and functions under acceptable duress. Yet even that verification can be automated in some situations as the Chaos Monkey and Chaos Lemur show.
Traditionally, this role has been responsible for conducting enterprise analysis, design, planning, and implementation using a big picture approach to ensure the successful development and execution of strategy. While those goals can still exist in many large organizations, the role of an architect is evolving to be an enabler for more self-sufficient, decoupled teams. Too often, this role has become a Dr. No in most large organizations, so care must be taken to ensure that the architect supports the team, not the other way around.
Architects are typically more senior technical staff who are domain experts. They may also be more technically astute, and in a consultative way, help ensure the long-term quality and flexibility of the software that the team creates. They may also share best practices and otherwise enable teams to be successful. This last point is crucial for, yet often ignored by, large organizations as we’ll discuss in the section titled dealing with legacy.
If your application requires a large amount of data analysis, you should consider including a data scientist role on the team. This role can follow the dedicated/designated pattern as discussed previously with the operations role.
Data science today is where design was a few years ago. It’s not considered to be a primary role on the product team, but more and more products today are introducing a level of insight not seen before now. Mobile notifications surface contextual information to buyers about flash sales nearby; users are offered deals on movie rentals tuned to their viewing behavior; GE uses fast modeling and analysis to tune wind and jet turbines; and trucking companies are using analytics to program their fleets. These features help turn “dumb,” transactional products into “smart,” differentiated products.
Check out my book, Monolithic Transformation for more on this topic and other tips on changing organizations.