Development Teams are cross-functional, with all of the skills as a team necessary to create a product Increment. - K. Schwaber and J. Sutherland, Scrum Guide, 2013
Except when they're not.
This topic is either ironic or timely, depending on your perspective. My last post was about the importance of vertically slicing stories. One of the most important elements needed to have all stories deliver working software is the cross-functional team. So let's just call this post an acknowledgment that the world is not perfect and move on.
Among the challenges of playing by the rules of the Scrum game is the problem of dependencies between teams.
Scrum teams are responsible for development work, but they are frequently part of a large project or program. So, the individual Scrum teams are not able to create something useful for the enterprise. Their work must first be integrated with the work of other teams to bring real value to the business.
At least once per Sprint, all work must be integrated. The preference should lean towards continuous integration, but daily integration with other teams should be the minimum. At first, continuously integrating each team's work is challenging and your organization might not even have the capability. Companies have to change the development organization in order to integrate multiple teams' work. The technology used to build and test products must change and engineering practices throughout the organization have to improve.
Some changes happen within individual Scrum teams or even at the developer level. However, most companies need considerable improvements across the organization. We build very complex products, no matter how simple the design. Building working software every Sprint is difficult under the best circumstances, but nearly impossible without knowing daily status of development. IT organizations will usually take this opportunity to complain about what they can't do within Scrum, but they are focused on the problem while they should be focusing on solutions. Maybe they can't do everything right now, but with the right engineering practices, some discipline, a lot of drive to get there, it can be done.
Let consider some common external dependency scenarios and see how they might be handled within Scrum.
Front-end functionality depends on infrastructure enhancements
The MyReports team selected a Product Backlog item to allow a user to view all transaction types over a user-defined period in a single view. The current situation required users to export data from multiple screens so they could summarize the data in Excel. The project work comprised changes to the screens, workflow, and database.
The workflow and database was part of the infrastructure that supported many of the division's products. To support the user requests, there would need to be more frequent data capture and additional fields added. When those enhancements were completed, the user interface could pass a single request with the required account and time period variables. Unfortunately, infrastructure changes go through a different team, so the MyReports team needed to make a decision: (a) continue without depending on infrastructure changes for testing and demonstrations of the front-end enhancements, or (b) select another item from the backlog to work on until the infrastructure team was ready to work together.
Luckily, Agile gives us guidance for this type of situation:
Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. - Agile Manifesto, 2001
So the only question the team needed to answer was whether option (a) would yield "valuable software." If the answer is yes, that would lead to a second question: is there really a dependency or is the infrastructure enhancement just preferable? In this example, the answer to (a) was no, so choice (b) was the only viable option.
Teams should not begin a story with an external dependency unless the other team is at the Sprint Planning Meeting to make their commitment. However, when a team supports the infrastructure enhancements for multiple customers, who prioritizes its work? One solution is an enterprise Product Owner who is responsible for overall ROI. By rolling up all the individual product backlogs into a single enterprise backlog, the uber Product Owner can prioritize the infrastructure Product Backlog to maximize overall ROI and minimize risk.
Work is divided by architectural layer
Sometimes it is not feasible to have development teams that are cross-functional across architectural layers. When working around this situation, don't lose sight of the three pillars upon which Scrum theory is founded: transparency, inspection, and adaptation. In order to inspect and adapt, teams always need to know where they are against the product backlog. Teams also need the ability to release working software as frequently as possible.
Amlar monitors financial transactions to identify suspicious banking activity. They deliver this functionality across five layers. The first layer collects and stores all transactions between banks and other entities. A second layer maintains client account information. The third layer maintains regulatory guidelines for financial transactions. The fourth layer compares transactions to regulations and creates message traffic. The fifth layer presents the messages as alerts for further investigation. Each of the layers was developed by separate organizations at different geographic locations: one layer in Georgia, one in New Hampshire, and three at different locations in Massachusetts. Each layer had its own Product Owner and development teams. The Amlar executives were pleased with the progress, but how did they do it? How do teams make sure they are adapting to changes, rather than continuing to build to the out-of-date specifications? How could each team know whether their increments would integrate into a useful product?
First, create an integration layer. Yes, a sixth! layer. The integration layer team had representatives from each of the other layers. The integration layer pulled the builds from each layer daily and tried to integrate them into a single build. Second, they developed integration tests and exercised the functionality as it would be when the final product was released. Failures were reported to the layer that had caused the failure. That team had to prioritize the problem before ahead of any more development. Third, a norm was established that all layers had to integrate into a working increment by the end of each Sprint. If not, nothing could be called Done and nothing could be demonstrated.
Scrum team depends on waterfall team
HipaaCare sells health insurance and needed to replace its legacy systems with software that could adapt to the Affordable Care Act and other regulatory changes. One team was responsible for building a CRM system that would replace the legacy mainframe components for four customer constituencies. To pull the relevant claims and billing data into the CRM and to make the customer data available to other system, shared web services would be developed by a dedicated team.
The web services team was using its own milestone-driven process as it worked from the SOA design specifications. The milestones were a design document, a prototype of each service, and then the finished product. The prototype was a crude imitation of the final product, but it provided a test environment for the CRM and could be used to verify every Sprint's increment of functionality. However, the service team's delivery milestone was three months into the project.
The CRM team used Scrum and completed three Sprints within the first three months. It built a mock service to the specifications of the web service. Once the real service was delivered, development done on the CRM could be fully tested.
Change and miscommunication are directly correlated to product complexity. That is why Scrum requires integration to occur at least once per Sprint so it can be demonstrated at the Sprint Review. When other teams don't use Scrum, the Scrum teams need to integrate to the best possible representations of the other components. These representations must be devised and used to minimize later rework.
The CRM team had to build a simulation of the new service and structure as best as it understood it until the prototype was ready. They selected several enhanced functions and several new functions for the first Sprint. It refactored the design to broadly take the anticipated changes into account. It also ensured that the previously developed functionality continued to work.
This solution required the CRM team to retest its work against the best possible representation of the completed product. Every time the design changed, these representations had to be changed for retesting. The rework was limited to that functionality and the design completed when the change occurred. The real services from the waterfall parts of the project then would replace the Scrum team's simulation layer.
Conclusion
There are many reasons why teams are not cross-functional and self-contained, not the least of which is that many large organizations have adopted Scrum, but are not yet structured that way. There are also enterprise-wide systems changes that require some level of dependencies. Whatever the cause, there isn't a one size fits all solution. I think you'll find, however, that many situations can be worked using one of the three examples given above.
What issues have you encountered with dependencies? Have you found a solution that everyone can be happy with?