Essential Scrum – Product Backlog, Estimation and Velocity, Technical Debt (4)
The product backlog is a prioritized list of desired product functionality. It provides a centralized and shared understanding of what to build and the order in which to build it. It is a highly visible artifact at the heart of the Scrum framework that is accessible to all project participants
Good Product Backlog Characteristics
- Detailed appropriately
- Emergent – As long as there is a product being developed or maintained, the product backlog is never complete or frozen. Instead, it is continuously updated based on a stream of economically valuable information that is constantly arriving.
- Creating and refining (adding details to) product backlog items
- Estimating product backlog items
- Prioritizing product backlog items
Who Does the Grooming?
Grooming the product backlog is an ongoing collaborative effort led by the product owner and including significant participation from internal and external stakeholders as well as the ScrumMaster and development team.
Ultimately there is one grooming decision maker: the product owner. However, good product owners understand that collaborative grooming fosters an important dialogue among all participants and leverages the collective intelligence and perspectives of a diverse group of individuals, thereby revealing important information that might otherwise be missed.
When Does Grooming Take Place?
Definition of Ready
Grooming the product backlog should ensure that items at the top of the backlog are ready to be moved into a sprint so that the development team can confidently commit and complete them by the end of a sprint.
Definition of Ready
|Business value is clearly articulated.|
|Details are sufficiently understood by the development team so it can make an informed decision as to whether it can complete the PBI|
|Dependencies are identified and no external dependencies would block the PBI from being completed.|
|Team is staffed appropriately to complete the PBI.|
|The PBI is estimated and small enough to comfortably be completed in one sprint.|
|Acceptance criteria are clear and testable.|
|Performance criteria, if any, are defined and testable.|
|Scrum team understands how to demonstrate the PBI at the sprint review.|
Uncertainty cannot be eliminated from product development. We must assume that a stream of economically important information will be constantly arriving and that we need to organize and manage the work (manage the product backlog) so that this information can be processed in a rapid, cost-effective way while maintaining good flow.
Release Flow Management
I have found it useful to actually partition the product backlog using two lines for each release, as illustrated in Figure 6.11.
Sprint Flow Management
When grooming for good sprint flow, it is helpful to view the product backlog as a pipeline of requirements that are flowing into sprints to be designed, built, and tested by the team (see Figure 6.12). In this figure we see that larger, less-well-understood requirements are being inserted into the pipeline. As they progress through the pipeline and move closer to the time when they will flow out to be worked on, they are progressively refined through the grooming activity. At the right side of the pipeline is the team. By the time an item flows out of the pipeline, it must be ready—detailed enough that the team can understand it and be comfortable delivering it during a sprint.
If there is ever a mismatch or unevenness between the inflow and outflow of items, we have a problem. If the flow of groomed, detailed, ready-to-implement items is too slow, eventually the pipeline will run dry and the team won’t be able to plan and execute the next sprint (a major flow disruption or waste in Scrum). On the other hand, putting too many items into the pipeline for refinement creates a large inventory of detailed requirements that we may have to rework or throw away once we learn more (a major source of waste). Therefore, the ideal situation is to have just enough product backlog items in inventory to create an even flow but not so many as to create waste.
One approach that Scrum teams use is to have an appropriate inventory of groomed and ready-to-implement items in the backlog. A heuristic that seems to work for many teams is to have about two to three sprints’ worth of stories ready to go.
Which and How Many Product Backlogs?
When deciding on which and how many product backlogs to form, I start with a simple rule: one product, one product backlog, meaning that each product should have its own single product backlog that allows for a product-wide description and prioritization of the work to be done.
What Is a Product?
A product is something of value that a customer would be willing to pay for and something we’re willing to package up and sell.
Large Products—Hierarchical Backlogs
Multiple Teams—One Product Backlog
One Team—Multiple Products
If an organization has multiple products, it will have multiple product backlogs. The best way to handle multiple product backlogs is to assign one or more teams to work exclusively on each product backlog.
In some instances, however, one team ends up working from multiple product backlogs (see the right side of Figure 6.16). Our goal should be to minimize the amount of multi-projecting that teams or team members perform. The first, and often the best, solution is to have the team work on one product at a time. In each sprint the team works only on the items from one product backlog.
However, if organizational impediments force us to have the single team work on multiple products concurrently, we might consider merging the PBIs for all three products into one product backlog. This would require that the product owners for the three products come together and reach a single prioritization across all of the products.
Estimation and Velocity
When and What to Estimate
Not all Scrum practitioners believe that PBI size estimation is a necessary activity. Their experience has shown that when Scrum teams become good enough, they are able to create PBIs that are small and of roughly the same size. Such practitioners have determined that it is wasteful to estimate small, similarly sized items. Instead, they just count the number of PBIs. They still use the concept of velocity, but it is measured as the number of PBIs that are completed in a sprint, instead of the sum of the sizes of the PBIs that are completed in a sprint.
PBI Estimation Concepts
- Estimate as a Team
Estimates are not commitments
- Focus on accuracy, not precision
- Use relative versus absolute size
PBI Estimation Units
- Story Points
Story points combine factors like complexity and physical size into one relative size measure. The goal is to be able to compare stories and say things like “Well, if the create-a-ticket story is a 2, then the search-for-a-ticket story is an 8,” implying that the searching story is roughly four times the size of the creation story.
- Ideal Days
- Consensus based
- Expert opinion
- Intense discussion
- Relative sizing
- Accurate grouping/binning
- Leverage estimating history
1, 2, 3, 5, 8, 13, 20, 40, and 100
|1/2||Used to size tiny items|
|1, 2, 3||Used to size small items|
|5, 8, 13||Used to size medium items. For many teams, an item of size 13 would be the largest they would schedule into a sprint. They would break any item larger than 13 into a set of smaller items.|
|20, 40||Used to size large items (for example, feature- or theme-level stories).|
|100||Either a very large feature or an epic.|
|∞ (infinity)||Used to indicate that the item is so large it doesn’t even make sense to put a number on it.|
|?||Indicates that a team member doesn’t understand the item and is asking the product owner to provide additional clarification. Some team members also use the question mark as a way of recusing themselves from the estimation of the current item—typically because the person is so far removed from the item he has no idea how to estimate it. Although it is acceptable not to estimate, it is unacceptable not to participate! So, just because someone doesn’t feel comfortable offering up an estimate, that doesn’t allow him to disengage from the conversation or responsibility of helping the team find a consensus estimate|
|π (pi)||The pi card is used when a team member wants to say, “I’m tired and hungry and I want to get some pie!” Some Planning Poker decks use a coffee cup image instead of pi. In either case, this card emphasizes an important point. The team members can engage in an intense estimation discussion for only a limited period of time (perhaps an hour or two). At that point, they really do need a break or the enthusiasm for the discussion will turn into an effort to figure out how to quickly get the estimates done, regardless of their accuracy or the learning that takes place. If people are playing the pi card, the team needs to take a break.|
The rules of Planning Poker are as follows:
- The product owner selects a PBI to be estimated and reads the item to the team.
- Development team members discuss the item and ask clarifying questions to the product owner, who answers the questions.
- Each estimator privately selects a card representing his estimate.
- Once each estimator has made a private selection, all private estimates are simultaneously exposed to all estimators.
- If everyone selects the same card, we have consensus, and that consensus number becomes the PBI estimate.
- If the estimates are not the same, the team members engage in a focused discussion to expose assumptions and misunderstandings. Typically we start by asking the high and low estimators to explain or justify their estimates.
- After the discussion, we return to step 3 and repeat until consensus is reached.
When a team does not have historical data the team can always forecast the estimates for 2 sprints ahead. Then a max and min value can be calculated.
- Unfit (bad) design—a design that once made sense but no longer does, given important changes to the business or technologies we now use
- Defects—known problems in the software that we haven’t yet invested time in removing
- Insufficient test coverage—areas where we know we should do more testing but don’t
- Excessive manual testing—testing by hand when we really should have auto- mated tests
- Poor integration and release management—performing these activities in a manner that is time-consuming and error-prone
- Lack of platform experience—for example, we have mainframe applications written in COBOL but we don’t have many experienced COBOL programmers around anymore
Type of Technical Debt
- Naive technical debt – Team member or business immaturity or process deficiencies that lead to sloppy design, poor engineering practices, and a lack of testing. This kind of debt can be eliminated through proper training, a good understanding of how to apply technical practices, and sound business decision making. This type of debt has an irresponsible and frequently accidental nature.
- Unavoidable technical debt – Usually unpredictable and unpreventable. For example, our understanding of what makes for a good design emerges from doing the design work and building user-valuable features on it.
- Strategic technical debt – This kind of debt is a tool that can be used to help organizations better quantify and leverage the economics of important, often time-sensitive, decisions. For example, an organization might deliberately make a strategic decision to take shortcuts during product development to achieve an important short-term goal, such as getting a time-sensitive product into the market- place.
Consequences of Technical Debt
Causes of Technical Debt
Pressure to Meet a Deadline
Both strategic and naive technical debt, however, are often driven by business pressure to meet an important looming deadline.
Attempting to Falsely Accelerate Velocity
Myth: Less Testing Can Accelerate Velocity
Technical Debt Must Be Managed
Managing the accrual of technical debt
- Use Good Technical Practices
- Use a Strong Definition of Done
- Properly Understand Technical Debt Economics
Making Technical Debt Visible
Make Technical Debt Visible at the Business Level
The problem in many organizations is that whereas the development team has at least some reasonable visibility into the product’s technical debt position, the business people typically do not. Ask any technical person who has knowledge of a product where the greatest concentration of technical debt in the product is, and chances are she can answer that question. Ask the same question of a business person and she will typically have no appreciable understanding of how much, or what type of, technical debt exists.
Make Technical Debt Visible at the Technical Level
- Technical debt could be logged like defects into an existing defect-tracking system
- Create product backlog items that represent technical debt. Doing so will give important technical debt visibility on a par with that of new features in the product backlog. Teams typically use this approach when the cost of servicing the technical debt is quite high and the product owner needs to be involved in deciding how that work should be ordered relative to value-added new features in the product backlog.
- A third approach to making technical debt visible is to create a special technical debt backlog that makes individual technical debt items visible. Whenever new technical debt is discovered or introduced into the product, a development team member can create a new technical debt item and add it to the technical debt backlog. By making the technical debt items visible, the development team can not only see its technical debt position but also can proactively determine when it wants to service each piece of technical debt.
Usually the technical debt board would be placed right next to the sprint backlog so that during sprint planning the team has visibility into the technical debt that it can consider servicing in the upcoming sprint.
Servicing the Technical Debt
The last activity in managing technical debt is to service or repay the debt. When discussing debt servicing, I find it helpful to use the following status categories:
- Happened-upon technical debt—debt that the development team was unaware existed until it was exposed during the normal course of performing work on the product. For example, the team is adding a new feature to the product and in doing so it realizes that a work-around had been built into the code years before by someone who has long since departed.
- Known technical debt—debt that is known to the development team and has been made visible using one of the previously discussed approaches.
- Targeted technical debt—debt that is known and has been targeted for servicing by the development team.
Based on these categories, I generally apply the following algorithm when servicing technical debt:
1. Determine if the known technical debt should be serviced (as I will discuss, not all debt should be serviced). If it should be serviced, go to step 2.
2. If you are in the code doing work and you discover happened-upon technical debt, clean it up. If the amount of happened-upon technical debt exceeds some reasonable threshold, clean it up until you reach that threshold. Then classify the nonserviced, happened-upon technical debt as known technical debt (for example, by creating entries in the technical debt backlog).
3. Every sprint, consider designating some amount of known technical debt as targeted technical debt to be serviced during the sprint. Favor servicing known technical debt with a high interest rate that is aligned with customer- valuable work.
Not All Technical Debt Should Be Repaid
- Product Nearing End of Life
- Throwaway Prototype
- Product Built for a Short Life