Technical Debt: Adding Math to the Metaphor

Ward Cunningham introduced the “debt metaphor” to describe what can happen when some part of software work is postponed in order to get other parts out the door faster. He used this metaphor to highlight the idea that such postponement, while often a good choice, could have higher costs than people suspect. He likened the ongoing cost of postponed work to financial interest, and the eventual need to complete the work to the repayment of principal, observing that the interest charges could eventually become high enough to reduce the capacity to do other important work.

There is no question that technical debt is a useful concept. It easily communicates the danger of postponing important work. Even people who understand little about product development quickly grasp the idea that postponed work can overwhelm them, like a constantly growing balance on a high interest credit card.

Unfortunately, metaphors can cause us to treat a large class of situations as being essentially similar – even when they are not. In this case, we can avoid this trap by using the metaphor to identify the economic forces at work and using math to assess the balance of these forces.  Adding math to the metaphor is particularly useful in the case of technical debt because it allows us to think more clearly about a very wide range of situations.

Let’s begin by reviewing the economic characteristics of financial and technical debt.

Technical Debt Table 2013bFinancial Debt

I’ll describe the simplest version of financial debt in which all principal is returned at the end of the loan period and recurring interest is paid regularly until the principal is returned. This is what most people have in mind when they use the debt metaphor.

When we incur a financial debt there are three economic consequences:

1. We receive access to an immediate benefit, the principal, which is of known magnitude and certain timing.

2. We incur an ongoing obligation to pay interest, which is an economic “rent” paid to the original owner of the principal. The timing and magnitude of these interest payments are also certain.

3. We undertake a future obligation to return 100 percent of the principal at a specific future time. Thus, the timing and magnitude of repayment is also certain.

Financial debt is not intrinsically evil.  It is economically sensible to incur financial debt when, by using the principal, we obtain risk-adjusted benefits that exceed the cost of the economic rent. For example, we might rationally borrow money to purchase a bicycle to avoid the recurring cost of paying daily taxi fares.

Technical Debt

If we consider technical debt to be analogous to financial debt, we could identify three similar economic consequences, although, as you will see, each one can take a wide range of forms:

1. We receive the benefit of postponing the expenditure of development effort. This benefit may include savings in both expenses and cycle time. The magnitude and timing of these benefits is uncertain.

2. We can incur an ongoing obligation to pay an economic “rent” on the deferred work. The magnitude and timing of this economic rent is also uncertain. It could range from positive to negative, and its timing can vary widely.

3. We incur a highly uncertain future obligation to repay the postponed development effort. Both the amount and timing of this future obligation is highly uncertain.

By analogy with financial debt, it is economically sensible to incur technical debt when the expected cost of the obligations incurred is lower than the benefits gained by deferring the work.  When we analyze the economics of technical debt we need to be particularly careful about two issues. First, we need to be sure we include the full economic costs. Second, we need to be sure we account for the uncertain nature of the costs. I’d like to illustrate how we can do this using three progressively more rigorous calculations.

First Iteration

Let’s take a simple case where we choose to introduce the product with technical debt rather than spending 2 months and $40,000 to design a debt-free product. In this case, assume we save $40,000 of development expense in year 1, pay extra support costs of $60,000 per year, and repay the deferred work in year 5 at twice the original effort for $80,000.  The table below shows these economics.Technical Debt Iteration1 2013
This is a bad economic proposition: our attempt to save $40,000 costs us $340,000. The answer looks compelling, but the calculation is faulty.

Second Iteration

Why? Because our first iteration ignored the cost of delaying the product launch by 2 months. We call the cost of delaying product introduction the Cost of Delay[1] (CoD), and it is quantifiable. So, I’ll add the delay costs for both the deferred and undeferred work on the lines highlighted in yellow.  In this case, we’ll assume our CoD is $250,000 per month and that the deferred work comprises 3 percent of the product value, and 3 percent of the delay cost. Therefore, the deferred work adds $90,000 per year of extra economic cost ($250,000 x 12 months x 3 percent), and the delivery of the undeferred work two months early saves us $485,000 ($250,000 x 2 months x 97 percent).

Technical Debt Iteration2 2013

Note the striking difference in our answer when we consider delay costs. We see that we can defer the work for up to three years with no net economic cost. Yet, deferring the work for a full 5 years does not make economic sense. While this calculation is better, it still fails to recognize the uncertain nature of the costs.

Third Iteration

Now, let’s recognize that some of these costs are uncertain and adjust them for their likelihood. In this case, for simplicity, I’ve weighted all the costs associated with deferred work, highlighted in yellow, with a 50 percent probability of occurrence.  As a general rule we’d expect to have less confidence in costs associated with the deferred work than the undeferred work, and less confidence in costs at longer time horizons.Technical Debt Iteration3 2013
When we take into account the uncertain nature of these costs, we get a very different view of the problem. We can economically justify the full 5 years of deferral, although the cumulative benefit of this deferral decreases every year.  This answer is very different than our first calculation. Our view has shifted from an expected loss every year to an expected gain every year. This is the advantage of using math.

The point of these calculations is not to provide you with a new universal truth that can be substituted for thinking – it is to show you how a little math can illuminate the problem of technical debt.  Now, let’s discuss what factors you might consider in doing this analysis.

Some Tips on Doing Better Analysis

If you choose to incorporate some economic analysis into your decisions to postpone work, here are some tips that might help you:

Use an Economic Frame

  1. Treat the decision to defer work as an economic choice rather than a philosophical one.  Weigh the likely economic benefits against likely costs.
  1. When evaluating the economic benefits of deferring work, be sure to include both the work that is deferred and the work that is not. The economic impact on the undeferred work is often the dominant economic factor.
  1. Since deferral affects development cycle time, it is crucial that we consider delay costs in our analysis. To assess this we must estimate the Cost of Delay for both deferred and undeferred work. Because 85 percent of developers do not quantify the Cost of Delay for their projects, they are poorly equipped to assess the real economics of deferring work.

Be Careful When Estimating Benefits Received

  1. Recognize that the benefits from deferring work are uncertain. Be willing to weight them by their probability of occurrence. Will we really deliver the software any earlier if we postpone a work item? Will omitting the work really reduce our effort?
  1. Focus on the net benefits.  Sometimes when we cripple a system by leaving out a key feature other activities will grow in cost and duration.  To assess net benefits we must look at both the deferred work and how this affects the rest of the project.
  1. Be sure to include the value of feedback. Early feedback on the work that is released early can significantly reduce risk, and this risk reduction is worth money. In many cases, risk reduction will be a more important economic factor than the expense savings.

Estimate “Interest” with Care

  1. Recognize that the recurring cost penalty associated with deferring work is uncertain. We are prone to overstate the benefits of our “debt-free” solution. This solution always appears to be far superior while it remains a figment of a programmer’s imagination, but sometimes, when instantiated into code, things are different.  Consequently, we should apply a discount factor on the savings that would arise from our “debt-free” solution.
  1. Recognize the time dependency of the “rent” associated with deferred work. For example, we might target a 30 percent performance advantage over our competition, but early in life we may be able to generate customer preference with a 20 percent performance edge. Later, when competition has improved, our 20 percent edge may degrade to mere parity. Under such circumstances it can make economic sense to immediately harvest the benefits of our 20 percent advantage, and move to 30 percent before competitors reach parity. In such a case, the period during which we have a sufficient performance advantage can be viewed as “rent-free”. In such cases we can exploit the interest-free period and retire the debt before our interest charges start.
  1. Recognize that at times there may never be any rent. If nobody actually cares about the feature, there will be no value created by delivering it early, and no cost when it is not yet available.
  1. Recognize that at times the rent can be negative.  The feature we leave out may actually add more cost than benefit. In such cases, leaving it out makes us money.  For example, consider the choice of using a piece of open source software immediately, versus developing a superior solution by writing our own clever code.  The open-source code might create less utility for our customer, but it might also lower our support costs. When the savings in support exceed the lost utility we are paying a negative rent—deferring work reduces our ongoing costs instead of increasing them.

Evaluate Need to Repay Principal with Great Care

  1. Don’t assume that it will cost the same to do work in the future as it does today. In most cases it will be cheaper to do it later because requirements will be better defined. However, in some important cases it can be very expensive to do the work later. When we create $1,000 of technical debt we might repay $2,000, $200, or absolutely nothing.
  1. Don’t assume doing the work immediately will protect us from doing it again later. We may actually find that we are paying twice, and that our preemptive effort to do a “debt-free” design does not preempt very much at all.  If the cost to repay principal is the same whether we defer work or not, it should not enter into our calculations.

Pay Attention to the Larger System

  1. Recognize how batch size reduction improves the economics of deferral . When we disaggregate the work into smaller chunks we dramatically increase the economic opportunity. If we defer 50 percent of work to permit 50 percent to be delivered early, then deferral costs will be high.  If we defer 1 percent of the work to permit 99 percent to be delivered earlier the economics are radically different.
  1. Recognize how product architecture alters the combined economics of deferred and undeferred work. We can create “landing pads” for deferred work that minimize our future pain. Often a small burden on the work we are releasing early leads to big savings for the follow-on work. This is particularly true when we provide extra margin in a product architecture aligned with its likely growth vectors.
  1. As mentioned earlier, recognize the feedback benefits of deferring work. Reductions in the scope of work permit us to deliver functionality faster. This gives us early feedback. Early feedback allows us to truncate unproductive paths more quickly. Such feedback can be exploited to alter the economics of subsequent decisions.
  1. Treat the economics of deferred work as something we can control, not an immutable characteristic of the work.  The value of understanding our economics is that it helps us find the best places to intervene in our system.

Consider Calling It Deferred Work

When we refer to postponed work as technical debt this automatically biases us to assume that both ongoing and future costs are more certain than they really are. This, in turn, causes us to overestimate these costs, leading us to be overly cautious about deferring work. If it is your intention to bias the decision against postponement, this is clearly the best term to use. However, if you are trying to carefully weigh of pros and cons of postponing work, I’d recommend using a more neutral term like deferred work. In finance the concept of deferral is well understood. There are economic reasons for deferring revenues and expenses; managers are familiar with deferred assets and deferred liabilities. Calling it deferred work captures the economic essence of what we are doing without any positive or negative connotation.

Conclusion

To be perfectly clear, I am not suggesting that the debt metaphor never applies, only that it is less reliable in the domain of product development than some of its users think. There will be times when technical debt behaves exactly like financial debt and others times when the differences are enormous. Reasoning purely by metaphor may lead to bad economic choices – adding some math can lead to good ones.

In fact, there is another important advantage gained by enhancing your view of technical debt with a bit of economics: the people you are trying to influence care about the economic effects of their decisions. When you frame choices in economic terms you can communicate your ideas to them in their preferred language. Don’t underestimate how much effort and frustration this will save you.

Don Reinertsen


[1] Cost of Delay will be a new idea to some readers, but it is discussed in all of my books.

One thought on “Technical Debt: Adding Math to the Metaphor

  1. Pingback: Technical Debt: Adding Math to the Metaphor - R...

Comments are closed.