Reduce technical debt

What is technical debt?

"...we accumulate the learnings we did about the application over time by modifying the program to look as if we had known what we were doing all along and to look as if it had been easy to do..." ~ WardCunningham[2]

“Any fool can write code that a computer can understand. Good programmers write code that humans can understand.” ~ Refactoring, Martin Fowler with Kent Beck, 1996

In software-intensive systems, technical debt is traditionally understood as a collection of design or implementation constructs that are expedient in the short term but create a technical context that can make future changes more costly or impossible[1]: It is when small changes take big efforts, and big changes require small miracles. But can we explore it from a different perspective? Let's consider technical debt through the lens of a knowledge-centric approach.

Ward Cunningham's seminal perspective on technical debt pertains to how we accumulate learnings about an application over time. The challenge lies in modifying the program to reflect as if we had known what we were doing all along and that it had been easy to do. This touches on the essence of what Martin Fowler and Kent Beck express in Refactoring - "Good programmers write code that humans can understand". They emphasize the human aspect - the need for code to be comprehensible not just to computers, but more importantly, to humans, including our future selves.

Technical debt, a term widely used in software development, refers to the implied cost of additional rework caused by choosing an easy or quick solution now instead of using a better approach that would take longer. From a knowledge-centric perspective, technical debt can be seen as the cost we pay when our initial decisions, due to lack of knowledge or deliberate strategy, lead to rework down the line.

Technical debt can be broadly categorized into two types: intentional and unintentional.

1. Intentional Technical Debt: This type of technical debt arises from deliberate decisions where short-term gains are prioritized over long-term code quality. A development team might choose to implement a quick but less optimal solution to meet a deadline or to quickly deliver a feature to the market. They know that the solution isn't the best in terms of maintainability or scalability, but they make a conscious decision to accept the technical debt and deal with it later. This is akin to taking out a loan knowingly, with a plan to repay it in the future.

1. Unintentional Technical Debt: This type of technical debt arises from a lack of knowledge or experience, resulting in suboptimal design or coding practices that later need to be corrected. It's like incurring a debt unknowingly, only to discover it later when the interest (in the form of rework) has compounded.

From a knowledge-centric perspective, both types of technical debt involve a replacement of existing knowledge — either the knowledge was insufficient at the time of the initial decision (unintentional), or the existing knowledge was deliberately set aside for short-term gains (intentional).

How to reduce the technical debt?

Managing and reducing the two types of technical debt might require different strategies. For intentional technical debt, it's about strategic planning and timely repayment. For unintentional technical debt, it's about improving knowledge acquisition and application processes to make better decisions in the first place. In both cases, the goal is to minimize the waste or rework necessary to deliver the desired software, thereby "maximizing the work not done".

This human-centric view lends itself well to understanding the nature of 'waste' and 'technical debt' in software development. Typically, 'waste' might refer to unnecessary features. However, when viewed from a knowledge-centric standpoint — where software development operates on knowledge much like a car runs on fuel — the definition of 'waste' evolves. We define 'waste' as a situation where existing knowledge (akin to laid bricks) is replaced with new information. It's the "death by a thousand cuts", where each cut is a piece of knowledge replaced.

This waste is primarily due to rework, which comes in two forms:

1. Changes to 'What' should be built, involving altering or removing features, which, in turn, changes the code and the user-facing functionality.
2. Changes to 'How' to build the 'What,' otherwise known as refactoring. This involves altering the code without changing the user-facing functionality.

Rework, thus defined, is distinct from design iterations intended for rapid learning, where design decisions are viewed as experimental and subject to change. It's also different from fast project cycles designed to accelerate customer feedback.

Therefore, the concept of 'technical debt', when viewed through a knowledge-centric lens, becomes a measure of the efficiency of our knowledge discovery and application process. It's about reducing the 'waste' or rework necessary to deliver the desired software. To put it another way, "minimizing waste" equates to "maximizing the work not done". Technical debt, then, is the cost we pay when our initial decisions, due to lack of knowledge, lead to wasteful rework down the line. The less we need to refactor, the more efficient we are, and the less 'debt' we accumulate. But to refector we need because of the very nature of software development!

Adding to the understanding of technical debt from a knowledge-centric perspective, it becomes clear that managing technical debt effectively often involves balancing between iterative and incremental development.

In software development, if we are unable to safely reiterate the existing design (refactor), then incrementing (adding functionality) may lead to the degradation of code and overall progress. Everything becomes progressively worse — it's slower, bug-ridden, complicated, unattractive, unpredictable — a host of issues arise. The takeaway is clear: there is no free lunch in the world of software development. If you can't iterate, you should avoid incrementing as well. If you do choose to increment, you must be prepared to iterate.

Applying a knowledge-centric view to this challenge brings further clarity. Typically, in the software development landscape, 'incremental' and 'iterative' refer to adding new features and refining existing ones, respectively. But from a knowledge discovery standpoint, these terms acquire a different nuance. 'Incremental' can be defined as a process where new knowledge is added without discarding existing knowledge — akin to laying bricks without removing any. Conversely, 'iterative' involves replacing existing knowledge with new information.

So, in terms of knowledge discovery, 'incremental' development involves consistently adding to the existing knowledge base, while 'iterative' development signifies the continuous cycle of refreshing knowledge, replacing old with new. This balancing act between iterative and incremental development is crucial in managing and minimizing technical debt, thereby maximizing the efficiency of the knowledge discovery and application process.