How to Run Predictable Projects

Abstract

In today's fast-paced and ever-evolving software development landscape, predictability stands as a cornerstone for successful project management. Predictable software development projects are essential for meeting delivery deadlines, setting realistic expectations, and maintaining stakeholder trust.

However, achieving predictability in such a dynamic environment is no small feat. It requires a thorough understanding of the project’s intricacies and a proactive approach to managing the various challenges that arise throughout the development lifecycle.

The essence of project management in software development lies in navigating the unknown. At the outset, many aspects of a project's outcome and the work required to achieve it remain undefined and uncertain. Acknowledging and effectively addressing knowledge gaps is crucial for successful project management and ensuring predictable delivery.

According to the Knowledge-Centric perspective on software development, knowledge serves as the primary resource in software development, driving the entire process from ideation to delivery. Effective management of knowledge acquisition, application, and transfer directly influences the predictability and success of a project.

The article discusses the dual nature of knowledge gaps—as hurdles to be overcome and as catalysts for innovation—emphasizing the need for a proactive and adaptive mindset. By distinguishing between different categories of knowledge gaps (known unknowns, unknown unknowns, and unknown knowns), and presenting a structured process for managing knowledge discovery, the article provides practical steps to enhance project predictability.

This process includes:

1. Understand your project's complexity
2. Staff the team with the talent needed to fill most of the knowledge gaps
3. Establish a Predictability Benchmark
4. Continuously fill the remaining knowledge gaps
5. Monitor progress and adapt strategies as needed

Through this iterative approach, teams can effectively manage uncertainties, solve complex problems, and achieve project goals efficiently.

By the end of this article, readers will have a comprehensive understanding of how to create and maintain a predictable software development environment, ensuring that our projects remain on track and deliver the expected outcomes.

Managing Knowledge Gaps for Predictable Delivery

"All software is simply a representation of knowledge and it is the acquisition of this knowledge that is the real activity of software development." ~ Phillip Glen Armour [1]

The very definition of a project implies a journey into the unknown. At the outset, many aspects of the project's outcome and the work required to achieve it remain undefined and uncertain. Acknowledging that any project will inevitably face knowledge gaps is crucial for effective project management. Addressing these gaps effectively is the largest part of project management and is key to ensuring predictable delivery in software development projects.

Unlike physical resources, knowledge is both the tool and the product of project management. The effectiveness with which a team acquires, applies, and transfers knowledge directly influences the predictability and success of a project. By effectively managing knowledge—understanding what is known, identifying what needs to be learned, and applying this evolving understanding—teams can navigate uncertainties, solve complex problems, and achieve project goals efficiently. Thus, knowledge is not merely an asset but the fundamental driver of project progress and success.

Central to this perspective is understanding the knowledge gap between what a developer knows and what they need to know to effectively complete tasks. Bridging this gap involves discovering and applying new knowledge.

Knowledge gaps can arise because the project team may lack the necessary knowledge and expertise required to address specific technical challenges or industry-specific requirements.

However, it's important to recognize that knowledge gaps also frequently represent opportunities for innovation, where the team must develop novel solutions to meet both business objectives and technological advancements.

Embracing this dual nature of knowledge gaps—as both hurdles to be overcome and as catalysts for creative problem-solving—enables teams to navigate the complexities of software development with a proactive and adaptive mindset.

We can distinguish between three categories of knowledge gaps i.e. “unknowns”:

• Known Unknowns: Gaps in our knowledge that we recognize and actively seek answers to; we are aware of the questions we need answered
• Unknown Unknowns: Requirements and technological challenges we are unaware of; we are oblivious to both the questions and their answers.
• Unknown Knowns: Subconscious or tacit knowledge—what we know but may not recognize we do; we are acquainted with the answers but not necessarily the questions they address.

Once one is aware that each project contains “unknown unknowns”, it becomes clear that a process is needed to reveal these “unknown unknowns” and turn them either into things we know or things of which we now know that we don’t know. We call this process “knowledge discovery”.

In the context of software development, "knowledge discovery" refers to the set of activities and methods that a software development organization uses to acquire, understand, and apply the necessary knowledge throughout the software development lifecycle.

Project complexity refers to the amount of knowledge that needs to be discovered, i.e., the missing information acquired by the team throughout the project lifecycle for the successful completion of the project.

The missing information can be broadly categorized into two types:

1. WHAT needs to be done (the objectives or requirements)
2. HOW to accomplish the WHAT (the processes, methods, and techniques to fulfill these requirements).

The difficulty of discovering “unknown unknowns” and closing the knowledge gaps lies in the fact that we are unaware of these unknowns.

It is clear that a process is needed to:

1. Show us what we know, what we don't know, and what we don't know we don't know
2. Turn the unknowns into things we know or things of which we now know that we don’t know

Here is a process that will help manage knowledge discovery in a project:

1. Understand your project's complexity
2. Staff the team with the talent needed to fill most of the knowledge gaps
3. Establish a Predictability Benchmark
4. Continuously fill the remaining knowledge gaps
5. Monitor progress and adapt strategies as needed

This iterative process aims to acquire new and missing knowledge with respect to the project, through experiments and training.

Understand your project's complexity

The traditional view is that project complexity refers to the degree of difficulty, uncertainty, and interdependence involved in managing and completing a project[3][4].

But what does "difficulty" mean in the context of software development projects? Does it imply developers face challenges while typing?

This is where the knowledge-centric approach to software project estimation and management comes in, which focuses on the acquisition of knowledge as the primary activity of software development[8].

Project complexity refers to the amount of knowledge that needs to be discovered, i.e., the missing information acquired by the team throughout the project lifecycle for the successful completion of the project.

The project is a system composed of the people who will deliver (Capability) and the people who define what to be delivered (Client). The Client is responsible for defining "what to do," while the Capability is responsible for defining "how to do the what."

The complexity of a project can be influenced by both Client and Capability. We group the factors into two clusters based on their ownership and influence by either the Client or the project team (Capability). Here's the breakdown:

Client-owned factors influencing project complexity:

• Requirements complexity: The number, degree of uncertainty, and intricacy of project requirements, as well as their potential for change, can impact the knowledge discovery process. In simpler terms, this factor asks whether the client has a clear understanding of what needs to be done. Projects with numerous or complex requirements, or with clients who are less certain about their desired outcomes, may necessitate more knowledge discovery and learning.
• Client collaboration: In addition to the internal characteristics of projects, it is important to consider the contexts in which projects are executed. The performance of the same project team may vary depending on whether the client is a startup or a Fortune 500 corporation, as the level of collaboration with stakeholders can differ significantly. For example, working with a small startup typically implies that questions are answered promptly and communication is more agile. On the other hand, working with a large organization like a bank often involves slower communication and more bureaucratic processes, which can impact the project's overall efficiency and progress.

Capability-owned factors influencing project complexity:

• Technology expertise: This includes the intricacy of the algorithms, technologies, tools, and frameworks used in the project, as well as the overall architectural design. A project with more advanced or unfamiliar technologies will require more knowledge discovery and learning.
• Codebase knowledge: This factor enables project managers to assess the potential risks, challenges, and dependencies associated with building upon an existing codebase or starting from scratch. It plays a significant role in determining project complexity, particularly when the team needs to familiarize themselves with an existing codebase. The time and effort required to understand the existing code, its architecture, and design patterns can slow down the project's progress, especially in the initial stages. Furthermore, dealing with an existing codebase may present additional challenges, such as technical debt, outdated dependencies, or a lack of proper documentation. These challenges can increase project complexity and affect overall project estimation and management processes. On the other hand, starting from scratch might necessitate more time for initial setup, architecture design, and development, but it also provides an opportunity to leverage the team's prior knowledge and expertise, potentially streamlining the development process.
• Domain Knowledge: Every software project operates within a specific domain, whether it's healthcare, finance, retail, or any other industry. Domain knowledge encompasses understanding the unique requirements, constraints, and best practices of that industry. Acquiring this knowledge can be particularly challenging for teams working in unfamiliar domains. The lack of domain expertise can lead to misunderstandings, misaligned priorities, and ultimately, a product that does not meet user needs or regulatory standards.

  The problem domain or industry the software is being developed for can influence the amount of knowledge that needs to be acquired. For example, a project in the healthcare industry may involve understanding specific medical terminology, regulations, and processes, which adds to the complexity.

• Team collaboration: The individual knowledge, skills, and expertise of team members, as well as the interdependence between teams or groups possessing specific information required for a project to progress, can influence the amount of knowledge that needs to be acquired during a project. For example, a project staffed primarily with junior developers will likely require more knowledge discovery and learning than a project with a more balanced mix of junior and senior developers. Having an experienced architect on the team can provide a valuable knowledge base and guidance, enabling developers to access the information they need more quickly. Furthermore, effective collaboration and communication between different teams or groups are essential for sharing vital information and addressing project challenges efficiently. Additionally, having dedicated QA team members can help ensure that any incorrect knowledge acquired and applied is identified and addressed more rapidly. This faster feedback loop can prevent the propagation of issues stemming from misunderstandings or lack of knowledge, ultimately improving the project's overall efficiency and quality. In this context, the interdependence among team members and groups plays a crucial role in managing project complexity and ensuring successful outcomes.
• Development process: The way the discovery of missing information is managed by the project team. When it comes to discovering missing information, Agile and Waterfall development processes handle this aspect differently:
  • Agile methodologies embrace change and encourage continuous learning and adaptation throughout the project. The iterative nature of Agile development enables teams to discover missing information and adjust their plans accordingly as the project progresses. By breaking the project down into smaller, manageable tasks, teams can identify gaps in their knowledge and understanding more quickly and address them before they become major issues. Frequent communication and collaboration among team members and stakeholders further facilitate the identification and resolution of missing information.
  • In the Waterfall development process, the discovery of missing information can be more challenging due to its linear and structured nature. The process relies heavily on upfront planning, with the assumption that all requirements and information are gathered and documented before the project begins. As a result, discovering missing information later in the development process can lead to delays, increased costs, and the need for significant rework. Teams following the Waterfall process may find it more difficult to adapt to change and acquire missing information, as doing so may require revisiting earlier stages in the project lifecycle.

It's important to note that while these factors can be categorized based on ownership and influence, there will still be some degree of interdependence between them. Collaboration and communication between the client and the project team (Capability) are crucial in managing project complexity effectively.

In this article we explore how to estimate project complexity using a knowledge-centric approach.

Staff the team with the talent needed to fill most of the knowledge gaps

Understanding your project's complexity would show the expertise you need to acquire certain knowledge areas. Each of these areas presents unique challenges that must be addressed to ensure project predictability.

In general, the team should include professionals with strong technological proficiency, who stay updated with the latest technologies, programming languages, and frameworks, through regular training and hands-on experimentation. Domain experts are crucial for understanding industry-specific requirements and constraints, ensuring the project aligns with industry standards. Business analysts or stakeholders with business acumen are needed to align the project with business goals, target audience needs, and market dynamics. Integration specialists are essential for seamlessly integrating new solutions with existing systems, maintaining compatibility and data integrity. Compliance experts are necessary to navigate legal and regulatory requirements, conducting regular audits and ensuring adherence to standards. Finally, security professionals are vital for implementing robust security measures, performing regular audits, and staying ahead of potential threats. By assembling a team with these diverse skills, project managers can effectively address knowledge gaps and enhance the predictability of software development projects.

We need different people to work on different knowledge gaps. The difference is the level of perplexity they can handle both from Capability and Client perspectives. For instance:

• If the problem is clear then we need a junior person, who can read a recipe book and follow it. She will need stable requirements and all the exact tools and technologies from the book.
• If the problem is complex we need a senior person, who can cope without a recipe, with changing requirements and with whatever is available in terms of tools and technologies.

When we need to adapt we need more diversity in the system because then we've got more evolutionary capacity. It's not a coincidence that for the living organisms the mutation rates increase under stress - more diversity more, adaptability. The requisite diversity is the optimal level of diversity for the context we are in. Diversity means inefficiency an organization needs to have. If overfocused on efficiency the organization loses adaptive capacity.

Diversity can help us answer the question posed by Brooks' Law[2]. The question is, "if I have a project that is running late, should I add people to it to speed it up?" Our answer is, "it depends on how you manage knowledge discovery." If the project scope is partitioned into work items classified using the Complexity Profile explained here, then you'll have Clear and Complicated work that can be given to the newcomers. That will free up your present developers to do more Complex work clarifying. Then the answer is "yes." The answer would be "No" if the addition of people will require that your current staff take time away from being productive to instruct the newcomers, resulting in a net drop in productivity. Hence, Brooks' Law: adding manpower to a late project makes it later.

Establish a Predictability Benchmark

A critical aspect of managing knowledge gaps is establishing a predictability benchmark. Since the main challenges are related to lack of knowledge, by observing how the current project's team acquires knowledge compared to teams from past projects, managers can identify early signs of potential issues.

If a team is not acquiring knowledge at an expected rate, it may indicate underlying problems within the project. This insight allows managers to investigate and address the root causes proactively. Conversely, if a team acquires the necessary knowledge at the expected rate, the process is considered predictable.

By continuously benchmarking and monitoring knowledge acquisition, managers can maintain a clear understanding of the project's trajectory and take timely corrective actions to ensure consistent progress and successful outcomes.

Implementation Strategy for Learning Curve Analysis

1. Identify a Suitable Reference Class: Select past projects that are similar to the current project to serve as a reference class. When determining similarity, consider factors that impact knowledge acquisition and project complexity, such as gaps in understanding, client type, team structure, and business domain expertise.
2. Historical Data: Collect historical data from past projects, including the knowledge discovered at each time point. For each project, calculate the cumulative growth rate of knowledge discovered over time. Compare the learning curves for all projects in the reference class using statistical methods and visual inspection. Employ Dynamic Time Warping (DTW) as the primary statistical method. DTW measures the similarity between two time series by minimizing the distance between their points, allowing for non-linear time alignment. A smaller DTW distance between two curves indicates a higher degree of similarity. This method is particularly beneficial when the curves have similar shapes but are misaligned in time or exhibit different progression speeds.
3. Establish Baseline: Examine the learning curves of the reference projects to gain insights into the learning and problem-solving activities likely to occur in your project. This analysis will provide a baseline learning curve, which will be used to anticipate the expected learning trajectory of the new project.

Continuously fill the remaining knowledge gaps

Focus on identifying and addressing key technology knowledge gaps that will provide significant value to the project stakeholders. For the rest of the technology stack, choose well-known, reliable components to ensure that the total knowledge gap for the project remains narrow and manageable..

Understanding your project's complexity would also shine a bright, glaring light on what people did not know. This could be quite uncomfortable for many and they could respond as many of us would do, by avoiding it altogether and working on something on which they could "make progress," on something they knew.

For most of us, given a straight choice of working on something that is easy, something that plays to our strengths and demonstrates how competent we are, and working on something that is hard, something we have no clue about, and ultimately shows us and everyone else just how little we understand about the system, we are probably going to choose the former[1]. This is not only a natural, human tendency; it is also exacerbated by some of the pressures we experience in development, the desire to demonstrate "progress" and most of all by the organization's and management's expectation of producing a product1].

The bottom line is always that processes do not do the work, people do. Not only do they come with their own proprietary thought processes and experience, they also come with a host of other characteristics: motivation, ethical basis, fortitude, helpfulness, resourcefulness, rapidity of thought, capacity to give and receive feedback, etc. These characteristics may be skills or they may be capabilities. These are not the same thing. People are created differently and through application of their skills, through life experience, and through their inclinations, end up with different capabilities. But the source of the problem is not in differences. In fact, it is the differences rather than the similarities that make project teams effective[1].

In order to successfully deliver the project we need to bridge all knowledge gaps. Since the team members would do that , their expertise will grow as a result.

Monitor predictability and adapt strategies as needed

As the new project progresses, benchmark the actual learning curve against the baseline learning curve from similar past projects. This process, involving the same statistical methods (DTW) and visual inspection used in reference class establishment, allows for early identification of deviations from expected learning rate. Such benchmarking is crucial for recognizing and addressing potential issues well before the testing and deployment phases of the new features.

Benchmarking the learning curve is key to early identification of potential issues and their root causes in areas where the learning rate lags behind expectations.

Visualizing the learning curve is a crucial step in understanding the dynamics of knowledge acquisition within a project. This visualization typically involves plotting the rate of knowledge acquisition over time, which can highlight various patterns:

• Rapid Learning Phases: Periods where knowledge acquisition is accelerated.
• Plateaus: Times when the rate of learning stabilizes.
• Declines in Learning Rates: Phases where the pace of knowledge acquisition slows down.

The below diagram provides a comparative view of learning curves.

Each line traces the cumulative growth rate for its respective project over the specified period.

• Reference Class Curves (in Blue): These curves represent the learning trajectories of similar past projects. They serve as a benchmark for expected knowledge growth.
• New Project Curve (in Red): This curve illustrates the learning rate of the current project.
• Axes Interpretation: The x-axis of the diagram represents the timeline, measured in weeks, while the y-axis indicates the cumulative growth rate (CR).
• Dates: Each learning curve starts at the project's inception and continues to the specified end date.

In this example, we observe that the CR of the new project is not aligning with the trend seen in the reference class. This discrepancy suggests potential areas within the new project that may require closer examination and intervention.

In the above we can visually check the distance between the learning curves of the new project and the reference class. It is handy to use an index for predictability index \(P\) with a bounded scale between 0 and 100, which is a measure of how closely the new project aligns with the reference class. In the diagram below we have presented a probability index between two of our reference projects and the new project.

On the x-axis we have weekly dates from the start of the new project. On the y-axis we have the predictability index \(P\) which ranges from 0 to 100. A predictability index of 100 indicates that the new project is perfectly aligned with the reference class, while a predictability index of 0 suggests that the new project is significantly diverging from the expected learning trajectory. By monitoring the predictability index over time, project managers can identify early signs of potential issues and take corrective actions to realign the project with the expected learning trajectory.

The diagram shows the probability index for both reference projects to be much less than 100. This indicates that the new project is not aligning with the reference class as expected. We can say the new project is unpredictable and requires immediate attention to realign it with the expected learning trajectory.

Proactive Interventions for Maintaining Predictability

Proactive interventions are essential to maintain predictability in software development projects. By addressing knowledge gaps and optimizing team performance through targeted strategies, project managers can ensure a smoother and more predictable project trajectory.

Targeted Training

The first step in targeted training is to identify specific knowledge gaps within the team. This can be achieved through regular skills assessments, feedback sessions, and performance reviews. By pinpointing areas where team members lack proficiency, managers can design training programs that directly address these gaps.

Once the knowledge gaps are identified, it's crucial to develop tailored training programs that focus on these areas. Customized workshops, online courses, and hands-on sessions can help team members quickly acquire the necessary skills and knowledge. Tailored training ensures that learning is relevant and immediately applicable, enhancing the team's overall capability.

Continuous Learning and Documentation

Create a comprehensive repository of domain knowledge, including industry standards, best practices, and key insights. Regular training sessions and knowledge-sharing forums can keep the team informed about the latest developments in the domain.

Close Collaboration with Stakeholders

Foster effective collaboration with business stakeholders to ensure that the development team fully understands the project's goals and objectives. Regular meetings, feedback sessions, and collaborative planning help align technical efforts with business needs.

Business-Oriented Workshops

Conduct workshops focused on business goals, market trends, and customer needs to enhance the team's understanding of the broader business context. This alignment helps in creating a product that meets both technical specifications and business objectives.

Addressing Mismatch Between Capabilities and Challenges

To address mismatches between team capabilities and project challenges, project managers should reallocate developers based on their strengths and expertise. By assigning tasks that align with individual skill sets, teams can work more efficiently and effectively, reducing the risk of errors and delays.

Ensuring a balanced mix of skills within the team is vital for tackling diverse project requirements. Regularly reviewing team composition and making necessary adjustments can help maintain a well-rounded skill set. Cross-training and encouraging skill diversification can also enhance team versatility and resilience.

Collaboration Improvement

Improving collaboration within the team can significantly enhance knowledge transfer and sharing. Implementing practices such as regular knowledge-sharing sessions, peer reviews, and collaborative problem-solving can foster a culture of continuous learning and mutual support.

Utilizing effective communication tools is essential for facilitating seamless collaboration. Tools like project management software, instant messaging platforms, and shared document repositories can streamline communication, ensure transparency, and keep everyone aligned on project goals and progress.

Comprehensive System Documentation

Ensure detailed documentation of existing systems to facilitate successful integration with new solutions. This should cover system architecture, data flows, interfaces, and dependencies.

Use of Integration Platforms and Tools

Leverage specialized integration platforms and tools to simplify the process of connecting old and new systems, improving data consistency, and enhancing overall system performance.

Modular Design Principles

Adopt modular design principles to allow for the addition of new functionalities without disrupting existing systems. This approach minimizes the risk of introducing bugs and ensures system stability.

Incremental Development Approaches

Utilize incremental development methods, such as Agile, to introduce new functionalities gradually. This allows for continuous feedback and improvement, reducing the risk of large-scale failures.

Prototyping and Experimentation

Encourage prototyping and experimentation to explore unclear technologies without committing significant resources. Prototypes help test feasibility and compatibility, providing valuable insights for informed decision-making.

Regular Compliance Audits

Conduct regular compliance audits to ensure adherence to relevant legal and regulatory requirements. Early identification of potential issues allows for timely corrective actions.

Training on Legal Standards

Provide ongoing training on legal and regulatory standards to keep team members informed about their responsibilities. This includes understanding data protection laws and industry-specific regulations.

Regular Security Training

Offer regular workshops and training sessions on secure coding, threat modeling, and incident response. Keeping security top of mind helps in building robust systems.

Security Best Practices and Audits

Implement and adhere to security best practices, including regular security audits. These audits help identify vulnerabilities and ensure compliance with security standards, protecting software from emerging threats.

Shortening Feedback Loops

Shortening feedback loops by incorporating frequent iterations and reviews helps teams identify and address issues promptly. Agile methodologies, with their emphasis on short development cycles and regular check-ins, enable teams to gather feedback, make adjustments, and continuously improve.

Establishing rapid problem-solving mechanisms, such as daily stand-up meetings and quick decision-making protocols, allows teams to tackle obstacles as soon as they arise. This proactive approach prevents minor issues from escalating into significant problems, maintaining project momentum.

Reducing Rework

Identifying knowledge gaps early in the project lifecycle can prevent costly rework later. Regularly reviewing project requirements, conducting thorough planning sessions, and involving key stakeholders from the beginning can help uncover potential gaps and address them before they impact the project.

Robust testing and validation processes are critical for reducing rework. Continuous integration and continuous delivery (CI/CD) practices, automated testing, and code reviews ensure that issues are detected and resolved promptly, maintaining high-quality standards and minimizing the need for rework.

Reducing Multitasking

In software development, avoiding multitasking isn't about having just one task in your hands—it's about having one task in your mind.

Creating focused work environments where team members can concentrate on one task at a time improves productivity and knowledge retention. Minimizing distractions, setting clear priorities, and establishing dedicated time blocks for specific tasks can enhance focus and efficiency.

Prioritizing tasks based on their importance and complexity helps in managing workloads effectively. By tackling high-priority tasks first and breaking down complex tasks into manageable steps, teams can improve their understanding and retention of knowledge, leading to more predictable project outcomes.

Implementing these proactive interventions can help maintain predictability in software development projects. By addressing knowledge gaps, optimizing team performance, and fostering a collaborative and focused work environment, project managers can ensure that their projects stay on track and deliver successful results.

In summary, by addressing these knowledge gaps through targeted strategies, project managers can enhance the predictability of software development projects. By focusing on these areas, project managers can proactively manage potential issues, ensure smooth project progression, deliver high-quality software that meets both technical and business objectives, and meet project deadlines consistently.

Conclusion

In this article, we have explored the importance of predictability in software development projects and how a knowledge-centric perspective can help achieve it.

The core of achieving predictability in software development lies in adopting a knowledge-centric perspective. Recognizing that knowledge gaps are an inherent part of any project, and addressing them proactively, ensures that teams are well-equipped to handle both expected and unforeseen challenges. By continuously enhancing the team's knowledge base and leveraging the learning curve as a benchmark, project managers can maintain a clear view of the project’s trajectory and make informed decisions to keep it on track.

In conclusion, running a predictable software development project requires a strategic focus on knowledge discovery management. By identifying and addressing knowledge gaps early, fostering a culture of continuous learning, and implementing proactive interventions, project managers can significantly enhance the predictability and success of their projects. We recommend regular skills assessments, tailored training programs, and close collaboration with stakeholders to ensure alignment with business goals.

By embracing a knowledge-centric perspective, project managers can navigate the complexities of software development with confidence and deliver high-quality software on time and within budget.