How to measure Capability in Software Development
A Knowledge-Centric Approach
Abstract
In the context of Quality Control, process capability is defined as a statistical measure of the inherent process variability of a given characteristic. A "characteristic" refers to any measurable feature of a product or process. The aim is to ensure that the process is capable enough to keep the variations of these characteristics within acceptable limits, thereby assuring the quality of the output.
For quality control purposes in manufacturing or similar processes, characteristics are typically physical or performance measures that can be precisely measured and statistically analyzed. In knowledge work, however, we adapt this concept to more abstract aspects. We use "Knowledge Discovery Efficiency" (KEDE), which quantifies the balance between individual capability and work complexity.
Thanks to KEDEHub, a specialized platform for assessing software development processes, we have a reliable and repeatable method to quantify KEDE. This approach enables organizations to apply statistical tools to assess how well their processes balance the capabilities of individuals and teams in relation to work complexity.
What is Capability?
There has been a lack of precision in the way capability and performance are applied and measured. Moreover, the terms capacity and capability have been used interchangeably. Thus, to clarify the terminology it is important to distinguish the three constructs of activity: capacity, capability, and performance. As most scientific knowledge on people and organizations have their roots in core behavioral sciences we will adopt their definitions.
Capacity describes what a person can do in a standardized, controlled environment. Capability describes what a person can do in a specific environment. Performance describes what a person actually does in a specific environment. The person-environment interaction is the discriminating element between capacity, capability, and performance[3].
When assessing the person's performance, both the physical and social environment play a role. There are factors in the context that impact how people with otherwise equal capacities actually perform. For two people of equal capacity, their performance may be different in different contexts. In view of this, we expect discrepancies between a person's capacity, capability, and performance.
Work comprises cognitive and manual labor. Manual labor uses physical effort to produce tangible results, while cognitive labor addresses the essential questions of "what," "how," and "why" a task is performed[2]:
- What needs to be done?
- How should it be done?
- Why is it being done?
Each of these questions can be broken down into more detailed questions in a fractal manner to infinitely small questions. Eventually, when these questions are answered, the person can proceed with manual labor. Importantly, the muscles are commanded by the brain, thus the cognitive labor always precedes the manual labor for a task.
For each of the questions, there is a required level of knowledge and prior level of knowledge. The Required Knowledge is the complexity of the task at hand i.e., what needs to be known. Prior Knowledge is the existing skills and knowledge an individual possesses i.e., what is known. The gap between required and prior knowledge is the Knowledge to Be Discovered - what the person thinks they don't know, given what they think they do know.
What is the problem with quantifying capability of an organization? In one word - measurement. Indeed, it is very difficult to measure the capability for two reasons. Firstly, because knowledge is invisible. It is in people's brains, in books, in manuals, in working procedures, in files on hard disks, in short - everywhere. When the organization invests in acquiring new talent or in training its current people it is a challenge to measure what they have got for the money.
Secondly, because acquiring knowledge is not enough. Attention must be paid if the knowledge is actually applied? If knowledge is acquired but not applied then there is no capability. Capability exists when applied to create economic value. Capability only adds value if it can be successfully converted into goods and services that will make a profit[4]. To increase the capability of knowledge workers means to increase the efficiency in acquisition and application of knowledge in their organizations.
If the knowledge is acquired but not fully utilized then the knowledge workers leave potential off the table.
How can you know if human potential is untapped? How do you know there is potential in the first place? How could we measure it? You have no way to get into people's heads and see the potential inside! Truth is you know the potential was really there only after you see knowledge applied.
We know there was knowledge discovered when we see at least one question asked. A Knowledge Discovery Process consists of asking questions in order to acquire the missing information about "what" tangible output to produce and how to produce it.
In the context of Quality Control, process capability is defined as a statistical measure of the inherent process variability of a given characteristic. A "characteristic" refers to any measurable feature of a product or process. The aim is to ensure that the process is capable enough to keep the variations of these characteristics within acceptable limits, thereby assuring the quality of the output.
According to our definition capability is not a static thing. Capability is a dynamic entity resulting from human potential, impersonal sources of information, collaboration and efficient work processes. It fluctuates and we need to manage it as such.
How to measure Capability?
For quality control purposes in manufacturing or similar processes, characteristics are typically physical or performance measures that can be precisely measured and statistically analyzed. In knowledge work, however, we adapt this concept to more abstract aspects. We use "Knowledge Discovery Efficiency" (KEDE), which quantifies the balance between individual capability and work complexity.
Thus, in order to measure capability we need to measure the Knowledge Discovery process that fuels it.
Fortunately, there is a way to measure the knowledge discovered (missing information) in software development. KEDEHub allows you to measure the capability of your capability using the scientifically backed and patented metric - Knowledge Discovery Efficiency (KEDE). It is pronounced [kiːd].
KEDE is a measure of how much of the required knowledge for completing tasks is covered by the prior knowledge.
Prior knowledge is the easiest and the fastest to discover - it is there, one just applies it. In other words, when prior knowledge is applied then there is the most efficient knowledge discovery. Conversely, when a lot of knowledge is missing then the knowledge discovery is less efficient. The more prior knowledge was applied i.e. the less knowledge was missing the more efficient a Knowledge Discovery process is.
KEDE can be used for comparisons between capability in different contexts.
Below is a time-series diagram of capability for the whole life of a crypto company.
On the diagram above we have presented Knowledge Discovery Efficiency (KEDE) for each of the developers who contributed to the capability. On the x-axis we have the quarter dates. On the y-axis we have Weekly KEDE values. The dark blue line is the average weekly KEDE for all developers calculated by EWMA.
A learning organization is skilled at creating, acquiring, and transferring knowledge, and at modifying its behavior to reflect new knowledge and insights[2]. It is responsive, understanding the need to keep up on things and to pivot from time to time[1]. How adaptive an organization is depends on its capability. Being a learning organization enables agility, adaptability, resiliency and the ability to meet the evolving needs of organization's stakeholders and the marketplace as a whole. The knowledge, skills and practices are held by people. Leaders must be capable of organizing individuals into a learning organization. A learning organization starts with the CEO and must be enabled through HR processes.
Works Cited
1. Senge, P.M. 1990: The Fifth Discipline , Random House, 1990
2. Garvin DA. Building a learning organization. Harv Bus Rev. 1993 Jul-Aug;71(4):78-91.
3. Holsbeeke, L., Ketelaar, M., Schoemaker, M. M., & Gorter, J. W. (2009). Capacity, Capability, and Performance: Different Constructs or Three of a Kind? Archives of Physical Medicine and Rehabilitation, 90(5), 849–855. https://doi.org/10.1016/j.apmr.2008.11.015
How to cite:
Bakardzhiev D.V. (2022) How to measure capability in software development? https://docs.kedehub.io/kede/kede-capability.html
Getting started
