The Knowledge-centric Perspective

Abstract

We subscribe to the knowledge-centric perspective on software development, treating knowledge as the fuel that drives the software development engine. Central to this perspective is the concept of the 'knowledge gap' - the difference between what a developer knows and what they need to know to effectively complete tasks. This gap directly influences developer's' work experience, talent utilization and productivity.

Bridging this knowledge gap is a dynamic process, defined as the Knowledge Discovery Process. This process is about discovering and transforming invisible knowledge into visible, tangible output. It is like a 'black box': its internal mechanisms may not be fully transparent, but the results it produces are clearly observable.

The tangible output contains the knowledge gained, encapsulating new information acquired through the process. Conversely, the input represents the missing knowledge or information that needs to be discovered. Knowledge discovery begins by identifying the gaps in understanding, essentially tackling the question, 'What do we need to know?' This highlights the gaps in understanding or information that need to be filled. By pinpointing the unknown – the questions without answers – the knowledge discovery process seeks to bridge these gaps. The knowledge discovery process is complete when the missing information is found and applied to the task at hand, thereby closing the knowledge gap.

Critical to illuminating the impact of the knowledge gaps on the developer work experience is the concept of Flow, as defined by psychologist Mihaly Csikszentmihalyi. Flow is a state of heightened focus and immersion in activities, crucial for optimizing productivity and job satisfaction. Achieving Flow requires balancing the complexity of tasks with the individual capabilities of the developer. When individual capability significantly exceeds task complexity, the knowledge gap becomes too narrow, leading to wasted potential and boredom. On the other hand, if task complexity surpasses individual capability, the knowledge gap widens, leading to stress and low productivity. A well-balanced gap keeps developers in the Flow state, fully immersed and focused on their work, leading to higher productivity and job satisfaction.

The Knowledge-centric perspective is a systems perspective, because it takes into account all of the behaviors of a system as a whole in the context of its ecosystem in which it operates. This includes team dynamics, organizational knowledge resources, development tools, and the overall work environment. Understanding and optimizing these systemic elements are essential for facilitating effective knowledge discovery and helping developers maintain a state of Flow. This systemic view underlines the importance of considering the interconnectedness of individual, team, and organizational factors in enhancing developer productivity and work experience.

A crucial aspect of our approach involves measuring the knowledge gaps that developers bridge in completing tasks, quantified in bits of information. This measurement approach allows for meaningful comparisons across different contexts, programming languages, and applications. It provides a tangible metric to assess the effectiveness of knowledge acquisition and application in the software development process.

The knowledge perspective applies to all human activities, not just software development. Insights gained from studying the knowledge discovery process in software development offer valuable implications for other fields where cognitive effort plays a significant role. This is apparent in tasks like furniture assembly, which, while seemingly manual, involves significant knowledge work.

Understanding the Nature of Knowledge Work

Knowledge work involves the cognitive effort to close the gap between what is known and what needs to be known in order to effectively complete a task. It becomes necessary when there is an imbalance between required knowledge and prior knowledge.

Understanding this concept is crucial as different professions demand varying proportions of manual and knowledge work. While some jobs embed much of the required knowledge within the tools and procedures used, others necessitate continuous learning and adaptation to bridge the gap between existing skills and the demands of the task. This variability highlights how the same profession, when executed by people with different levels of prior knowledge, can require different proportions of knowledge work.

In software development, distinguishing between 'knowledge' work and 'manual' work is essential. Software developers engage in extensive knowledge acquisition for problem-solving and innovation, thus we may speculate that there is 95% of knowledge work and 5% manual work..

This is exemplified by Margaret Hamilton's contribution to the Apollo 11 program in 1969. In the picture below to the right we have Margaret being awarded the Presidential Medal of Freedom by President Obama.

On the picture to the left we have Margaret standing next to part of the computer code she and her team developed for the Moon landing. What we witness is an example of the manual part of the knowledge work. The software developers had to type all of those pages manually. All the knowledge went from their brains, through their fingers and ended up as symbols on sheets of paper. The difficult and time consuming activity in creating Moon landing software was not in typing their already-available knowledge into source code. It was in effectively acquiring knowledge they did not already have i.e. getting answers to the questions they already had. Even more specifically, it was in discovering the knowledge necessary to make the system work that they did not know they were missing. The symbols are the tangible output of their knowledge work. The symbols are not the knowledge itself, but the trace of the knowledge left on the paper.

Take a Systems Perspective

The Knowledge-centric perspective is a systems perspective, because it takes into account all of the behaviors of a system as a whole in the context of its ecosystem in which it operates. This includes team dynamics, organizational knowledge resources, development tools, and the overall work environment.

It's a common misconception to overestimate how much of the knowledge discovery efficiency is a function of the individual developers' skills and abilities and underestimate how much is determined by the system they operate in. Knowledge is a property of the organization the software developer operates in. This includes the knowledge of the developer, but also the knowledge of their teammates, the Product Owner, the architect, the QA, the support, the documentation available, the applicable knowledge in StackOverflow, and so on.

While it's true that designers, architects, and QAs may not produce code, they contribute significantly to helping developers produce code that meets customer expectations. Therefore, it's not recommended to measure the capability of individual developers. Instead, you should always measure the capability of an organization, whether that be a team, a project, or a company.

Organizational Knowledge Resources

A critical component of the Knowledge-centric perspective is the effective utilization and management of organizational knowledge resources[8]. These resources encompass the existing knowledge, expertise, and insights available within an organization. They include not only formal documentation like databases, manuals, and standard operating procedures but also the tacit knowledge that resides in the experience and skills of individual team members.

Organizational knowledge resources play a pivotal role in bridging the knowledge gaps identified in the software development process. They provide a rich reservoir of knowledge that can be tapped into for problem-solving, innovation, and learning. Effective management of these resources involves not just cataloging and storing knowledge but also ensuring it is accessible and relevant to the needs of developers. This includes fostering a culture of knowledge sharing, where insights and experiences are openly exchanged, and collaboration is encouraged.

Moreover, in the context of software development, these resources are not static. They evolve as projects progress, technologies change, and new challenges emerge. Therefore, maintaining up-to-date and dynamic knowledge repositories is crucial. Additionally, leveraging tools and platforms that facilitate knowledge sharing and collaboration, such as internal wikis, forums, or collaborative software, can significantly enhance the efficiency and effectiveness of the knowledge discovery process.

The integration of these organizational knowledge resources into the daily workflow of software development teams is a key factor in sustaining a productive and innovative work environment. It enables developers to quickly access the information they need, reducing the time spent in rediscovering existing solutions or reinventing the wheel.

Furthermore, by aligning these resources with the concept of Flow, organizations can create an environment where developers are consistently engaged, challenged, and productive, thereby optimizing the overall software development process.

Dependencies

A 'dependency' refers to the relationships and interconnections between different entities (individuals, groups, or organizations) that rely on each other for resources, information, or capabilities to produce tangible outputs. Understanding dependencies is crucial for optimizing value creation, enhancing collaboration, and managing risks.

Types of Dependencies:

• Resource Dependencies: Occur when one entity needs resources (e.g., data, materials, tools) from another to carry out its activities. For example, in software development, a team might need access to a testing environment to run tests.
• Knowledge Resource Dependencies: Occur when entities require information or knowledge from knowledge resources to make decisions or perform tasks.
• Capability Dependencies: Occur when one entity relies on the unique capabilities or expertise of another to achieve specific outcomes. For instance, a software development team might need the expertise of a DBA for specific tasks.

From a Knowledge-Centric perspective, we focus on the last two knowledge-related dependencies. These dependencies highlight how the knowledge discovery and outputs of one entity impact and are influenced by others within the organization.

Our dependencies differ from handoffs. Handoffs involve the transfer of something tangible from one entity to another (send or pick-up). This is the domain of inventory management, not knowledge work value creation.

From a knowledge-centric perspective Knowledge is in humans' minds. A handoff there means asking a question and waiting for an answer.

Dependencies involve relationships where one entity requires the output of another entity to perform their work. In knowledge work, the output is or contains knowledge, which is the key interest rather than its physical, tangible wrapping.

Thus, in knowledge work, dependencies are relationships between entities where one entity requires the knowledge of another entity to perform their tasks.

Dependencies illustrate how knowledge flows through the organizational network, identifying which entities have missing knowledge (i.e., questions to be answered) and which provide the answers. For example, a software development team relies on input from product managers and QA testers to verify if a functionality works as expected. It is important to note that QA testers depending on developers to provide code to test is a handoff because it views the code as something tangible.

Let's give examples. Imagine a software developer must deliver a feature that calculates the distance between two points on a map. To do this, the developer needs to know the formula for calculating the distance between two points. If the developer does not have this knowledge but can ask a knowledgeable colleague, the colleague becomes an information dependency. Alternatively, if the developer uses a library with this knowledge built-in, the library becomes a dependency, representing a capability dependency. If the library is still under development, the developer would have to wait for its completion, introducing a risk due to potential delays.

Dependencies in knowledge work are fundamental to understanding how different entities interact and rely on each other to create value. In software development, managing dependencies effectively is crucial for efficient operations, seamless collaboration, and risk mitigation. Identifying Critical Dependencies involves understanding which dependencies are critical for managing risks and ensuring that disruptions in one part of the network do not halt the entire value creation process. Developing strategies to mitigate risks associated with dependencies, such as having backup resources or alternative information channels, is essential.

By mapping and analyzing dependencies, organizations can optimize their value networks, enhance collaboration, and drive sustainable value creation.

Collaboration

In knowledge work, collaboration is fundamentally about learning. When two or more people collaborate, they learn from each other's knowledge, skills, and experiences.

The knowledge yet to be discovered can be acquired from two types of sources[10]:

• Personal Sources: Those that communicate information personally including insights, experiences, and skills of team members. When people collaborate, they share and combine their knowledge, skills, and experiences, leading to a more effective problem-solving process and achievement of shared goals. In this sense, collaboration amplifies individual contributions by creating a collective intelligence that exceeds the sum of its parts.
• Impersonal Sources: These are non-human sources that communicate information to broad audiences, including books, research papers, digital databases, API documentation, StackOverflow responses, coding libraries, and online audio and video. This type of information is vital as it provides a broader, often more objective viewpoint, and can provide data and insights that may not be available within the collaborating group.

In a well-functioning collaborative environment, both personal and impersonal sources of information are considered and used to acquire knowledge, solve problems, and make informed decisions. When looking at a knowledge discovery process, it is important to account for both these categories of sources. This means we account for all channels through which new information can be gained, thereby providing a comprehensive view of the organization's level of knowledge discovery efficiency.

Collaboration is a knowledge discovery process that acquires missing information from personal sources.

It is not possible for two or more people to type on the same keyboard simultaneously. Thus, when software developers collaborate, they build on each other's knowledge to accomplish something beyond their individual capabilities.

This is evident in practices such as pair programming and mob programming. Pair programming is a software development technique in which two programmers work together at one computer. One, the driver, writes code while the other, the observer or navigator, reviews each line of code as it is typed in. The two programmers switch roles frequently.[11]. Mob programming is a software development approach where the whole team works on the same thing, at the same time, in the same space, and on the same computer. With mob programming, collaboration is extended to everyone on the team, still using a single computer for writing the code and adding it to the source code repository.[12].

We can distinguish between instant availability (of collaborators) and non-instant availability (of cooperators). Collaboration accelerates progress by eliminating delays between people skilled in different information discovery activities.

We define the Level of Collaboration as the probability with which a person can acquire the missing knowledge from a personal source.

Introducing the Concept of Flow

The concept of Flow,, as coined by psychologist Mihaly Csikszentmihalyi, refers to a state of heightened focus and immersion in activities. People in a state of Flow lose their sense of time, feel in control of the task at hand, and experience an intrinsic sense of enjoyment and fulfillment[6].

To enter and maintain a flow state, the following conditions should be met[7]:

1. The opportunity for action, with a clear and attainable goal. That doesn't mean having an overall goal for the activity, but knowing what to do next from moment to moment
2. An action with a balance between skill level and challenge, meaning that the activity is neither too easy nor too difficult.
3. Clear and immediate feedback, so that successes and failures are apparent and behavior can be adjusted as needed.

The balance should be established for each task the developer will face. Once the conditions for action are in place, the developer is able to engage in a series of challenging tasks that are neither too difficult (not overwhelming) nor too easy (not boring). By balancing each task, we can create the conditions for entering flow for each set of tasks, each feature, and ultimately, for the scope of the project.

To achieve flow, one must find a balance between individual capability and work complexity, with a slight inclination towards challenges.

When the conditions for flow are met, they create a feedback loop between action and feedback that allows for continuous, effortless tuning of performance while taking action. This feedback loop makes an activity worth doing for its own sake[6].

A state of Flow is achieved when there is a balance between individual capability and work complexity, with a slight inclination towards challenges. This is measured by gauging the knowledge discovered after successful work completion.

Applying the Knowledge-centric Perspective Across Human Activities

The knowledge perspective applies to all human activities, not just software development.

Both manual and knowledge work result in outputs, outcomes, and impacts, but they differ significantly in their primary inputs. While manual work's primary input is muscle effort, software development thrives on cognitive effort. This becomes evident when we realize that in manual tasks, much of the necessary knowledge is already embedded in the machines and procedures. In contrast, software development demands continuous knowledge acquisition and application, necessitating a deeper level of problem-solving and innovation.

There is no requirement for the tangible output of a Knowledge Discovery process to be symbols of source code contributed via a keyboard. This is apparent in tasks like furniture assembly, which, while seemingly manual, involves significant knowledge work. This task requires reading instructions and learning to use tools to assemble parts into a tangible piece of furniture, such as tightening a screw. Each tightened screw represents a quantum of tangible output in this knowledge discovery process. Thus, knowledge work extends beyond typical IT jobs, encompassing any activity where learning and applying new knowledge are essential to produce a tangible result.

An application in assembling furniture

Let's imagine in front of us are two people who assemble a different model of IKEA furniture. We are told their names are Joe and Don. We are also informed that Don is a professional furniture assembler and Joe is a real novice assembling his first furniture. Our task is to tell who is Joe and who is Don just by looking at them assembling furniture.

Many of us have the frustrating experience of assembling a Billy. You know how the furniture arrived at your house, all neatly packed in a box? Yeah, that doesn't last long. Within minutes of opening the box, the contents seem to have multiplied. Suddenly, there's not enough space to stack all the pieces, and you're struggling to keep all those screws and fasteners in a neat, little pile. You pick up the instructions printed without words, using only simple illustrations, and suddenly the lack of text makes you feel extremely perplexed.

The first few steps are done and nothing can stop you. Except the next step you are doing it totally right but it's not fitting together. So you read, and then you re-read, and then you pick up the screws and try again. It's still not working. Then it's working! Nope, still not working. You hang up and look around. Your room is in shambles and random screws are rolling everywhere. Finally, the pieces screw in. It's coming together. And then you realize, on the very last step, that you're one screw short. For 10 minutes, you're on your hands and knees, going through the garbage, trying to locate the missing screw. And then you spot it. The last screw. Your key to freedom. You're sore and tired and your brain hurts, but you're done. You expected to have all your questions answered by the manual. And yet, you spent time asking yourself many more additional questions.

Having our past experience in mind while looking at the two men we can easily tell who is finding the manual informative enough, and who is asking himself many more additional questions. The former should be Don, because his training has provided him with all the knowledge needed to assemble furniture. The latter should be Joe, because he's spending a lot of time sitting on the floor with an allen wrench in one hand and the manual in the other.

In addition, can we tell Joe or Don even without looking at them while the men worked? Yes, because asking questions takes time. Prior knowledge takes less time to discover because it is in our heads. Hence, per unit time the professional should fix more screws, barrel bolts, and dowels(tacks) than a novice. Thus, by just counting at the screws two men fixed per unit time we can tell who's more efficient in discovering and applying their knowledge about assembling furniture.

An Application in Driving Cars

Joe is a novice driver, who has just received his driving licence. Joe has a total of 20 hours experience driving a car. Next to him is Don, who had received his driving licence 25 years ago. It is not clear how many hours Don spent driving a car, but it is in the thousands. Let's imagine Joe and Don take part in a car race. The race track features short straights that climb sharply uphill for a short time, then suddenly drop downhill and very fast left and right turns. In short - it is a challenging track. Drivers need to make only one lap - from Start to Finish. They are given the same car model to drive on the same racetrack - hence, they are on an equal footing. We are watching the race, but we are not aware which is Joe's car and which is Don's car. It is like a black box to us. Our goal is to tell which car is Joe's and which car is Don's. How can we tell? Well, if the track is the same and the cars are the same, then the best driver will finish first. Finishing first means less time to get from Start to Finish i.e. more meters per unit time.

But what factors into being the "best" driver in this scenario? It boils down to knowledge. He knows precisely when to engage the clutch, which gear to select for various sections of the track, and how to navigate turns efficiently. His extensive knowledge directly translates into a faster, more proficient driving performance.