The Knowledge-centric Perspective
On software development
Abstract
We subscribe to the knowledge-centric perspective on software development, treating knowledge as the fuel that drives the software development engine. Central to this perspective is the concept of the 'knowledge gap' - the difference between what a developer knows and what they need to know to effectively complete tasks. This gap directly influences developer's' work experience, talent utilization and productivity.
Bridging this knowledge gap is a dynamic process, defined as the Knowledge Discovery Process. This process is about discovering and transforming invisible knowledge into visible, tangible output. It is like a 'black box': its internal mechanisms may not be fully transparent, but the results it produces are clearly observable.
The tangible output contains the knowledge gained, encapsulating new information acquired through the process. Conversely, the input represents the missing knowledge or information that needs to be discovered. Knowledge discovery begins by identifying the gaps in understanding, essentially tackling the question, 'What do we need to know?' This highlights the gaps in understanding or information that need to be filled. By pinpointing the unknown – the questions without answers – the knowledge discovery process seeks to bridge these gaps. The knowledge discovery process is complete when the missing information is found and applied to the task at hand, thereby closing the knowledge gap.
Critical to illuminating the impact of the knowledge gaps on the developer work experience is the concept of Flow, as defined by psychologist Mihaly Csikszentmihalyi. Flow is a state of heightened focus and immersion in activities, crucial for optimizing productivity and job satisfaction. Achieving Flow requires balancing the complexity of tasks with the individual capabilities of the developer. When individual capability significantly exceeds task complexity, the knowledge gap becomes too narrow, leading to wasted potential and boredom. On the other hand, if task complexity surpasses individual capability, the knowledge gap widens, leading to stress and low productivity. A well-balanced gap keeps developers in the Flow state, fully immersed and focused on their work, leading to higher productivity and job satisfaction.
The Knowledge-centric perspective is a systems perspective, because it takes into account all of the behaviors of a system as a whole in the context of its ecosystem in which it operates. This includes team dynamics, organizational knowledge resources, development tools, and the overall work environment. Understanding and optimizing these systemic elements are essential for facilitating effective knowledge discovery and helping developers maintain a state of Flow. This systemic view underlines the importance of considering the interconnectedness of individual, team, and organizational factors in enhancing developer productivity and work experience.
A crucial aspect of our approach involves measuring the knowledge gaps that developers bridge in completing tasks, quantified in bits of information. This measurement approach allows for meaningful comparisons across different contexts, programming languages, and applications. It provides a tangible metric to assess the effectiveness of knowledge acquisition and application in the software development process.
The knowledge perspective applies to all human activities, not just software development. Insights gained from studying the knowledge discovery process in software development offer valuable implications for other fields where cognitive effort plays a significant role. This is apparent in tasks like furniture assembly, which, while seemingly manual, involves significant knowledge work.
Knowledge Discovery in Software Development
"The completion of a mental activity requires two types of input to the corresponding structure: an information input specific to that structure, and a non-specific input, which may be variously labeled “effort,” “capacity,” or “attention.” To explain man's limited ability to carry out multiple activities at the same time, a capacity theory assumes that the total amount of attention which can be deployed at any time is limited.." ~ Daniel Kahneman [4]
Knowledge is invisible, but as Drucker noted that a very large number of knowledge workers do both knowledge work and manual work[2]. He called those people “technologists” and stated they may be the single biggest group of knowledge workers. As examples for technologists Drucker provided surgeons, office clerks and dentists[2]. The first example I could think of are technology jobs in general and software developers in particular. Manual work results in tangible output which is visible. That makes the manual part of knowledge work visible. Importantly the tangible output shows that the knowledge was actually applied. That helps us to define what is a knowledge worker.
The key word here is 'knowledge,' particularly the knowledge workers lacked before starting a task—the knowledge they needed to gather to successfully complete it.
Understanding the Nature of Knowledge Work
In software development, distinguishing between 'knowledge' workers and 'manual' workers is essential. For manual workers, much of the required knowledge is embedded in the machines they operate and the procedures they follow. In contrast, knowledge workers, like software developers, engage in extensive knowledge acquisition for problem-solving and innovation.
This is exemplified by Margaret Hamilton's contribution to the Apollo 11 program in 1969. In the picture below to the right we have Margaret being awarded the Presidential Medal of Freedom by President Obama.
On the picture to the left we have Margaret standing next to part of the computer code she and her team developed for the Moon landing. What we witness is an example of the manual part of the knowledge work. The software developers had to type all of those pages manually. All the knowledge went from their brains, through their fingers and ended up as symbols on sheets of paper. The difficult and time consuming activity in creating Moon landing software was not in typing their already-available knowledge into source code. It was in effectively acquiring knowledge they did not already have i.e. getting answers to the questions they already had. Even more specifically, it was in discovering the knowledge necessary to make the system work that they did not know they were missing. The symbols are the tangible output of their knowledge work. The symbols are not the knowledge itself, but the trace of the knowledge left on the paper.
The Knowledge Discovery process
A key aspect of knowledge work is the 'knowledge gap' - the difference between what a developer currently knows and what they need to learn to effectively complete a task. Bridging this gap involves a process of Knowledge discovery, which encompasses both the discovery and application of new knowledge.
We can turn to physics for an analogy. A quantum (plural: quanta) is the smallest discrete unit of a physical phenomenon. For example, a quantum of light is a photon, and a quantum of electricity is an electron.
The tangible output represents the knowledge discovered through the knowledge discovery process, encapsulating the new information that has been acquired. We do not address the question of the quality of the tangible output, assuming that it meets appropriate standards.
A Knowledge Discovery Process transforms invisible knowledge into visible, tangible output.
Think of it as a black box, which may contain senior and junior developers potentially using AI tools like ChatGPT. To learn more about it, please refer to this article.
Knowledge is here divided into four categories[9]:
- Know-what: refers to the knowledge of facts and concepts. In software development that's usually business knowledge about WHAT needs to be delivered
- Know-why: refers to the knowledge of principles and laws.
- Know-how: refers to th skills - i.e. knowledge of how to do something. In software development that's usually technical knowledge about HOW to deliver the WHAT.
- Know-who: refers to the knowledge of people and relationships.
To explain the tangible output, we can use an analogy from physics where a quantum is the smallest discrete unit of a physical phenomenon. In this context, tangible output comprises symbols produced..The quality of the output is assumed to meet target standards.
Inputs represent the knowledge developers lack before starting a task i.e. the missing information or knowledge that needs to be discovered, which is measured in bits.
In the case of knowledge discovery we are not able to calculate the amount of missing information H the way we did it with the gold coin, because there is no way to know the number of boxes n and the size of each box pi. In real life knowledge discovery is a constant stream of questions asked and answers received. The number of boxes, their size and what's hidden in them constantly fluctuates based on the answers received. Each question may enlarge or shrink the set of possibilities. That is why in knowledge work we cannot apply the Shannon's formula and calculate the missing information H.
For that we will turn to Psychology where they study mental effort and cognitive load. Fortunately, there is scientific research that can help!
Different mental activities impose different demands on the limited cognitive capacity. An easy task demands little mental effort, and a difficult task demands much. Because the total quantity of mental effort which can be exerted at any time is limited, concurrent activities which require attention tend to interfere with one another. Human capacity to perform concurrent perceptual tasks is limited.
Easy tasks can be performed together with little interference, but more difficult tasks cannot. An even distribution of attention among concurrent activities is possible only at a low level of total mental effort. When total effort is high, one of the activities typically draws most of the attention, leaving little for the others. This suggestion implies that attention is divisible at low levels and more nearly unitary at high levels of effort[4].
As humans become skilled in a task, its demand for energy diminishes. The pattern of activity associated with an action changes as skill increases, with fewer brain regions involved. Talent has similar effects. Highly intelligent individuals need less effort to solve the same problems. The knowledge is stored in memory and accessed without intention and without mental effort[3].
Since humans can either type or ask a question then we conclude:
Now we have a means to infer the number of questions asked during a knowledge discovery process.
Introducing the Concept of Flow
The concept of Flow,, as coined by psychologist Mihaly Csikszentmihalyi, refers to a state of heightened focus and immersion in activities. People in a state of Flow lose their sense of time, feel in control of the task at hand, and experience an intrinsic sense of enjoyment and fulfillment[6].
To enter and maintain a flow state, the following conditions should be met[7]:
- The opportunity for action, with a clear and attainable goal. That doesn't meant having an overall goal for the activity, but knowing what to do next from moment to moment
- An action with a balance between skill level and challenge, meaning that the activity is neither too easy nor too difficult.
- Clear and immediate feedback, so that successes and failures are apparent and behavior can be adjusted as needed.
The balance should be established for each task the developer will face. Once the conditions for action are in place, the developer is able to engage in a series of challenging tasks that are neither too difficult (not overwhelming) nor too easy (not boring). By balancing each task, we can create the conditions for entering flow for each set of tasks, each feature, and ultimately, for the scope of the project.
When the conditions for flow are met, they create a feedback loop between action and feedback that allows for continuous, effortless tuning of performance while taking action. This feedback loop makes an activity worth doing for its own sake[6].
A state of Flow is achieved when there is a balance between individual capability and work complexity, with a slight inclination towards challenges. This is measured by gauging the knowledge discovered after successful work completion.
Take a Systems Perspective
The Knowledge-centric perspective is a systems perspective, because it takes into account all of the behaviors of a system as a whole in the context of its ecosystem in which it operates. This includes team dynamics, organizational knowledge resources, development tools, and the overall work environment.
It's a common misconception to overestimate how much of the knowledge discovery efficiency is a function of the individual developers' skills and abilities and underestimate how much is determined by the system they operate in. Knowledge is a property of the organization the software developer operates in. This includes the knowledge of the developer, but also the knowledge of their teammates, the Product Owner, the architect, the QA, the support, the documentation available, the applicable knowledge in StackOverflow, and so on.
While it's true that designers, architects, and QAs may not produce code, they contribute significantly to helping developers produce code that meets customer expectations. Therefore, it's not recommended to measure the capability of individual developers. Instead, you should always measure the capability of an organization, whether that be a team, a project, or a company.
Organizational Knowledge Resources
A critical component of the Knowledge-centric perspective is the effective utilization and management of organizational knowledge resources[8]. These resources encompass the existing knowledge, expertise, and insights available within an organization. They include not only formal documentation like databases, manuals, and standard operating procedures but also the tacit knowledge that resides in the experience and skills of individual team members.
Organizational knowledge resources play a pivotal role in bridging the knowledge gaps identified in the software development process. They provide a rich reservoir of knowledge that can be tapped into for problem-solving, innovation, and learning. Effective management of these resources involves not just cataloging and storing knowledge but also ensuring it is accessible and relevant to the needs of developers. This includes fostering a culture of knowledge sharing, where insights and experiences are openly exchanged, and collaboration is encouraged.
Moreover, in the context of software development, these resources are not static. They evolve as projects progress, technologies change, and new challenges emerge. Therefore, maintaining up-to-date and dynamic knowledge repositories is crucial. Additionally, leveraging tools and platforms that facilitate knowledge sharing and collaboration, such as internal wikis, forums, or collaborative software, can significantly enhance the efficiency and effectiveness of the knowledge discovery process.
The integration of these organizational knowledge resources into the daily workflow of software development teams is a key factor in sustaining a productive and innovative work environment. It enables developers to quickly access the information they need, reducing the time spent in rediscovering existing solutions or reinventing the wheel.
Furthermore, by aligning these resources with the concept of Flow, organizations can create an environment where developers are consistently engaged, challenged, and productive, thereby optimizing the overall software development process.
Categorization
We adopt a categorization of Organizational Knowledge Resources into those providing knowledge alone versus those offering both knowledge and tangible output While there isn't a universally accepted categorization specifically labeled in these exact terms, we can certainly distinguish between these types of resources based on their functionality and output. Here's how these categories might be conceptualized:
-
Knowledge-Only Resources:
These resources primarily provide information, insights, or knowledge without directly producing a tangible output. They require the user to actively interpret and apply this knowledge in their specific context. They are crucial for foundational learning, concept understanding, and gaining theoretical insights. Examples include:
- Books and Manuals: Offer theoretical knowledge or instructions which users must interpret and apply.
- Educational Resources: Such as online courses or training modules that impart skills or knowledge.
-
Knowledge and Tangible Output Resources:
These resources not only provide knowledge but also assist in creating a direct, tangible output, often leveraging technology or tools. They are instrumental in practical application, efficiency improvement, and bridging the gap between theory and practice. Examples include:
- AI-Assisted Tools like GitHub Copilot: Provide coding suggestions and can generate code snippets, offering both knowledge (through code examples) and tangible output (actual code).
- Software Development Frameworks and Libraries: Offer pre-written code or functions that developers can use directly in their projects.
-
Automation tools:
Help automate parts of the software development process and workflow e.g. Continuous Deployment/Integration like Jenkins, CircleCI, etc.
These tools typically fall into this category for the following reasons:
- Tangible Output: Automation tools in software development, such as Continuous Integration/Continuous Deployment (CI/CD) platforms, automated testing frameworks, and build tools, actively perform tasks that would otherwise be done manually. The tangible outputs are the results from executed tasks and processes, which are essential for the software development lifecycle.
- Knowledge Facilitation: While these tools are not primary sources of theoretical knowledge, they significantly contribute to the practical understanding of software processes. They not only provide knowledge or information on how to execute these processes (scripting) but provide insights into the development process through logs, reports, and real-time monitoring. This operational knowledge is crucial for identifying issues, improving code quality, and continuous improvement in software development practices.
This distinction is valuable for organizations to understand as it helps them curate a balanced set of resources. While knowledge-only resources lay the theoretical groundwork and deepen understanding, resources providing both knowledge and tangible output are crucial for practical application, time-saving, and enhancing productivity.
Organizations benefit from a mix of both types of resources, ensuring a comprehensive knowledge ecosystem that supports both learning and efficient execution.
Applying the Knowledge-centric Perspective Across Human Activities
The knowledge perspective applies to all human activities, not just software development.
Both manual and knowledge work result in outputs, outcomes, and impacts, but they differ significantly in their primary inputs. While manual work's primary input is muscle effort, software development thrives on cognitive effort. This becomes evident when we realize that in manual tasks, much of the necessary knowledge is already embedded in the machines and procedures. In contrast, software development demands continuous knowledge acquisition and application, necessitating a deeper level of problem-solving and innovation.
There is no requirement for the tangible output of a Knowledge Discovery process to be symbols of source code contributed via a keyboard. This is apparent in tasks like furniture assembly, which, while seemingly manual, involves significant knowledge work. This task requires reading instructions and learning to use tools to assemble parts into a tangible piece of furniture, such as tightening a screw. Each tightened screw represents a quantum of tangible output in this knowledge discovery process. Thus, knowledge work extends beyond typical IT jobs, encompassing any activity where learning and applying new knowledge are essential to produce a tangible result.
An application in assembling furniture
Let's imagine in front of us are two people who assemble a different model of IKEA furniture. We are told their names are Joe and Don. We are also informed that Don is a professional furniture assembler and Joe is a real novice assembling his first furniture. Our task is to tell who is Joe and who is Don just by looking at them assembling furniture.Many of us have the frustrating experience of assembling a Billy. You know how the furniture arrived at your house, all neatly packed in a box? Yeah, that doesn't last long. Within minutes of opening the box, the contents seem to have multiplied. Suddenly, there's not enough space to stack all the pieces, and you're struggling to keep all those screws and fasteners in a neat, little pile. You pick up the instructions printed without words, using only simple illustrations, and suddenly the lack of text makes you feel extremely perplexed.
The first few steps are done and nothing can stop you. Except the next step you are doing it totally right but it's not fitting together. So you read, and then you re-read, and then you pick up the screws and try again. It's still not working. Then it's working! Nope, still not working. You hang up and look around. Your room is in shambles and random screws are rolling everywhere. Finally, the pieces screw in. It's coming together. And then you realize, on the very last step, that you're one screw short. For 10 minutes, you're on your hands and knees, going through the garbage, trying to locate the missing screw. And then you spot it. The last screw. Your key to freedom. You're sore and tired and your brain hurts, but you're done. You expected to have all your questions answered by the manual. And yet, you spent time asking yourself many more additional questions.
Having our past experience in mind while looking at the two men we can easily tell who is finding the manual informative enough, and who is asking himself many more additional questions. The former should be Don, because his training has provided him with all the knowledge needed to assemble furniture. The latter should be Joe, because he's spending a lot of time sitting on the floor with an allen wrench in one hand and the manual in the other.
In addition, can we tell Joe or Don even without looking at them while the men worked? Yes, because asking questions takes time. Prior knowledge takes less time to discover because it is in our heads. Hence, per unit time the professional should fix more screws, barrel bolts, and dowels(tacks) than a novice. Thus, by just counting at the screws two men fixed per unit time we can tell who's more efficient in discovering and applying their knowledge about assembling furniture.
An Application in Driving Cars
Joe is a novice driver, who has just received his driving licence. Joe has a total of 20 hours experience driving a car. Next to him is Don, who had received his driving licence 25 years ago. It is not clear how many hours Don spent driving a car, but it is in the thousands. Let's imagine Joe and Don take part in a car race. The race track features short straights that climb sharply uphill for a short time, then suddenly drop downhill and very fast left and right turns. In short - it is a challenging track. Drivers need to make only one lap - from Start to Finish. They are given the same car model to drive on the same racetrack - hence, they are on an equal footing. We are watching the race, but we are not aware which is Joe's car and which is Don's car. It is like a black box to us. Our goal is to tell which car is Joe's and which car is Don's. How can we tell? Well, if the track is the same and the cars are the same, then the best driver will finish first. Finishing first means less time to get from Start to Finish i.e. more meters per unit time.But what factors into being the "best" driver in this scenario? It boils down to knowledge. He knows precisely when to engage the clutch, which gear to select for various sections of the track, and how to navigate turns efficiently. His extensive knowledge directly translates into a faster, more proficient driving performance.
Conclusion
In conclusion, the knowledge-centric perspective sheds light on software development as a field driven by continuous knowledge acquisition and application. highlights the importance of knowledge work in software development, focusing on the knowledge discovery process. This process, which is integral to both software development and everyday tasks like assembling furniture, involves bridging the 'knowledge gap' between existing knowledge and what needs to be learned for task completion. Recognizing and measuring this gap is key to enhancing work experience and unlocking team potential.
Applying a knowledge-centric perspective becomes crucial not just in software development but across various fields, fostering more efficient and innovative work environments.
Works Cited
1. Shannon, C. E. (1948). A Mathematical Theory of Communication. Bell System Technical Journal. 1948;27(3):379-423. doi:10.1002/j.1538-7305.1948.tb01338.x
2. Drucker , Peter F, “Knowledge-Worker Productivity: The Biggest Challenge,California Management Review, vol. 41, no. 2, pp. 79–94, Jan. 1999, doi: 10.2307/41165987.x
3. Kahneman, D. (2011). Thinking, fast and slow. Farrar, Straus and Giroux.
4. Kahneman D. (1973). Attention and Effort. Englewood Cliffs, NJ: Prentice-Hall
5. Alves, R. A., Castro, S. L., & Olive, T. (2008). Execution and pauses in writing narratives : Processing time, cognitive effort and typing skill. International Journal of Psychology.
6. Csikszentmihalyi, M. 1990. Flow: the psychology of optimal experience. United States of America: Harper & Row.
7. Schaffer, O. 2013. Crafting Fun User Experiences: A Method to Facilitate Flow. Human Factors International Whitepaper. Retrieved from: https://scholar.google.com/scholar?cluster=9760324565241258858
8. Holsapple, C. W., & Joshi, K. D. (2001). Organizational knowledge resources. Decision Support Systems, 31, 39-54. https://doi.org/10.1016/S0167-9236(00)00118-4
9. Lundvall, B. Å and Johnson, B. (1994), “The learning economy”, Journal of Industry Studies, Vol. 1, No. 2,December 1994, pp. 23-42.
How to cite:
Bakardzhiev D.V. (2023) A Knowledge-centric Perspective On all human activities https://docs.kedehub.io/kede/knowledge-centric-perspective.html
Getting started
