Perplexity in knowledge work
How does it feel to do knowledge work?
"Socrates: But do you suppose that he would ever have enquired into or learned what he fancied that he knew, though he was really ignorant of it, until he had fallen into perplexity [aporia], under the idea that he did not know, and had desired to know?
Meno: "I do not think so, Socrates"" ~ Plato, Meno
Cambridge Dictionary defines "perplexity" as “a state of confusion or a complicated and difficult situation or thing.”. Merriam-Webster defines "perplexed" as “the state of being filled with uncertainty and full of difficulty." Perplexity is described as a subjective state, a feeling or experience.
Socrates holds perplexity in higher regard. In a state of aporia or perplexity, Socrates explains, one is best positioned to begin to learn.
For Dewey, the origin of thinking is some perplexity, confusion, or doubt.
Perplexity has three cognitive dimensions: ignorance, uncertainty and contradiction. They represent knowledge worker's evaluations about her own actual state.
In order to understand the perplexity knowledge workers face we need to consider all of its dimensions.
Strength of beliefs
Belief is the cognitive act or state in which a proposition is taken to be true. Belief, considered as the propositional attitude implicit in every cognition, has been a prominent topic in quite different traditions of psychological research. Belief, as a subjective probability, has naturally been a central topic in the judgmental or decision-theoretical tradition, since it is one of two free variables in the definition of utility. There are long traditions of research on the formation of belief through induction and deduction, and modest paradigms such as the concept-learning paradigm have given empirical access to the interaction of belief and metabelief.
We live in an open world with possibly infinite source of information such as:
- direct experience (such as perceptive evidences);
- information provided by other people;
- reasoning (about other beliefs);
- categorization (reasoning about classes and similarities).
Knowledge workers can make “queries” to information sources i.e. ask questions. The more you know an environment, the less you need to question.
Humans hold beliefs to describe or capture the way things actually are. When one forms a belief, one is seeking a match between one's mind and the world.
Knowledge is a kind of belief. If one has no beliefs about a particular matter, one cannot have knowledge about it. Thoughts that an individual has never entertained are not among his beliefs, and thus cannot be included in his body of knowledge. Knowledge, then, requires belief.
Belief is necessary but not sufficient for knowledge. We are all sometimes mistaken in what we believe; in other words, while some of our beliefs are true, others are false. As we try to acquire knowledge, then, we are trying to increase our stock of true beliefs The knowledge worker asks herself: how much can I be certain of a certain belief from a source? That results in assigning a reliability score to each information source. The strength of a belief shows how much the knowledge worker relies on the perceived reliability of its information sources. In statistical terms strength of beliefs are probabilities of possible states in a distribution.
Ignorance is when the knowledge worker is lacking knowledge or missing information. Since knowledge is acquired when there is no missing information left then ignorance is the opposite of knowledge.
Ignorance is a subjective evaluation of actual missing information.
The notion of missing information implies we have an information source and just have to question it. But are we sure we have all the sources? How many are there in the open world? How much information there is that we have not considered but we should? Ignorance depends on how much information we have with respect to how much there is in the open world.
Ignorance also includes the notion that there may be information sources we are not aware of.
The open world also offers conflicting beliefs from multiple sources of information. We prefer to question very reliable sources. Our ignorance includes the notion that some of the information sources we are aware of are not reliable.
Ignorance is also determined by the lack of information sources that provide strong beliefs.
In an open world in theory there are an infinite number of information sources to take into account. It is not possible to perform a full search. Hence the knowledge worker can never conclude that her own ignorance is zero. In reality it is enough to make the ignorance acceptably close to zero.
Uncertainty is a measure of the difference between the strengths of a set of beliefs.
It is equivalent to the value of missing information. For a probability distribution it can be calculated using Shannon's formula. Accordingly, when the difference is 0 the uncertainty is maximal, when the difference is 1 the uncertainty is minimal.
Contradiction is a logical inconsistency in the strength of a set of beliefs.
For example, if the knowledge worker believes that a ball is both black and white that is a contradiction. In statistical terms there is contradiction if the sum of the strengths of a set of beliefs is more than 1.
A knowledge worker may rationally subscribe to contradictory beliefs for the simple reason that more than rationality may be needed to discern the contradiction.
Perplexity is multivariate
As we see Perplexity is multivariate. A single problem can be perplexed along multiple axes. For each task a knowledge worker can be perplexed about:
- The particular technologies used or to be used. That is very common in software development where technologies are constantly changing and maturing.
- The breadth of technology options available
- The business domain.
- The way of articulating the problem — a better model could make the solution obvious.
- The people in the team — their aspirations or fears, their motivation, their relationships with one another and out into the wider organization.
- The people to build relationships with.
- The organizational constraints i.e. policies, rules, procedures.
- The culture of the organization.
- The third party dependencies and associated risks.
We are just scratching the surface here. Anyone could imagine many other factors that knowledge workers could be perplexed about.
"The more possibilities we are confronted with, the more perplexed and challenged we are likely to be.” ~ Aurelian Craiutu, Faces of Moderation
Perplexity is a term that is commonly used in the field of natural language processing and machine learning to refer to the uncertainty or unpredictability of a model's output. In information theory, perplexity is a measurement of how variable a prediction model is. It is typically calculated as the inverse of the probability of the model's output, and is used as a measure of the model's performance. Perplexity is often used as a way to compare the performance of different prediction models, with lower perplexity values indicating that a model is more certain or predictable and higher perplexity values indicating that a model is more uncertain or unpredictable. Perplexity provides a bridge between the average probability and the missing information of a distribution.
In a broader sense, perplexity might be understood as a measure of the complexity, difficulty, uncertainty or unpredictability of a problem or task. For example, a person might be said to be in a state of perplexity if they are trying to solve a difficult puzzle or find an object in a large search space and are having trouble figuring out the best way to proceed. In this context, perplexity might be thought of as a measure of the search space i.e. average number of possible states, in the sense that a more complex or difficult task might have a greater number of possible states or options to consider, which might increase the feeling of uncertainty or confusion. For example, if one is trying to find a specific item in a large room or warehouse, the search space would be the number of potential locations where the item could be found.
To quantify perplexity, we need to know how many options a software developer sees when looking at a work item before starting the work. This task can be challenging, if not impossible, since it involves understanding the individual's thought process!
However, we can measure the knowledge discovered after completing the work. This knowledge equals the difference between the prior knowledge a software developer had and the knowledge required to develop the work item.
Let's look back at the game where we searched for a gold coin in a number of boxes. The number of boxes is the size of the search space. Recall that when we have n equally sized boxes the average number of questions needed to find the coin location is:
Hence if we know the number of questions asked we can calculate the number of boxes:
That is what information theory calls “perplexity”. It is good to point out that perplexity cannot be zero. At the minimum it can be one.
Below is an animated example of calculating perplexity when we search for a gold coin hidden in 1 of 64 boxes. In all six cases the required knowledge is 6 bits. That means, we need to ask six binary questions on average to find the gold coin. You can see that the size of the search space is determined by the knowledge discovered. The less knowledge discovered the lower the perplexity for the humans involved.
Knowledge Discovery Efficiency (KEDE) measures the discovered knowledge. From KEDE, we can assess the perplexity a software developer experienced while working.
Taking into consideration how KEDE correlates with perplexity, we can say that::
KEDE is calculated per knowledge worker. KEDE reflects how efficiently knowledge workers discover and apply knowledge. KEDE is low when the knowledge worker does not possess the knowledge needed for the challenges . KEDE is high when the knowledge worker possesses the knowledge needed for the challenges. The less knowledge to be discovered the lower the perplexity for the human involved.
For instance, if a knowledge worker has KEDE=100, it means she had to find a coin in only one box.
This situation reflects a perplexity of 1, indicating there was only one possible answer to the question: "Where is the gold coin?" This is the minimum value.
However, if a knowledge worker has KEDE=20, it implies she had to find a coin among 16 boxes, indicating a tolerable perplexity level.
But if a knowledge worker had KEDE=1, equivalent to answering 99 binary questions, it means she had to find a coin among approximately 6.3x1029 boxes!
To contextualize, there are an estimated 1022 to 1024 stars in the Universe. The number of boxes was significantly greater than the number of stars in the Universe! We can infer that the software developer faced an exceedingly high level of perplexity.
How perplexity affects humans?
Perplexity, as it is typically used in the field of psychology, refers to a state of confusion or uncertainty.
In general, it is thought that humans prefer environments that are predictable and provide clear goals and feedback, as this can help them to feel more in control and motivated. High levels of uncertainty or unpredictability can create a sense of instability and insecurity, which can have negative effects on cognition and behavior.
It is possible that experiencing high levels of uncertainty or unpredictability in one's environment or in a task or problem that they are attempting to solve could potentially affect human cognition and behavior in various ways. For example, high levels of perplexity might lead to feelings of confusion, frustration, or stress, which could affect a person's ability to think clearly and make decisions. It could also lead to a lack of motivation or interest in the task, as the person might feel that their efforts are unlikely to be successful.
There is some research suggesting that a lower level of confusion or uncertainty may be associated with higher levels of happiness while working. For example, studies have found that people who have a greater sense of control over their work environment and their work tasks tend to report higher levels of job satisfaction and well-being. However, it is important to note that the relationship between perplexity and happiness is likely to be complex and multifaceted. Perplexity, or confusion and uncertainty, can be a normal and necessary part of the learning and growth process, and may not necessarily be associated with negative emotions. In fact, some research has suggested that experiencing a certain level of uncertainty and novelty can actually be beneficial for well-being and can lead to feelings of excitement and curiosity.
The feeling of perplexity, or confusion and uncertainty, is generally seen as being incompatible with the state of flow. Flow is a mental state characterized by a sense of complete immersion and focus in an activity. It is often described as a state of "optimal experience," in which a person becomes fully absorbed in an activity and loses track of time. It is difficult to experience flow when one is in a state of perplexity, as the mental state of flow requires a clear sense of purpose and a high level of focus and concentration. Perplexity is an indicator of whether a knowledge worker was happy while working.
However, it is possible that experiencing a certain level of novelty or uncertainty in an activity may help to stimulate creativity and can lead to feelings of excitement and curiosity, which may contribute to the experience of flow. This is particularly true in activities that involve problem-solving or decision-making, as the process of working through and resolving uncertainty can be engaging and rewarding. However, it is important to note that there is a balance to be struck, and too much uncertainty or confusion may make it difficult to experience flow.
Perplexity may be related to the size of the search space in the sense that a larger search space may increase the difficulty of finding a solution to a problem or answering a question, which could potentially lead to feelings of confusion or uncertainty. The size of the search space may be one factor that influences a person's level of perplexity, but it is not the only factor, and the relationship between perplexity and the size of the search space is likely to be complex and multifaceted. It will depend on a wide range of factors, such as working memory capacity, attention, and the specific demands of the task at hand, may also influence a person's ability to search effectively and may affect their level of perplexity.
Limited Working memory
Working memory is a cognitive system that allows people to temporarily store and manipulate information in their minds. It plays an important role in a variety of mental tasks, including problem-solving, decision-making, and learning, and is often referred to as the "mental scratchpad" because it allows us to temporarily hold information in mind while we manipulate it or use it to guide our actions. In 1956, the renowned cognitive psychologist George Miller argued that although the brain can store a whole lifetime of knowledge in its trillions of connections, the number of items that humans can actively hold in their conscious awareness at once is limited. Only seven of them can fit in what's called working memory, where they are available for our focused attention and other cognitive processes. This has been interpreted to mean that the human brain can hold about 2 to 3 bits of information in working memory at a time, where each bit represents a binary decision (question). Those items might be a series of digits, a handful of objects scattered around a room, words in a list, or overlapping sounds. Their retention in working memory is short-lived and when they're no longer actively being thought about, they're stored elsewhere or forgotten. There are ways in which people work around this constraint - we can remember all the digits of a phone number by “chunking” digits (remembering 1, then 4, as the single item 14, for instance), or develop mnemonic devices for shuffling random digits of pi out of longer-term storage. Since Miller's time, neuroscientists and psychologists have continued to study working memory. They have found that the limit may really be closer to four or five items than seven]. The precise limit is determined by each individual's memory capacity, such that the activity from low-capacity individuals reaches this plateau much sooner than that from high-capacity individuals. Cognitive capacity is directly related to cognitive ability.
Searching in a large search space can be challenging when working memory capacity is limited, as it may be more difficult to hold multiple pieces of information in mind at once, which can make it harder to effectively search through a large search space. For example, if you are trying to search for a specific item in a large room or warehouse, you may need to remember the location of other items or landmarks as you search, which can be challenging if your working memory capacity is limited. However, it is important to note that there are many other factors that can contribute to feelings of perplexity, and working memory capacity is just one potential influence. A person's knowledge and prior experience, as well as their emotional state and the context in which they are encountering the information, can all play a role in their ability to understand and make sense of new information. It is also worth noting that working memory capacity is not the only cognitive ability that can affect a person's ability to process and understand information. Other factors, such as attention, processing capacity, and executive function, can also play a role in a person's ability to learn and understand new information.
Limited processing capacity of the conscious mind
The processing capacity of the conscious mind refers to the amount of mental resources that are available to process and understand information. While a great deal occurs below the threshold of our awareness, and this has an impact on how we feel and what our life is going to be like, in order for something to become encoded as part of your experience, you need to have paid conscious attention to it.
That is the speed limit for the traffic of information we can pay conscious attention to at any one time. Since processing capacity is limited, it can be more difficult to effectively search through a large search space, as it may be harder to hold relevant information in mind and to process new information as it is encountered. This could potentially lead to feelings of confusion or uncertainty, or a sense of being overwhelmed.
While the terms "processing capacity of the conscious mind" and "working memory capacity" are often used interchangeably, they are not exactly the same thing. Here's a more nuanced explanation:
Here's a summary explanation:
- Processing Capacity of the Conscious Mind refers to the ability of the conscious mind to process information. It's a broad term that encompasses several cognitive functions, including attention, perception, and memory. It's about how much information the conscious mind can handle at any given moment.
- Working Memory Capacity is a more specific term that refers to the amount of information that can be temporarily held and manipulated in the mind. It's a component of the overall processing capacity of the conscious mind.
So, while working memory capacity is a part of the processing capacity of the conscious mind, it's not the entirety of it. The processing capacity of the conscious mind also includes other cognitive functions beyond just working memory.
1. Nelson, K. P. (2016). Reduced Perplexity: A simplified perspective on assessing probabilistic forecasts, in Chen, M., Dunn, J. M., Golan, A., & Ullah, A. (Eds.). (2021). Advances in Info-Metrics: Information and Information Processing across Disciplines. Oxford University Press.
2. Pezzulo, G., Lorini, E., & Calvi, G. (2004). How do I Know how much I don't Know? A cognitive approach about Uncertainty and Ignorance. In Proceedings of the Annual Meeting of the Cognitive Science Society (Vol. 26, No. 26).
3. How many stars are there in the Universe? (n.d.). Retrieved January 11, 2022, from https://www.esa.int/Science_Exploration/Space_Science/Herschel/How_many_stars_are_there_in_the_Universe
4. Castelfranchi, C., Lorini, E. (2003). Cognitive Anatomy and Functions of Expectations. IJCAI03 Workshop on Cognitive modeling of agents and multi-agent interaction, Acapulco, Mexico.
5. Egan O. (1986) The Concept of Belief in Cognitive Theory. In: Mos L.P. (eds) Annals of Theoretical Psychology. Annals of Theoretical Psychology, vol 4. Springer, Boston, MA. https://doi.org/10.1007/978-1-4615-6453-9_23
6. Castelfranchi C. (1997) Representation and Integration of Multiple Knowledge Sources: Issues and Questions. In: Cantoni V., Di Gesù V., Setti A., Tegolo D. (eds) Human and Machine Perception. Springer, Boston, MA. https://doi.org/10.1007/978-1-4615-5965-8_17
7. Owens, J. I. (1989). Contradictory Belief and Cognitive Access. Midwest Studies in Philosophy, 14, 289–316. https://doi.org/10.1111/j.1475-4975.1989.tb00194.x
8. Dewey J. (1910). "What is thought?" Chapter 1 in How we think. Lexington, Mass: D.C. Heath: 1-13.
9. Miller, G. A. The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychol Rev 63, 81 (1956).
10. Vogel EK, Machizawa MG. Neural activity predicts individual differences in visual working memory capacity. Nature. 2004 Apr 15;428(6984):748-51. doi: 10.1038/nature02447. PMID: 15085132.
11. Fukuda K, Vogel E, Mayr U, Awh E. Quantity, not quality: the relationship between fluid intelligence and working memory capacity. Psychon Bull Rev. 2010 Oct;17(5):673-9. doi: 10.3758/17.5.673. PMID: 21037165; PMCID: PMC3050565.
12. Unsworth N, Fukuda K, Awh E, Vogel EK. Working memory and fluid intelligence: capacity, attention control, and secondary memory retrieval. Cogn Psychol. 2014 Jun;71:1-26. doi: 10.1016/j.cogpsych.2014.01.003. Epub 2014 Feb 14. PMID: 24531497; PMCID: PMC4484859.
13. Cowan, N. (2001). The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and Brain Sciences, 24(1), 87-114. doi:10.1017/S0140525X01003922
14. Levitin, D. J. (2015). The Organized Mind: Thinking Straight in the Age of Information Overload (Illustrated edition). Dutton.
“120 bits per second This estimate derives independently from Csikszentmihalyi (2007) and the Bell Labs engineer Robert Lucky, who made an independent estimate that regardless of the modality, the cortex cannot take in more than 50 bits/second—within an order of magnitude of Csikszentmihalyi’s. Csikszentmihalyi explains his estimate: “As George Miller and others have suggested, we can process 5–7 bits of information in one apperception; each apperception takes at least 1/15th of a second; hence 7 × 15=105 bits/second. Nusbaum has calculated that understanding verbal material takes on the average 60 bits/second.”
How to cite:
Bakardzhiev D.V. (2021) Perplexity in knowledge work https://docs.kedehub.io/kede/perplexity-knowledge.html