Perplexity in knowledge work

Perplexity

"Socrates: But do you suppose that he would ever have enquired into or learned what he fancied that he knew, though he was really ignorant of it, until he had fallen into perplexity [aporia], under the idea that he did not know, and had desired to know?

Meno: "I do not think so, Socrates"" ~ Plato, Meno

Cambridge Dictionary defines "perplexity" as “a state of confusion or a complicated and difficult situation or thing.”. Merriam-Webster defines "perplexed" as “the state of being filled with uncertainty and full of difficulty." Perplexity is described as a subjective state, a feeling or experience.

Socrates holds perplexity in higher regard. In a state of aporia or perplexity, Socrates explains, one is best positioned to begin to learn.

For Dewey, the origin of thinking is some perplexity, confusion, or doubt[8].

The only way to reduce perplexity is through asking questions.

Perplexity has three cognitive dimensions: ignorance, uncertainty and contradiction[2]. They represent knowledge worker's evaluations about her own actual state.

In order to understand the perplexity knowledge workers face we need to consider all of its dimensions.

Strength of beliefs

Belief is the cognitive act or state in which a proposition is taken to be true. Belief, considered as the propositional attitude implicit in every cognition, has been a prominent topic in quite different traditions of psychological research. Belief, as a subjective probability, has naturally been a central topic in the judgmental or decision-theoretical tradition, since it is one of two free variables in the definition of utility. There are long traditions of research on the formation of belief through induction and deduction, and modest paradigms such as the concept-learning paradigm have given empirical access to the interaction of belief and metabelief[5].

We live in an open world with possibly infinite source of information such as:

• direct experience (such as perceptive evidences);
• information provided by other people;
• reasoning (about other beliefs);
• categorization (reasoning about classes and similarities)[4].

Knowledge workers can make “queries” to information sources i.e. ask questions. The more you know an environment, the less you need to question.

Humans hold beliefs to describe or capture the way things actually are. When one forms a belief, one is seeking a match between one's mind and the world.

Knowledge is a kind of belief. If one has no beliefs about a particular matter, one cannot have knowledge about it. Thoughts that an individual has never entertained are not among his beliefs, and thus cannot be included in his body of knowledge. Knowledge, then, requires belief.

Belief is necessary but not sufficient for knowledge. We are all sometimes mistaken in what we believe; in other words, while some of our beliefs are true, others are false. As we try to acquire knowledge, then, we are trying to increase our stock of true beliefs The knowledge worker asks herself: how much can I be certain of a certain belief from a source? That results in assigning a reliability score to each information source. The strength of a belief shows how much the knowledge worker relies on the perceived reliability of its information sources[2]. In statistical terms strength of beliefs are probabilities of possible states in a distribution.

Ignorance

Ignorance is when the knowledge worker is lacking knowledge or missing information. Since knowledge is acquired when there is no missing information left then ignorance is the opposite of knowledge.

Ignorance is a subjective evaluation of actual missing information[2].

The notion of missing information implies we have an information source and just have to question it. But are we sure we have all the sources? How many are there in the open world? How much information there is that we have not considered but we should? Ignorance depends on how much information we have with respect to how much there is in the open world.

Ignorance also includes the notion that there may be information sources we are not aware of[2].

The open world also offers conflicting beliefs from multiple sources of information. We prefer to question very reliable sources. Our ignorance includes the notion that some of the information sources we are aware of are not reliable.

Ignorance is also determined by the lack of information sources that provide strong beliefs[2].

In an open world in theory there are an infinite number of information sources to take into account. It is not possible to perform a full search. Hence the knowledge worker can never conclude that her own ignorance is zero. In reality it is enough to make the ignorance acceptably close to zero.

Uncertainty

Uncertainty is a measure of the difference between the strengths of a set of beliefs.

It is equivalent to the value of missing information. For a probability distribution it can be calculated using Shannon's formula. Accordingly, when the difference is 0 the uncertainty is maximal, when the difference is 1 the uncertainty is minimal.

Contradiction

Contradiction is a logical inconsistency in the strength of a set of beliefs.

For example, if the knowledge worker believes that a ball is both black and white that is a contradiction. In statistical terms there is contradiction if the sum of the strengths of a set of beliefs is more than 1.

A knowledge worker may rationally subscribe to contradictory beliefs for the simple reason that more than rationality may be needed to discern the contradiction[7].

Perplexity is multivariate

As we see Perplexity is multivariate. A single problem can be perplexed along multiple axes. For each task a knowledge worker can be perplexed about:

• The particular technologies used or to be used. That is very common in software development where technologies are constantly changing and maturing.
• The breadth of technology options available
• The business domain.
• The way of articulating the problem — a better model could make the solution obvious.
• The people in the team — their aspirations or fears, their motivation, their relationships with one another and out into the wider organization.
• The people to build relationships with.
• The organizational constraints i.e. policies, rules, procedures.
• The culture of the organization.
• The third party dependencies and associated risks.

We are just scratching the surface here. Anyone could imagine many other factors that knowledge workers could be perplexed about.

How perplexity affects humans?

Perplexity, as it is typically used in the field of psychology, refers to a state of confusion or uncertainty.

In general, it is thought that humans prefer environments that are predictable and provide clear goals and feedback, as this can help them to feel more in control and motivated. High levels of uncertainty or unpredictability can create a sense of instability and insecurity, which can have negative effects on cognition and behavior.

It is possible that experiencing high levels of uncertainty or unpredictability in one's environment or in a task or problem that they are attempting to solve could potentially affect human cognition and behavior in various ways. For example, high levels of perplexity might lead to feelings of confusion, frustration, or stress, which could affect a person's ability to think clearly and make decisions. It could also lead to a lack of motivation or interest in the task, as the person might feel that their efforts are unlikely to be successful.

There is some research suggesting that a lower level of confusion or uncertainty may be associated with higher levels of happiness while working. For example, studies have found that people who have a greater sense of control over their work environment and their work tasks tend to report higher levels of job satisfaction and well-being. However, it is important to note that the relationship between perplexity and happiness is likely to be complex and multifaceted. Perplexity, or confusion and uncertainty, can be a normal and necessary part of the learning and growth process, and may not necessarily be associated with negative emotions. In fact, some research has suggested that experiencing a certain level of uncertainty and novelty can actually be beneficial for well-being and can lead to feelings of excitement and curiosity.

The feeling of perplexity, or confusion and uncertainty, is generally seen as being incompatible with the state of flow. Flow is a mental state characterized by a sense of complete immersion and focus in an activity. It is often described as a state of "optimal experience," in which a person becomes fully absorbed in an activity and loses track of time. It is difficult to experience flow when one is in a state of perplexity, as the mental state of flow requires a clear sense of purpose and a high level of focus and concentration. Perplexity is an indicator of whether a knowledge worker was happy while working.

The less perplexed the happier.

However, it is possible that experiencing a certain level of novelty or uncertainty in an activity may help to stimulate creativity and can lead to feelings of excitement and curiosity, which may contribute to the experience of flow. This is particularly true in activities that involve problem-solving or decision-making, as the process of working through and resolving uncertainty can be engaging and rewarding. However, it is important to note that there is a balance to be struck, and too much uncertainty or confusion may make it difficult to experience flow.

Perplexity may be related to the size of the search space in the sense that a larger search space may increase the difficulty of finding a solution to a problem or answering a question, which could potentially lead to feelings of confusion or uncertainty. The size of the search space may be one factor that influences a person's level of perplexity, but it is not the only factor, and the relationship between perplexity and the size of the search space is likely to be complex and multifaceted. It will depend on a wide range of factors, such as working memory capacity, attention, and the specific demands of the task at hand, may also influence a person's ability to search effectively and may affect their level of perplexity.

Limited Working memory

Working memory is a cognitive system that allows people to temporarily store and manipulate information in their minds. It plays an important role in a variety of mental tasks, including problem-solving, decision-making, and learning, and is often referred to as the "mental scratchpad" because it allows us to temporarily hold information in mind while we manipulate it or use it to guide our actions. In 1956, the renowned cognitive psychologist George Miller argued that although the brain can store a whole lifetime of knowledge in its trillions of connections, the number of items that humans can actively hold in their conscious awareness at once is limited[9]. Only seven of them can fit in what's called working memory, where they are available for our focused attention and other cognitive processes. This has been interpreted to mean that the human brain can hold about 2 to 3 bits of information in working memory at a time, where each bit represents a binary decision (question). Those items might be a series of digits, a handful of objects scattered around a room, words in a list, or overlapping sounds. Their retention in working memory is short-lived and when they're no longer actively being thought about, they're stored elsewhere or forgotten. There are ways in which people work around this constraint - we can remember all the digits of a phone number by “chunking” digits (remembering 1, then 4, as the single item 14, for instance), or develop mnemonic devices for shuffling random digits of pi out of longer-term storage. Since Miller's time, neuroscientists and psychologists have continued to study working memory. They have found that the limit may really be closer to four or five items than seven]. The precise limit is determined by each individual's memory capacity, such that the activity from low-capacity individuals reaches this plateau much sooner than that from high-capacity individuals[10]. Cognitive capacity is directly related to cognitive ability[11][12].

The capacity of working memory is limited to a few items, with estimates ranging from three to seven items.

Searching in a large search space can be challenging when working memory capacity is limited, as it may be more difficult to hold multiple pieces of information in mind at once, which can make it harder to effectively search through a large search space. For example, if you are trying to search for a specific item in a large room or warehouse, you may need to remember the location of other items or landmarks as you search, which can be challenging if your working memory capacity is limited. However, it is important to note that there are many other factors that can contribute to feelings of perplexity, and working memory capacity is just one potential influence. A person's knowledge and prior experience, as well as their emotional state and the context in which they are encountering the information, can all play a role in their ability to understand and make sense of new information. It is also worth noting that working memory capacity is not the only cognitive ability that can affect a person's ability to process and understand information. Other factors, such as attention, processing capacity, and executive function, can also play a role in a person's ability to learn and understand new information.

Limited processing capacity of the conscious mind

The processing capacity of the conscious mind refers to the amount of mental resources that are available to process and understand information. While a great deal occurs below the threshold of our awareness, and this has an impact on how we feel and what our life is going to be like, in order for something to become encoded as part of your experience, you need to have paid conscious attention to it[14].

The processing capacity of the conscious mind has been estimated at 120 bits per second.

That is the speed limit for the traffic of information we can pay conscious attention to at any one time. Since processing capacity is limited, it can be more difficult to effectively search through a large search space, as it may be harder to hold relevant information in mind and to process new information as it is encountered. This could potentially lead to feelings of confusion or uncertainty, or a sense of being overwhelmed.

While the terms "processing capacity of the conscious mind" and "working memory capacity" are often used interchangeably, they are not exactly the same thing. Here's a more nuanced explanation:

Here's a summary explanation:

• Processing Capacity of the Conscious Mind refers to the ability of the conscious mind to process information. It's a broad term that encompasses several cognitive functions, including attention, perception, and memory. It's about how much information the conscious mind can handle at any given moment.
• Working Memory Capacity is a more specific term that refers to the amount of information that can be temporarily held and manipulated in the mind. It's a component of the overall processing capacity of the conscious mind.

So, while working memory capacity is a part of the processing capacity of the conscious mind, it's not the entirety of it. The processing capacity of the conscious mind also includes other cognitive functions beyond just working memory.