Inger Sverreson Holmes

Is This An Overview?
There are aspects of life in which people want diversity of views, with disagreement expected. But, when the expectation is that the decision makers are supposed to provide a similar judgement within similar contexts, the diversity of views is harmful. These are noisy judgements. While biased judgements are systematically off, noisy judgements are those in which agreement is expected but not attained.

Whether in a public or private organization, their representatives are meant to provide a similar product no matter who is using their service. In practice, those using their services enter a lottery as to whom they receive as a representative. The outcomes depend on who is asked, for the person can receive someone favorable or unfavorable to them. Leading to very different outcomes for people within similar circumstances, rather than the expected reliable judgements.

Noise is the unwanted divergent judgements. Noise is more disagreement in a system than what is expected. Noise leads to unfairness in society, and a loss of profit for firms. Decision hygiene is meant to reducing noise which leads to better decision making. There are practical steps that everyone can take to reduce the amount of noise in the system, and take a noise audit to find out how much noise there is.

What Is An Example of Noise?
Judges are expected to deliver similar sentences to similar cases. But, there is a lot of noise in the sentences that judges make. Judges use their discretion to tailor the sentence with various factors. Although this discretion is meant to enable better outcomes, the discretion also creates discrimination due to arbitrary cruelties.

The noisy sentences received attention, leading to sentence guidelines. The guidelines reduced noise, but judges objected due to their lack of discretion. When the guidelines were removed, noise came back into sentences given. This created law without order.

How To Understand Noise?
A judgement is the conclusion. It is a process of mental activity and the product. A judgement is never certain. It includes reasonable disagreement. A judgement has an expectation of bounded disagreement. The amount of disagreement that is acceptable depends on the problem. Large disagreement violates expectations of fairness and consistency when representatives of public or private institutions are meant to be interchangeable and assigned quasi-randomly. Noise in the judgements are errors, and in a noisy system, the errors do not cancel each other out.

Organization and people tend to maintain an illusion of agreement, even though they disagree in their judgements. People tend to think that others share their beliefs, that they understand reality the way the individual does. With naïve realism, people assume that there is a single interpretation, which is rarely challenged. Organizations prefer consensus and harmony over dissent and conflict. Procedures are designed to minimize exposure to disagreement, and explain disagreement away.

Noise is unwanted, and noise is not always unwanted. Variability in judgement is acceptable when it comes to experiences with expected diverse views. Such as innovative solutions to problems, in competition, and art.

Noise is undesirable variability in judgement to the same problem, which does not apply to singular problems that are not repeated. But, there could be counterfactuals, as different decision makers with the same competencies could have made different decisions.

Why Is There Noise?
Noise can occur even with the same facts, as the same facts on different occasions produce different results. It is not just different people that can have different judgements, but also the individual. Mood affects what the individual thinks, and how the individual thinks. Making people less consistent than they think.

Overconfidence in predictions reduces the quality of the predictions. Perfect predictions are impossible, but that does not prevent overconfidence in predictions. Experts tend not to do much better than everyone else when making predictions. What experts know is how to explain themselves and see the different issues involved, but not make better predictions. Better forecasters tend to be those who continuously update their beliefs.

People jump to conclusions based on little information while believing that their views are based on appropriate evidence. Building evidence when a conclusion has been made, rather than seek alternative explanations. People reply on empty explanations to enable coherence of events.

People can have different views based on earlier impressions. Judgements are affected by prior attitudes. Interpretation of facts depends on prior impressions. The affect heuristic, also known as the halo effect, occurs when people use their emotions to make decisions. Applying the same favorable or unfavorable emotions to a person, even though the person is complex.

How To Reduce Noise?
Decision hygiene is the term meant to indicate when there is an attempt to reduce noise. This can include sequencing information, independent assessments, referencing the outside view, and aggregating various independent judgments. A noise audit can be used to understand the amount of noise in the system. Within a noise audit, the same case is evaluated by different individuals.

When making a collective decision, better to apply a wisdom of the crowd’s approach. To gain a wisdom of the crowd, judgements need to be independent of others. Individual judgement needs to not be influenced by other people’s judgements. What influences judgement is popularity for popularity is self-reinforcing as people do what they see others doing.

Simple rules are better than human judgment. Rules do worse when the person has decisive information that the model did not consider, which is called the broken-leg principle. The reason why rules do better is due to the amount of noise in human judgement. Rules do better but they are not perfect. Models do better, but not by much. Resistance to rules tend to be that humans are allowed to err, while machines are not given that permission.

Rules are complicated. Rules try to eliminate discretion, while standards provide discretion. Some rules restrict behavior without specifying the behavior. This creates a problem of arbitrary decisions. But if the behavior would be specified, then people would be able to behave in inappropriate manner with behavior not covered by the rule.

Not all noise needs to be removed. Removing noise can be costly, create their own errors, reduce dignity, and noise can be needed for evolution of values.

Caveats?
Some parts of the book are related to the authors prior works. The prior work is referenced, without going into detail. There is a bit of statistics, which could be better understood by those who already have some knowledge of statistics.… (plus d'informations)

This book has some merits, but being interesting isn't one of them. It is repetitive and filled with statistical discussions. I love, absolutely love, statistics, but there are ways to discuss them that isn't just plain boring. Also, some of the statistical data they presented seem to support their conclusion, but...and this is a big but...the effect was small enough that it likely didn't meet the criterion of being important. Significance isn't enough; is the difference big enough to cover the deviation and the overlap? And even if it is, does it matter? If I'd finished the book, perhaps they'd have convinced me it did, but I couldn't slog through any more of it, even though their major premise is accurate. The world does have a lot of noise in our judgement, causing one person to judge vastly different than another, and even the same person to vary depending on the environment. I'm not sure AI is the answer, though, even though they are enthusiastic. The biases that develop quickly in AI seem to make that a risky proposition. Overall, I don't recommend it.… (plus d'informations)

I found the first 200 pages of this book to be almost impenetrable and frequently forgot a sentence shortly after reading it.

That said, the book and its import improve.

If you’ve read Kahneman’s earlier work, Thinking Fast and Slow, you’ll be familiar with the use of a core metaphor to the argument. While the book says it’s about “Noise” it’s really about the statistical sources of bad judgments.

Noise is the shorthand systems engineers use to explain flaws in the system.

Kahneman et al want us to take a systems view of bad judgments, and bad judges. There is hope for them yet.

Forestalling judgment until the evidence is collected, breaking down complex judgments to their constituent parts, employing baseline comparisons, and employing objective referees will all yield better judgments in business, in law and medicine, and in life.

I certainly hope so. I have trouble just dealing with the volume of judgments I am called upon to make everyday in business.

There is a lot here to think about, especially about the people who are the experts we rely upon, and how they frequently get important things wrong.… (plus d'informations)