To confirm or not to confirm? That was not Hamlet’s question, partly because he was pondering weightier matters and partially because people, including presumably even Hamlet, have a very strong tendency to subconsciously choose “confirm.” We simply tend not to ask ourselves my opening question. We tend to confirm.
“Confirmation bias” as it is technically known is a real problem for people in general but it is especially important for anyone doing research. Confirmation bias deals with problems that people have when testing hypotheses. Anyone researching their genealogy should be constantly testing their ideas, both as part of moving forward and as part of improving the ground they have already covered.
There are at least three places where confirmation biases can appear. They are things to watch out for in yourself as you research. Introspection as a researcher is a skill to be nurtured.
The Search
When we look for evidence do we look for all kinds of evidence? If we believe that someone died in 1897, do we look for a death certificate in 1897 and, not finding it, do we look in another year? Often we look in another place because we are “know” when the person died. Worse yet, do we simply conclude that the certificate was never prepared or became lost and not pursue the matter? That is one way confirmation bias appears, we hunt for information in a biased way. A too narrow search can simply turn up a record for the John Doe that most closely matches your expectations, as opposed to a record of the John Doe that was your ancestor.
The Record
Second, what if we are biased in the way we record or fail to record the data? What if when we might make use of the data we remember some of the evidence, the evidence we prefer, and forget what contradicts our expectations? People often selectively forget things that do not fit their notions and have an easier time remembering things that confirm their hypotheses.
When researching a person that we think we have finally understood, what happens when a bit of evidence is found that contradicts what we have come to believe? One possibility is that we decide that we don’t know what to do with it so it isn’t worth recording. Suddenly an opportunity for greater understanding of a complex and contradictory ancestor is lost. A human being remains hidden behind a paper cutout.
The Thinking
The third way has to do with what we do with the evidence once we have collected it. If we find a death certificate with a date that is hard to read, do we decide that the year that seems to end in “2″ must end in a “7″ because that is what we want it to read or do we keep an open mind and consider the possibility that this person died too early to be the one we are hunting?
We ought to try to actively prove that we have the wrong person, not confirm but instead deny. We ought to question the assumption that the year cannot be five years earlier than we thought. We should not simply assume confirmation.
People are not always as simple as our hypothesis
People are complex beings. If all one can find about a person is a few shreds of information, that complexity does not have a chance to come through. Perhaps we become fooled into thinking that long ago, when times were simpler, that people should be simpler as well. If there is more than a trivial amount of evidence remaining about a person, it is likely to be a complex set of evidence full of contradictions. It will contain information that supports several hypotheses. One needs to be aware of those possibilities and not simply focus on and confirm the one we currently favor.
The same thing can happen when we need to bring a great deal of indirect evidence to bear on a problem. Often several hypothesis can be supported by that complex collection of information. One factor in confirmation bias is that, given a wealth of complex information to choose from, people will generally emphasize the information that backs their current notions. The whole point of research is to check those notions. The natural tendency to select only supporting evidence defeats that point. Disproving is as important as proving but people find that mindset very difficult to retain.
Nothing New
Grappling with confirmation bias is nothing new. It has long been seen in human thought. We have tended to effortlessly accept confirmation and either ignore or strenuously deny repudiation for a very long time. Thucydides, an Athenian general and historian of the 5th century BC wrote in his History of the Peloponnesian War—”their judgment was based more upon blind wishing than upon any sound prediction; for it is a habit of mankind to entrust to careless hope what they long for, and to use sovereign reason to thrust aside what they do not fancy.” In research we need to do better than that.