Looking for Clues

Nov 21 2010

“Our observation of nature must be diligent, our reflection profound, and our experiments exact.  We rarely see these three means combined; and for this reason, creative geniuses are not common.”  - Denis Diderot

Sherlock’s comeback

I have loved the Sherlock Holmes stories since I was young, and looked forward to the BBC’s latest updating of the stories eagerly. I was not disappointed, and consider the three episodes to be one of the best adaptations I have ever seen (and in a different league from Guy Ritchie’s dismal efforts).

The most impressive part of the adaptation is that the completely authentic setting of London in 2010 is complemented by a faithfulness to the spirit of the original characters, situations and stories which shows a profound love of the original books and impressive narrative and empathic skills. John Watson is still a veteran of the Afghanistan war who publishes their adventures (this time as a blog), Sherlock is still a (possibly slightly more weird) “high-functioning sociopath” who uses the latest advances in mobile technology and forensic science, and their relationship a strange coincidence and continuously evolving surprise based at the same address in Baker Street.

Observe and deduce

What can we learn from Sherlock’s methods?  The one striking thing about Sherlock Holmes is his painstaking observation of the world, linked to continuous generation and testing of new hypotheses about the problems he seeks to solve.  He is supremely proactive in seeking clues through simple observation.  In Sherlock Holmes Was Wrong, Pierre Bayard discusses the different ways in which Sherlock Holmes observes.

Firstly, he looks for physical signs of evidence such as his first introduction to Watson, which is closely modelled on one of their early encounters in Conan Doyle’s original version of “A Study in Scarlet” (updated to A Study in Pink):

“You appeared to be surprised when I told you, on our first meeting, that you had come from Afghanistan.”

“You were told no doubt.”

“Nothing of the sort.  I knew you came from Afghanistan.  From long habit the train of thoughts ran so swiftly through my mind that I arrived at the conclusion without being conscious of the intermediate steps.  There were such steps, however.  The train of reasoning ran, ‘Here is a gentleman of a medical type, but with the air of a military man,  Clearly, an army doctor then.  He has just come from the tropics, for his face is dark, and that is not the natural tint of his skin, for his wrists are fair.  He has undergone hardship and sickness, as his haggard face says clearly.  His left arm has been injured.  he holds it in a stiff and unnatural manner.  Where in the tropics could an English army doctor have seen much hardship and got his arm wounded?  Clearly in Afghanistan.’ The whole train of thought did not occupy a second.  I then remarked that you came from Afghanistan, and you were astonished.”

In the remake, Sherlock’s deductions are cleverly shown onscreen, so we can see his thinking as it evolves.

What to observe?

Sherlock does not observe randomly, but always has a theory (or theories) in mind. As well as looking for physical signs, he searches for traces left in the environment (fingerprints etc), objects which may tell a story (read Snoop for more on this side of detection), written documents which may have important information, and indirect clues such as cigar or cigarette ash which can reveal something through analysis of their source.  In 2010, many of the traces relate to digital technology, and Sherlock makes prodigious use of mobile phones (messages, numbers) and the internet (ISP addresses and instant searches) to reveal the traces of a crime and its perpetrators.

He also uses psychology (although not always as well as he could, as we see below), to interpret people’s behaviour and also reactions to situations and interrogations.  It is telling that he rarely if ever asks someone directly about their behaviour (let alone if they ‘did it’), and relies on indirect clues from their actions and mental states as revealed in traces and responses. Above all, Sherlock is a great ethnographic researcher, using clues about the consumer based on what they do, their environment and context, and how they behave in different situations to understand their motivations.

Detection is backwards

Importantly, while Sherlock uses ‘deduction’ (in its loosest sense), he generally reasons backwards, as in the example of understanding Watson’s history, or in the episode A Study in Pink where he works out that a victim must have travelled from South Wales to London, because her coat is damp and London has not had any rain! He also makes frequent use of comparisons to better understand evidence and its meaning, showing the importance of constantly seeking context and relationships to make sense of individual data points.

Above all, he is a great example of Bayes theory, and the importance of prior knowledge to the interpretation of data.  Data can only make sense as far as you already have a frame of reference with which to interpret it – without the frame of reference, even the most important clues may be meaningless.  In market research, the frame of reference is always the client’s business issue and financial motivation as we have written about previously. And market researchers have the added complication that they are sometimes asked to predict the future, as well as explain the past (which is much more straightforward).

Why does Sherlock makes mistakes?

Pierre Bayard points out that Sherlock frequently gets things wrong (and there are examples of that in the BBC’s production).  He discusses in great detail supposed flaws in the case of the Hound of the Baskervilles, but there are two main reasons why this happens.

Firstly, Sherlock often confuses science with statistics, attributing too much confidence to results based on probability.  This is also a frequent failing of market researchers, who focus too much on statistical testing and too little on the patterns of data and consistency across different information sources.  For instance, if 20 statistical tests are conducted and one shows a significant difference, that does not make it worth reporting let alone a key finding.  Although this is a (perhaps) trivial observation, it is nevertheless a frequent phenomena (and one I find disturbing).  Market researchers (and clients) would do far better to focus on the real world impact of any difference instead of its statistical significance.

It’s the people stupid

Sherlock’s other failing is his poor understanding of his fellow human beings and their motivations.  He tends to assume everyone is like himself (hence his surprise when others do not reach the same conclusions, “what is it like to be in your brains?”), and his dealings with Watson and his housekeeper are often awkward.  He shares many characteristics with those who model the financial markets, in taking too rational view of the world.  The lesson is that you can only make sense of your observations and the physical evidence, if you have a sound understanding of what makes people tick.

This is a critical lesson for research and researchers.  You can have as much data as you want, but unless you understand the situations and motivations of the people involved, you will never be able to understand their past behaviour, let alone predict their next move.

REFERENCES

Sherlock by Paul McGuigan (2010)

Sherlock Holmes was Wrong: Reopening the Case of the Hound of the Baskervilles by Pierre Bayard (2008)

Snoop: What Your Stuff Says About You by Sam Gosling (2009)

The Complete Sherlock Holmes by Arthur Conan Doyle

No responses yet

Leave a Reply