The Devil is in the System

Jul 20 2010

I recall on holiday last year reading Elliot Aronson’s sociology course text book, “The Social Animal”. Heavy reading perhaps for the beach in Bali, but I devoured it faster than any Stephen King pulp fiction. What amazed me at the time was how clearly it demonstrates the malleability of human beings to social influence. It left me feeling that we were all, well rather daft creatures, and that our sense of autonomy was probably often an illusion.

Almost a year on and last week I was watching a BBC documentary about the Holocaust. I was reminded of the apparent normality of Auschwitz boss, Adolf Eichmann. There he was living in a little cottage next to the Auschwitz camp system, killing Jews by day, reading bed time stories to his kids by night. Much as I wanted to just label him ‘a devil’, but it didn’t quite seem to add any explanation as to why aparently normal people end up doing such devlish things. It seems it has been troubling my unconscious; without thinking about it, I picked up Philip Zimbardo’s, the Lucifer Effect – How Good People Turn Evil, at the airport this week.

Professor of psychology at Stanford University, Zimbardo, conducted a classic sociology experiment in 1971, called the Stanford Prison Experiment. He created a mock up prison in the basement of the campus, and randomly assigned guard and prisoner roles to 24 normal middle class volunteers.

He had to stop the experiment before the first week was up because of the emotional distress prisoners were under. The experiment emerged as a powerful illustration of the potential toxic effect of bad systems and situations in making good people behave in pathological ways. The intensity of the guards domination and the speed with which it appeared in the wake of a prisoner rebellion, were surprising. Healthy and normal young men started losing a sense of personal identity as they were subjected to arbitrary control of behavior, sleep and privacy deprivation. More on the experiment can be found on

Zimbardo sums up that neither the guards nor the prisoners could be considered ‘bad apples’. In fact they were selected for their normal profiles. It was then the ‘bad barrel’ of the situation which created powerful forces to change their behavior – the roles the rules, norms, anonymity of person and place, the dehumanizing process, conformity pressures, group identity and more.

He draws a striking parallel to the events at the Abu Ghraib prison in Baghdad. Originally Saddam’s torture prison, it was taken over in the Iraq War by US forces. It subsequently became famous when pictures of abuses conducted on Iraqi’s by US military personnel began appearing in the media. If you recall it became obvious this was no isolated incident. Nor was it limited to US soldiers, it seemed the British army were also shaming themselves with similar immoral acts. Prime Minister’s Blair and President’s Bush’s so called “War on Terror” started to look very suspicious.

In an attempt to distance themselves and the system from what they called at the time, “a few bad apples” the Bush administration ensured a few soldiers at the bottom of the chain of command received stiff sentences. Zimbardo argues that many of these soldiers showed no signs of pathology before they entered the Abu Ghraib situation. He shows that the hellish conditions (they were under constant attack) combined with the Bush administration’s open acceptance of torture for intelligence investigations had much more to answer (Guantanamo Bay set the scene for the spreading abuses elsewhere in the World …).. A red flag should be raised in your mind when your own democratically elected president starts to see the Geneva Convention on the rules of war as an inconvenience…

It is tempting to interpret the “evil acts” of people as a result of their individuality, or often more superstitiously as the work of the devil or Satan. The Stanford Prison Experiment, and many other sociology experiments, along with the reports of victims and abusers from many terrible events over history, suggest otherwise. We need to understand the Person in the Situation. It is all too easy to create evil traps for good people. In a sense the devil does reside in all of us; the line between good and evil is in the center of every human heart.

So what are these social dynamics that can exert such a powerful effect on our behavior, and unleash the devil in us?

Peer pressure. There is a huge pressure to feel part of the group, to not feel left outside or rejected. Groups can get us to do things we might otherwise not do on our own. Indeed recent neuroscience imaging has shown that conformity or independence light up different parts of the brain.. If you make independent judgments that go against the group, your brain lights up in areas which are associated with emotional salience..autonomy has a psychic cost.

Obedience to authority. The classic Stanley Milgram experiment where volunteers gave electric shocks to other human beings is also detailed in Zimbardo’s book. Most subjects acquiesce to the experimenter’s encouragement to continue administering increasing shocks, even as the pain rises. It works in different cultures, on puppies as well as human beings. When we give up our freewill to authority figures we can all too easily surrender our humanity.

What social psychology has given to an understanding of human nature is the discovery that forces larger than ourselves determine our mental life and our actions – chief among these forces is the power of the social situation. (Harvard psychologist – Mahrzarin Banaji).

Insights from research into other domains highlight other forces at work: deindividuation dehumanization, and bystander apathy. Experiments have shown that when our identity is anonymous we are more likely to engage in antisocial behavior. In the Stanford Prison Experiment mirrored glasses did the trick for the guards. It is no coincidence that when people go to war they don the equivalent of war paints. In the Rape of Nanking a Japanese General reported it was all too easy for his soldiers to brutally massacre Chinese civilians; “because we thought of them as things, not people like us”. At My Lai it was “gooks”, in Iraq it was “towel heads”. By seeing the enemy as inhuman people are able to disengage morally – time and again governments have set out to reframe the enemies of the state in this way.

So how can we transform our world into one with a more positive future, and resist unwanted social influences? Zimbardo provides a ten step program:

1. Admit mistakes, don’t go make errors to save face

2. Be mindful, too often we run on autopilot, never go mindlessly into new situations

3. Be responsible for your decisions and actions - no one will accept your pleas of following orders

4. Be yourself, the best you can – don’t let others put you in box, don’t be anonymous

5. Respect just authority but rebel against unjust authority

6. Value your independence, and appreciate that different groups have different things to offer

7. Be more frame vigilant – look out for the way media frame events for example

8. Balance time perspective – never lose sight of the past or your future

9. Don’t sacrifice personal freedoms for an illusion of security…classic Bush tactics

10. Oppose unjust systems – gangs, cults, corporations etc, be a hero.

What commercial implications does this have?

Perhaps an obvious one is the way we conduct and interpret focus groups. It is clear that there are some strong forces at play, both among participants and with the moderator. Some of the advice above could be adapted for instructing focus group participants to be themselves and not be swayed by others.

The need for social belonging is well established in many understanding consumer need states. We might be inclined to give it yet more focus. Most surveys focus on the individual, without really taking the time to look at the system of interactions and social dynamics that surround product purchase or consumption. The advent of social media has opened up a whole new world to understand how these forces might be playing out in popular culture.

There also seem to be lessons for organizations and brands seeking to behave in more responsible and effective ways. As has often shown to be the case, it is all too easy for a culture of dishonest complicity to develop within a company. Strong and morally guided leadership, an open culture, empowered and respected employees are all keys to keeping the devils out of the system.


The Lucifer Effect: How Good People Turn Evil by Philip Zimbardo (2008)

No responses yet

Leave a Reply