Every day we are bombarded with orders. Some big, some small, some coming from our minds, some coming from somebody else’s, and most, recently, some coming from devices, that sit on the kitchen or office table, appearing to be harmless and unsuspectingly quiet.
As we like to think that we are quite familiar with the orders issued by people, how should we take orders issued by a device?
When I first heard about the Milgram study, I wasn’t aware of the lasting impression it would leave on me. In 1961, following the Nuremberg trials — in which main convicts used the argument that they were “just obeying orders” to justify mass killings in World War II — Milgram, a professor at Yale, conducted a study, researching how far people would go in obeying an instruction that involved harming another person.
In order to this, he invited common and unsuspecting people as volunteers to participate in a experiment labeled differently than the one that in reality would take place. He assigned some of the staff to the role of “experimenter” — a person that would give the participants orders to administer electrical shocks to other people (in this case actors that would emulate sounds of pain), each time their answers to word pairing questions were wrong.
Invitation for the study
There are many interesting details in the setup of the study, and I think it benefits all to learn more about it (you can easily find the paper and loads of other information online), but making a long story short the surprising and somewhat shocking results were that, despite being warned of the gradually increasing pain caused by the shocks as they progressed on the questionary and the voltage scale, participants still obeyed to the orders to admnuster them (without much further elaboration on why they should do so provided by the experimenters).
In fact 65% proceeded to administer shocks until the highest level of electrical discharge, meaning that much fewer than initially expected offered some sort of resistance and refusal to go any further.
This revealed some some sort of automatic obeying reaction to authority in human nature, that I found highly disturbing and continue to find to date.
I believe that it is since then that I regard authority with a grain of salt, specially that one that asserts itself by raising the voice, and not offering any justification to why its commands should be obeyed. (This was to be further fueled in teenage years by the words of the song “Minority” by Greenday, replayed over and over again, oh yeah 🤘).
“I don’t need your authority / Gone with the moral majority/ Cause I want to be the minority”
It seems, in principle, that history could in some respects have been better, if people nurtured that kind of healthy disrespect for authority, for complying with orders they don’t fully understand. Perhaps as individuals it would still be hard to face some of the pressures, but if masses opted for this behavior of questioning authority, some things could have gone other way. Well, I admit, “what if” exercises are always somewhat superfluous to do when it comes to history, and yes, hindsight wisdom can be particularly annoying…
This brings me to that particular day, when my watch gave me the blinking order: “fill your day with physical activity”… That would have been quite ok, if facebook, hadn’t, just some days earlier, given me the somewhat contrary order to “post more regularly”.
What to do?
There was this sudden realization that if devices were getting smarter about us, recognizing patterns perhaps sometimes not even evident to ourselves, and shouting out orders, it was absolutely crucial that we as humans, were able to sort out the good orders from the less good ones, and not fall prey to obeying everything sheepishly, running the danger, as the volunteers of the Milgram studies did, to be willing to inflict harm on others and ourselves by doing so.
If machines are getting smarter about our now-less-intimate thoughts, then we must not forget that most work at the service of others, their creators and it is not clear that our health and wellbeing are the objectives they are ultimately optimizing for.
Don’t eat the ice cream (unless it does you good and you want to)
On the verge of a new year, I would dare say that perhaps one of the critical skills for human beings going forward, if it wasn’t before, is to be able the dissect orders into their basic components, trying to assess what will be most helpful to us, being aware that (gladly) now machines simply cannot (perhaps yet) force us to do whatever they want us to do — they can mostly shout out orders, like other humans do. It is still up to us, to decide to comply or not, having a part in what gets done to us.
Unlike one of the Adweek’s 10 Best Ads of 2017(perhaps also a strong contender to one of the creepiest) from ice cream brand Halo Top, in which a woman is fed icecreams by a machine that obnoxiously decides what is best for her and seems quite unable to really refuse the offering, we still have a say on this.
And talking about commercials, almost everyone is familiar with the technique of having someone put in a lab coat, using his/hers makeshift “authority” to sell us more toothbrushes or washing powder. This has been used for decades, and is one of the forms in which authority speaks to us, triggering once more our instincts to believe it and obey.
Perhaps more pressing than the so much talked about jobs that are going to be stolen by machines and what not, our major concern should be with the thousands of orders that are already being issued by thousands of devices, and applications, that to some extent do not act in our best interest, although definitely acting in the interest of somebody…else.
Which leads me to other point, if machines are one day able to know ourselves better than we do — I was so surprised to see that a machine recognized a person in an image of an algorithm, that the human eye wouldn’t be able to — what happens when they know all the buttons to push, and words to use to get something done by us?
Only the development of observation and critical thinking skills, and perhaps a bit of knowledge of history, can prevent us, from automatically agreeing with what is being asked of us. Or what would be the price of the ultimate system that would take only our best interest into account? And in this day and age, who would be willing to sell it, when they could be receiving daily rents, optimizing their interest first?
Having become acquainted with the classical dystopias by Bradbury, Huxley and Orwell, and observing how they have been in the current year been quoted alongside the series Black Mirror, as plausible portraits of a not so far future, these are my wishes for the new year:
That you may be able to refuse the ice cream, unless it does you good and you really want to eat it.
Update: Just yesterday, 13th Jan, I went to see a play by Gotthold Ephraim Lessing, Nathan the Wise, from 1779. One of the lines from the dialogue struck a chord, seeming particularly timely and relevant for this discussion…such a coincidence and written more that 200 years ago:
“So — And you obey without much scrupulous questioning?
FRIAR.
Were it obedience else, good sir?
TEMPLAR.”