When Artificial Intelligence Takes Over, We'll Say, "You didn't take it. I gave it to you."

One of the earliest posts I wrote on this blog as about how technology -- particularly drones -- have put us under the the gaze of the state in a way we thought was only possible in science fiction: "Drones, 1984, and Foucault's Panopticon." I've written a lot more about drones, 1984, Foucault, and surveillance in subsequent posts.

Since that time, I've also internalized what I understand to be Foucault's central premise: the temptation to have power over other people is enormous, and it's particularly difficult to resist when it is accomplished in the guise of something else, something that we tend to consider very positive -- observing, comprehending, recording, organizing.

That has spiritual implications at the level of individual experience -- the observing, comprehending, recording, organizing that we do as we seek to navigate the external world -- and that is something I plan to write about in a future post.

But it also has implications for the ways in which we are subjected to observing, comprehending, recording, organizing. I've become more and more aware of the way that the many tiny bits of data that we now slough off each day, like so many dead skin cells, add up  -- in the hands of large computer operators (Big Data) -- to terrifyingly complete and intimate pictures of our personal lives. 

Some time in late 2020, I watched the documentary, The Social Dilemma, on Netflix: "This documentary-drama hybrid explores the dangerous human impact of social networking, with tech experts sounding the alarm on their own creations." The whole documentary is insightful, and the commentary by Jaron Lanier is particularly good. My key takeaway from that documentary was: don't talk about "if" artificial intelligence (A.I.) takes over, as if it is something in the distant future; AI is already operating on us in the form of the social networks that we participate in day after day.

What I've particularly struggled with since then is this: how can we ordinary people use social media to benefit us, without all of it falling into the maw of A.I.?

It made me think of this picture that one of my kids drew when they were about three (above).

(At the time it was amusing, and my main feeling was pride at how clever my child was. It reminded me of Grendel, and I wondered if they had been reading Beowulf behind my back. Now I'm more terrified .... )

The thing that is so painful is that, each day as any of us interacts with Facebook, Twitter, Instagram, Linkedin -- or even this blog -- not to mention every Google search or credit card purchase, we are not being strong-armed into coughing up our deepest secrets -- we're willingly surrendering them! (To quote Roy in the movie Matchstick Men, "You didn't take it. I gave it to you.")

So, yes, Foucault ... power, observing, comprehending, recording, organizing. Yes: drones. Yes: Big Brother; yes: surveillance. 

But maybe the BIG question we each need to ask ourselves when we wake up each morning is:

"How much – and what type – of data will I surrender today?"

Drones, ISIS, and Permawar

 Let's be clear: there will always be an "ISIS" out there.

The Obama administration had no sooner re-entered the Iraq War, ordering air strikes in northern Iraq against ISIS, than it announced that it "needed" to send the military into Syria.

Oh, we're not doing anything "risky" in Syria - just having a look. In other words, using spy planes and drones to get visual control over even more of the Middle East.

The U.S. narrative goes something like this:

* Somebody "bad" (e.g. ISIS) is doing bad stuff.

* The U.S. wants to "help" -- without overcommitting. We'll just start with a few advisers (to instruct, not to fight) and a few drones (to survey, not to kill).

* One thing leads to another and there's yet another fight. (Lucky we were there . . . )

Does it every occur to us that we've got the narrative (and the causality) backwards? That the truth is something like this?

* Permanent war is what suits us.

* All we need is the next excuse for more war.

* To find an excuse, we need "eyes" there. (Send in the drones.)

* Any bad actor will do as a reason for drone surveillance.

Once again, I return to the wisdom of Michel Foucault. Total surveillance is not just playing an incidental or supporting role in total state control and permanent war. Total surveillance is the infallible and essential heart of total state control and permanent war.


Related posts

In the old order of things, power places itself on display, and hopes that the population sees fit to obey. In the new order of things, power compels every member of the population to display himself or herself . . .  In the new order of things, the courts are bypassed and the instruments of discipline -- observe, classify, examine -- run rampant.

(See "Surveiller et Punir" Indeed!)

The U.S. can get more "bang for the buck" out of each pair of boots it puts on the ground, because -- through the magic of robotics -- it can back up those boots with Hellfire missiles and 500-lb. bombs. For the folks back home, it helps maintain the illusion that the U.S. isn't really intervening in a way that risks escalation. For the population of the affected areas of Iraq, it helps maintain the balance of terror -- because those armed drones are just part of a much larger fleet of drones that is patrolling the skies over Baghdad.  ("Is that drone overhead aiming . . . or just 'looking'?" From the ground, one has to assume they're all aiming . . . . )

(See Armed Drones Over Iraq: A Force Multiplier (Which Is Precisely Why They Are So Dangerous) )


The United States perpetuates a state of permanent war. The names change -- hell, sometimes they change by just a single letter -- but the result is the same. Call it "permawar."

(See #Permawar) 

Why U.S. Society is Caving in to the Lure of Wall-to-Wall Surveillance

I was reuning with high school friends last week. One of them is now a principal in a suburban elementary school. He loves his work -- and I can't think of a better person to be shaping young lives.

Over a diner breakfast one morning, he said, "You know, Joe, we now have cameras throughout the school. So when a parent calls and says their kid had a problem, we can pull up the video and see exactly what happened." He told me that many schools installed camera systems in the wake of the Newtown shootings. (See for instance Schools beef up security after Newtown; cameras, panic buttons installed)

For him, as a principal, total visibility into the school is great. (Just imagine trying to settle a he-said, she-said dispute without being able to go back to replay.)

And the technology supports crisis management, if that should become necessary. (Local police are able to link in to the system.)

Of course, there are limits. There are no cameras in the bathrooms. (Yet.)

My friend is aware of my work on the issues of surveillance and drones. We talked about some of the philosophical and ethical issues. In general, the children in an elementary school are assumed to require the oversight and direction of school staff, so it is not unreasonable to have adults watching the children. On the other hand . . . .

I proposed a thought experiment: "Can you envision a situation in which a parent came to you about an issue, and you might elect to not make use of the video record? What would it be like to engage the people involved in resolving the conflict without being impacted by that video?"

(I think that my proposal was stimulated by the fact that what had been really interesting to me was listening to his descriptions of engaging with young boys who were engaging in troublesome behavior. As he recounted the way he talked to them, I thought, "He is not coming at them from a position of power and threat; he's using empathy, humor, modeling.")

We talked about the work of Michel Foucault, and his insights about how observation -- especially total observation -- is the tip of the iceberg in a system of one-way control. From the time I started focusing on the problem of drones -- about four years ago -- the work of Foucault and the alarm he raised about the "panopticon" society has been in the front of my mind.

How does someone who holds in his hands the power to see everything resist the temptation to control, and instead focus on the need to understand?

This conversation helped crystallized for me the broad insight: "Surveillance is useful and also threatening."

I hope a large number of people will take this up and struggle with it.


Here's more on the issue as it relates to schools: "Privacy vs. Security: Are you prepared for the thorny issues surrounding student surveillance?" by David Rapp in Scholastic. 


Related posts

The panopticon was a prison design that reversed the old paradigm, in which prisoners were stored away, "out of sight, out of mind," and instead arrayed them in a way in which they could be observed as efficiently as possible by the fewest number of managers.

(See Drones, 1984, and Foucault's Panopticon)

In the old order of things, power places itself on display, and hopes that the population sees fit to obey. In the new order of things, power compels every member of the population to display himself or herself . . .  In the new order of things, the courts are bypassed and the instruments of discipline -- observe, classify, examine -- run rampant.

(See "Surveiller et Punir" Indeed!)



A large number of people are marked for exclusion and deprivation -- and worse -- because they have characteristics that are susceptible to the whole apparatus of power:  they are easily recognizable as  NOT "normal" or "right" or "acceptable" . . . under the gaze of surveillance this condition is recorded and propagated . . . for perpetual recording and processing within the data centers of power . . . accompanied by intermittent acts of physical and cultural injury -- random, senseless -- to reinforce their unshakeable status. 

(See Drone Gaze, Drone Injury: The War on Communities of Color)

Re-reading George Orwell's 1984 recently made me see at least 15 ways 2013 is like the world he describes in the book . . . .

See 2013 = 1984 ?

Foucault and Drones: "Surveiller et Punir" Indeed!

More at: Can we stop the DRONES?

It was several years ago that I first drew the connection between the work of Michel Foucault and drone surveillance and drone killing. At the time I pointed out that "Foucault understood [surveillance] to be symptomatic of the much larger project of societal rule. To Foucault, prior to the physical and bodily aspects of control and manipulation, there are aspects that have to do with seeing, knowing, naming, and categorizing."

We now know far more about how the Obama administration carries out drone surveillance and drone killing, especially in light of the New York Times article exposing Obama's "secret kill lists" several weeks ago. It seems like an appropriate time to revisit Foucault's dissection of surveillance and punishment.

"THE MEANS OF CORRECT TRAINING"
The fundamental insight of Foucault's work, "Surveiller et Punir: Naissance de la prison" (published in English as "Discipline & Punish: The Birth of the Prison") is that through a set of practices (observation, norm-imputation, examination), extra-legal compulsion in diverse forms comes to permeate society. These practices are, in part, mimetic of the truth-finding and justice-dispensing practices of a true legal system, but fall far short of judicial due process.

Why speak of "rights" and "justice" when you have the means to impose power -- i.e. "train" your subjects -- unilaterally?

"HIERARCHICAL OBSERVATION"
According to Foucault, one of the principal marks of modern social organization is that observation is so important that whole categories of roles are reserved for simply watching what others are doing; in fact, organizations now institutionalize armies of watchers, as well as watchers of the watchers. (Got supervisors?)

Drone operators are certainly quintessential "watchers" of this kind. But Foucault's insight that these systems tend toward hierarchical (and total) observation alerts us to the fact that what lies ahead will almost certainly be worse. Work is being done to make drones "autonomous," that is, fully computer-controlled. "Signature strikes" -- in which observations are processed according to an algorithm to determine if a strike should be carried out, independent of human judgement about the actual identities of the people observed -- are a first indication of how this will be done.

In other words: "drone surveillance"? You ain't seen nothing yet!

"NORMALIZING JUDGEMENT"
All of the surveillance done by the watchers can only be put to use if there are standards of "normal" or "right" or "acceptable" behavior against which to compare the observations. This sets up both a fiction that there is a "right" way to be, and that the watchers are qualified to sit in judgement over the judged.

As the New York Times' description of Obama's secret kill list makes clear, Obama and his national security team sit as judge, jury, and executioner over a process of judgement of people subject to drone observation in Pakistan.

What is less clear is the "norm" against which the people there are judged. Is any adult male in the relevant areas of Pakistan assumed to be a "militant," unless proven otherwise?

What is similarly unclear is the qualifications of the "judges." As Eric Holder made clear when he (partially) divulged the rationale for the Administration's killings during his speech several months ago in Chicago, "due process" does not necessarily mean "judicial process."

"THE EXAMINATION"
Inherent in the scheme of "training" that Foucault understands to be at the heart of modern imposition of control is a third step: examination. If everyone is subject to observation, and everyone is subject to comparison to some "acceptable" measure of behavior, there must be some ritual(s) wherein the individual is required to stand and prove that they "pass the test."

The strange thing about the current state of U.S. drone killings is that they do not yet hint at what the "examination" stage will look like. At present, people are observed, and judged, and then unilaterally killed. But is that where it will all end?

If Foucault is right, the killings are just the beginning; they are just the leading edge of a much broader pattern of social control.

* * * * *

In the old order of things, power places itself on display, and hopes that the population sees fit to obey. In the new order of things, power compels every member of the population to display himself or herself.

In the old order of things, individualism is the way of the world. In the new order of things, individualism is relegated to the margins - the abnormal - and the cost of extreme individualism is extermination.

In the old order of things, all of society's capability for science and humanism is focused on the operating of due judicial process. In the new order of things, the courts are bypassed and the instruments of discipline -- observe, classify, examine -- run rampant.


Related posts

The biggest idea coming out of the 2013 Drone Summit? We will only deal successfully with the crimes being committed using drones when we understand them as part of the much larger war against communities of color . . . .

(See Drone Gaze, Drone Injury: The War on Communities of Color )







The panopticon was a prison design that reversed the old paradigm, in which prisoners were stored away, "out of sight, out of mind," and instead arrayed them in a way in which they could be observed as efficiently as possible by the fewest number of managers.

(See Drones, 1984, and Foucault's Panopticon)






Re-reading George Orwell's 1984 recently made me see at least 15 ways 2013 is like the world he describes in the book . . . .

See 2013 = 1984 ?






More at: Can we stop the DRONES?

* * * * *

Image sources:
Predator cockpit from Drone Wars UK
Situation room from Churls Gone Wild

Drones, 1984, and Foucault's Panopticon

Are drones such a menace because of their weapons? Or because of their surveillance? Or is it even bigger? Ask Foucault . . . .

More: "Why focus on drone attacks?"

For those of us who have been focused on the boots on the ground in Afghanistan, data published by the New York Times yesterday added a whole new dimension: it published statistics on drone sorties and strikes in Afghanistan in 2009.

The article is an important counterpoint to the headline-grabbing drone strikes by the CIA in Pakistan, and is worth reading online. What you miss by looking at the online version of the story, however, is the graphs of month-by-month activity, which show several striking trends. Strikes by missiles and bombs from drones were up sharply in the first half of the year; the second half of the year, by and large, showed a slight tapering off. (This was consistent with the overall trend in attacks by aircraft and drones.) At the same time, the number of drone sorties -- that is, missions of all types, including "just surveillance" -- showed a very steady upward trend month by month throughout the year. As the article reports:

"Predators and Reapers [are] now supplying more than 400 hours of video a day to troops in Afghanistan . . . Some of the Reapers will soon carry 10 cameras instead of just one, and 30 by 2011, adding to the profusion of video."

Up until now, when I have thought about drones, I have thought about the ethics of drones as a weapon in general, including the specific problem of whether drone strikes constitute extrajudicial executions under the Geneva Conventions.

I have always also had a nagging sense that there is something wrong with the surveillance aspects of drones, as well. But I didn't become so focused on it until confronted with the numbers.

Science fiction? 15 Ways George Orwell Was Right . . .

There is perhaps no more powerful portrayal of the problem of extreme surveillance than George Orwell's novel 1984. One of my strongest memories of 1984 was the efforts that Winston had to make simply to find a place in his own apartment where he could escape the observation of Big Brother. Perhaps it was because I, like so many other people, encountered the book as a teenager, when privacy suddenly becomes simultaneously so important as well as so hard to come by, that I remember this so clearly.

While popular culture makes frequent explicit references to "Orwellian" situations that involve doublethink, Newspeak, the Thought Police, and the other ideological nightmares of 1984, I wonder if the real nightmare isn't simply the constant surveillance. I, for one, have always thought that lack of privacy is not an absolute evil, but can only be evaluated in the context of what happens as a result of loss of privacy. I'm beginning to rethink that view.

The French philospher Michel Foucault, in his groundbreaking work Discipline and Punish, reviewed the history of society's efforts to isolate and control elements that caused it problems. Ultimately, Foucault zeroed in on the efforts of Jeremy Bentham and the Utilitarians to apply analysis and scientific thinking to prisons. The result was the panopticon.

Jeremy Bentham's "panopticon" (drafted by Willey Reveley)

The panopticon was a prison design that reversed the old paradigm, in which prisoners were stored away, "out of sight, out of mind," and instead arrayed them in a way in which they could be observed as efficiently as possible by the fewest number of managers.

Presidio Modelo prison, Cuba (2005),
followed the panopticon design.

Foucault understood this to be symptomatic of the much larger project of societal rule. To Foucault, prior to the physical and bodily aspects of control and manipulation, there are aspects that have to do with seeing, knowing, naming, and categorizing.

At the end of the day, is surveillance bad if no harm comes of it? Or is it even possible to separate the two? Consider this quote from the New York Times article. Speaking of Stephen P. Mueller, top air commander in Afghanistan, the article stated:

He said the strikes typically came when troops were caught in firefights or the drones came across people who appeared to be planting homemade bombs, the biggest source of allied casualties. The counterinsurgency strategy "isn't about going out and finding those," he said. "But when we do find them, we obviously do what's necessary."

Obviously.


MORE:

Foucault and Drones: "Surveiller et Punir" Indeed!

Drone Gaze, Drone Injury: The War on Communities of Color

Why U.S. Society is Caving in to the Lure of Wall-to-Wall Surveillance

When the President says, "The American people don't have a Big Brother who is snooping into their business," it probably means the American people have a Big Brother who is snooping into their business . . . .

Fed Up With Being Spied On

DRONES: Build a Foundation for Our 3-D Future

Drone Killings: Come Clean

Orwell Was Right