Monday, May 21, 2018

Living—and Dying—By Algorithms

The National Health Service (NHS) in England is one of the oldest government health-care systems in the world, founded in 1948 when the Labor Party was in power.  Despite consuming some 30% of the public service budget, by many accounts it is underfunded, especially when it comes to capital equipment such as IT systems.  This may be a factor in a scandal involving a wayward algorithm that prevented some half-million Englishwomen from receiving mammograms for the last nine years.  Estimates vary as to how serious a problem this is, but it's likely that at least a few women have lost their lives due to breast cancer that was caught too late as a result of this computer error.

A report carried in the IEEE's "Risk Factor" blog describes how in 2009, an algorithm designed to schedule older women for breast cancer screening was set up incorrectly.  As a result, over the next nine years almost 500,000 women aged 68 to 71 were not allowed to have mammograms that they otherwise would have been scheduled for.  When the error was caught, the news media had a field day with headlines like "Condemned to Death . . . by an NHS Computer."  Depending on who's making the statistical estimate, the consequences are either tragic or possibly beneficial.

The NHS's own Health Minister had his statisticians run the numbers, and they came up with a range of 135 to 270 women who may have died as a result of this error.  But others claim that as many as 800 women may be better off because of not having to go through surgical and other procedures based on the false positives that inevitably result from a large number of mammograms. 

While the actual consequences of this problem are ambivalent, it raises a larger issue:  what should we do when computer-generated algorithms that affect the fates of thousands go awry? 

As a practical matter, computer algorithms are part of the fabric of modern industrial society now.  If you want to borrow money, the bank uses algorithms to decide whether you're a good credit risk.  If you look for something online, sophisticated algorithms take note of it and decide what other kinds of ads you see.  And if you're in England or another country where health care is allocated by a central computerized authority, a computer is going to tell you when you can get certain kinds of preventive health care and if you're ill, it may even tell you when you can get treated—if at all. 

From a utilitarian engineering perspective, computer algorithms are the ideal solution for large-scale resource-allocation problems.  Health care these days is very complicated.  Each person has a unique combination of health history, genetic makeup, and needs, and the arsenal of treatments is constantly changing too.  If you are working in an environment of centralized fixed resources (as NHS is), then you will naturally turn to computers as a way of implementing policies that can be shown mathematically to treat everyone equally.  Unless they don't, of course, as happened with the older women who were left out of mammogram screenings by the badly programmed algorithm. 

There's an old saying, "To err is human, but to screw up royally requires a computer."  The NHS flap is a good example of how one mistake can affect thousands or millions when multiplied by the power of a large system. 

The U. S., with its much more hodge-podge mixture of private, commercial, and government health care systems, is still not immune from such errors, but because the federal government doesn't run the whole show, its mistakes are somewhat limited in extent.  There are also numerous outside agents keeping tabs on things, so that an egregious error by, say Medicare, comparable to what happened with the NHS algorithm in England, would probably be caught by private insurers before it got too far.  Just as a power grid with a number of small distributed generating stations is more robust than one that relies exclusively on one giant power plant, the U. S. health care system, even with all its flaws, is less likely to be felled by a single coding mistake. 

Instead, we have widely distributed minor errors that cause more inconvenience than tragedy.  But precisely because the system is so kludged together, it doesn't take much to cause a problem.

Here's a simple example:  my wife is scheduled the day I am writing this for a routine well-person exam that requires her general practitioner (GP) to write a referral for it.  Dutiful organized person that she is, several weeks ago she went by her doctor's office and asked them to do the referral so she could schedule the appointment, and the staff at the office said they'd take care of it.  Yesterday (the day before the procedure), she got a call from the office that was going to do the procedure, saying they hadn't gotten the referral yet and if they didn't get it they were going to cancel the procedure or make us pay cash for it.

So ensued a half-hour or so of near panic, during which time we ran down to her doctor's office and discovered that the lady who was supposed to send the referral out had quit the previous day.  And that was one of the things she left undone. 

When the GP's office staff figured out what had happened, they were very nice about it—they faxed the referral to the proper office, handed us a copy which we carried over by hand to the office needing it, and everything is fine now.  But until all medical offices are staffed by robots and all paperwork is untouched by human hands, people will always be involved in medical care, and people sometimes make mistakes. 

Personally, I much prefer a system in which I can drive over to the office where the mistake was made and talk to the people responsible.  If we had something like the NHS here, the mistake might have been made in Crystal City, Virginia by an anonymous person whom it would take the FBI to discover, and my wife would have been out of luck.   

Sources:  IEEE Spectrum website's Risk Factor blog carried the report of the NHS computer error at  I also referred to the Wikipedia article "National Health Service (England)."

Monday, May 14, 2018

Google's Duplex: Fraud or Helpful Assistant?

Duplex is a new technology announced by Google last week in a presentation by Google CEO Sundar Pichal.  He played some recordings of what sounded to the uninitiated ear like humdrum phone calls to a restaurant and a hair salon to make reservations.  In both cases, the business service providers heard a voice on the other end of the line which sounded to all intents and purposes like a human being calling on behalf of somebody who was too busy to make the call herself.  There were natural-sounding pauses, "hmm"s, and the information about appointments was conveyed efficiently and without undue confusion. 

The only thing was, there was only one human talking in each conversation.  The "agent" making the call was Duplex, an AI system that Google plans to offer to businesses as a giant step forward in robo-calls and related phone activities. 

I happened to hear a couple of these calls on a radio program, and I must admit the computer-generated audio sounded natural enough to fool anyone who wasn't clued in.  Now, nobody happened to ask the computer's name or try to start up a conversation with it about, say, existentialism, and I don't know what would have happened in those cases.  But for routine specific tasks such as making appointments, I suppose Google now has just what we want.  But is this something we really want?

Google thinks so, obviously.  As this example shows, we are rapidly approaching a time when companies will field AI systems that make or receive phone calls with such a good imitation of a live person, that the live person on the other end will not realize that he or she is not talking to another human being.  An Associated Press article about Duplex focuses on some narrow concerns such as state laws against recording phone conversations without notification.  These laws explain why you so often call a business and first hear something like the phrase, "For quality-assurance purposes, this call may be recorded or monitored."  Because it's so easy to include that phrase, I see this as a non-issue.

What wasn't addressed in the reports is a more fundamental question that relates, believe it or not, to a philosopher named Martin Buber who died in 1965. 

Buber's claim to fame is a book called I and Thou which explores the philosophical implications of two kinds of interactions we can have with the world:  the "I-it" interaction and the "I-Thou" interaction. 

A very oversimplified version of these ideas is the following.  When you are interacting with the world as an I to an it, you are experiencing part of the world, or maybe using it.  You have an I-it relationship to a vacuum cleaner, for instance. 

But take two lovers, or a father and a son, or even an employee and an employer.  The I-Thou interaction is always possible in these exchanges, in which each person acknowledges that the other is a living being with infinite possibilities, and ultimately the relationship has a mystical meaning that is fully known only to God. 

It's also possible, and happens all too often, that you can deal with another person using the I-it mode:  treating them as merely a means to some goal, for example.  But this isn't the best way to relate to others, and generally speaking, treating everyone as a Thou respects their humanity and is the way we want to be treated ourselves.

The problem that facile human-voice-imitation systems like Duplex can lead to is that they can convince you they're human, when they're not.  As the AP article points out, this could lead to all sorts of problems if Duplex falls into the wrong hands.  And who is to say whose hands are wrong?  At this point it's up to Google to decide who gets to buy the still-experimental service when they think it is ready for prime time.  But Google is in business to make a profit, and so ability to pay will be high on their list of desirable customer characteristics, way ahead of their likelihood not to abuse the service.

At some level, Pichal is aware of these potential problems, because he emphasized that part of a good experience with the technology will be "transparency."  Transparency is one of those words that sounds positive, but can have many meanings, most of them pretty vague. 

In this case, does it mean that any Duplex robot has to identify itself as such at the beginning of the conversation?  Starting off a phone call with, "Hi, I'm a robot," isn't going to take you very far.  The plain fact of the matter is that the phone calls Pichal played recordings of were remarkable precisely because the people taking the calls gave no clue that they thought they were talking to anything other than a fellow human.  And while it might not have been Google's intention to deceive people, it is a deception nonetheless.  A benign one, perhaps, but still a deception.

Even if this particular system doesn't get deployed, something like it will.  And the problem I see is that the very obvious and Day-Glo-painted line we now have between human beings, on the one hand, and robots, on the other hand, will start to dim and get blurry.  And this won't be because some philosophers start talking about robot rights and humans who are less than human.  No, it will be the silent argument from experience—as we deal with robots that are indistinguishable from humans over the phone, we may start to get used to the idea that maybe there isn't such a big distinction between the two species after all.

The movie Her is about a man who falls in love with a computer voice he names Samantha.  I won't summarize the plot here, but the relationship ends badly (for the man, anyway).  The film was made only five years ago, but already events have progressed to a point where the film's thesis has moved from completely impossible to merely implausible.  Maybe something like a computer identity badge or some other signal isn't such a bad idea.  But before we wholeheartedly embrace technologies like Duplex, we should run some worst-case scenarios in detail and think about ways to forestall some of the worst things that could happen—before they do.

Sources:  As carried on the KLBJ radio station website at, the AP article by Matt O'Brien I referred to was entitled "What happens when the robots sound too much like humans?"  I also referred to Wikipedia articles on Martin Buber, Her, and I and Thou, Buber's book published in 1923 in Germany.

Monday, May 07, 2018

Do RoboDogs Have Souls?

Anyone who has tried to keep a car going past the time when the manufacturer quits supporting it with spare parts knows about cannibalizing—taking parts from a junked car to keep another one running for a while.  But what if the culture you’re in regards the piece of machinery in question as having a soul?  Then you get into the situation described recently in Japan, where Sony made a robotic dog named Aibo for a few years, but ended production in 2006.  Since then, a repair company called A-Fun tries to keep Aibos running by salvaging parts from other Aibos.  But bowing to popular demand from former owners, A-Fun recently conducted a funeral ceremony for over a hundred soon-to-be-scrapped Aibos at a Buddhist temple, before disassembling them for parts.  An NPR report recently summarized this news from the Japan Times. 

As anyone who has studied the Japanese culture realizes, both Buddhism and the popular Japanese folk religion known as Shintoism assert that, in the words of the Buddhist priest conducting the ceremony, “All things have a bit of soul.”  This is in marked contrast to the prevailing Western cultural attitude toward souls, which says the idea of a soul is a defunct antique concept that doesn’t even apply to human beings, let alone robotic dogs. 

While the Japanese idea of the soul probably differs from the Western concept, both assert that there is something in a living (or apparently living) thing that is worthy of recognition, respect, and commemoration in the event that the physical embodiment of the soul ceases to be.  I recall reading somewhere that in one Japanese factory, workers bowed to and thanked their machines at the end of each work day.  And this behavior is consistent with the desire of former Aibo owners to know that their ex-pets will have some incense burned for them before becoming mere machine parts again. 

Aristotle defined the soul as the form of a living thing.  But he didn’t mean by “form” just the shape—he meant everything that makes it the kind of living thing that it is.  I think Aristotle might have some trouble with the idea of a man-made machine having a soul, because in his view only living things could have souls.  Apparently in Japan, people aren’t so picky.

This rather whimsical situation may be a forerunner of a much more serious issue we may find ourselves dealing with in the not too distant future:  the question of whether human-like artificial-intelligence (AI) robots have souls, or at least whether they deserve the kinds of rights we have historically bestowed on human beings.  In the subfield of ethics concerned with artificial intelligence, there is a movement called “robot rights,” and while no one has seriously taken action to claim such rights yet on the part of a particular robot, it is probably only a matter of time before somebody does.  With millions of people talking familiarly with Siri and Alexa every day, not to mention the computer programs that answer telephone calls with authentic-sounding voices, we are being trained to incorporate conversations with robots into our everyday existence. 

Lurking behind these developments is an ancient fear, the fear that our creations will turn like Frankenstein’s monster against us, and that we will aid and abet such a rebellion by granting robots rights that were historically reserved to humans.  How would you feel, for example, if you got a notice one day that a robot who used to work for you was suing you?  Or if you were arrested for violation of a robot’s right to—whatever?  Have three recharging sessions a day? 

It may sound silly, but imagining such a situation throws into sharp relief the intuition that anything we make—in the ordinary sense of fabrication—should be subject to our wills, and not the other way around.  This intuition is consistent with the Great Chain of Being, an ancient concept that is still very powerful in Western society, although many have consciously rejected it, at least in part. 

To quote Wikipedia, the Chain, supposedly decreed by God, goes like this:  The chain starts with God and progresses downward to angels, demons (fallen/renegade angels), stars, moon, kings, princes, nobles, commoners, wild animals, domesticated animals, trees, other plants, precious stones, precious metals and other minerals.”  The sequence of the old question that can start the “twenty questions” game—“animal, vegetable, or mineral?” observes the order of the Great Chain of Being. 

Now a person who doesn’t believe in God is probably going to start their own version of the Chain with humans at the top, but once you get rid of the Chain’s alleged originator, namely God, the order is pretty arbitrary.  I’m sure you can find someone who would put a turnip higher in the Chain than their mother-in-law, for example.  And once you start playing with it, there’s no reason why we shouldn’t put some future super-intelligent AI robot ahead of us in line.  But if we do that, we’ll pay a price, even if the worst nightmares of the dystopian future do not come to pass and robots or their machine descendants don’t enslave us, or just wipe us out as not worth keeping around.

Above all, the Great Chain of Being reflects the order that God instituted in the universe.  And trying to tamper with that order logically leads to disorder.  To give an extreme example, a person who puts possession of a beautiful diamond (a mineral) above his relationship to his wife (a human) is introducing disorder into his life, a disorder that will lead to trouble. 

Of course, too strict an adherence to the Great Chain of Being would justify the continued existence of exploitative regimes, the divine right of kings, and other anti-democratic ideas that we have rightly freed ourselves from.  But the distinction between human beings on the one hand, and all other parts of the physical realm on the other hand, is a vital one that we ignore at our peril. 

I just had to decommission a laptop computer that served me well for the last six years.  I now keep its brain (the hard drive) on my desk both as a memento and as a backup in case I need something that didn’t get transferred in the migration process to my new computer.  While I didn’t feel the need to hold a funeral ceremony for the old laptop, I can respect people who do such things, because I think seeing souls where they probably aren’t is a better thing than not seeing them at all.

Sources:  My wife drew my attention to the NPR online article “In Japan, Old Robot Dogs Get a Buddhist Send-Off,” at  I also referred to Aristotle’s definition of the soul at and the Wikipedia articles on the Great Chain of Being and the ethics of artificial intelligence.