Navy report warns of robot uprising, suggests a strong moral compass
By Joseph L. Flatley 
posted February 18th 2009 5:55PM
posted February 18th 2009 5:55PM
You know, when armchair futurists (and jive talkin' bloggists) make note of some of the scary new tech making the rounds in defense circles these days it's one thing, but when the Doomsday Scenarios come from official channels, that's when we start to get nervous. According to a report published by the California State Polytechnic University (with data made available by the U.S. Navy's Office of Naval Research) the sheer scope of the military's various AI projects is so vast that it is impossible for anyone to fully understand exactly what's going on. "With hundreds of programmers working on millions of lines of code for a single war robot," says Patrick Lin, the chief compiler of the report, "no one has a clear understanding of what's going on, at a small scale, across the entire code base." And what we don't understand can eventually hunt us down and kill us. This isn't idle talk, either -- a software malfunction just last year caused US. Army robots to aim at friendly targets (fortunately, no shots were fired). The solution, Dr. Lin continues, is to teach robots "battlefield ethics... a warrior code." Of course, the government has had absolutely no problems with ethics over the years -- so programming its killer robots with some rudimentary values should prove relatively simple.
you forgot the 4th law. under no circumstances may any A.I try to help the human race by enslaving them.
Those asmiov rules would waste army research a bit would it not? Not harm human beings.
Besides since the whitehouse has in the past re-defined torture and human rights andsoforth they could also redefine 'human beings'
And another thing, what's with having some respect for animals too? Robots can just do anything to your pets with those rules.
Stock up on ammo people.
I wonder what a .30-06 would do to Asimo? Guess we'll just have to wait and find out!
50 BMG Fuck Yeah!
"no one has a clear understanding of what's going on, at a small scale, across the entire code base."
We will, when movies start to become true.
"battlefield ethics... a warrior code."
If it moves, kill it.
A "warrior code"? Great, now we'll have robotic Klingons, which are even MORE bloodthirsty.
I for one welcome our new robot overlords.
How has no one said this yet?
just teach them right from wrong and set them free, then we'll have nothing to fear. Right?
We might teach them right from left, but that's about it I think, plus that's flawed anyway because it depends on its orientation, wait so does right and wrong, oh well..
Why won't you ever die Sigma?
Well I happen to go to Cal Poly, and from what I know we have a rather good Computer Science department. However Dr. Patrick Lin seems to be a philosophy teacher, why should we trust his judgment of "millions of lines" of code.
Though many sci-fi stories warn us of the impending doom brought on by AI and it's improper usage.
can a robot resit a shot gun blast? if not im getting a rocket launcher
So if a robot can't resist a shotgun blast, you'll get a rocket launcher. But what if it _can_? You'll get a tactical nuke?
They should work in an irony sub-routine too: a strong moral compass would be a bit of a burden, if your primary function is to, well, kill people.
Why don't they just input the Bible into its memory-banks? Surely there is no better source of morality?
would love to see what a killing robot would do with references to sodomy
Oh, sure. Great idea. Are you not aware that more violence has been committed in the name of religion than for any other cause, including freedom? Then we would probably have robots on a bizarre religious crusade of some type, spouting sermons as they fire their weapons. Aside from the fact that there are many forms of religion, if you are referring to the Christian Bible, it does in some places contradict itself, and humans have knotty theologial discussions about what some passages really mean. Not only does a robot not have the mental capacity to use its own judgement, they also do not have the one thing inherent to any religion, faith. But then again, maybe this idea may not be all bad either, because could just program them to pray for our souls after they kill us and leave it at that.
Wow, looks like some people don't recognize sarcasm when it's staring them in the face...
@stacymailbox
You actually thought he was being serious?!? I couldn't care less about religion but after seeing someone get that angry about nothing I have an odd urge to suddenly donate to churches.
http://www.urbandictionary.com/define.php?term=poe%27s+law
:-D
Haven't you seen the new Battlestar Galactica, some toasters are religious fanatics... of course they wiped out most of the human race, so not sure that would help.
@Chaos Theory: Best response ever.
Havent we all seen a lot of Robot/End of world movies to know that a lot of times those base codes still make them crap out and kill everyone?
Uh. We're boned!
Our new robot overlords can bite my shiny metal hat.
Finally- robotic beings rule the world!
http://www.youtube.com/watch?v=WGoi1MSGu64
MoralCompass::disableDirective('NiceNess');
3 Prime directives inserted into all military robots (2009)
1- Serve the public trust
2- Protect the innocent
3- Uphold the law
3 directives erased after military computers were infected with the RoboCop worm (2010)
Judgment Day is at hand.
To match that DoD robot army we must create a Clone army to protect the republic.
Hahahaha, yes!
Does it come with pre-installed AVG?? It would be pretty bad if someone hacked and took control of a robot with lasers. As smart as their AI gets, at this stage... still probably can't defeat the mind of a hacker....
Crap - all of this *is* happening again. Once the toasters figure out how to resurrect, we're screwed.
Agreed
We must be aware AI is a threat!
Thanks
Anonymous.good.friend
MGS4 SPOILER ALERT
Snake, you have to destroy the AI stationed at NONR (Navy's Office of Naval Research) and destroy the AI. You'll have to go through a microwave chamber which serves no meaning whatsoever because my tiny robot will be the only key to implementing the virus that kills it.
They could just not write any code remotely hinting at aiming at friendly's
True Ai doesn'exist and won't for a long time so moral codes are well something more for sci-fi afictionado's then people working on actual AI systems (for now, so much fr asimov's 3 rules, the problem with the 3 laws is they are human concept, and since we're no where close to creating a true AI that can do anything with complex human concepts.....)
Basicly a robot is a pc with a body, they still can't do anything we don't specificaly program them to do, so I don't really see the problem on a programming logic level at all, unless the programmers are idiots.
Do not write code for e.g. being able to detect an enemy dressed as a friendly or some such that obfuscates code and creates possibilities for bad identification with all the bad consequences that might entail on a bad day.
It's not the bot's job to do advanced human thinking like that, it's the human controller's responsibility.
Bot's in this respect should be grunts, but even dummer. the ultimate cannon fodder while the humans do the things that create the real victory.
Also I fear for a world where war does not appear to have to guarantee dead people anymore, would we be more or less prone to start a war that might get out of hand in such a world?
The relative peace since the cold war is there for only one reason. Mutual Assured Destruction. Humans are sad beings that need to create situations where we either don't fight a war or if we do everyone dies, just to not wage war in the first place.
We definitely do not need to be led to thinking we can get away with a war without casualties amongst our troops and civs because we can send our bots.
You have to pinpoint friendlies to avoid firing on them though, and to avoid collateral damage, simply completely ignoring friendlies would make the AI shoot through friendly 'targets' to hit enemies behind them too.
Alot of people are writing stuff such as:
"Program it to not harm people"
Tho its still the military who is develping this robot.. isnt it quite obvious that this is the exact working duty it WILL have?
Who will they call for in the case of riots? The military who have friends and family among the rioters?
Yet another example of a way they can implent RFID Chips
"well without them the robot guards who are here to protect us and free us (lol) cannot identify you"
gogo team america!
Well said, albeit in somewhat imperfect english, but don't bother about it, the points are clear.
(You could get firefox, it checks spelling as you type.)
Im from Sweden. I have firefox, it has not corrected me so far.. O.o,
My written english is worse than my spoken sadly.
Some good further reading on the topic:
Gianmarco Veruggio: Roboethics - http://lis.epfl.ch/resources/podcast/2008/01/gianmarco-veruggio-roboethics.html
Noel Sharkey: Robot Ethics (Part 1) - http://www.robotspodcast.com/podcast/2009/02/robots-robot-ethics-part-1.html
Ronald Arkin: Robot Ethics (Part 2) (coming up) - http://www.robotspodcast.com/
We need more public debate on this!