Navy report warns of robot uprising, suggests a strong moral compass
By Joseph L. Flatley 
posted Feb 18th 2009 5:55PM
posted Feb 18th 2009 5:55PM
You know, when armchair futurists (and jive talkin' bloggists) make note of some of the scary new tech making the rounds in defense circles these days it's one thing, but when the Doomsday Scenarios come from official channels, that's when we start to get nervous. According to a report published by the California State Polytechnic University (with data made available by the U.S. Navy's Office of Naval Research) the sheer scope of the military's various AI projects is so vast that it is impossible for anyone to fully understand exactly what's going on. "With hundreds of programmers working on millions of lines of code for a single war robot," says Patrick Lin, the chief compiler of the report, "no one has a clear understanding of what's going on, at a small scale, across the entire code base." And what we don't understand can eventually hunt us down and kill us. This isn't idle talk, either -- a software malfunction just last year caused US. Army robots to aim at friendly targets (fortunately, no shots were fired). The solution, Dr. Lin continues, is to teach robots "battlefield ethics... a warrior code." Of course, the government has had absolutely no problems with ethics over the years -- so programming its killer robots with some rudimentary values should prove relatively simple.
lol, im supposed to be afraid of a honda...
The Honda were created by man.
They rebelled.
13 Live among us.
One will be revealed.
Dramatic reenactment:
http://www.youtube.com/watch?v=v90PVDOIt7c
The Hondas look just like us now, only hotter!
You must embrace our new Honda MKII overlords or you will be deleted.
Wii60:
I hear they look like us now.
I think GM had the same response to the Civic and look where it got them.
And they have a plan.
Last time I heard the words "strong moral compass," something tells me it didn't end well.
Why is my brain associating "moral compass" with bad... OOOhhhh...
my moral compass is in my pants
[dot] [dot] [dot]
Love the graphic - all fear chunky cheese drummer bot! ;-]
we have nothing to fear but fear itself
and R.O.B. the robot
Time to start making EMP guns!!!
KIL
i see your hollow ichigo and raise you a renji drinking wine (or blood. never knew which.)
Sorry, I just wanted to get my avatar in there. Take that Ichigo.
It seems like our government is hard at work trying to create skynet.
Military AI Programmer cup. Golden!
I see what you did there, shaka-kon XD
*points to avatar*
need I say more? perish. all of you.
http://www.youtube.com/watch?v=aUf6aMYeHyI
Menos
Like fricken laser beams much?
negative.
(funny vid, though.)
something tells me i don't belong here...
(I knew I shoulda taken that left turn at Albuquerque)
So, the solution of not being able to understand millions of lines of existing code is to add few more million lines of code of "moral compass", which would prevent the existing millions of lines of code from aiming at us?
Or, they could just hard-wire the 3 principles in each robot.
If person = friendly
Then turn gun around.
Small problem: these are military robots, so the programmers won't be able to get past the first part of the first law without a clash with project, er, requirements.
the first of the first three rules would negate the very thing the robots are being built to do: kill people.
where is Isaac Asimov when you need him =[
I'm sorry, but your display picture is making it impossible for my eyes to rotate over to your comment.
lmao
Great, now he won't be able to do anything for the rest of the day.
our only hope is EMP's
Nope. All the critical systems on the Air Force planes I work on are EMP shielded.
We're screwed.
Help us, John Connor, you are our only hope.
Will Smith knew it would happen
Just what we need: Robots that say, "No."
Remove their higher-brain inhibitors.
Looks like anyone who bought a nuclear bunker will now have a reason to use it.
And what's with Patrick Lin being tagged four times?
Dr with a space, Dr without a space, name with a space, name without a space.
Redundant. =/
I demand a Bender reference somewhere.
Kiss my shiny metal butt!
Surprised everyone missed the sarcasm in that last sentence.
Right? Right...?
I submitted this story to you guys yesterday!
All your moral compass are belong to us.
I'm sure they'll program it not to kill Americans. Leaving us Brits in the firing line. Given the lack of emotion from our new robot overlords, is 'friendly fire' the correct term?
Of course, the AI could learn this simple program and we're then we're stuffed.
10 - Kill All Humans
20 Goto 10
Probably get a Div/0 error once we'd been eliminated though, but hey, we're not around so who cares! :)
Billy Bob!!!
How about: we use humans to aim guns.
Lots of problems solved.
Just program this in. It worked most of the time in Asimov's future.
1.A robot may not injure a human being or, through inaction, allow a human being to come to harm.
2.A robot must obey orders given to it by human beings, except where such orders would conflict with the First Law.
3.A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.
I just can't believe no one else already referenced this.
Yes, but have you not seen I,Robot? Will Smith can't say no lies!
I think you missed the line that said "a software malfunction just last year caused US. Army robots to aim at friendly targets." Asimov's rules don't really apply when programming tells the robot to shoot people from the word "go."
you forgot the 4th law. under no circumstances may any A.I try to help the human race by enslaving them.
Those asmiov rules would waste army research a bit would it not? Not harm human beings.
Besides since the whitehouse has in the past re-defined torture and human rights andsoforth they could also redefine 'human beings'
And another thing, what's with having some respect for animals too? Robots can just do anything to your pets with those rules.
Stock up on ammo people.
I wonder what a .30-06 would do to Asimo? Guess we'll just have to wait and find out!
50 BMG Fuck Yeah!
"no one has a clear understanding of what's going on, at a small scale, across the entire code base."
We will, when movies start to become true.
"battlefield ethics... a warrior code."
If it moves, kill it.
A "warrior code"? Great, now we'll have robotic Klingons, which are even MORE bloodthirsty.
I for one welcome our new robot overlords.
How has no one said this yet?
just teach them right from wrong and set them free, then we'll have nothing to fear. Right?
We might teach them right from left, but that's about it I think, plus that's flawed anyway because it depends on its orientation, wait so does right and wrong, oh well..
Why won't you ever die Sigma?
Well I happen to go to Cal Poly, and from what I know we have a rather good Computer Science department. However Dr. Patrick Lin seems to be a philosophy teacher, why should we trust his judgment of "millions of lines" of code.
Though many sci-fi stories warn us of the impending doom brought on by AI and it's improper usage.
can a robot resit a shot gun blast? if not im getting a rocket launcher
So if a robot can't resist a shotgun blast, you'll get a rocket launcher. But what if it _can_? You'll get a tactical nuke?
They should work in an irony sub-routine too: a strong moral compass would be a bit of a burden, if your primary function is to, well, kill people.
Why don't they just input the Bible into its memory-banks? Surely there is no better source of morality?
would love to see what a killing robot would do with references to sodomy
Oh, sure. Great idea. Are you not aware that more violence has been committed in the name of religion than for any other cause, including freedom? Then we would probably have robots on a bizarre religious crusade of some type, spouting sermons as they fire their weapons. Aside from the fact that there are many forms of religion, if you are referring to the Christian Bible, it does in some places contradict itself, and humans have knotty theologial discussions about what some passages really mean. Not only does a robot not have the mental capacity to use its own judgement, they also do not have the one thing inherent to any religion, faith. But then again, maybe this idea may not be all bad either, because could just program them to pray for our souls after they kill us and leave it at that.
Wow, looks like some people don't recognize sarcasm when it's staring them in the face...
@stacymailbox
You actually thought he was being serious?!? I couldn't care less about religion but after seeing someone get that angry about nothing I have an odd urge to suddenly donate to churches.
http://www.urbandictionary.com/define.php?term=poe%27s+law
:-D
Haven't you seen the new Battlestar Galactica, some toasters are religious fanatics... of course they wiped out most of the human race, so not sure that would help.
@Chaos Theory: Best response ever.
Havent we all seen a lot of Robot/End of world movies to know that a lot of times those base codes still make them crap out and kill everyone?
Uh. We're boned!
Our new robot overlords can bite my shiny metal hat.
Finally- robotic beings rule the world!
http://www.youtube.com/watch?v=WGoi1MSGu64
MoralCompass::disableDirective('NiceNess');
3 Prime directives inserted into all military robots (2009)
1- Serve the public trust
2- Protect the innocent
3- Uphold the law
3 directives erased after military computers were infected with the RoboCop worm (2010)
Judgment Day is at hand.
To match that DoD robot army we must create a Clone army to protect the republic.
Hahahaha, yes!
Does it come with pre-installed AVG?? It would be pretty bad if someone hacked and took control of a robot with lasers. As smart as their AI gets, at this stage... still probably can't defeat the mind of a hacker....
Crap - all of this *is* happening again. Once the toasters figure out how to resurrect, we're screwed.
Agreed
We must be aware AI is a threat!
Thanks
Anonymous.good.friend
MGS4 SPOILER ALERT
Snake, you have to destroy the AI stationed at NONR (Navy's Office of Naval Research) and destroy the AI. You'll have to go through a microwave chamber which serves no meaning whatsoever because my tiny robot will be the only key to implementing the virus that kills it.
They could just not write any code remotely hinting at aiming at friendly's
True Ai doesn'exist and won't for a long time so moral codes are well something more for sci-fi afictionado's then people working on actual AI systems (for now, so much fr asimov's 3 rules, the problem with the 3 laws is they are human concept, and since we're no where close to creating a true AI that can do anything with complex human concepts.....)
Basicly a robot is a pc with a body, they still can't do anything we don't specificaly program them to do, so I don't really see the problem on a programming logic level at all, unless the programmers are idiots.
Do not write code for e.g. being able to detect an enemy dressed as a friendly or some such that obfuscates code and creates possibilities for bad identification with all the bad consequences that might entail on a bad day.
It's not the bot's job to do advanced human thinking like that, it's the human controller's responsibility.
Bot's in this respect should be grunts, but even dummer. the ultimate cannon fodder while the humans do the things that create the real victory.
Also I fear for a world where war does not appear to have to guarantee dead people anymore, would we be more or less prone to start a war that might get out of hand in such a world?
The relative peace since the cold war is there for only one reason. Mutual Assured Destruction. Humans are sad beings that need to create situations where we either don't fight a war or if we do everyone dies, just to not wage war in the first place.
We definitely do not need to be led to thinking we can get away with a war without casualties amongst our troops and civs because we can send our bots.
You have to pinpoint friendlies to avoid firing on them though, and to avoid collateral damage, simply completely ignoring friendlies would make the AI shoot through friendly 'targets' to hit enemies behind them too.
Alot of people are writing stuff such as:
"Program it to not harm people"
Tho its still the military who is develping this robot.. isnt it quite obvious that this is the exact working duty it WILL have?
Who will they call for in the case of riots? The military who have friends and family among the rioters?
Yet another example of a way they can implent RFID Chips
"well without them the robot guards who are here to protect us and free us (lol) cannot identify you"
gogo team america!
Well said, albeit in somewhat imperfect english, but don't bother about it, the points are clear.
(You could get firefox, it checks spelling as you type.)
Im from Sweden. I have firefox, it has not corrected me so far.. O.o,
My written english is worse than my spoken sadly.
Some good further reading on the topic:
Gianmarco Veruggio: Roboethics - http://lis.epfl.ch/resources/podcast/2008/01/gianmarco-veruggio-roboethics.html
Noel Sharkey: Robot Ethics (Part 1) - http://www.robotspodcast.com/podcast/2009/02/robots-robot-ethics-part-1.html
Ronald Arkin: Robot Ethics (Part 2) (coming up) - http://www.robotspodcast.com/
We need more public debate on this!