Safe product design, forensic engineering, and Asimov's Laws of Robotics

Abstract

Isaac Asimov wrote a series of science fiction stories regarding failure analysis of complex systems: his fictional positronic brained robots. The stories revolved around his “Three Laws of Robotics”. One, a robot may not injure a human being, or, through inaction allow a human being to come to harm. Two, a robot must obey the orders given it by human beings except where such orders would conflict with the First Law. Three, a robot must protect its own existence as long as such protection does not conflict with the First or Second Laws. We are surrounded by automated systems that routinely violate these Three Laws, yet some systems, such as implanted pacemakers and defibrillators have specific and distinct circuitry and firmware that implement exactly these rules. Furthermore, as engineers we are called upon to evaluate systems that have failed, determine root cause, and assist the courts in determining culpability. This paper presents a series of examples of systems that are well implemented examples of Asimov's Three Laws, of systems that categorically fail to implement the Three Laws, tie the Three Laws into the Criticality and Severity Analysis (FMEA/CA/SA) Failure Modes and Effects Analysis standards, and examine the use of The Laws in forensic engineering and failure analysis.