Requirement for human judgment in autonomous weapon systems

(a) Policy requirement The Secretary of Defense shall— (1) ensure that any autonomous weapon system or artificial intelligence capability developed, acquired, or operated by the Department of Defense— (A) is designed and employed in a manner that enables commanders and operators to exercise ultimate human responsibility over the use of force; and (B) is developed, fielded and employed using ethical and legal principles that would delineate unlawful uses of artificial intelligence; and (2) develop technical and operational and organizational guidelines around the use of artificial intelligence. (b) Design requirements To meet the requirements under subsection (a), the Secretary of Defense shall ensure that any autonomous weapon system or artificial intelligence capability described in such subsection, to the maximum extent practicable— (1) allows human operators to supervise the operation of the weapon system during mission execution when appropriate given system design; (2) includes mechanisms that allow commanders or operators to intervene in or terminate the use of force by the weapon system during mission execution when appropriate given system design; (3) includes fail-safe mechanisms to enable manual control when autonomous systems are degraded, jammed, spoofed, or under adversarial attack when appropriate given system design; (4) provides sufficient information regarding weapon system status and decision logic to enable operators to understand, oversee, and exercise functional control over system behavior; (5) maintains records of relevant target selection data, decision logic, and human operator actions sufficient to enable post-engagement review of compliance with the requirements under subsection (a); and (6) operates consistent with applicable United States law and international law, rules of engagement, the law of armed conflict, applicable treaties, and Department of Defense policy. (c) Operational oversight (1) In general The Secretary of Defense shall establish procedures to ensure that commanders and operators are rigorously trained to appropriately supervise autonomous weapon systems or artificial intelligence capabilities in operational environments, including regular proficiency assessments in manual target identification, threat assessment, and engagement procedures, to ensure operators can effectively execute missions using other capabilities when autonomous systems are unavailable or unreliable. (2) Reporting mechanism The procedures established under paragraph (1) shall include a mechanism for operators of autonomous weapon systems or artificial intelligence capability to report, without fear of reprisal, concerns regarding weapon system reliability or the adequacy of human-machine interfaces.