Testing and evaluation requirements

(a) General requirement The Secretary shall ensure that, regardless of the acquisition pathway or testing and evaluation oversight status for a weapon system, each system described in section 3(a)(2) undergoes— (1) rigorous hardware and software verification and validation; and (2) realistic system developmental and operational test and evaluation, including analysis of unanticipated emergent behavior. (b) Specific considerations Testing and evaluation of a system under subsection (a)(2) may include testing on how human operators respond to ensure they are provided with enough time to exercise judgment and can reject or challenge suggestions or recommendations. (c) Specific requirements Testing and evaluation of a system under subsection (a) shall include the following: (1) Verification that the system functions as anticipated in realistic operational environments against adaptive adversaries, including with realistic civilian presence, activities, actions, and reactions, and are sufficiently robust to minimize failures. (2) For a system incorporating artificial intelligence capabilities, rigorous developmental and operational test and evaluation to verify and validate that the artificial intelligence is robust according to design requirements. (3) Testing to confirm that autonomy algorithms in systems incorporating artificial intelligence capabilities can be rapidly reprogrammed on new input data. (d) Post-Fielding testing The Secretary shall ensure that, after initial operational test and evaluation of a system under subsection (a), as directed by the Director of Operational Test and Evaluation— (1) system data is collected and any further changes to the system undergo appropriate verification and validation and test and evaluation to ensure that critical safety features have not been degraded; (2) system software is tested using best-available Department means and methods to validate that critical safety features have not been degraded, and automated testing tools, such as modeling and simulation, are used whenever feasible; (3) any new or revised operating states or other relevant changes in the system are identified and undergoes appropriate and tailored additional test and evaluation to characterize the system behavior in that new operating state; and (4) changes to the state transition matrix are evaluated to determine whether they require whole system follow-on operational test and evaluation. (e) Iterative cyber testing The Secretary shall ensure that hardware and software verification and validation of a system under subsection (a) includes quarterly cyber test and evaluation to verify that the system is resilient and survivable in contested cyberspace. (f) Role of the Director of Operational Test and Evaluation Under this section, the Director of Operational Test and Evaluation shall— (1) oversee development of realistic operational test, risk assessments, and evaluation standards for autonomous and semi-autonomous weapon systems, including requirements for data collection and standards for test and evaluation of any changes to the system following initial operational test and evaluation; (2) evaluate whether autonomous and semi-autonomous weapon systems under the Director's oversight have met standards after being tested for rigorous verification, validation, and evaluation in realistic operational conditions, including potential adversary action, to ensure that the system is robust to minimize failures; (3) establish standards for data collection post-fielding and monitoring and assessment by programs; (4) establish and maintain a centralized repository for reporting, collecting, and analyzing operational incidents, weapon system failures, and unintended weapon system behaviors; (5) review and approve operational and live fire test plans for autonomous and semi-autonomous weapon systems; and (6) coordinate with the Under Secretary of Defense for Research and Engineering and the appropriate Secretary of a military department or Assistant Secretary for Low-Intensity Conflict to provide for monitoring to identify and address when changes to the system design or operational environment require additional testing and evaluation to ensure that the system is robust to minimize failures such as unintended engagements with civilians and civilian infrastructure, densely populated areas, and resist interference by unauthorized parties.