Pre-deployment evaluations and post-deployment impact assessments

(a) Pre-Deployment evaluations Prior to deploying, licensing, or offering a covered algorithm (including deploying a material change to a previously deployed covered algorithm or a material change made prior to deployment) for a consequential action, a developer or deployer shall conduct a pre-deployment evaluation in accordance with the following: (1) Preliminary evaluation (A) Plausibility of harm (i) Developers The developer shall conduct a preliminary evaluation of the plausibility that any expected use of the covered algorithm may result in a harm. (ii) Deployers The deployer shall conduct a preliminary evaluation of the plausibility that any intended use of the covered algorithm may result in a harm. (B) Results Based on the results of the preliminary evaluation, the developer or deployer shall— (i) in the event that a harm is not plausible, record a finding of no plausible harm, including a description of the developer’s expected use or the deployer’s intended use of the covered algorithm, how the preliminary evaluation was conducted, and an explanation for the finding, and submit such record to the Commission; and (ii) in the event that a harm is plausible, conduct a full pre-deployment evaluation as described in paragraph (2). (C) Previously deployed covered algorithms When conducting a preliminary evaluation of a material change to, or new use of, a previously deployed covered algorithm, the developer or deployer may limit the scope of the evaluation to whether use of the covered algorithm may result in a harm as a result of the material change or new use. (2) Full pre-deployment evaluation (A) For developers (i) Independent auditor evaluation If a developer determines a harm is plausible during the preliminary evaluation described in paragraph (1), the developer shall engage an independent auditor to conduct a pre-deployment evaluation. (ii) Pre-deployment evaluation requirements The evaluation required under clause (i) shall include a detailed review and description, sufficient for an individual having ordinary skill in the art to understand the functioning, risks, uses, benefits, limitations, and other pertinent attributes of the covered algorithm, including— (I) the covered algorithm’s design and methodology, including the inputs the covered algorithm is designed to use to produce an output and the outputs the covered algorithm is designed to produce; (II) how the covered algorithm was created, trained, and tested, including— (aa) any metric used to test the performance of the covered algorithm; (bb) defined benchmarks and goals that correspond to such metrics, including whether there was sufficient representation of demographic groups that are reasonably likely to use or be affected by the covered algorithm in the data used to create or train the algorithm, and whether there was reasonable testing, if any, across such demographic groups; (cc) the outputs the covered algorithm actually produces in testing; (dd) a description of any consultation with relevant stakeholders, including any communities that will be impacted by the covered algorithm, regarding the development of the covered algorithm, or a disclosure that no such consultation occurred; (ee) a description of which protected characteristics, if any, were used for testing and evaluation, and how and why such characteristics were used, including— (AA) whether the testing occurred in comparable contextual conditions to the conditions in which the covered algorithm is expected to be used; and (BB) if protected characteristics were not available to conduct such testing, a description of alternative methods the developer used to conduct the required assessment; (ff) any other computational algorithm incorporated into the development of the covered algorithm, regardless of whether such precursor computational algorithm involves a consequential action; (gg) a description of the data and information used to develop, test, maintain, or update the covered algorithm, including— (AA) each type of personal data used, each source from which the personal data was collected, and how each type of personal data was inferred and processed; (BB) the legal authorization for collecting and processing the personal data; and (CC) an explanation of how the data (including personal data) used is representative, proportional, and appropriate to the development and intended uses of the covered algorithm; and (hh) a description of the training process for the covered algorithm which includes the training, validation, and test data utilized to confirm the intended outputs; (III) the potential for the covered algorithm to produce a harm or to have a disparate impact in the equal enjoyment of goods, services, or other activities or opportunities, and a description of such potential harm or disparate impact; (IV) alternative practices and recommendations to prevent or mitigate harm and recommendations for how the developer could monitor for harm after offering, licensing, or deploying the covered algorithm; and (V) any other information the Commission deems pertinent to prevent the covered algorithm from causing harm or having a disparate impact in the equal enjoyment of goods, services, or other activities or opportunities, as prescribed by rules promulgated by the Commission pursuant to section 553 of title 5, United States Code. (iii) Report The independent auditor shall submit to the developer a report on the evaluation conducted under this subparagraph, including the findings and recommendations of such independent auditor. (B) For deployers (i) Independent auditor evaluation If a deployer determines a harm is plausible during the preliminary evaluation described in paragraph (1), the deployer shall engage an independent auditor to conduct a pre-deployment evaluation. (ii) Pre-deployment evaluation requirements The evaluation required under clause (i) shall include a detailed review and description, sufficient for an individual having ordinary skill in the art to understand the functioning, risks, uses, benefits, limitations, and other pertinent attributes of the covered algorithm, including— (I) the manner in which the covered algorithm makes or contributes to a consequential action and the purpose for which the covered algorithm will be deployed; (II) the necessity and proportionality of the covered algorithm in relation to its planned use, including the intended benefits and limitations of the covered algorithm and a description of the baseline process being enhanced or replaced by the covered algorithm, if applicable; (III) the inputs that the deployer plans to use to produce an output, including— (aa) the type of personal data and information used and how the personal data and information will be collected, inferred, and processed; (bb) the legal authorization for collecting and processing the personal data; and (cc) an explanation of how the data used is representative, proportional, and appropriate to the deployment of the covered algorithm; (IV) the outputs the covered algorithm is expected to produce and the outputs the covered algorithm actually produces in testing; (V) a description of any additional testing or training completed by the deployer for the context in which the covered algorithm will be deployed; (VI) a description of any consultation with relevant stakeholders, including any communities that will be impacted by the covered algorithm, regarding the deployment of the covered algorithm; (VII) the potential for the covered algorithm to produce a harm or to have a disparate impact in the equal enjoyment of goods, services, or other activities or opportunities in the context in which the covered algorithm will be deployed and a description of such potential harm or disparate impact; (VIII) alternative practices and recommendations to prevent or mitigate harm in the context in which the covered algorithm will be deployed and recommendations for how the deployer could monitor for harm after offering, licensing, or deploying the covered algorithm; and (IX) any other information the Commission deems pertinent to prevent the covered algorithm from causing harm or having a disparate impact in the equal enjoyment of goods, services, or other activities or opportunities as prescribed by rules promulgated by the Commission pursuant to section 553 of title 5, United States Code. (iii) Report The independent auditor shall submit to the deployer a report on the evaluation conducted under this subparagraph, including the findings and recommendations of such independent auditor. (b) Deployer annual impact assessment After the deployment of a covered algorithm, a deployer shall, on an annual basis, conduct an impact assessment in accordance with the following: (1) Preliminary impact assessment The deployer shall conduct a preliminary impact assessment of the covered algorithm to identify any harm that resulted from the covered algorithm during the reporting period and— (A) if no resulting harm is identified by such assessment, shall record a finding of no harm, including a description of the developer's expected use or the deployer's intended use of the covered algorithm, how the preliminary evaluation was conducted, and an explanation for such finding, and submit such finding to the Commission; and (B) if a resulting harm is identified by such assessment, shall conduct a full impact assessment as described in paragraph (2). (2) Full impact assessment In the event that the covered algorithm resulted in harm during the reporting period, the deployer shall engage an independent auditor to conduct a full impact assessment with respect to the reporting period, including— (A) an assessment of the harm that resulted or was reasonably likely to have been produced during the reporting period; (B) a description of the extent to which the covered algorithm produced a disparate impact in the equal enjoyment of goods, services, or other activities or opportunities, including the methodology for such evaluation, of how the covered algorithm produced or likely produced such disparity; (C) a description of the types of data input into the covered algorithm during the reporting period to produce an output, including— (i) documentation of how data input into the covered algorithm to produce an output is represented and complete descriptions of each field of data; and (ii) whether and to what extent the data input into the covered algorithm to produce an output was used to train or otherwise modify the covered algorithm; (D) whether and to what extent the covered algorithm produced the outputs it was expected to produce; (E) a detailed description of how the covered algorithm was used to make a consequential action; (F) any action taken to prevent or mitigate harms, including how relevant staff are informed of, trained about, and implement harm mitigation policies and practices, and recommendations for how the deployer could monitor for and prevent harm after offering, licensing, or deploying the covered algorithm; and (G) any other information the Commission deems pertinent to prevent the covered algorithm from causing harm or having a disparate impact in the equal enjoyment of goods, services, or other activities or opportunities as prescribed by rules promulgated by the Commission pursuant to section 553 of title 5, United States Code. (3) Reports (A) To the deployer After the engagement of the independent auditor, the independent auditor shall submit to the deployer a report on the impact assessment conducted under paragraph (2), including the findings and recommendations of such independent auditor. (B) To the developer Not later than 30 days after the submission of a report on an impact assessment under subparagraph (A), a deployer shall submit to the developer of the covered algorithm a summary of such report, subject to the trade secret and privacy protections described in subsection (e)(3). (c) Developer annual review of assessments A developer shall, on an annual basis, review each impact assessment summary submitted by a deployer of its covered algorithm under subsection (b)(3)(B) for the following purposes: (1) To assess how the deployer is using the covered algorithm, including the methodology for assessing such use. (2) To assess the type of data the deployer is inputting into the covered algorithm to produce an output and the types of outputs the covered algorithm is producing. (3) To assess whether the deployer is complying with any relevant contractual agreement with the developer and whether any remedial action is necessary. (4) To compare the covered algorithm’s performance in real-world conditions versus pre-deployment testing, including the methodology used to evaluate such performance. (5) To assess whether the covered algorithm is causing harm or is reasonably likely to be causing harm. (6) To assess whether the covered algorithm is causing, or is reasonably likely to be causing, a disparate impact in the equal enjoyment of goods, services, or other activities or opportunities, and, if so, how and with respect to which protected characteristic. (7) To determine whether the covered algorithm needs modification. (8) To determine whether any other action is appropriate to ensure that the covered algorithm remains safe and effective. (9) To undertake any other assessment or responsive action the Commission deems pertinent to prevent the covered algorithm from causing harm or having a disparate impact in the equal enjoyment of goods, services, or other activities or opportunities, as prescribed by rules promulgated by the Commission pursuant to section 553 of title 5, United States Code. (d) Joint developer and deployer obligations If a person is both the developer and deployer of a covered algorithm, the person may conduct combined pre-deployment evaluations and annual assessments, provided that each combined evaluation or assessment satisfies all requirements for both developers and deployers. (e) Reporting and retention requirements (1) Reporting A developer or deployer that conducts a full pre-deployment evaluation, full impact assessment, or developer annual review of assessments shall— (A) not later than 30 days after completion, submit the evaluation, assessment, or review to the Commission; (B) upon request, make the evaluation, assessment, or review available to Congress; and (C) not later than 30 days after completion— (i) publish a summary of the evaluation, assessment, or review on the website of the developer or deployer in a manner that is easily accessible to individuals; and (ii) submit such summary to the Commission. (2) Retention A developer or deployer shall retain all evaluations, assessments, and reviews described in this section for a period of not fewer than 10 years. (3) Trade secrets and privacy A developer or deployer— (A) may redact and segregate any trade secret (as defined in section 1839 of title 18, United States Code) from public disclosure under this subsection; and (B) shall redact and segregate personal data from public disclosure under this subsection. (f) Rulemaking (1) Authority The Commission may, in accordance with section 553 of title 5, United States Code, promulgate such rules as may be necessary to carry out this section. (2) Additional regulations Not later than 2 years after the date of enactment of this Act, the Commission shall— (A) promulgate rules, pursuant to section 553 of title 5, United States Code, specifying— (i) what information and factors a developer or deployer shall consider in making the preliminary evaluation or preliminary impact assessment described in subsections (a)(1) and (b)(1), respectively; (ii) what information a developer or deployer shall include in a summary of an evaluation, assessment, or developer review described in subsection (e)(1)(C); and (iii) the extent to and process by which a developer may request additional information from a deployer, including the purposes for which a developer is permitted to use such additional information; and (B) in promulgating such rules, consider the need to protect the privacy of personal data, as well as the need for information sharing by developers and deployers to comply with this section and inform the public.