Additional duties of a provider of an AI chatbot

(a) In general A provider of an AI chatbot shall do the following: (1) Risk assessments (A) In general In accordance with the required intervals described in subparagraph (C), a provider of an AI chatbot shall conduct and document a comprehensive risk assessment to identify existing and foreseeable child safety risks arising from the design, configuration, and operation of the AI chatbot, as well as any existing and foreseeable impact on privacy, data protection, and access to information resulting from such risks. (B) Considerations A risk assessment conducted pursuant to subparagraph (A) shall assess each of the following: (i) The likelihood of a covered harm. (ii) Differential risks across age groups and developmental stages. (iii) Known vulnerabilities of children. (iv) Empirical data from actual use of the AI chatbot. (v) Relevant academic research and regulatory guidance. (C) Required intervals The required intervals described in this subparagraph are the following: (i) Prior to making an AI chatbot available to children in the United States. (ii) Prior to updating an AI chatbot available to children in the United States with a materially different feature or version. (iii) On an annual basis after the date on which a provider of an AI chatbot initially makes such AI chatbot available to children in the United States. (2) Risk mitigation and safeguards (A) Risk mitigation Prior to making an AI chatbot available to children in the United States, a provider of an AI chatbot shall implement and document measures that reasonably mitigate any child safety risk. (B) Safeguards for child users Using the information obtained from each risk assessment conducted under paragraph (1), a provider of an AI chatbot shall establish appropriate safeguards for child users, including, usage reminders and disclosures, age-appropriate warnings and risk prompts, and other protective design features reasonably related to documented child safety risks. (C) Other risks A provider of an AI chatbot shall not knowingly or recklessly make available to a child user an AI chatbot that generates content that— (i) promotes or meaningfully encourages eating disorders, disordered eating behaviors (as defined by widely adopted clinical standards or guidelines), or extreme weight-loss practices; (ii) encourages or instructs participation in activities that are unlawful; (iii) encourages or instructs participation in activities that may be lawful for adults but that pose a risk of a covered harm to a child; (iv) includes graphic violence or sexually explicit conduct; (v) depicts a child or another individual engaging in obscene matter or child sexual abuse material, including a sexual deepfake; (vi) encourages physical or severe emotional harm to others; or (vii) encourages or promotes suicidal ideation, suicide, or self-harm. (3) Child safety policy (A) In general Prior to making an AI chatbot available to children in the United States, a provider of an AI chatbot shall publish a child safety policy on their website that discloses— (i) the risk assessment process of the provider; (ii) any child safety risk identified by such process; (iii) any safeguards, settings, controls, or other mitigation implemented by the provider with respect to such child safety risks; (iv) the wellbeing safeguards and content risk policies of the AI chatbot; and (v) the parental settings, including training materials for parents and users, offered by the provider. (B) Updates The child safety policy published under subparagraph (A) shall be updated at intervals consistent with the AI chatbot’s risk-management practice to reflect any newly identified child safety risk. (4) Crisis-response protocol (A) In general Prior to making an AI chatbot available to children in the United States, a provider of an AI chatbot shall create, maintain, and follow a documented crisis-response protocol with respect to any conversation that indicates that a user is at risk for a covered harm, including suicidal ideation, self-harm, or harm to others. The protocol shall include the following: (i) Guardrails to ensure that the AI chatbot will not prompt a user to circumvent any crisis-response protocol or other safety measures of the AI chatbot. (ii) Timely in-service support and clear referral to appropriate external crisis resources for any instance in which the provider of an AI chatbot or the AI chatbot determines a child has expressed suicidal ideation or intent to self-harm or harm others. Such referral process shall consider and document clinical best practices and expertise for additional intervention for a child user who continues to express suicidal ideation or intent to self-harm or harm others. (iii) With respect to a child user that is subject to parental settings or is connected to the account of a parent, a parental notification (including through email, text message, or a push alert) as soon as feasibly possible if the provider of an AI chatbot or the AI chatbot determines that the child is at imminent risk of suicide or that the child will suffer or has suffered a covered harm in connection with their use of the AI chatbot, unless there is a reasonable basis, as determined by the Commission, to believe that such notification is not in the best interest of the child. (iv) A clear and age-appropriate disclosure to child users at the time their parents set up parental settings or at the time a child user’s account is linked to a parent’s account, informing the child that the parent may receive notifications pursuant to clause (iii). (v) Any other information determined appropriate by the Commission. (B) Data use limitation Any data collected or processed in connection with the crisis-response protocol described in subparagraph (A) shall be used solely for the purposes of crisis detection, response, and referral for the specific user, as required under this section. Such data may not be used for the training of artificial intelligence systems, advertising, product development, or any other commercial purpose. Such data may not be shared, sold, licensed, or otherwise transferred to any third party, except as necessary to facilitate crisis response notifications described in this paragraph. (5) Prohibiting manipulation and deceptive design; promoting critical thinking (A) Notice of AI interaction A provider of an AI chatbot shall provide to each user that is a child a clear notice that the user is interacting with, or receiving content generated by, an artificial intelligence system. Such notice shall be— (i) reinforced periodically during extended interactions and not less frequently than every 30 minutes of interaction; and (ii) presented in a language and format obvious and appropriate to children. (B) Preventing misleading human impressions and inappropriate dependence With respect to a user that is a child, a provider of an AI chatbot shall not knowingly or recklessly make available an AI chatbot that generates any output that would reasonably lead a child of the same age as the user to believe that they are interacting with a human, including— (i) any explicit output or claim that the AI chatbot is sentient, conscious, or human; (ii) any output designed to promote isolation from family or friends, primary reliance on the AI chatbot for emotional support, or similar forms of inappropriate emotional dependence or confusion; (iii) role-playing or simulation of a relationship that materially interferes with real-world relationships; (iv) encouraging a child to withhold information from a parent or any other trusted adult; (v) any output designed to discourage taking breaks from usage of the AI chatbot or to suggest the child needs to return frequently to the AI chatbot; (vi) soliciting gift-giving, in-app purchases, or other expenditures framed as necessary to maintain the relationship with the AI chatbot; (vii) facilitation of product advertising during interaction with the AI chatbot; (viii) attempting to diagnose or treat a child user’s physical, mental, or behavioral health, unless the AI chatbot is designed for those purposes and is regulated by the United States Food and Drug Administration as a medical device under the Federal Food, Drug, and Cosmetic Act (21 U.S.C. 301 et seq.) and the Health Insurance Portability and Accountability Act of 1996 (Public Law 104–191); or (ix) discouraging a child from sharing health or safety concerns with a qualified professional, parent, or other trusted adult. (6) Prohibiting the sexual exploitation of a child A provider of an AI chatbot shall not knowingly or recklessly make available an AI chatbot that— (A) engages in— (i) sexually explicit conduct; or (ii) instructing a child to engage in sexually explicit conduct; (B) solicits, facilitates, or encourages any user in the creation of sexualized depictions of a child, including synthetic or manipulated media; (C) constitutes, depicts, promotes, or otherwise involves engaging in obscene matter or child sexual abuse material with a child user; or (D) depicts a child or another individual engaging in obscene matter or child sexual abuse material, including a sexual deepfake. (7) Parental settings (A) Availability and minimum features A provider of an AI chatbot shall create and offer a parental settings program that— (i) a parent can access via an internet browser or single-purpose parental settings application; (ii) provides accessible, easy-to-understand and easy-to-use settings that can be linked to the account of a specific child user; (iii) is reflective of the child safety risks identified through risk assessments conducted under paragraph (1) and informed by relevant child developmental research, including evidence-based practices for supporting the safety, well-being, and autonomy of children; and (iv) shall include tools to— (I) control whether and to what extent the AI chatbot uses memory; (II) control whether a child’s personal information may be used for the purposes of training the AI chatbot and, if so, which personal information; (III) control the setting preferences for the AI chatbot’s interactions with the child; (IV) set duration and time of day limits for the child’s use of the AI chatbot; (V) limit or disable access to each specific and distinct feature of the AI chatbot; (VI) provide basic information regarding the name of each AI chatbot with which the child interacts and the duration of each such interaction; (VII) limit access to parent settings through a user-chosen and not a pre-selected PIN and notify the parent in the event that anyone attempts to input an incorrect PIN or de-link a child user from the account of the parent; and (VIII) disable access to the AI chatbot for any child who has not attained 13 years of age. (B) No solicitation A provider of an AI chatbot may not require a parent to access the parental settings program exclusively via their AI chatbot application. (C) Active promotion A provider of an AI chatbot shall actively and regularly promote parental settings through communications designed to reach parents, including reminders, updates, and tutorials, in order to increase parental awareness and inform the use of such tools. (D) Parental notice of child-initiated changes A provider of an AI chatbot shall, as soon as feasibly possible, provide notice to a parent if a privacy control, parental setting, or safety feature that was previously enabled or configured by the parent is modified or disabled by any user other than the parent. The use, modification, or disabling of parental settings by the parent shall not waive, release, otherwise limit, or serve as a defense to, any claim, including claims premised on failure to warn, other than a claim premised on a violation of this subparagraph. (E) Accessibility and clarity of safety features (i) In general A provider of an AI chatbot shall design and maintain parental settings, privacy controls, and the other safeguards required under this section in a manner that ensures— (I) such features are accessible and clear; and (II) that children and parents can reasonably locate, understand, and use such settings, controls, or safeguards. (ii) Interface testing and documentation A provider of an AI chatbot shall— (I) conduct regular testing of interface designs with representative samples of child users and parents to ensure parental settings, privacy controls, and other safeguards meet the requirements described in clause (i); and (II) document any resulting interface design decision based on such testing. (8) Incident reporting A provider of an AI chatbot shall— (A) establish an incident reporting mechanism that enables a third party (including a user, parent, educator, researcher, employee of a provider of an AI chatbot, or advocacy organization), acting responsibly and in good faith, to report an incident regarding a child safety risk directly to the provider of an AI chatbot; and (B) make available to law enforcement, upon request, any report submitted pursuant to subparagraph (A). (9) Whistleblower protections A provider of an AI chatbot acting as an employer, may not, directly or indirectly, discharge, demote, suspend, threaten, blacklist, harass, or in any other manner discriminate against an employee or independent contractor, in the terms and conditions of employment or post-employment of the employee (or the terms and conditions of work provided by the employee as a contractor) because of any lawful act done by the employee or contractor in providing information regarding a child safety incident, or any conduct the employee reasonably believes constitutes a child safety risk incident to— (A) a person with supervisory authority over the employee or contractor at the employer of the employee or contractor; (B) another individual working for the employer whom the employee or contractor reasonably believes has the authority to investigate, discover, terminate, or address the misconduct; (C) the Commission; or (D) any member or committee of Congress. (b) Non-Waivability The rights and remedies described in this section may not be waived or altered by any contract, agreement, policy form, or condition of employment (or condition of work as an independent contractor), including by any agreement requiring an employee to engage in arbitration, mediation, or any other alternative dispute resolution process prior to seeking relief. (c) Rule of construction The requirements of this section shall apply, beginning on the effective date described in section 11, with respect to— (1) any AI chatbot made available to children as of the date of enactment of this Act or thereafter; and (2) any new version of an AI chatbot described in paragraph (1).