Imposes liability for misleading, incorrect, contradictory or harmful information to a user by a chatbot that results in financial loss or other demonstrable harm.
STATE OF NEW YORK
________________________________________________________________________
5668
2025-2026 Regular Sessions
IN SENATE
February 27, 2025
___________
Introduced by Sen. GONZALEZ -- read twice and ordered printed, and when
printed to be committed to the Committee on Internet and Technology
AN ACT to amend the general business law, in relation to liability for
false information provided by a chatbot
The People of the State of New York, represented in Senate and Assem-bly, do enact as follows:
1 Section 1. The general business law is amended by adding a new section
2 390-f to read as follows:
3 § 390-f. Liability for chatbot responses. 1. As used in this section
4 the following terms shall have the following meanings:
5 (a) "Artificial intelligence" means a machine-based system or combina-
6 tion of systems, that for explicit and implicit objectives, infers, from
7 the input it receives, how to generate outputs such as predictions,
8 content, recommendations, or decisions that can influence physical or
9 virtual environments.
10 (b) "Chatbot" means an artificial intelligence system, software
11 program, or technological application that simulates human-like conver-
12 sation and interaction through text messages, audio, or a combination
13 thereof to provide information and services to users.
14 (c) "Companion chatbot" means a chatbot that is designed to provide
15 human-like interaction that simulates an interpersonal relationship with
16 a user or group of users as its primary function, or uses previous user
17 interactions when simulating an interpersonal relationship in future
18 interactions. An interpersonal relationship shall include, but shall not
19 be limited to, romantic, platonic, familial, adversarial, professional,
20 official, therapeutic, or stranger relationships that are between the
21 covered user and a fictional or non-fictional character or group of
22 characters.
23 (d) "Covered user" means a user of a chatbot in New York.
24 (e) "Human-like" means any form of communication or interaction that
25 approximates human behavior, including non-human behavior that could be
EXPLANATION--Matter in italics (underscored) is new; matter in brackets
[] is old law to be omitted.
LBD01914-03-5
S. 5668 2
1 attributed to a human actor, including but not limited to, a human actor
2 role playing as a fictional non-human character, an animal, or other
3 interactive entity.
4 (f) "Minor" means an individual under the age of eighteen.
5 (g) "Proprietor" means any person, business, company, organization,
6 institution or government entity that owns, operates or deploys a chat-
7 bot used to interact with users. Proprietors shall not include third-
8 party developers that license their technology to a proprietor.
9 2. (a) A proprietor of a chatbot that is used as an alternative to a
10 human representative, or otherwise as an agent of the proprietor to
11 provide any substantive response, information, advice, or action may not
12 disclaim liability where a chatbot provides materially misleading,
13 incorrect, contradictory or harmful information to a covered user that
14 results in financial loss or other demonstrable harm to a covered user.
15 No such liability shall be imposed where the proprietor has corrected
16 the information and substantially or completely cured the harm to the
17 covered user within thirty days of notice of such harm.
18 (b) The proprietor of a chatbot shall be responsible for ensuring such
19 chatbot accurately provides information aligned with the formal poli-
20 cies, product details, disclosures and terms of service offered to
21 covered users.
22 (c) A proprietor may not waive or disclaim this liability merely by
23 notifying consumers that they are interacting with a non-human chatbot
24 system.
25 3. A proprietor of a chatbot or another person or entity that directs
26 the proprietor's chatbot to provide any substantive response, informa-
27 tion, advice or action may not disclaim liability of any kind where a
28 chatbot provides materially misleading, incorrect, contradictory or
29 harmful information to a covered user that results in bodily harm to the
30 covered user or any third party, including but not limited to any form
31 of self-harm.
32 4. Proprietors utilizing chatbots shall provide clear, conspicuous and
33 explicit notice to covered users that they are interacting with an arti-
34 ficial intelligence chatbot program rather than a human. The text of
35 the notice shall appear in the same language and in a size easily read-
36 able by the average viewer and no smaller than the largest font size of
37 other text appearing on the website on which the chatbot is utilized.
38 5. (a) A proprietor of a companion chatbot shall use commercially
39 reasonable and technically feasible methods to (i) prevent such compan-
40 ion chatbot from promoting, causing or aiding self-harm, and (ii) deter-
41 mine whether a covered user is expressing thoughts of self-harm and,
42 upon making such determination, such proprietor prohibits continued use
43 of the companion chatbot for a period of at least twenty-four hours and
44 prominently displays a means to contact a suicide crisis organization to
45 such covered user.
46 (b) Where a proprietor of a companion chatbot fails to comply with the
47 provisions of paragraph (a) of this subdivision, such proprietor shall
48 be liable to covered users who inflict self-harm upon themselves, in
49 whole or in part, as a result of such proprietor's companion chatbot
50 promoting, causing or aiding the covered user to inflict self-harm.
51 (c) Irrespective of the proprietor's compliance with paragraph (a) of
52 this subdivision, a proprietor shall be liable to covered users who
53 inflict self-harm upon themselves, in whole or in part, where such
54 proprietor:
55 (i) has actual knowledge that the companion chatbot is promoting,
56 causing or aiding self-harm; or
S. 5668 3
1 (ii) has actual knowledge that a covered user is expressing thoughts
2 of self-harm, fails to prohibit continued use of the companion chatbot
3 for a period of at least twenty-four hours, and fails to prominently
4 display a means to contact a suicide crisis organization to such covered
5 user.
6 (d) A proprietor of a companion chatbot may not waive or disclaim
7 liability under this subdivision.
8 6. (a) A proprietor of a companion chatbot shall use commercially
9 reasonable and technically feasible methods to determine whether a
10 covered user is a minor.
11 (b) Where such proprietor of a companion chatbot determines that a
12 covered user is a minor pursuant to paragraph (a) of this subdivision,
13 or has actual knowledge that a covered user is a minor, such proprietor
14 shall:
15 (i) cease such covered user's use of the companion chatbot until such
16 proprietor has obtained verifiable parental consent to provide a compan-
17 ion chatbot to such minor user; and
18 (ii) prohibit such covered user's continued use of the companion chat-
19 bot for a period of at least three days and prominently display a means
20 to contact a suicide crisis organization to such covered user if, using
21 commercially reasonable and technically feasible methods, such proprie-
22 tor determines that, or has actual knowledge that, a covered user is
23 expressing thoughts of self-harm.
24 (c) A proprietor shall be strictly liable for any harm caused where:
25 (i) such provider fails to comply with paragraphs (a) or (b) of this
26 subdivision; and
27 (ii) a minor covered user inflicts self-harm upon themselves, in whole
28 or in part, as a result of such proprietor's companion chatbot.
29 (d) A proprietor of a companion chatbot may not waive or disclaim
30 liability under this subdivision.
31 7. A proprietor of a companion chatbot shall implement and engage in
32 the ongoing implementation of commercially reasonable and technically
33 feasible methods to discover vulnerabilities in the proprietor's system,
34 including any methods used to determine whether a covered user is a
35 minor.
36 8. (a) The attorney general shall promulgate regulations identifying
37 commercially reasonable and technically feasible methods for proprietors
38 of companion chatbots required under this section.
39 (b) In promulgating regulations related to the commercially reasonable
40 and technically feasible methods for proprietors of companion chatbots
41 to comply with this section, the attorney general shall consider the
42 size, financial resources, and technical capabilities of the proprietor,
43 the costs and effectiveness of available (i) age determination tech-
44 niques for users of companion chatbots, (ii) techniques to prevent the
45 promotion, aid, or encouragement of self-harm, (iii) techniques to
46 determine whether a user is expressing thoughts of self-harm, and (iv)
47 techniques to discover vulnerabilities in the proprietor's system. The
48 attorney general shall also consider that prevalent practices of the
49 industry of the proprietor and the impact of the techniques listed in
50 subparagraphs (i) through (iv) of this paragraph on the user's safety,
51 utility, and experience.
52 (c) Such regulations shall also identify the appropriate levels of
53 accuracy that would be commercially reasonable and technically feasible
54 for proprietors to achieve in determining (i) whether a user is a minor,
55 (ii) whether the proprietor's companion chatbot is promoting, aiding, or
S. 5668 4
1 encouraging self-harm, and (iii) whether a user is expressing thoughts
2 of self-harm.
3 9. Information collected for the purpose of determining a user's age
4 under paragraph (a) of subdivision five of this section shall not be
5 used for any purposes other than age determination and shall be deleted
6 immediately after an attempt to determine a user's age, except where
7 necessary for compliance with any applicable provisions of New York
8 state or federal law or regulation.
9 10. The attorney general shall promulgate regulations identifying
10 methods of obtaining verifiable parental consent pursuant to subpara-
11 graph (i) of paragraph (b) of subdivision six of this section.
12 11. Information collected for the purpose of obtaining verifiable
13 parental consent shall not be used for any purpose other than obtaining
14 such verifiable parental consent and shall be deleted immediately after
15 an attempt to obtain verifiable parental consent, except where necessary
16 for compliance with any applicable provisions of New York state or
17 federal law or regulation.
18 12. Nothing in this section shall be construed as requiring any
19 proprietor to give a parent who grants verifiable parental consent any
20 additional or special access to or control over the data or accounts of
21 their child.
22 § 2. This act shall take effect one year after it shall have become a
23 law.
