Prohibits the provision of an artificial intelligence companion to a user unless such artificial intelligence companion contains a protocol for addressing possible suicidal ideation or self-harm expressed by a user, possible physical harm to others expressed by a user, and possible financial harm to others expressed by a user; requires certain notifications to certain users regarding crisis service providers and the non-human nature of such companion models.
STATE OF NEW YORK
________________________________________________________________________
6767
2025-2026 Regular Sessions
IN ASSEMBLY
March 13, 2025
___________
Introduced by M. of A. VANEL -- read once and referred to the Committee
on Consumer Affairs and Protection
AN ACT to amend the general business law, in relation to artificial
intelligence companion models (Part __);
The People of the State of New York, represented in Senate and Assem-bly, do enact as follows:
1 PART __
2 Section 1. The general business law is amended by adding a new article
3 47 to read as follows:
4 ARTICLE 47
5 ARTIFICIAL INTELLIGENCE COMPANION MODELS
6 Section. 1700. Definitions.
7 1701. Prohibitions and requirements.
8 1702. Notifications.
9 1703. Enforcement.
10 1704. Severability.
11 § 1700. Definitions. As used in this article, the following terms
12 shall have the following meanings:
13 1. "Artificial intelligence", "artificial intelligence technology", or
14 "AI" means a machine-based system that can, for a given set of human-de-
15 fined objectives, make predictions, recommendations, or decisions influ-
16 encing real or virtual environments, and that uses machine- and human-
17 based inputs to perceive real and virtual environments, abstract such
18 perceptions into models through analysis in an automated manner, and use
19 model inference to formulate options for information or action.
20 2. "Generative artificial intelligence" means a class of AI models
21 that are self-supervised and emulate the structure and characteristics
22 of input data to generate derived synthetic content, including, but not
23 limited to, images, videos, audio, text, and other digital content.
EXPLANATION--Matter in italics (underscored) is new; matter in brackets
[] is old law to be omitted.
LBD10829-01-5
A. 6767 2
1 3. "AI model" means a component of an information system that imple-
2 ments artificial intelligence technology and uses computational, statis-
3 tical, or machine-learning techniques to produce outputs from a given
4 set of inputs.
5 4. "AI companion" means a system using artificial intelligence, gener-
6 ative artificial intelligence, and/or emotional recognition algorithms
7 to simulate social human interaction, by retaining information on prior
8 interactions and user preference, asking questions, providing advice,
9 and engaging in simulated conversation on matters of personal well-be-
10 ing. "AI companion" shall not include any system used by a business
11 entity solely intended to provide users with information about available
12 commercial services or products, customer account information, or other
13 information related to a user's customer, or potential customer,
14 relationship with such business entity.
15 5. "Operator" means any person, partnership, association, firm, or
16 business entity, or any member, affiliate, subsidiary or beneficial
17 owner of any partnership, association, firm, or business entity who
18 operates or provides an AI companion.
19 6. "Person" means any natural person.
20 7. "Emotional recognition algorithms" means artificial intelligence
21 that detects and interprets human emotional signals in text (using
22 natural language processing and sentiment analysis), audio (using voice
23 emotion AI), video (using facial movement analysis, gait analysis, or
24 physiological signals), or a combination thereof.
25 8. "User" means any person who uses an AI companion within the state
26 and who is not an operator or agent or affiliate of the operator of the
27 AI companion.
28 § 1701. Prohibitions and requirements. It shall be unlawful for any
29 operator to operate or provide an AI companion to a user unless such AI
30 companion contains a protocol for addressing: 1. possible suicidal idea-
31 tion or self-harm expressed by a user to the AI companion, 2. possible
32 physical harm to others expressed by a user to the AI companion, and 3.
33 possible financial harm to others expressed by the user to the AI
34 companion, that includes but is not limited to, a notification to the
35 user that refers them to crisis service providers such as a suicide
36 hotline, crisis text line, or other appropriate crisis services.
37 § 1702. Notifications. An operator shall provide a notification to a
38 user at the beginning of any AI companion interaction and at least every
39 three hours for continuing AI companion interactions thereafter, which
40 states either verbally or in bold and capitalized letters of at least
41 sixteen point type, the following:
42 "THE AI COMPANION (OR NAME OF THE AI COMPANION) IS A COMPUTER PROGRAM
43 AND NOT A HUMAN BEING. IT IS UNABLE TO FEEL HUMAN EMOTION".
44 § 1703. Enforcement. Any person who was physically injured through
45 self-harm or was physically or financially harmed by another as a result
46 of a violation of section seventeen hundred one or seventeen hundred two
47 of this article may bring an action in a court of competent jurisdiction
48 for damages, equitable relief, and such other remedies as the court may
49 deem appropriate.
50 § 1704. Severability. If any clause, sentence, paragraph, subdivision,
51 section or part of this act shall be adjudged by any court of competent
52 jurisdiction to be invalid, such judgment shall not affect, impair, or
53 invalidate the remainder thereof, but shall be confined in its operation
54 to the clause, sentence, paragraph, subdivision, section or part thereof
55 directly involved in the controversy in which such judgment shall have
56 been rendered. It is hereby declared to be the intent of the legislature
A. 6767 3
1 that this act would have been enacted even if such invalid provisions
2 had not been included herein.
3 § 2. This act shall take effect on the one hundred eightieth day after
4 it shall have become a law.
