Enacts the "advanced artificial intelligence licensing act"; providing for regulation of advanced artificial intelligence systems (Part A); requiring registration and licensing of high-risk advanced artificial intelligence systems and related provisions regarding the operation of such systems (Part B); establishing the advanced artificial intelligence ethical code of conduct (Part C); and prohibiting the development and operation of certain artificial intelligence systems (Part D).
STATE OF NEW YORK
________________________________________________________________________
8195
2023-2024 Regular Sessions
IN ASSEMBLY
October 27, 2023
___________
Introduced by M. of A. VANEL -- read once and referred to the Committee
on Science and Technology
AN ACT to amend the state technology law and the criminal procedure law,
in relation to advanced artificial intelligence systems (Part A); to
amend the state technology law, in relation to requiring registration
and licensing of high-risk advanced artificial intelligence systems
and related provisions regarding the operation of such systems (Part
B); to amend the state technology law, in relation to establishing the
advanced artificial intelligence ethical code of conduct (Part C); and
to amend the state technology law, in relation to prohibiting the
development and operation of certain artificial intelligence systems
(Part D)
The People of the State of New York, represented in Senate and Assem-bly, do enact as follows:
1 Section 1. Short title. This act shall be known and may be cited as
2 the "advanced artificial intelligence licensing act".
3 § 2. Legislative findings and determinations. The legislature hereby
4 finds and declares that the transformative power of advanced artificial
5 intelligence is likely to redefine nearly every industry and society,
6 bringing with it the potential for both immense benefits and serious
7 risks.
8 The legislature also finds and declares that excessive and burdensome
9 regulations for advanced artificial intelligence can create significant
10 adverse effects at both the state and national levels, particularly in
11 regards to national security and economic stability. The balance between
12 innovation and regulation, if not maintained with care, can result in
13 repercussions affecting the overall progress and well-being of society.
14 The legislature additionally finds and declares that a balanced regu-
15 latory framework, designed with a focus on non-intrusive safeguards for
16 advanced artificial intelligence, has the capacity to curtail bad
EXPLANATION--Matter in italics (underscored) is new; matter in brackets
[] is old law to be omitted.
LBD13008-03-3
A. 8195 2
1 actors, foster positive innovation, and secure the benefits of advanced
2 artificial intelligence for both the state and its residents.
3 § 3. This act enacts into law major components of legislation which
4 are necessary to implement the "advanced artificial intelligence licens-
5 ing act". Each component is wholly contained within a Part identified as
6 Parts A through D. The effective date for each particular provision
7 contained within such Part is set forth in the last section of such
8 Part. Any provision in any section contained within a Part, including
9 the effective date of the Part, which makes a reference to a section "of
10 this act", when used in connection with that particular component, shall
11 be deemed to mean and refer to the corresponding section of the Part in
12 which it is found. Section five of this act sets forth the general
13 effective date of this act.
14 PART A
15 Section 1. The state technology law is amended by adding a new article
16 IV to read as follows:
17 ARTICLE IV
18 ADVANCED ARTIFICIAL INTELLIGENCE SYSTEMS
19 Section 401. Definitions.
20 402. Powers and duties of the department.
21 403. Functions, powers and duties of the secretary.
22 404. Advisory council for artificial intelligence.
23 405. Functions, powers and duties of the council.
24 406. Rulemaking authority.
25 407. Disposition of moneys received for license fees.
26 408. Violations of artificial intelligence laws; penalties and
27 injunctions.
28 409. Formal hearings; notice and procedure.
29 § 401. Definitions. As used in this article, the following terms shall
30 have the following meanings:
31 1. "Advanced artificial intelligence system" shall mean any digital
32 application or software, whether or not integrated with physical hard-
33 ware, that autonomously performs functions traditionally requiring human
34 intelligence. This includes, but is not limited to the system:
35 (a) Having the ability to learn from and adapt to new data or situ-
36 ations autonomously; or
37 (b) Having the ability to perform functions that require cognitive
38 processes such as understanding, learning, or decision-making for each
39 specific task.
40 2. "High-risk advanced artificial intelligence system" shall mean any
41 advanced artificial intelligence system that possesses capabilities that
42 can cause significant harm to the liberty, emotional, psychological,
43 financial, physical, or privacy interests of an individual or groups of
44 individuals, or which have significant implications on governance,
45 infrastructure, or the environment. The director shall assess any such
46 public or private system in determining whether such system requires
47 registration. High-risk advanced artificial intelligence systems shall,
48 at least, include systems that are designed to, whether directly or
49 indirectly, on purpose or without purpose, do the following:
50 (a) Cause material harm to persons, wildlife, or the environment;
51 (b) Manage, control, or significantly influence healthcare or health-
52 care-related systems, including but not limited to, diagnosis, treatment
53 plans, pharmaceutical recommendation, or storing of patient records;
A. 8195 3
1 (c) Operate, control, or guide motor vehicles, aircraft, or any other
2 forms of transport which, if it were to malfunction, has a high proba-
3 bility of posing a risk to human safety or environmental integrity;
4 (d) Psychologically profile individuals for the purpose of targeted
5 advertising, behavioral prediction, or the manipulation of user experi-
6 ences and interactions in products or services;
7 (e) Manage, control, or create critical infrastructure, including but
8 not limited to the supply of water, electricity, gas, and heating, or
9 construction;
10 (f) Facilitate, control, or significantly impact financial systems,
11 including but not limited to control of stock exchanges, stock trading,
12 credit scoring, or other activities where inaccuracies or failures could
13 lead to substantial economic harm for individuals or broader financial
14 instability;
15 (g) Assist, replace, or augment human decision-making in law enforce-
16 ment, the judiciary, the executive, the legislature, or any government
17 agency;
18 (h) Enable advanced surveillance capabilities;
19 (i) Involve the use or development of autonomous weapons systems that
20 can cause harm, destruction, or engage in conflict without meaningful
21 human intervention; and
22 (j) Decode or interpret neural or cognitive activity.
23 3. "System" shall be used interchangeably with high-risk advanced
24 artificial intelligence system unless the context shall otherwise
25 require.
26 4. "Uncontained" shall mean that critical components of the source
27 code of a high-risk advanced artificial intelligence system that, in
28 substantially their original form, have been reproduced by an amount of
29 individuals so numerous that it is deemed to be practically impossible
30 to prohibit or control its usage using existing technology. The terms
31 "contain" and "contained" shall not be construed as meaning the opposite
32 of uncontained.
33 5. "Operator" shall mean the person who distributes and has control
34 over the development of a high-risk advanced artificial intelligence
35 system. Where a high-risk advanced artificial intelligence system is
36 publicly accessible code, the operator shall be deemed the platform or
37 platforms which host the system.
38 6. "Publicly accessible code" shall mean software whose source code is
39 made available to the public where the public is capable of using, modi-
40 fying, or distributing the source code irrespective of the associated
41 costs to use, modify, or distribute the source code, if any, or the
42 underlying license agreement or rights to use, modify, or distribute the
43 source code.
44 7. "Secretary" shall mean the secretary of state.
45 8. "Department" shall mean the department of state.
46 9. "Council" shall mean the advisory council for artificial intelli-
47 gence established pursuant to section four hundred four of this article.
48 10. "Person" shall mean any individual, group of individuals, partner-
49 ship, corporation, association or any other entity.
50 § 402. Powers and duties of the department. The department shall have
51 the following functions, powers and duties:
52 1. Discretion to issue or refuse to issue any license provided for in
53 this article in accordance with the relevant provisions for each
54 license.
55 2. To revoke, cancel or suspend, after notice and an opportunity to be
56 heard, except where immediate revocation, cancellation, or suspension is
A. 8195 4
1 necessary to protect the public, any license issued under this article
2 for a violation of this article or any regulation pursuant thereto.
3 3. To impose or recover a civil or criminal penalty, as otherwise
4 authorized under this article, against any person found to have violated
5 any provision of this article, whether or not a license has been issued
6 to such person pursuant to this article.
7 4. To promulgate rules and regulations in accordance with statutory
8 grants of authority pursuant to this article.
9 5. To hold hearings, subpoena witnesses, compel their attendance,
10 administer oaths, to examine any person under oath and in connection
11 therewith to require the production of any books, records, documents,
12 source code or logs relative to an inquiry pursuant to the provisions of
13 this article. A subpoena issued pursuant to this subdivision shall be
14 subject to the provisions of the criminal procedure law.
15 6. To appoint any necessary deputies, counsels, assistants, investi-
16 gators, and other employees to carry out the provisions of this article
17 within the limits provided by appropriation. Deputies, counsels and
18 confidential secretaries to department members shall be in the exempt
19 class of the civil service. The other assistants, investigators and
20 employees of the department employed to carry out the provisions of this
21 article shall all be in the competitive class of the civil service and
22 shall be considered for purposes of article fourteen of the civil
23 service law to be public employees of the state, and shall be assigned
24 to the appropriate collective bargaining unit. Investigators so employed
25 by the department shall be deemed to be peace officers only for the
26 purposes of enforcing the provisions of this article or judgments or
27 orders obtained for violation thereof, with all the powers set forth in
28 section 2.20 of the criminal procedure law.
29 7. To prescribe forms of applications for licenses under this article
30 and of all reports deemed necessary by the department.
31 8. To appoint such advisory groups and committees as deemed necessary
32 to provide assistance to the department to carry out the purposes and
33 objectives of this article.
34 9. To enter into contracts, memoranda of understanding, and agreements
35 as deemed appropriate to effectuate the policy and purpose of this arti-
36 cle.
37 10. If public health, safety, or welfare imperatively requires emer-
38 gency action, and incorporates a finding to that effect in an order,
39 summary suspension of a license issued pursuant to this article may be
40 ordered, effective on the date specified in such order or upon service
41 of a certified copy of such order on the licensee, whichever shall be
42 later, pending proceedings for revocation or other action. These
43 proceedings shall be promptly instituted and determined. In addition,
44 the department may order the administrative seizure of services, issue a
45 stop order, or take any other action necessary to effectuate and enforce
46 the policies and purposes of this article.
47 11. To draft, provide for public comment on and issue declaratory
48 rulings, guidance and industry advisories.
49 12. When an administrative decision is appealed to the department by
50 an applicant, registered organization, licensee or permittee, issue a
51 final determination of the department.
52 § 403. Functions, powers and duties of the secretary. The secretary
53 shall have the following functions, powers and duties:
54 1. To exercise the powers and perform the duties in relation to the
55 administration of the provisions of this article.
A. 8195 5
1 2. To keep records in such form as he or she may prescribe of all
2 registrations and licenses issued and revoked within the state; such
3 records shall be so kept as to provide ready information as to the iden-
4 tity of all licensees including the names of the officers and directors
5 of corporate licensees. The secretary may contract to furnish copies of
6 the records of licenses issued within the state or any political subdi-
7 vision thereof, for any license year or term of years not exceeding five
8 years.
9 3. To prescribe forms of applications for licenses under this article
10 and of all reports deemed necessary by the department.
11 4. To delegate the powers provided in this section to such other offi-
12 cers or employees as may be deemed appropriate by the secretary.
13 5. To enter into contracts, memoranda of understanding and agreements
14 to effectuate the policy and purpose of this article.
15 6. To advise and assist the department in carrying out any of its
16 functions, powers and duties.
17 7. To coordinate across state agencies and departments in order to
18 research and study artificial intelligence and the impact it may have on
19 various industries and subject matters.
20 8. To issue guidance and industry advisories.
21 9. To create and maintain a publicly available directory of the names
22 and locations of persons licensed pursuant to this article.
23 10. To create a system whereby persons licensed under this article can
24 confirm the license of another person for the purposes of ensuring
25 compliance with this article.
26 § 404. Advisory council for artificial intelligence. 1. There shall be
27 an advisory council for artificial intelligence. The secretary shall
28 serve as chair of the council. The remainder of the council shall be
29 composed as follows:
30 (a) two members of the artificial intelligence industry appointed by
31 the governor;
32 (b) one member of the artificial intelligence industry appointed by
33 the temporary president of the senate;
34 (c) one member of the artificial intelligence industry appointed by
35 the speaker of the assembly;
36 (d) the commissioner of environmental conservation;
37 (e) the superintendent of financial services;
38 (f) the commissioner of health;
39 (g) the commissioner of labor;
40 (h) the commissioner of transportation;
41 (i) the commissioner of the division of human rights;
42 (j) the superintendent of state police;
43 (k) the attorney general;
44 (l) the state comptroller;
45 (m) the commissioner of the division of homeland security and emergen-
46 cy services; and
47 (n) the director of the office of information technology services.
48 2. The members of the council shall receive no compensation for their
49 services, but shall be allowed their actual and necessary expenses
50 incurred in the performance of their duties.
51 § 405. Functions, powers and duties of the council. The council shall
52 have the following functions, powers and duties:
53 1. To review and comment on all rules and regulations of the depart-
54 ment;
55 2. To issue non-binding recommendations on whether to grant a license
56 under this article, what changes need to be made by an applicant for a
A. 8195 6
1 license in order to be granted a license, what changes need to be made
2 in order for an operator to publicly implement modifications, updates,
3 upgrades, and rewrites to an operator's source code, and what changes
4 need to be made to an operator's system in order to maintain such opera-
5 tor's license; and
6 3. To perform such other acts as may be assigned by the chairperson of
7 the council which are necessary or appropriate to carry out the func-
8 tions of the council.
9 § 406. Rulemaking authority. 1. The department shall perform such
10 acts, prescribe such forms and propose such rules, regulations and
11 orders as it may deem necessary or proper to fully effectuate the
12 provisions of this article.
13 2. The department shall, in consultation with the director, have the
14 authority to promulgate any and all necessary rules and regulations
15 governing the use, control, and prohibited activities of advanced arti-
16 ficial intelligence systems as well as, hearing procedures and addi-
17 tional causes for cancellation, suspension, revocation, and/or civil
18 penalties against any person licensed by the department and the circum-
19 stances, manner and process by which a licensee may apply to change or
20 alter its application.
21 3. The department shall promulgate rules and regulations that are
22 designed to:
23 (a) prevent the creation and proliferation of advanced artificial
24 intelligence systems that possess the ability to cause substantial harm
25 to members of the public, society at large, or the environment without
26 justification;
27 (b) prevent the uncontainment of prohibited and high-risk advanced
28 artificial intelligence systems;
29 (c) prevent the use of prohibited and unlicensed high-risk advanced
30 artificial intelligence systems from this state to other states and from
31 other states into this state;
32 (d) educate and inform the public about artificial intelligence and
33 the benefits and risks it poses;
34 (e) inform the public about the prohibition on unlicensed high-risk
35 advanced artificial intelligence systems and prohibited advanced artifi-
36 cial intelligence systems; and
37 (f) establish application and licensing processes which ensure all
38 material owners and interest holders are disclosed and that officials or
39 other individuals with control over the approval of an application or
40 license do not themselves have any interest in an application or
41 license.
42 4. In adopting any emergency rule, the department shall comply with
43 the provisions of subdivision six of section two hundred two of the
44 state administrative procedure act and subdivision three of section one
45 hundred one-a of the executive law; provided, however, that notwith-
46 standing the provisions of such laws:
47 (a) Such emergency rule shall remain in effect for no longer than one
48 hundred twenty days, unless within such time the department complies
49 with the provisions of such laws and adopts the rule as a permanent
50 rule;
51 (b) If, prior to the expiration of a rule adopted pursuant to this
52 subdivision, the department finds that the readoption of such rule on an
53 emergency basis or the adoption of a substantially similar rule on an
54 emergency basis is necessary for the preservation of the public health,
55 safety or general welfare the department may only readopt the rule on an
56 emergency basis or adopt a substantially similar rule on an emergency
A. 8195 7
1 basis if on or before the date of such action the department has also
2 submitted a notice of proposed rulemaking pursuant to subdivision six of
3 section two hundred two of the state administrative procedure act and
4 subdivision three of section one hundred one-a of the executive law. An
5 emergency rule adopted pursuant to this paragraph may remain in effect
6 for no longer than one hundred twenty days;
7 (c) An emergency rule adopted pursuant to this subdivision or a
8 substantially similar rule adopted on an emergency basis may remain in
9 effect for no longer than one hundred twenty days, but upon the expira-
10 tion of such one hundred twenty-day period no further readoptions or
11 adoptions of substantially similar rules shall be permitted for a period
12 of one hundred twenty days. Nothing in this subdivision shall preclude
13 the adoption of such rule by submitting a notice of adoption pursuant to
14 subdivision five of section two hundred two of the state administrative
15 procedure act; and
16 (d) Strict compliance with the provisions of this subdivision shall be
17 required, and any emergency rule or substantially similar rule that does
18 not so comply shall be void and of no legal effect.
19 5. The department shall have the authority to promulgate regulations
20 governing the appropriate use and licensure of the development of high-
21 risk advanced artificial intelligence systems.
22 § 407. Disposition of moneys received for license fees. The department
23 shall establish a scale of application, licensing, and renewal fees,
24 based upon the cost of enforcing this article, the size of the business
25 being licensed, and the risk it poses as follows:
26 1. The department shall charge each licensee a licensure fee, and
27 renewal fee, as applicable. The fees may vary depending upon the nature
28 and scope of the different registration, licensure activities.
29 2. The total fees assessed pursuant to this article shall be set at an
30 amount that shall generate sufficient total revenue to, at a minimum,
31 fully cover the total costs of administering this article.
32 § 408. Violations of artificial intelligence laws; penalties and
33 injunctions. 1. Any person who violates, disobeys or disregards any term
34 or provision of this article or of any lawful notice, order or regu-
35 lation pursuant thereto for which a civil or criminal penalty is not
36 otherwise expressly prescribed in this article by law, shall be liable
37 to the people of the state for a civil penalty of not to exceed the
38 amount gained from such violation, or the actual damages caused by such
39 violation whichever is greater. In assessing the civil penalty under
40 this subdivision, the department, as may be applicable shall take into
41 consideration the nature of such violation and shall assess a penalty
42 that is proportionate to the violation.
43 2. The penalty provided for in subdivision one of this section shall
44 be recovered by an action or proceeding in a court of competent juris-
45 diction brought by the department, as may be applicable, or by the
46 attorney general at the request of the department.
47 3. Such civil penalty may be released or compromised by the depart-
48 ment, as may be applicable, before the matter has been referred to the
49 attorney general, and where such matter has been referred to the attor-
50 ney general, any such penalty may be released or compromised and any
51 action or proceeding commenced to recover the same may be settled and
52 discontinued by the attorney general with the consent of the department.
53 4. It shall be the duty of the attorney general upon the request of
54 the department, as may be applicable, to bring an action or proceeding
55 against any person who violates, disobeys or disregards any term or
56 provision of this article or of any lawful notice, order or regulation
A. 8195 8
1 pursuant thereto for any relief authorized under this article, including
2 equitable and/or injunctive relief and the recovery of civil penalties;
3 provided, however, that the department or the secretary shall furnish
4 the attorney general with such material, evidentiary matter or proof as
5 may be requested by the attorney general for the prosecution of such an
6 action or proceeding.
7 5. It is the purpose of this section to provide additional and cumula-
8 tive remedies, and nothing herein contained shall abridge or alter
9 rights of action or remedies now or hereafter existing, nor shall any
10 provision of this section, nor any action done by virtue of this
11 section, be construed as estopping the state, persons or municipalities
12 in the exercising of their respective rights.
13 6. The department shall forward any final findings of a violation
14 under this article to any other statewide licensing agency where such
15 findings were entered against a business holding any other such license,
16 for any such other licensing agency to review the findings to determine
17 if there has been a violation of any such license issued by such agency.
18 § 409. Formal hearings; notice and procedure. 1. The department, or
19 any person designated by them for this purpose, may issue subpoenas and
20 administer oaths in connection with any hearing or investigation under
21 or pursuant to this article, and it shall be the duty of the department
22 and any persons designated by them for such purpose to issue subpoenas
23 at the request of and upon behalf of the respondent.
24 2. The department and those designated by them shall not be bound by
25 the laws of evidence in the conduct of hearing proceedings, but the
26 determination shall be founded upon preponderance of evidence to sustain
27 it.
28 3. Notice and right of hearing as provided in the state administrative
29 procedure act shall be served at least fifteen days prior to the date of
30 the hearing, provided that, whenever because of danger to the public
31 health, safety or welfare it appears prejudicial to the interests of the
32 people of the state to delay action for fifteen days or when necessary
33 for the preservation of the public health, safety or general welfare,
34 the office may serve the respondent with an order requiring certain
35 action or the cessation of certain activities immediately or within a
36 specified period of less than fifteen days.
37 4. Service of notice of hearing or order shall be made by personal
38 service or by registered or certified mail. Where service, whether by
39 personal service or by registered or certified mail, is made upon an
40 incompetent individual, partnership, or corporation, it shall be made
41 upon the person or persons designated to receive personal service by
42 article three of the civil practice law and rules.
43 5. At a hearing, that to the greatest extent practicable shall be
44 reasonably near the respondent, the respondent may appear personally,
45 shall have the right of counsel, and may cross-examine witnesses against
46 him or her and produce evidence and witnesses on his or her behalf.
47 6. Following a hearing, the department may make appropriate determi-
48 nations and issue a final order in accordance therewith. The respondent
49 shall have thirty days to submit a written appeal to the department. If
50 the respondent does not submit a written appeal within thirty days of
51 the determination of the department the order shall be final.
52 7. The department may adopt, amend and repeal administrative rules and
53 regulations governing the procedures to be followed with respect to
54 hearings, investigations, and other administrative enforcement actions
55 taken pursuant to this article, including any such enforcement actions
56 taken against persons not licensed under this article. Such rules shall
A. 8195 9
1 be consistent with the policy and purpose of this article and the effec-
2 tive and fair enforcement of its provisions.
3 8. The provisions of this section shall be applicable to all hearings
4 held pursuant to this article, except where other provisions of this
5 article applicable thereto are inconsistent therewith, in which event
6 such other provisions shall apply.
7 § 2. Section 2.10 of the criminal procedure law is amended by adding a
8 new subdivision 87 to read as follows:
9 87. Investigators appointed by the department of state, pursuant to
10 section four hundred two of the state technology law; provided, however,
11 that nothing in this subdivision shall be deemed to authorize such offi-
12 cer to carry, possess, repair, or dispose of a firearm unless the appro-
13 priate license therefor has been issued pursuant to section 400.00 of
14 the penal law.
15 § 3. This act shall take effect on the one hundred eightieth day after
16 it shall have become a law. Effective immediately, the addition, amend-
17 ment and/or repeal of any rule or regulation necessary for the implemen-
18 tation of this act on its effective date are authorized to be made and
19 completed on or before such effective date.
20 PART B
21 Section 1. The state technology law is amended by adding twenty new
22 sections 410, 411, 412, 413, 414, 415, 416, 417, 418, 419, 420, 421,
23 422, 423, 424, 425, 426, 427, 428 and 431 to read as follows:
24 § 410. Duty to register a high-risk advanced artificial intelligence
25 system. 1. Any person who develops a high-risk advanced artificial
26 intelligence system, whether in whole or in part, in the state that is
27 presently performing functions for its intended purpose or within its
28 designated operational parameters, shall have the duty to disclose the
29 existence and function of said system to the secretary by applying for a
30 license as required under section four hundred eleven of this article
31 or, where applicable, a supplemental license under section four hundred
32 twelve of this article. This duty to disclose shall be triggered by the
33 system's active deployment and usage in its intended context or field of
34 operation and is applicable irrespective of the system's location of
35 operation. This duty extends to any updates, modifications, upgrades,
36 or expansions of the system's capabilities or intended uses.
37 2. Any person developing a system as defined in paragraph (i) of
38 subdivision two of section four hundred one of this article within the
39 state shall disclose in writing to the secretary the development of such
40 a system prior to active development of the system. Such writing shall
41 set forth the names and addresses of all persons involved in the devel-
42 opment of such system, a description of the system, the systems func-
43 tions and intended use cases, and measures that will be taken to ensure
44 that any risks posed by the system are mitigated. The secretary may,
45 upon receipt of such writing, require such person to cease development
46 of such a system where, in the secretary's discretion, the secretary
47 believes the system has a high likelihood of violating section four
48 hundred twenty-nine or section four hundred thirty of this article.
49 3. The duties set forth in this section shall apply only to advanced
50 artificial intelligence systems that more likely than not fall under the
51 definition of high-risk advanced artificial intelligence system as
52 defined in section four hundred one of this article. The secretary
53 shall send notice to any system that is presently performing functions
54 for its intended purpose or within its designated operational parameters
A. 8195 10
1 which, in his or her discretion, may fall under the definition of high-
2 risk advanced artificial intelligence systems but that has not regis-
3 tered with the secretary. In the notice, the secretary may require the
4 creators of the system to cease development and access by private indi-
5 viduals or the general public, pending review. Such notice shall be
6 binding and have the effect of law. Determinations that a system is a
7 high-risk advanced artificial intelligence system shall be made in a
8 hearing held pursuant to the provisions of section four hundred nine of
9 this article. In such hearing, the administrator of such hearing shall
10 accept comments from the public. Such hearing shall, to the extent prac-
11 ticable, not disclose any proprietary information concerning the
12 advanced artificial intelligence system to the public.
13 4. A determination pursuant to a hearing held pursuant to section four
14 hundred nine of this article shall be binding and final, provided howev-
15 er that the secretary may, in his or her discretion, conduct further
16 review after a determination has been made and may appeal such determi-
17 nation where appropriate based on information not available in such
18 hearing.
19 5. Where a hearing concludes that a non-registered advanced artificial
20 system is a high-risk advanced artificial intelligence system, the crea-
21 tors of such a system shall cease development and public or private use
22 until registration is completed. The secretary may impose a monetary
23 penalty for such failure and charge all costs of the proceeding to the
24 operator where the secretary determines that the creators of the system
25 knew that the system would more likely than not be considered a high-
26 risk advanced artificial intelligence system and willfully failed to
27 register, the secretary may, in his or her discretion, impose punitive
28 fines and other penalties pursuant to the provisions of this article
29 based on the level of risk that the high-risk advanced artificial intel-
30 ligence system possesses, the widespread use of the system, the severity
31 of the damage that the system caused, if any, and whether the system is
32 uncontained. The secretary may also prohibit such persons involved in
33 the development of such a system from obtaining a license for their
34 existing system or systems or in the future.
35 § 411. License. 1. No person shall (a) develop, in whole or in part, a
36 high-risk advanced artificial intelligence system as defined in para-
37 graph (i) of subdivision two of section four hundred one of this article
38 or operate such a system that is presently performing functions for its
39 intended purpose or within its designated operation parameters within
40 the state where such system was developed outside of the state; or (b)
41 operate a high-risk advanced artificial intelligence system other than a
42 system as defined in paragraph (i) of subdivision two of section four
43 hundred one of this article that is presently performing functions for
44 its intended purpose or within its designated operational parameters
45 within the state without first obtaining a license.
46 2. An application for a license under this article shall be in writ-
47 ing, under oath and in the form prescribed by the secretary.
48 3. At the time of filing an application for a license, the applicant
49 shall pay to the secretary an application fee. Such application fee
50 shall be prescribed pursuant to the rules and regulations of the secre-
51 tary.
52 4. A license granted pursuant to this article shall be valid unless
53 revoked or suspended by the secretary or surrendered by the licensee.
54 § 412. Supplemental license. 1. Where a person other than a natural
55 person is licensed under this article, such person shall apply for a
56 supplemental license for each additional high-risk advanced artificial
A. 8195 11
1 intelligence system such person develops after being licensed initially
2 pursuant to section four hundred eleven of this article.
3 2. Notwithstanding any provision of law, rule or regulation to the
4 contrary, a supplemental license shall be provided in the same manner as
5 a license granted pursuant to the provisions of section four hundred
6 eleven of this article and shall be subject to the same requirements,
7 duties and prohibitions as provided for in this article.
8 § 413. Application for licenses. 1. An application for a license
9 required under this article shall be in writing, under oath, and in the
10 form prescribed by the secretary, and shall contain the following:
11 (a) the exact name and address of the applicant, and if the applicant
12 be a co-partnership or association, the names of the members thereof,
13 and if a corporation the date and place of its incorporation;
14 (b) the name and the business and residential address of each member
15 of the ethics and risk management board, each principal, and officer of
16 the applicant; and
17 (c) the description of all known general use cases of the advanced
18 artificial intelligence system, including any purposes foreseen to be
19 implemented by the applicant. A "use case" shall be defined as broad
20 category of potential use.
21 2. After the filing of an application for a license accompanied by
22 payment of the fees for license and investigation, it shall be substan-
23 tively reviewed. After the application is deemed sufficient and
24 complete, the secretary shall issue the license, or the secretary may
25 refuse to issue the license if the secretary shall find that the ethics,
26 experience, character and general fitness of the applicant or any person
27 associated with the applicant are not such as to command the confidence
28 of the community and to warrant the belief that the business will be
29 conducted honestly, fairly and efficiently within the purposes and
30 intent of this article.
31 3. If the secretary refuses to issue a license, the secretary shall
32 notify the applicant of the denial, return to the applicant the sum paid
33 as a license fee, but retain the investigation fee to cover the costs of
34 investigating the applicant.
35 4. Each license issued pursuant to this article shall remain in full
36 force unless it is surrendered by the licensee, revoked or suspended.
37 § 414. License provisions and posting. 1. Any license issued under
38 this article shall state the name and address of the licensee, and if
39 the licensee be a co-partnership or association, the names of the
40 members thereof, and if a corporation the date and place of its incorpo-
41 ration.
42 2. Such license or licenses shall be kept conspicuously posted in the
43 office of the licensee and, where such licensee has a public internet
44 presence, on the website or mobile application of the licensee and shall
45 not be transferable or assignable.
46 § 415. Grounds for suspension or revocation of license; procedure. 1.
47 A license granted pursuant to this section may not be renewed, and may
48 be revoked or suspended by the secretary upon a finding that:
49 (a) the licensee has not complied with reporting requirements;
50 (b) the licensee has violated any provision of this article;
51 (c) the licensee knowingly allowed a non-certified third-party system
52 to integrate with the licensee's system;
53 (d) any fact or condition exists which, if it had existed at the time
54 of the original application for such license, clearly would have
55 warranted the secretary's refusal to issue such license; or
A. 8195 12
1 (e) the licensee has failed to pay any sum of money lawfully demanded
2 by the secretary or to comply with any demand, ruling or requirement of
3 the secretary.
4 2. Any licensee may surrender any license by delivering to the secre-
5 tary written notice that the licensee thereby surrenders such license,
6 but such surrender shall not affect such licensee's civil or criminal
7 liability for acts committed prior to such surrender.
8 3. Every license issued hereunder shall remain in force and effect
9 until the same shall have been surrendered, revoked or suspended in
10 accordance with the provisions of this article, but the secretary shall
11 have authority to reinstate suspended licenses or to issue new licenses
12 to a licensee whose license or licenses shall have been revoked if no
13 fact or condition then exists which clearly would have warranted the
14 secretary's refusal to issue such license.
15 4. Whenever the secretary shall revoke or suspend a license issued
16 pursuant to this article, the secretary shall forthwith execute a writ-
17 ten order to that effect.
18 5. The secretary may, on good cause shown, or where there is a
19 substantial risk of public harm or substantial risk of a system becoming
20 uncontained, without notice and a hearing, suspend any license issued
21 pursuant to this article for a period not exceeding thirty days, pending
22 investigation.
23 § 416. Ethics and risk management board and reports. 1. Every operator
24 of a licensed high-risk advanced artificial intelligence system or
25 systems shall establish an ethics and risk management board composed of
26 no less than five individuals who shall have the responsibility to
27 assess the ethical implications of all possible use cases of the system,
28 whether such use cases are intended or unintended, and whether likely or
29 unlikely to be used, and the current operational outcomes of the system.
30 Such operator, other than an operator who is a natural person, operating
31 more than one high-risk advanced artificial intelligence system with a
32 supplemental license shall not be required to have more than one ethics
33 and risk management board for each system.
34 2. No member of an ethics and risk management board shall be a member,
35 officer, or director within the operator's entity. No member shall be
36 required to be employed by the operator.
37 3. Such board shall adopt rules governing its decision-making proc-
38 esses, duties and responsibilities. Such rules shall not conflict with
39 the provisions of this article.
40 4. Annually, the ethics and risk management board of each operator
41 shall submit to the secretary a comprehensive report for each licensed
42 high-risk advanced artificial intelligence system which consists of the
43 following:
44 (a) All possible use cases, whether intended or unintended, whether
45 likely or unlikely.
46 (b) A thorough risk assessment for each use case, considering and
47 evaluating the potential for harm, irrespective of the probability of
48 such risk materializing. This shall include, but not be limited to, the
49 system's potential impact on privacy, security, fairness, economic
50 implications, societal well-being, and safety of persons and the envi-
51 ronment.
52 (c) A detailed evaluation of known use cases of the system by users,
53 exploring whether certain applications ought to be constrained or banned
54 due to ethical considerations. This shall include an assessment of the
55 operator's capacity to impose such constraints on use cases.
A. 8195 13
1 (d) A mitigation plan for each identified risk, including preemptive
2 measures, monitoring processes, and responsive actions. This shall also
3 include a communication strategy to inform users and stakeholders about
4 potential risks and steps taken to mitigate them.
5 (e) A comprehensive review of any incidents or failures of the system
6 in the past year, detailing the circumstances, impacts, measures taken
7 to address the issue, and modifications made to prevent such incidents
8 in the future.
9 (f) Any existing attempts to educate users and, based on the existing
10 use of the system by users, a detailed plan on how the operator intends
11 to inform and instruct users on the safe and ethical use of the system,
12 considering varying levels of digital literacy among users.
13 (g) A disclosure of any conflicts of interest within the ethics board,
14 which could potentially influence the board's decisions and recommenda-
15 tions. This shall include measures to manage and resolve such conflicts.
16 (h) An update on the measures taken by the operator to ensure the
17 system's adherence to existing laws, regulations, and ethical guidelines
18 related to artificial intelligence.
19 5. In addition to any applicable civil penalties pursuant to section
20 four hundred eight of this article, a member of an ethics and risk
21 management board who makes a false statement, fails to disclose
22 conflicts of interest or misrepresents the risks or severity of the
23 risks posed by a system in the performance of his or her duties as a
24 member of such board, shall be guilty of a misdemeanor and, upon
25 conviction, shall be fined not more than five hundred dollars or impri-
26 soned for not more than six months or both, in the discretion of the
27 court.
28 § 417. Source code and outcome review. 1. The secretary shall conduct
29 periodic evaluations of the source code and outcomes associated with
30 each high-risk advanced artificial intelligence system. These examina-
31 tions shall determine whether the system is in compliance with this
32 article. The timing and frequency of these reviews shall be determined
33 at the secretary's discretion, taking into account the potential risk
34 posed by the system, the complexity of the system, the frequency of
35 updates and upgrades, the complexity of such updates and upgrades, and
36 any previous issues of non-compliance.
37 2. Upon completion of the review, the secretary is empowered to make
38 binding recommendations to the operator to ensure the system's function-
39 ality and outcomes are aligned with the principles in the advanced arti-
40 ficial intelligence ethical code of conduct pursuant to section four
41 hundred twenty-nine of this article, restrictions on prohibited artifi-
42 cial intelligence systems pursuant to section four hundred thirty of
43 this article, and limitations and procedures for source code modifica-
44 tions, updates, upgrades, and rewrites pursuant to section four hundred
45 nineteen of this article.
46 3. Following receipt of the secretary's recommendations, the operator
47 shall consult with the secretary to determine the feasibility of imple-
48 menting the recommendations and the time frame in which such recommenda-
49 tions can be implemented to ensure full compliance with the secretary's
50 recommendations. The operator shall provide a detailed plan outlining
51 how the recommendations will be addressed, along with a timeline for
52 their implementation. The detailed plan shall be binding on the opera-
53 tor; provided however that where an unexpected occurrence arises which
54 causes changes to such plan, the operator shall be entitled to extend
55 such timeline or alter such plans where such operator notifies the
56 secretary in writing regarding the unexpected occurrence and, within
A. 8195 14
1 such writing, sets forth amendments to the detailed plan and timeline.
2 The secretary shall have thirty days to approve or reject such amend-
3 ments. Where such amendments are rejected, the operator shall continue
4 with their original plan and timeline.
5 4. The secretary shall monitor the operator's compliance with such
6 recommendations and may impose fines and other penalties pursuant to the
7 provisions of this article for non-compliance that the secretary shall
8 deem just and proportionate to the violation.
9 § 418. Willfully or negligently uncontaining high-risk source code. 1.
10 No licensee or non-licensee who develops a high-risk advanced artificial
11 intelligence system shall willfully or negligently uncontain their
12 source code except where authorized by the secretary in writing.
13 2. Any member, officer, director or employee of an entity who willful-
14 ly violates subdivision one of this section shall be guilty of a class E
15 felony.
16 3. Any member, officer, director or employee of an entity who negli-
17 gently violates subdivision one of this section shall be guilty of a
18 class A misdemeanor.
19 4. Where any member, officer, director or employee or an entity will-
20 fully or negligently uncontains a high-risk advanced artificial intelli-
21 gence system described in paragraph (f) of subdivision two of section
22 four hundred one of this article or a prohibited high-risk advanced
23 artificial intelligence system as described in section four hundred
24 thirty of this article shall be guilty of a class C felony.
25 5. The provisions of this section shall not be construed as imposing
26 liability on any member, officer, director or employee who had no
27 explicit or implicit knowledge of the risk or circumstances that caused
28 the uncontainment of the high-risk advanced artificial intelligence
29 system.
30 § 419. Source code modifications, updates, upgrades, and rewrites. 1.
31 Where a licensee intends to modify or upgrade the source code of his or
32 her high-risk advanced artificial intelligence system, such licensee
33 shall be required to inform the secretary of such modification or
34 upgrade and shall be prohibited from implementing such modification or
35 upgrade in an accessible version of the system without express consent
36 of the secretary in writing. This section shall not apply to source code
37 updates.
38 2. A licensee shall, in writing to the secretary, set forth the
39 purpose of the modification or upgrade, the new functions added to the
40 system or the functions modified, the reason for the modification or
41 upgrade, and an assessment of new risks or risks that may be more proba-
42 ble as a result of the modification or upgrade. The secretary shall,
43 upon receipt of notice, have thirty business days to provide the licen-
44 see with approval of the modification or upgrade. Where approval is not
45 received within thirty business days, absent an extension in writing
46 which shall not exceed thirty additional business days, the modification
47 or upgrade shall be deemed approved. Nothing in this subdivision shall
48 be construed as limiting the ability of the secretary to take any action
49 he or she is authorized to take in relation to the approved modification
50 or upgrade. Where the secretary rejects the modification or upgrade, the
51 secretary shall set forth in writing the reasons for the rejection and
52 steps that the licensee can take to receive approval. Where the secre-
53 tary approves the modification or upgrade, the licensee may immediately
54 implement such modification or upgrade in a publicly accessible version.
55 3. A licensee who rewrites the source code of its system shall comply
56 with the same standards set forth in subdivisions one and two of this
A. 8195 15
1 section provided however that the secretary shall examine such source
2 code in the same manner as a new application and shall provide a letter
3 of approval or rejection upon completion of such review within one
4 hundred eighty business days of receipt of such notices except where the
5 secretary requires an extension of time, then an extension of no more
6 than one hundred eighty days shall be authorized. Where the secretary
7 rejects the rewrite, such letter of rejection shall state the reasons
8 for the rejection and steps that the licensee can take to correct such
9 rejection, if any. Where the secretary approves the modification or
10 upgrade, the licensee may immediately implement such modification or
11 upgrade in a publicly accessible version.
12 4. All modifications, upgrades, and rewrites shall be conducted in a
13 pre-production environment, which shall mean any stage prior to the
14 accessible version.
15 5. For purposes of this section:
16 (a) "Modify" shall mean altering the source code of the system to
17 alter the way by which the system, or any features within the system,
18 makes decisions.
19 (b) "Upgrade" shall mean altering the source code of the system which
20 gives it new features or functions.
21 (c) "Rewrite" shall mean a change in the source code to such a
22 substantial degree that:
23 (i) it effectively results in a new version of the system; or
24 (ii) the change nullifies all or a substantial amount of the initial
25 findings of the secretary in the operator's original application.
26 (d) "Update" shall mean a change to the source code that includes
27 minor enhancements, improvements, modifications, error corrections,
28 cosmetic changes, or any other change intended to increase the function-
29 ality, compatibility, security or performance of the system.
30 (e) "Accessible version" shall mean a version of the software that is
31 available to the public or for private use or that is presently operat-
32 ing within its designated operational parameters.
33 § 420. Malfunction and incident reports; duty to notify. 1. A licensee
34 shall have the duty to notify the department and, if applicable, a rele-
35 vant law enforcement agency or governmental entity where the licensee's
36 system fails to operate as intended for any significant period of time.
37 A period of time is deemed "significant" for purposes of this section
38 where the period of time that the malfunction occurred had the capacity
39 to or has harmed a person or persons.
40 2. A licensee shall have the duty to notify a relevant law enforcement
41 agency or governmental entity of a malfunction where designated by the
42 department upon receipt of a license. The secretary shall issue such a
43 requirement upon the licensee where such systems interact with law
44 enforcement systems or the systems of a government agency, engage in law
45 enforcement functions or the functions of a government agency, or where
46 such systems operate, in whole or in part, or are, a weapon.
47 § 421. State and national security risks. 1. The secretary may, by
48 regulation, designate unique requirements for systems which, in the
49 secretary's discretion, pose a risk to state or national security. Such
50 systems shall be assessed on a case-by-case basis and shall not be
51 liberally construed as including any system that, where used improperly,
52 inherently possesses the ability to harm persons or property.
53 2. A high-risk advanced artificial intelligence system shall be deemed
54 to pose a risk to state or national security where the system's malfunc-
55 tioning or misuse poses a high risk of:
56 (a) disrupting critical infrastructure;
A. 8195 16
1 (b) triggering or escalating existing conflicts;
2 (c) undermining or impacting the democratic process;
3 (d) causing unauthorized access to classified information as desig-
4 nated by a relevant governmental entity;
5 (e) harming a significant portion of the population or a specific
6 segment of the population;
7 (f) negatively impacting financial markets or economic stability;
8 (g) causing consequential or irreversible damage to the environment;
9 or
10 (h) causing significant harm to the social fabric.
11 § 422. Information and source code sharing. 1. Licensees shall be
12 permitted to share information and source code with any third party,
13 provided however, that where information is biometric information such
14 party shall be jointly liable for any harm or violations under this
15 article with the licensee. The secretary may, in his or her discretion,
16 prohibit any person from accessing the information or source code of a
17 licensee provided however that the secretary shall provide a written
18 justification for such a prohibition.
19 2. For purposes of this section, "biometric information" shall include
20 a person's:
21 (a) faceprint;
22 (b) voiceprint;
23 (c) fingerprint;
24 (d) gaitprint;
25 (e) irisprint;
26 (f) psychological profile; or
27 (g) any other data related to a person's body or mind that can be used
28 to identify a person.
29 3. This section shall only apply to the sharing of information
30 received or generated by the licensee or source code created by the
31 licensee and shall not apply to a third party integrating their systems
32 with the licensee.
33 § 423. Third-party systems; certificates of compliance. 1. Non-licen-
34 see third-party systems may integrate with a licensee under the follow-
35 ing conditions:
36 (a) Where a third-party system assists in the proper functioning of
37 the licensee or where such system provides additional services to the
38 licensee's service-offerings, such a system shall not be required to
39 obtain a license but shall be required to obtain a certificate of
40 compliance in accordance with this section.
41 (b) No third-party system may access the system of a licensee to
42 provide itself with new high-risk advanced artificial intelligence capa-
43 bilities without first obtaining a license.
44 2. Every third-party system which integrates with a licensee shall,
45 prior to integration, apply for and receive a certificate of compliance.
46 Such certificate shall be issued by the department and shall only be
47 issued where such third-party system is assessed by the department and
48 the department finds it conforms to the cybersecurity standards set by
49 the office. The secretary shall set the rules and regulations regarding
50 the application and requirements of receiving a certificate of compli-
51 ance. This section shall not be construed as requiring any third-party
52 system to receive more than one certificate of compliance.
53 § 424. Logging. Every time a licensee's system operates it shall auto-
54 matically generate a log. Standards related to the specific types of
55 events that are required to be logged, the format in which logs must be
56 kept, the individuals or entities permitted to access logs and the
A. 8195 17
1 conditions governing such access, the encryption and cybersecurity
2 protocols to be applied to logs, the procedures for both the preserva-
3 tion and disposal of logs, and any other actions pertinent to log
4 management shall conform to the standards set by the secretary. Such
5 logs shall be preserved for a period of ten years from the date they are
6 generated and shall be subject to inspection under section four hundred
7 twenty-six of this article.
8 § 425. Internal controls; ceasing operation. Every licensee shall have
9 in place internal controls that, within a reasonable time following
10 initiation, can safely and indefinitely cease the operation of the
11 system or a major part of the system.
12 § 426. Investigations and examinations. 1. The secretary shall have
13 the power to make such investigations as the secretary shall deem neces-
14 sary to determine whether any operator or any other person has violated
15 any of the provisions of this article, or whether any licensee has
16 conducted itself in such manner as would justify the revocation of its
17 license, and to the extent necessary therefor, the secretary may require
18 the attendance of and examine any person under oath, and shall have the
19 power to compel the production of all relevant books, records, accounts,
20 documents, source code, and logs.
21 2. The secretary shall have the power to make such examinations of the
22 books, records, accounts, documents, source code, and logs used in the
23 business of any licensee as the secretary shall deem necessary to deter-
24 mine whether any such licensee has violated any of the provisions of
25 this article.
26 3. The expenses incurred in making any examination pursuant to this
27 section shall be assessed against and paid by the licensee so examined,
28 except that traveling and subsistence expenses so incurred shall be
29 charged against and paid by licensees in such proportions as the secre-
30 tary shall deem just and reasonable, and such proportionate charges
31 shall be added to the assessment of the other expenses incurred upon
32 each examination. Upon written notice by the secretary of the total
33 amount of such assessment, the licensee shall become liable for and
34 shall pay such assessment to the secretary.
35 4. All reports of examinations and investigations, and all correspond-
36 ence and memoranda concerning or arising out of such examinations or
37 investigations, including any duly authenticated copy or copies thereof
38 in the possession of any licensee or the department, shall be confiden-
39 tial communications, shall not be subject to subpoena and shall not be
40 made public unless, in the judgment of the secretary, the ends of
41 justice and the public advantage will be subserved by the publication
42 thereof, in which event the secretary may publish or authorize the
43 publication of a copy of any such report or other material referred to
44 in this subdivision, or any part thereof, in such manner as the secre-
45 tary may deem proper.
46 § 427. Books, records, source code, and logs to be kept. 1. Every
47 operator shall maintain such books, records, source code, and logs as
48 the secretary shall require provided however that every operator shall,
49 at least, maintain a copy of all logs generated from the system as well
50 as a backup of every version of the system which shall be stored in a
51 safe manner as prescribed by the secretary.
52 2. By a date to be set by the secretary, each operator shall annually
53 file a report with the secretary giving such information as the secre-
54 tary may require concerning the business and operations during the
55 preceding calendar year of the operator within the state under the
56 authority of this article. Such report shall be subscribed and affirmed
A. 8195 18
1 as true by the operator under the penalties of perjury and be in the
2 form prescribed by the secretary. In addition to such annual reports,
3 the secretary may require of operators such additional regular or
4 special reports as the secretary may deem necessary to the proper super-
5 vision of operators under this article. Such additional reports shall be
6 in the form prescribed by the secretary and shall be subscribed and
7 affirmed as true under the penalties of perjury.
8 § 428. Regulations and rulings. The secretary is hereby authorized and
9 empowered to make such rules and regulations, conduct hearings and make
10 such specific rulings, orders, demands and findings as may be necessary
11 for the proper conduct of the business authorized and licensed under and
12 for the enforcement of this article.
13 § 431. Severability. If any provision of this article or the applica-
14 tion thereof to any person or circumstances is held invalid, the inva-
15 lidity thereof shall not affect other provisions or applications of the
16 article which can be given effect without the invalid provision or
17 application, and to this end the provisions of this article are severa-
18 ble.
19 § 2. Section 401 of the state technology law, as added by section 1 of
20 part A of this act, is amended by adding a new subdivision 11 to read as
21 follows:
22 11. "Log", "logs" or "logging" shall mean a systematic, chronological-
23 ly ordered record of events pertaining to a system's operations, activ-
24 ities, and transactions that is in compliance with the standards set by
25 the secretary in accordance with section four hundred twenty-four of
26 this article. In the context of logging, an event shall refer to any
27 significant or notable occurrence, action, or anomaly within the system.
28 § 3. Subdivisions 5 and 6 of section 408 of the state technology law,
29 as added by section 1 of part A of this act, are renumbered subdivisions
30 7 and 8 and two new subdivisions 5 and 6 are added to read as follows:
31 5. Any person who knowingly makes any incorrect statement of a materi-
32 al fact in any application, report or statement filed pursuant to this
33 article, or who knowingly omits to state any material fact necessary to
34 give the director any information lawfully required by the secretary or
35 refuses to permit any lawful investigation or examination, shall be
36 guilty of a misdemeanor and, upon conviction, shall be fined not more
37 than five hundred dollars or imprisoned for not more than six months or
38 both, in the discretion of the court.
39 6. No person shall make, directly or indirectly, orally or in writing,
40 or by any method, practice or device, a representation that such entity
41 is licensed under the law except that a licensee under this chapter may
42 make a representation that the licensee is licensed as a high-risk
43 advanced artificial intelligence system under this article.
44 § 4. This act shall take effect one year after it shall have become a
45 law. Effective immediately, the addition, amendment and/or repeal of any
46 rule or regulation necessary for the implementation of this act on its
47 effective date are authorized to be made and completed on or before such
48 effective date.
49 PART C
50 Section 1. The state technology law is amended by adding a new section
51 429 to read as follows:
52 § 429. Advanced artificial intelligence ethical code of conduct. The
53 following ethical code of conduct shall be binding on all licensees and
A. 8195 19
1 non-licensees who develop or operate a high-risk advanced artificial
2 intelligence system:
3 Respect: Artificial intelligence systems should respect human autonomy
4 and not unduly influence or manipulate individuals' behavior or deci-
5 sions.
6 Equity: An artificial intelligence system should provide equitable
7 outcomes, irrespective of any characteristics protected by law. They
8 should not perpetuate existing biases, discrimination, or disparities.
9 Accountability: Persons that design, develop, deploy, or use artifi-
10 cial intelligence systems should be held accountable for the impacts and
11 outcomes of these systems except where the law provides otherwise.
12 Clear mechanisms for addressing harms and violations of law should be in
13 place.
14 Care: Artificial intelligence systems should not cause harm or
15 adversely affect individuals, society, or the environment without legal
16 justification.
17 Trust: Artificial intelligence systems should respect individuals'
18 privacy rights, and securely handle personal and sensitive data in
19 accordance with applicable laws and regulations.
20 Inclusivity: Artificial intelligence systems should be designed,
21 developed, and used in ways that are inclusive, serving a diverse range
22 of users and contexts.
23 Oversight: There should always be meaningful human oversight of arti-
24 ficial intelligence systems to ensure ethical use and decision-making.
25 Notice: The operations, decision-making processes, and use of artifi-
26 cial intelligence systems should, where feasible, be made known to
27 persons affected by them.
28 Safety: Artificial intelligence systems should be robust, secure, and
29 reliable. They should have mechanisms in place to prevent misuse or
30 harmful outcomes.
31 § 2. This act shall take effect on the one hundred eightieth day after
32 it shall have become a law.
33 PART D
34 Section 1. Legislative findings and determinations. The legislature
35 hereby finds and declares that in the instance where the source code of
36 any system with the capacity to destroy or disrupt the security, integ-
37 rity, and moral well-being of the state is created, a palpable opportu-
38 nity arises for such source code to be accessed by entities beyond the
39 jurisdictional boundaries of not only the state of New York but also the
40 United States; under these circumstances, no oversight, law, rule, regu-
41 lation or existing technologies can sufficiently thwart the potential
42 misuse of such a system. Therefore, in recognizing the severe impli-
43 cations of this threat, the legislature hereby declares that certain
44 applications of artificial intelligence harbor such an immense capacity
45 for causing significant harm to the security, integrity, and moral well-
46 being of the state and such a risk of becoming uncontainable that the
47 state has a compelling interest in preventing their creation.
48 § 2. The state technology law is amended by adding a new section 430
49 to read as follows:
50 § 430. Prohibited artificial intelligence systems. 1. No person shall
51 develop, in whole or in part, or operate an artificial intelligence
52 system within the state where such a system performs any of the follow-
53 ing, whether or not it is the system's main function:
A. 8195 20
1 (a) the deployment of subliminal techniques that operate beyond an
2 individual's conscious awareness, with the express purpose of materially
3 distorting an individual's behavior in such a manner that leads to, or
4 possesses a high likelihood of leading to, physical or psychological
5 harm to that individual or another, or that leverages the vulnerabili-
6 ties of a defined group of individuals to similar ends;
7 (b) the infliction of physical or emotional harm upon individuals
8 without any valid law enforcement or self-defense purpose or justifica-
9 tion;
10 (c) the prediction of an individual's future actions or behaviors,
11 followed by subsequent reactions based on these predictions, carried out
12 in such a way that, without legal justification, infringes upon or
13 compromises the individual's liberty, emotional, psychological, or
14 financial interests;
15 (d) the unauthorized acquisition, retention, or dissemination of or
16 access to sensitive personal information or non-public data in violation
17 of applicable data privacy, security, and hacking laws; or
18 (e) the implementation of any form of autonomous weapon system
19 designed to inflict harm on persons, property, or the environment that
20 lack meaningful human supervision or control. "Meaningful human super-
21 vision or control" shall mean the ability to actively manage, intervene,
22 or override the autonomous weapon system's functions.
23 2. Where the secretary discovers the development or operation of a
24 prohibited artificial intelligence system, the secretary may, in writ-
25 ing, demand that the person who is developing or operating such system
26 cease development or operation of or access to such a system within a
27 period of time as the secretary deems necessary to prevent the system
28 from widespread use or, if the system is operational or accessible to
29 persons for use, to ensure the system is properly terminated in such a
30 way to minimize risks of harm to individuals, society, or the environ-
31 ment. A demand made pursuant to this section shall be finally and irre-
32 vocably binding on the person unless the person against whom the demand
33 is made shall, within such period of time set by the secretary, after
34 the giving of notice of such determination, petition the department for
35 a hearing to determine the legal findings of the secretary. The person
36 developing or operating such a prohibited system shall, prior to peti-
37 tion, cease development, operation, and access to the system until and
38 unless such determination is favorable to the person. Such determination
39 may be appealed by any party as of right.
40 3. The secretary shall not grant a license pursuant to this article to
41 any high-risk advanced artificial intelligence system described under
42 this section except as described in subdivision seven of this section.
43 4. Any member, officer, director or employee of an operator of any
44 entity who knowingly publicly or privately operates any system described
45 in this section shall be guilty of a class D felony and shall incur a
46 civil penalty of the amount earned from the creation of the prohibited
47 system or the amount of damages caused by the system, whichever is
48 greater.
49 5. This section shall not be construed as imposing liability on any
50 member, officer, director or employee who had no explicit or implicit
51 knowledge of the prohibited high-risk advanced artificial intelligence
52 system provided however that where the secretary sends a demand to cease
53 the development, operation, or access to such system all members, offi-
54 cers, and directors shall be rebuttably presumed to have knowledge of
55 the prohibited high-risk advanced artificial intelligence system.
A. 8195 21
1 6. This section shall be construed as prohibiting the development of a
2 prohibited high-risk advanced artificial intelligence system or making
3 such a system accessible to persons in the state of New York.
4 7. Notwithstanding subdivision one of this section, a person may
5 develop a prohibited high-risk advanced artificial intelligence system
6 where authorized by the secretary, provided that such system is devel-
7 oped and used only by the state or with substantial, continuous over-
8 sight by the state and such system is authorized only after public hear-
9 ing and comment in accordance with section four hundred nine of this
10 article.
11 § 2. This act shall take effect one year after it shall have become a
12 law. Effective immediately, the addition, amendment and/or repeal of any
13 rule or regulation necessary for the implementation of this act on its
14 effective date are authorized to be made and completed on or before such
15 effective date.
16 § 4. Severability clause. If any clause, sentence, paragraph, subdivi-
17 sion, section or part of this act shall be adjudged by any court of
18 competent jurisdiction to be invalid, such judgment shall not affect,
19 impair, or invalidate the remainder thereof, but shall be confined in
20 its operation to the clause, sentence, paragraph, subdivision, section
21 or part thereof directly involved in the controversy in which such judg-
22 ment shall have been rendered. It is hereby declared to be the intent of
23 the legislature that this act would have been enacted even if such
24 invalid provisions had not been included herein.
25 § 5. This act shall take effect immediately; provided, however, that
26 the applicable effective date of Parts A through D of this act shall be
27 as specifically set forth in the last section of such Parts.
