Requires the division of criminal justice services to formulate a protocol for the regulation of the use of artificial intelligence and facial recognition technology in criminal investigations; restricts the use of artificial intelligence-generated outputs in court.
STATE OF NEW YORK
________________________________________________________________________
10625
IN ASSEMBLY
June 20, 2024
___________
Introduced by COMMITTEE ON RULES -- (at request of M. of A.
Bichotte Hermelyn) -- read once and referred to the Committee on Codes
AN ACT to amend the executive law, in relation to the regulation of the
use of artificial intelligence and facial recognition technology in
criminal investigations; and to amend the criminal procedure law, in
relation to limiting the use of artificial intelligence-generated
outputs in court
The People of the State of New York, represented in Senate and Assem-bly, do enact as follows:
1 Section 1. Legislative intent. Artificial intelligence tools like
2 Cybercheck and facial recognition technologies have been used extensive-
3 ly in criminal investigations across the U.S., including nearly 8,000
4 cases in 40 states. However, their use has raised significant legal
5 challenges due to the lack of transparency in their methodologies and
6 concerns over accuracy and reliability. Defense attorneys have success-
7 fully argued in several cases that AI-generated evidence should be
8 excluded from court proceedings due to these issues. This act aims to
9 address these concerns by prohibiting the use of AI outputs, including
10 facial recognition, as evidence while allowing their use for investi-
11 gative purposes, thereby protecting the rights of defendants and main-
12 taining the integrity of the judicial process.
13 § 2. Section 837 of the executive law is amended by adding a new
14 subdivision 24 to read as follows:
15 24. (a) Promulgate a standardized and detailed written protocol for
16 the utilization of artificial intelligence by police agencies in the
17 investigation of criminal activity. The protocol shall address the
18 following topics:
19 (i) authorizing law enforcement agencies to utilize AI systems and FRT
20 for investigative purposes, including but not limited to data analysis,
21 pattern recognition, facial recognition, and predictive analytics;
22 (ii) authorizing law enforcement agencies to utilize AI systems and
23 FRT to assist in identifying potential suspects, uncovering evidence,
24 and generating investigative leads;
EXPLANATION--Matter in italics (underscored) is new; matter in brackets
[] is old law to be omitted.
LBD15799-01-4
A. 10625 2
1 (iii) requiring law enforcement agencies utilizing AI systems and FRT
2 to implement transparency measures, including maintaining records of
3 AI-generated outputs and the parameters used by the AI system. Such
4 records shall be subject to audit and review by an independent oversight
5 body designated by the department to ensure compliance with the proto-
6 col;
7 (iv) requiring law enforcement agencies utilizing AI systems and FRT
8 to undergo regular independent audits of FRT systems to assess their
9 accuracy and biases. The results of these audits shall be publicly
10 accessible and disclosed in any case where FRT evidence is used; and
11 (v) requiring that law enforcement officers utilizing AI systems and
12 FRT receive thorough training on the limitations and proper use of AI
13 systems and FRT, emphasizing the importance of understanding and miti-
14 gating biases and errors.
15 (b) For the purposes of this subdivision:
16 (i) "artificial intelligence" or "AI" means a machine-based system
17 that can, for a given set of human-defined objectives, make predictions,
18 recommendations, or decisions influencing real or virtual environments;
19 (ii) "AI-generated output" means any information, analysis, or deci-
20 sion produced by an AI system during a criminal investigation, including
21 but not limited to facial recognition, predictive policing models, and
22 other machine learning algorithms; and
23 (iii) "facial recognition technology" or "FRT" means a technology that
24 analyzes facial features and is capable of identifying or verifying a
25 person from a digital image or a video frame from a video source.
26 § 3. The criminal procedure law is amended by adding a new section
27 60.77 to read as follows:
28 § 60.77 Rules of evidence; use of artificial intelligence.
29 1. Artificial intelligence-generated outputs, including facial recog-
30 nition results, shall not be admissible as evidence in any criminal or
31 civil court proceeding.
32 2. No party in a criminal or civil case shall reference or introduce
33 artificial intelligence-generated outputs, including facial recognition
34 results, during questioning of witnesses, suspects, or defendants.
35 3. Artificial intelligence-generated outputs shall not form the basis
36 for any legal decision or judgment.
37 4. Defendants shall have the right to expert witnesses who can testify
38 about the reliability and limitations of artificial intelligence and
39 facial recognition technology systems used in the investigation of their
40 cases. Funding shall be available through the office of indigent legal
41 services to indigent defendants to access such experts.
42 5. All prosecutorial offices must disclose detailed information about
43 any artificial intelligence systems and facial recognition technology
44 used in investigations, including error rates, known biases, source
45 code, and algorithmic transparency, where possible.
46 § 4. This act shall take effect on the ninetieth day after it shall
47 have become a law.