Senator Hashim proposes amendment to require human review of AI output used in mental-health services

Senator Hashim offered an amendment to the bill addressing use of artificial intelligence in mental-health services. The amendment would require that any AI-generated output used for mental-health services in Vermont be reviewed and approved by a mental-health professional before being shared with a patient, client or AI user.

Under the amendment, mental-health services "shall be provided in Vermont through artificial intelligence only if a mental health professional reviews and approves the artificial intelligence generated output prior to the information being transmitted to a patient, client or artificial intelligence user," according to the text presented. Members discussed whether the amendment would require corporations hosting chatbots or other AI platforms to employ Vermont-licensed mental-health professionals to perform that review and noted the absence of an individual private right of action in the draft consumer-protection language.

Supporters said the amendment is intended to ensure clinical oversight and patient safety where automated tools are used. Some members expressed concerns about enforcement mechanisms and consumer protections if commercial actors rely on out-of-state AI systems. The committee indicated Senator Hashim plans to present the amendment on the floor and that staff will follow up on drafting and any needed consumer protection language.