US lawyers raise concerns over how AI conversations may be used

US lawyers are raising fresh warnings that conversations with AI chatbots may no longer be treated as private, especially when legal risk is involved. The concern is growing after recent court rulings suggested that AI-generated chats can be accessed and used in criminal and civil proceedings, depending on how they were created and shared.

The issue has gained attention as more people rely on tools like ChatGPT and Claude for advice, including in sensitive legal situations. Attorneys now say users should assume anything shared with an AI system could potentially be requested in court, similar to other digital communications that lack traditional legal privilege protections.

Court ruling in New York sets key precedent

The alarm intensified after a federal judge in New York ruled in a case involving former financial services executive Bradley Heppner that AI chat records could not be shielded from prosecutors. Heppner had used Anthropic’s Claude chatbot to prepare materials connected to his defense in a securities and wire fraud case.

Judge Jed Rakoff ordered that 31 documents created with the chatbot be handed over to prosecutors. He stated that no attorney-client relationship exists between a user and an AI platform, making the chats potentially discoverable in legal proceedings.

Little-known fact: A Stanford study highlights that major AI companies often rely on user conversations to train and improve their models, which raises concerns about how much personal data is quietly fed back into AI systems during normal usage.

Judges differ on how AI chats should be treated

While the New York ruling raised concerns, another court decision showed a more nuanced approach. In Michigan, US Magistrate Judge Anthony Patti ruled that a self-represented plaintiff did not have to turn over her ChatGPT conversations related to her employment lawsuit.

Patti treated the AI chats as part of the user’s personal work product rather than communications equivalent to a third-party disclosure. He also noted that generative AI tools are “tools, not persons,” highlighting that courts are still divided on how to classify AI interactions.

Lawyers warn clients not to treat AI like legal counsel

Following these rulings, law firms across the United States have begun advising clients to avoid sharing sensitive case details with AI chatbots. Attorneys say users often assume these tools are private or protected, similar to conversations with a lawyer, but that assumption may not hold up in court.

One lawyer, Alexandria Gutiérrez Swette of Kobre & Kim, said clients are being told to proceed with caution when using AI tools. Firms emphasize that anything entered into a chatbot could later be demanded by prosecutors or opposing parties in litigation.

Attorney-client privilege does not automatically apply to AI

Legal experts stress that attorney-client privilege, which protects confidential communication between lawyers and clients, does not extend to AI systems. Because chatbots are not human legal professionals, their outputs and inputs may be treated as third-party communications.

Lawyers are also warning that sharing attorney advice with an AI platform could weaken or eliminate legal protections. Some firms have updated client contracts to state that disclosing privileged material to an AI platform may waive attorney-client privilege.

Law firms begin updating contracts and internal guidance

In response to the uncertainty, some law firms have updated client agreements to address AI use directly. Sher Tremonte, for example, added a language warning that sharing legal communications with an AI platform could compromise attorney-client privilege.

Other firms are advising clients to use only controlled or “closed” AI systems designed for enterprise use, although even those protections remain legally untested. Some guidance also suggests that AI use in legal research may be safer when conducted under the direct supervision of an attorney.

Courts also face AI-related errors and misuse cases

Judges are increasingly encountering issues tied to AI-generated legal content, including filings that contain fabricated case citations created by chatbots. These incidents have added to concerns about how AI is being used without proper oversight in legal settings.

At the same time, courts are beginning to distinguish between misuse and legitimate assistance. Some rulings suggest that AI can be treated as a productivity tool, but its outputs may still be scrutinized if they become part of legal evidence or discovery requests.

AI companies acknowledge data and privacy limits

Companies behind leading AI tools, including OpenAI and Anthropic, have noted in their policies that user data may be shared under certain conditions. They also advise users not to rely on chatbots for legal advice without consulting qualified professionals.

Court records and judicial remarks also highlight that users may not have a reasonable expectation of privacy when interacting with AI platforms. These disclosures are now being cited by lawyers to reinforce warnings about potential exposure in litigation.

Little-known fact: Many AI platforms, including tools like Claude and ChatGPT, collect and store user inputs as part of their service design, and in some cases, those conversations may be reviewed, retained, or reused for model training.

Legal landscape remains unsettled as AI use grows

Experts say the legal system is still adapting to the rapid rise of generative AI in everyday decision-making and professional work. Future court rulings are expected to further clarify when AI chats can be considered protected or discoverable evidence.

Until then, attorneys are reinforcing a simple message that is increasingly echoed across firms. Do not assume AI conversations are private, especially when legal liability or criminal exposure is involved.

TL;DR

• US lawyers warn AI chatbot conversations may be used as evidence in court.
• A New York ruling allowed AI-generated documents to be accessed in a criminal case, saying no legal privilege applies to AI tools.
• Courts are split, with another ruling treating AI chats as personal work product in a civil case.
• Attorney-client privilege does not extend to AI platforms, increasing risk for users sharing sensitive legal info.
• Law firms are updating contracts and advising clients to avoid sharing confidential case details with AI.
• Some guidance suggests AI use may be safer only when done under a lawyer’s supervision and clearly labeled.
• Courts are also dealing with AI-generated errors in filings, raising concerns about reliability.
• AI companies record user data that may be shared under certain conditions and is not fully private.

This article was made with AI assistance and human editing.

Don’t forget to follow us for more exclusive content.

If you liked this, you might also like:

• Lawsuit over AI chatbot suicide settled by Google and Character.AI
• California governor signs law requiring AI companies to disclose safety plans
• Musk’s xAI challenges Colorado AI law that could reshape the industry