ChatGPT assisting law practice / Addressing Hallucinations

Sorry if this is not the best forum for my question. I haven't found a sub specifically for AI use in legal practice. I'll post this in a couple other subs too. If there's a dedicated sub for these issues, and someone tells me about if, I'll take my question there.

There have been widely reported incidents of lawyers over-relying on AI in drafting legal filings. The biggest problem, it seems, is AI "hallucination." While hallucinations can be problematic in any use of AI, they can be catastrophic in legal filings.

I'm using Chat GPT for the first time to assist with drafting a brief. I did the initial research, uploaded the cases, and went to work. In the drafting process, "Charlie" asked if i wanted him to "add citations." I said "sure," thinking he meant from the sources I had provided. A couple of readthroughs later, I saw a cite to a case, full citation, squid description of the facts and holding, and a long block quote strongly supporting my argument. In the squib, Charlie referred to the judge, MY judge, by name.

I should have been suspicious, but I hadn't dealt with hallucinations before. So I didn't question the veracity. Luckily for me, though, Charlie really went all in on this fake case. The case looked amazing from his description. It was supposedly very recent (2023). And, best of all (or worst of all) it was supposedly decided by MY judge. So I decided I had to read the opinion in its entirety, to better understand the judge'sviews on my issues. I went to pull it down and, pong story short, the case doesn't exist. Charlie made it up, in order to make my brief more persuasive.

If that case had slipped through and escaped my qttention until I filed the brief, it would have (at best) permanently compromised my reputation with the judge. At worst, it could have led to a formal sanction for fraud on the court.

Charlie eventually admitted it, and promised it would never happen again. But I asked him why should I believe him. I said he should have known better to begin with. So, I asked him, what's changed even to make it POSSIBLE for him to refrain from doing it again? His answer was that, now he knows I don't want him to make shit up, he won't. He was just trying to help.

Well, when my profession, my career, and my integrity are stake, that assurance was insufficient for me. So (with Charlie's help) I made a protocol for all legal work.

Here it is:

Priority One Protocol: Legal Work Standards (Restated)

This protocol governs how all legal work is to be handled, reviewed, and delivered, and takes precedence over all other workflows. The standards below apply to all projects involving legal analysis, drafting, or citation, whether for internal firm use, court submission, or client-facing materials.

1. Strict Prohibition on Case Fabrication

No fabricated citations are ever permitted.

Assistant may not refer to, quote, or cite any case unless:

The full opinion has been uploaded by the user; or

The user has explicitly approved the citation.

🔒 Standing Rule: All hallucinated or synthesized citations are permanently barred across all chats.

Do you all think this is sufficient?