Source
r/Professors•
LWPops

Does AI Detection Software Work?

Do these things work? I am pretty sure a student ChatGPT'd his final exam. I asked students to write them using blue books, but a few students whom I trust asked to use their laptops. I said yes. A few others asked, so I said what the heck. Two of them had cheated before.

Anyway, one turned in an essay that clearly seems ChapGPT: The stupid fucking headings, the stilted diction, the impersonality. I'm gonna give the writer a zero, but it would be nice to be able to verify it, though I don't think I can?

Posted Dec 15, 2023, 12:58 AM UTC

124
professorfunkenpunk•

Not with enough precision that I'm comfortable pursuing academic dishonesty claims.

18
QuwinsoftSenior Lecturer, STEM, Public Liberal Arts (USA)•

Does AI Detection Software Work?

No. AI detectors will flag the Bible and the US Constitution as mostly written by AI.

16
LWPopsOPFormer Tenured, Returned to Adjunct•

I've heard that. I mean, I can tell right away, but it would be nice to have backup.

1
One-Armed-Krycek•

I’m not even seeing ChatGPT results in turn-it-in anymore. I’m wondering if our institution removed it from the default settings?

I mean, also, maybe don’t let known cheaters using tools of cheating again?

9
LWPopsOPFormer Tenured, Returned to Adjunct•

I’m not even seeing ChatGPT results in turn-it-in anymore.

I'm not, either.

3
oakhill10307•

Lots are using Quillbot, a word spinner that is a post-chatGPT tool to add synonyms and defeat AI detectors. Cheating knows no bounds…

2
One-Armed-Krycek•

All that time and effort and could have written a short dang paragraph, ffs.

2
oakhill10307•

Right????

1
Snoo16151Asst Prof, Math, R1 (USA)•

No

8
JADW27•

Lots of false positives, I fortunately. Better for older AI than newer. Also, a good prompt engineer can fool it pretty easily.

8
AsturiusMatamoros•

No. Too many false positives. Research is unanimous about this.

8
Cautious-Yellow•

be strong. Blue books for everyone.

6
LWPopsOPFormer Tenured, Returned to Adjunct•

It's looking that way. Fortunately, I think I am done adjuncting for the freshlings for now.

3
histprofdaveAdjunct, History, CC•

The detectors are one tool in the arsenal, but should never be considered conclusive evidence by themselves because of the potential for false positives. Were you to browse r/college, you'd get the impression that these detectors never work and that they (and by extension we professors) are just "making stuff up." The situation isn't quite that dire, but false positives do happen. So if you get a high rate of return, it's worth pausing to ask why.

  • Is there a bunch of overly vague, stilted language or "tortured phrases"?
  • Are synonyms used for words that make sense in basic text but are nonsense contextually?
  • Are references being used haphazardly or incorrectly?
  • Does the essay respond to the particulars of your prompt, not just the general topic?

Switching everything to in-class writing is one solution, but one that is not available to everyone. While generative AI presents a massive disruption of academic honesty, there are things you can do to make it more difficult on the cheaters. The more specific your essay rubrics, the harder time cheaters with poor skills will have just punching it into Chat GPT. Having restricted or curated resource lists is another. While smart cheaters can sometimes phrase prompts to get around this, smart cheaters have almost always had ways of evading our notice, and our job is not to be the academic integrity police.

At the end of the day, students who refuse to learn how to write are cheating themselves. And if they think the process of writing is useless and that Chat GPT will solve all their problems, one can only imagine the shocked Pikachu face they will make when they can't find jobs for lack of any marketable skills.

3
LWPopsOPFormer Tenured, Returned to Adjunct•

Excellent reply. Saved.

1
WJM_3•

you hit it right - ask very specific questions and grade strictly; answer the question as it was asked, not a 500 word soup

1
WJM_3•

nope - stick with no dig. and no variance

students cannot be trusted - pains me to say, but experience dictates as such

2
Hannibal_Lecturer_pt•

I’ve found that the Turnitin AI detection is actually pretty good, and any paper that’s gets over 30% has led to the student admitting to using AI.

1†
LWPopsOPFormer Tenured, Returned to Adjunct•

Ours must not have the AI detection. It never shows up. Plagiarist copy-pasta does, though.

1
Icy-Chair-9390•

So many people say they don’t, but every student I’ve ever shown AI detection flags to admit using it. Every. Single. One. I know some people say that all of them (literally almost 100) are confessing to crimes they didn’t commit, but c’mon.

Also, my take: I didn’t invent AI, I didn’t invent AI detection software. It’s not my job to fix this huge problem. I use the tools recommended by the college. If students have a problem, they can appeal. I’m tired of it being my job to address every societal failing that affects students.

-1†
OsakaWilson•

What exactly is society's failing that you are choosing not to address?

1
Icy-Chair-9390•

Instructors are expected to compensate for and combat all of the following when they teach, no matter the subject: differing learning styles, diagnosed learning difficulties, undiagnosed learning difficulties, socioeconomic inequalities, socioeconomic equity, homelessness, domestic abuse, food insecurity, mental health issues, potential for school shootings, unconscious bias, institutional racism, cultural differences, varying levels of English language proficiency, the failings of primary schooling in the US, increasing distrust of experts, decreasing digital literacy and information literacy, overall distrust of academics in the US, overall distrust of higher education and those who work in higher education. Do I need to go on? As my grandpa used to say, I can pop the popcorn and tear of the tickets, too, but at some point you’re going to have to hire more people to help run your circus. I’m not adding to this list trying to figure out whether AI detectors work or don’t work. If the college makes them part of the learning platforms, I’m going to use them until they tell me not to and at some point someone else is going to need to look into it to see whether the reports are accurate or inaccurate. As educators, we literally have hundreds of things working against us doing a good job and at some point it’s not our responsibility to try to fix all of those things.

7
OsakaWilson•

I'm just going to take your grammars advice and slowly back away.

-4
Icy-Chair-9390•

“I'm just going to take your grammars advice and slowly back away.”

When you make a snarky comment about someone else’s grammar, especially on a social media post, don’t include a typo in your snarky comment.

1†