Recent Posts

Recent Comments

Archives

Categories

Subscribe to this blog's feed
September 2024
Sun Mon Tue Wed Thu Fri Sat
1 2 3 4 5 6 7
8 9 10 11 12 13 14
15 16 17 18 19 20 21
22 23 24 25 26 27 28
29 30          

Search Site


Add me to your TypePad People list

« Labor Day Music Post | Main | Justice Thomas's Disclosure Form Conceals the Value of His Gift from Harlan Crow »

Tuesday, September 05, 2023

Does ChatGPT Read Prawfs?

The following comment was posted to last week's The Peer Review Follies:

Sometime Peer said:
 
Fascinating story. Thank you for relaying it. Wow, that is an ordeal for the author. Two years to publish!! We need to reform how we do things... Replying to Anon Sept 1, 11:31 - Wow (again), that is worse than I thought the state of things. I heard there is lots of fraud in scientific publishing, but did not know the same was going on in empirical legal studies. And as you say, the problem is compounded when courts/policymakers rely on bad empirical scholarship. Some say that "evidence-based policy-making" has become "policy-based evidence-making."
 
Although it was not captured by the spam filter, it struck me as having been written by AI, so I unpublished it. Maybe I was wrong. Or maybe it was posted as an experiment. In either case, perhaps the person behind it will come forward.
 
Comments are open and will be monitored for relevance and civility.

Posted by Steve Lubet on September 5, 2023 at 04:47 AM | Permalink

Comments

What I've found most often about ChatGPT responses is that they offer plain vanilla generalities, never really answer the question (BingChat is somewhat better), and end by advising you to consult a real lawyer.

Which, of course, is the same problem one often finds when grading law school exams (well, not the last but never say never).

Hence, in response to the panic over ChatGPT sounding like a student, I've responded that I'm more concerned about students who sound like ChatGPT.

Posted by: Jeff Lipshaw | Sep 6, 2023 12:05:26 PM

That's exactly what ChapGPT would say.

Posted by: Howard Wasserman | Sep 5, 2023 4:05:54 PM

Hi, I wrote this comment. And I am human. I was genuinely surprised at the story and the anonymous commentor, which is why I used the words "wow" and exclamation points. And I have done peer reviews before.

Posted by: SometimePeer | Sep 5, 2023 12:47:28 PM

Good question, Marty. At first, I thought it was spam. When I realized that it was probably AI, I decided to make it a separate post, inviting the human originator to come forward.

Posted by: Steve L. | Sep 5, 2023 8:35:57 AM

Just curious: Let's say it was generated, in whole or in part (e.g., a "first draft"), by ChatGPT. Nevertheless, "Sometime Peer" thought it was worth posting. To be sure, it doesn't say much, but nor do many similar "me too" comments here and at other blogs (including by anonymous commenters). So why delete it? Would you delete a ChatGPT-generated post that offered a really useful substantive response?

Posted by: Marty Lederman | Sep 5, 2023 8:26:39 AM

The comments to this entry are closed.