Research Bites AI retweeted
We looked at all arXiv machine learning papers last month. 74% are open source ๐
For my first post, Iโm sharing a letter @NVIDIA signed on why open models matter.
AI will transform every industry, power every company, and be built by every country.
Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty.
The world needs both frontier closed models and frontier open models.
images.nvidia.com/pdf/Open-Wโฆ
Research Bites AI retweeted
Does your paper contradict itself?
We just launched an internal consistency reviewer! It will catch when numbers, terminology, notation, text, or figures contradict each other across a manuscript.
For example, an equation that is inconsistent with its plot, a p-value rounded two different ways, or cell dimensions that are inconsistent with reported volume.
Try it out today!
Research Bites AI retweeted
Last month, arXiv announced a one year ban for authors who submit papers with unchecked AI use, including hallucinated references.
We decided to test how this new policy was working. Using Reviewer3โs reference checker followed by manual verification, we sampled 200 papers submitted to arXiv on June 10th, well after the ban announcement, and found some concerning results:
1 in 4 arXiv papers still contain at least one partially or fully hallucinated reference.
Results:
- 55/200 (27% of papers) still contained at least one partially or fully hallucinated reference.
-29/200 (14%) had more than one.
-117 total invalid references were identified.
-The most common hallucination was an author mismatch. 25/117 fabricated references contained titles of real manuscripts but fumbled the listed authors.
See more below.
Research Bites AI retweeted
One month after the arXiv ban and there are still plenty of hallucinated references.
Last month, arXiv announced they will begin banning authors for one year if they submit manuscripts with unchecked AI generation, particularly hallucinated references.
We decided to test how this new policy was working. Using Reviewer3โs reference checker, we sampled 200 papers submitted to arXiv on June 10th, well after the
ban announcement, and found some concerning results:
- 55/200 (27%) had at least 1 hallucinated reference.
- 29/200 (14%) had more than 1.
- In total, we identified 117 hallucinated references.
Research Bites AI retweeted
Last month, arXiv announced they will begin banning authors for one year if they submit manuscripts with unchecked AI generation, particularly hallucinated references.
We decided to test how this new policy was working. Using Reviewer3โs reference checker, we sampled 200 papers submitted to arXiv on June 10th, well after the
ban announcement, and found some concerning results:
- 55/200 (27%) had at least 1 hallucinated reference.
- 29/200 (14%) had more than 1.
- In total, we identified 117 hallucinated references.
Research Bites AI retweeted
arXiv just announced a one-year ban for authors who submit papers with hallucinated references.
The most severe issues in research papers need to be caught before they enter the scientific record.
So this week, we rolled out hallucinated reference checks and fatal flaw assessments to all authors on Reviewer3. These tools were previously only available in our journal workflows.
AI is making research faster and more accessible. Our goal is to scale verification with it.
If your lab, department, or journal wants access, reach out.
Research Bites AI retweeted
These findings argue against framing AI as a replacement to human reviewers.
Instead, they are complementary. AI reviewers can surface major, technical issues to support human judgment.
RT @reviewer3com: It's finally out! The first benchmark of a dedicated AI manuscript reviewer.
We compared publicly available human revieโฆ
Research Bites AI retweeted
No, AI should not be used to make decisions about manuscripts or "score" novelty or significance.
That's not what it's good at, anyways.
Data from our benchmark of over 145,000+ comments shows AI focuses primarily on technical verification - like the validity, sufficiency, and transparency of the work.
Evaluating contribution is uniquely human.
Research Bites AI retweeted
Peer review should not be about filtering through slop.
We started reviewer3.com to address biases and delays in peer review. But since then, AI has flooded the scientific literature. So, we've expanded into integrity checks.
We now catch AI text, hallucinated references, and fatal design flaws. New look below!
Research Bites AI retweeted
More than half of researchers now use AI in peer review, according to Nature.
The question is whether AI can actually doanything useful in that process, or whether it just suggests adding a few emdashesโฆ
So I tested it. Hereโs what I found ๐ #ad
Research Bites AI retweeted
Had a paper rejected and am feeling salty, so sure, this is sour grapes. But it is *insane* that reviewers treat "I have a substantive philosophical disagreement with the argument of this paper" as a reason to reject.
Research Bites AI retweeted
Hallucinated references are showing up everywhere!
So we brought our reference checker back. It will:
- search each reference
- link the ones it can find
- flag invalid ones
Here's a real example on a paper that was recently retracted by Springer Nature for 24 hallucinated references.
Research Bites AI retweeted
What the reviewers will say about your research paper?
Find it out before you submit your paper to a journal.
Here is how it works.
1. Go to reviewer3.com
2. Click on โTry it for Freeโ
3. Select โauthorโ and submit your paper
4. The following 3 reviewers will review your paper.
๐) ๐๐ญ๐ฎ๐๐ฒ ๐๐๐ฌ๐ข๐ ๐ง ๐๐๐ฏ๐ข๐๐ฐ๐๐ซ
โ This reviewer answers the question.
โณ Are the conclusions logically supported by the data?
๐) ๐๐๐ฉ๐ซ๐จ๐๐ฎ๐๐ข๐๐ข๐ฅ๐ข๐ญ๐ฒ ๐๐๐ฏ๐ข๐๐ฐ๐๐ซ
โ This reviewer answers the question.
โณ Is the study statistically sound and reproducible?
๐) ๐๐ข๐ฆ๐ข๐ญ๐๐ญ๐ข๐จ๐ง๐ฌ ๐๐ง๐ ๐๐จ๐ง๐ญ๐๐ฑ๐ญ ๐๐๐ฏ๐ข๐๐ฐ๐๐ซ
โ This reviewer answers the question
โณ Are the limitations and context adequately discussed?
Try @reviewer3com ---> reviewer3.com
Research Bites AI retweeted
Happy Halloween! What's spookier than feedback from Reviewer number 2? ๐ป
Today only: Upload a paper, and hear from a special surprise visitor. ๐ง
Upload, if you dare...
Research Bites AI retweeted
We had a flood of uploads this weekend. Researchers everywhere are trying AI peer review for the first time.
We're closing in on 10,000 uploads and celebrating with a little surprise:
A free โAnti Journal Journal Clubโ T-shirt from @Research_Bites ๐
To enter:
- Retweet this post
- Upload your next paper
Weโll draw the winner when we hit #10,000! ๐
Research Bites AI retweeted
Replying to @drmichaellevin
Weโre not currently writing for readability by humans, weโre writing to pass peer review. Unless peer review changes, papers will likely stay the same























