Phelim Bradley (@Phelimb) header
Phelim Bradley (@Phelimb) profile picture

CEO @Prolific https://teapawt.net/t.co/hmoUnNjHDh

London, England
Joined February 2011
Choice of sample matters!
I'm increasingly nervous about how selection bias in domain expert data influences model behavior. Domain experts lean junior in experience because they're looking for side income. This doesn't really make sense as we design longer horizon tasks, since we're missing the judgment and intuition senior people have.
1
131
Phelim Bradley retweeted
Our midterm poll has Democrats ahead by about 11 points among likely voters. @orioljbosch, who led the poll, explains how we got there and why the same data could have given us 5 or 15. If you want to run your own analysis, you can get the data tables from our article.
1
3
5
734
Phelim Bradley retweeted
Replying to @Prolific
@Prolific's midterm projections are now out: Democrats lead by about 11 points among likely voters (D+10.7) And we are telling you exactly how we got there and what those decisions mean: prolific.com/resources/poll-… Greater transparency in polling is essential #polling #midterms
1
2
3
78
Phelim Bradley retweeted
Our midterm poll has Democrats ahead by about 11 points among likely voters (D+10.7), 53% to 42%. A few defensible choices move that from about 5 to about 15, so we show what each one is worth. ↓
2
7
5
16
6,499
Phelim Bradley retweeted
Authenticity checks are now available to use with LimeSurvey and Pavlovia in addition to @Qualtrics. This includes both LLM and bot detection. Check our guide to see how you can get trusted data from the next study you launch.
2
1
12
1,171
Some heavy hitters on this list. Proud to see @Prolific right in the mix! If you want to build with one of the fastest-growing teams in tech, we're hiring: prolific.com/careers
Spring was an abstraction. Summer was the stack. Fall? The “chips” fall where they may. The 25 fastest-growing software vendors on Brex this fall👇
1
2
11
475
Phelim Bradley retweeted
Changing minds and driving action are different jobs. Kobi and the team show it with real petitions and real donations from almost 15K participants (via Prolific), the latest in a series of AI persuasion studies. Insights from the study below ↓
New in @PNASNews: In two large experiments (~18k obs from ~14.8k people), we find that brief conversations with AI can get people to take real political action - incl. signing real petitions and giving money to charity. Surprisingly, the persuasion strategy that worked best for changing people's attitudes - packing convos with info - was the least effective at changing behavior! 🧵
3
4
914
Phelim Bradley retweeted
Congratulations to our AI research team on 2 papers accepted at @NeurIPSConf 2026 🎉 2x accepted papers, MetaLoop and Theory of Mind and Persuasion Beyond Conversation, plus a workshop paper being presented at BiAlign Workshop.
1
4
16
1,911
Interesting new UPenn paper below. Only 1% of 712,930 Prolific submissions (across 127k+ studies) involved AI assistance. Rare, but there are important signals around safeguards, something we’re always treating seriously.
🚨 New paper 🚨 Are your participants using AI to take your study? We conducted the first study directly observing AI assistance in online research, spanning 700K+ Prolific submissions. We found very rare, concentrated AI use, but gaps in safeguards. arxiv.org/abs/2610.09279
2
8
1,008
Phelim Bradley retweeted
Digital twins are being sold as a faster, cheaper alternative to live participants. On Nov 4, @andrew_j_gordon presents two studies on how accurate twins are and how the people behind them feel about it. Sign up below. riverside.com/webinar/regist…
2
6
882
Phelim Bradley retweeted
Today we're launching Odyssey-3, the most powerful foundation world model yet. It sets a new state of the art on Physics-IQ, and powers robots, trains AIs, and generates interactive experiences. It's really cool. Experience the model today, all for free!
209
388
308
3,008
1,383,983
Phelim Bradley retweeted
Excited to share ResearchTrails, a dataset of human research decision trajectories extracted from Git commit histories. Final papers show the outcome, but they often do not capture the full path: abandoned methods, omitted experiments, and ablations. ResearchTrails recovers these intermediate decisions, and we show that learning from them improves LLMs’ ability to assist with research exploration! Blog, code, data, and the trajectory visualizer are at researchtrails.org 🧵
19
44
8
387
24,176
Phelim Bradley retweeted
We're excited to be at @COLM_conf 2026 from Oct 6-9, the conference spotlighting cutting-edge LM research. Find us at Booth #200, plus explore COLM papers that cite Prolific in their data collection or human evaluation process. ↓
2
4
23
1,947
Phelim Bradley retweeted
MSR & AAPOR’s @DavMicRot doesn't believe synthetic data will ever replace human respondents. He expects it to reshape the survey workflow: run synthetics first, validate with real humans, then iterate, using as many or more humans than before. Watch on The Signal.
3
4
960
Today we launched The Signal, @Prolific's new podcast on research evidence, not research tools or trends. David and Andrew discussed what research quality looks like once you remove the human from the process. Give it a watch!
Has AI been a net positive or negative for survey research? In episode 1 of The Signal, Dr. David Rothschild (@MSFTResearch, @AAPOR AI task force Co-chair) joins @andrew_j_gordon on where AI helps, where synthetic data breaks down, and why transparency beats standards.
2
7
499
Our team recently tested whether AI can replace human survey respondents, using simulated LLM personas against a real US survey (N = 996). Turns out demographic personas make it worse, idiographic information helps a little, and output format is the biggest lever of all. Synthetic data predicts human opinion; it's not a substitute for measuring it. Pre-print: papers.ssrn.com/sol3/papers.…
2
8
346
Can an LLM fake being a human survey respondent? We tested it on 996 real people using a poll plus a 10-minute qualitative interview (via Outset). New paper: "Do Richer Personas Improve LLM Survey Simulation? A Fidelity Paradox"
3
5
1
33
4,184
Takeaway: synthetic respondents are fine for cheap, directional signal. They're not a replacement for real human data.
2
1
297
Authors: Nora Petrova, John Burden, Oriol Jover, Ning Ding. Paper: papers.ssrn.com/sol3/papers.…
300