The open-source platform for designing, running, and analyzing research with digital personas and people.

Cambridge, MA
Joined February 2024
Expected Parrot (YC F25) retweeted
A new very creative method for measuring ambiguity with LLMs co-led by @dsallydt and @LukeThorburn_ (EMNLP Oral 2026). When I worked with @mhtessler @summerfieldlab et al on the Habermas Machine (a project in which we use LLMs to find statements that maximize agreement within groups of people), we often found that LLMs maximize consensus by simply making statements more ambiguous. Eg a statement like "peaceful protests should be allowed" will likely lead to high agreement but that's because a lot of disagreement hides in different interpretations of what's "peaceful". One way to measure ambiguity is to ask lots of people how they would interpret a statement and see if those people disagree. But this is obviously impossible to do at scale. In this work we show that we can use LLMs to help approximate this. Read Sally's thread or the paper for more details. I think work like this is super important, for example to stress tests AI constitutions and model specs to see which rules are ambigious and in which situations. If you are in Budapest, please stop by Sally's presentation.
🚨 New paper: how to measure ambiguity in laws, public proposals, and even AI constitutions! with @LukeThorburn_ @Werdnamai @bakkermichiel 🧵
4
5
35
2,677
Expected Parrot (YC F25) retweeted
One of my favorite papers in the LLMs & Social Sciences literature is Ashokkumar et al. which shows that LLMs (GPT-4) asked to simulate how people respond in survey experiments is predictive of actual trt effects & the result is not likely to be memorization; their data & code are public and I used it to do a follow-up w/ more recent models 1/
10
39
1
231
16,098
Really excited to make some @ExpectedParrot Research Agent playbooks using our @firecrawl integration for this!
Introducing the free Government & Legal Index for ChatGPT & Claude. Give your AI agents 30M+ US government pages, from federal statutes and court opinions to city codes and permits. Research tax rules, permits & case law with the Firecrawl plugin. firecrawl.dev/alexandria/gov…
2
2
9
854
Expected Parrot (YC F25) retweeted
Aside from just the point estimates, I like seeing how it reasons through the evidence and makes adjustments. It's more interesting than just seeing a prediction market price. It you want to try it for some question, use this playbook URL: expectedparrot.com/research-… My report: expectedparrot.com/research-…. /end
3
1
3
635
Expected Parrot (YC F25) retweeted
One of my new favorite things is having AI agents assign probabilities to rumors. Our @ExpectedParrot agent put the Instacart acquisition at 35% before next November 1/
SCOOP: Instacart $CART is in talks to sell itself w/ @coryweinberg (his @theinformation swan song) + @Jessicalessin
1
1
7
1,503
Expected Parrot (YC F25) retweeted
John and @ExpectedParrot remain gems (P.S., Pointing my agent at the report in the repo w. "I asked John's academic website reviewer to run checks. Here's the report. Fix what you can. Please make sure to ask me and discuss before doing anything that might be controversial.")
I created a 'playbook' (link in next tweet) for doing a review of your academic website: it explores navigation, accessibility, organization, link freshness etc. it can also simulate potential visitors & whether they can accomplish their goals. Ran it on my own website & found several issues 1/
2
3
537
Expected Parrot (YC F25) retweeted
re-upping this - lots of people using the academic website reviewer; if you use it & share your results/thoughts, there might be some free credits in it for you :). Link to try in original thread
I created a 'playbook' (link in next tweet) for doing a review of your academic website: it explores navigation, accessibility, organization, link freshness etc. it can also simulate potential visitors & whether they can accomplish their goals. Ran it on my own website & found several issues 1/
3
19
1,652
Expected Parrot (YC F25) retweeted
shoutout to @ExpectedParrot for making it a breeze to run LLM surveys at scale. cc: @jamescham
1
2
5
382
Expected Parrot (YC F25) retweeted
They used to consistently return 37, and our Research Agent has interesting thoughts on why that has changed. I asked it to vary the instruction, add frontier models, and then field to mortals via @Prolific to compare. Models got all the versions independently; mortals are randomized (for real) to question versions. Web survey: expectedparrot.com/respond/h… Research Agent chat (can be cloned and extended): expectedparrot.com/research-…
1
5
184
Expected Parrot (YC F25) retweeted
PSA: If you're a student and need @ExpectedParrot credits for your research, don't create a bunch of fake accounts w/ emails and make self-referrals to get credits. Just email us! We run a very loose, highly progressive fiscal policy & can make credits by fiat. We also have a soft-spot for students!
4
12
79
5,919
Expected Parrot (YC F25) retweeted
you don't have to be up for tenure to give this a go! this is super useful. also, took me three clicks (ok, three taps, I am on my phone) to get to business. link: expectedparrot.com/login?red…
lol - a *lot* of you are apparently on P&T committees or going up for tenure. If you went through our AI interviewer to get some free credits, just hold tight - we'll get them to you ASAP (Ms @ExpectedParrot is making soup at the moment & she reviews all of these requests.) Thanks for signing up!
1
5
1,435
We designed our AI interviewer to make it easy to experiment transparently - with response modes, models, speeds, languages, progress indicators, piping, personalization, combining un/structured question types, and more. Our Research Agent can help you explore options, create and test interview guides, analyze responses, and, critically, document and reproduce your work.
I think AI Interviewers are very cool & will have a big impact in the social sciences. However, the best ones are very much aimed at enterprises, not academics. Expected Parrot recently created a self-serve, usage-based AI interviewer we hope you'll try out 1/
2
3
15
1,430
Expected Parrot (YC F25) retweeted
I think AI Interviewers are very cool & will have a big impact in the social sciences. However, the best ones are very much aimed at enterprises, not academics. Expected Parrot recently created a self-serve, usage-based AI interviewer we hope you'll try out 1/
In my own personal celebration of Bob Hall Nobel Prize, I'm asking AI agents pretending to be people how'd they react to sudden windfall cash using another AI playing the role of interviewer. One nerdy AI apparently went *deep* on estate planning...
2
14
3
65
8,054
Expected Parrot (YC F25) retweeted
💀
6
2
1
27
2,419
Expected Parrot (YC F25) retweeted
"Sure, sure, your survey software has question types like free text, multiple choice, matrix...but does it have 'Dino Runner' w/ extensive JSON-based performance logging? Hmm, well, @ExpectedParrot does. Frankly, it's table-stakes for today's enterprise-grade tooling."
1
11
908
Expected Parrot (YC F25) retweeted
Replying to @Ingar30 @CFCamerer
I've been working on this for my own purposes: github.com/expectedparrot/ka… it's an agent-facing tool for reviewing. It's oriented around improving a paper (substantive issues --> git issues) & uses @ExpectedParrot for inference but that's not actually necessary - can be run w/ local model or API keys
2
10
322
Expected Parrot (YC F25) retweeted
Hot take: From about GPT-5 onward, if humans and LLMs disagree in a data labeling task, when you look at the actual disagreeing cases, it's almost always that the LLM is right and the human(s) are wrong OR the humans also substantively disagree with each other
48
38
16
877
84,944
Expected Parrot (YC F25) retweeted
On the claim that AI is adversely affecting NYC entry level labor market; I had my research agent build on @alexolegimas & @FutureEconJacob 's work to assess the report - links below 1/
Early warning signs are mounting that AI is already impacting the job market in NYC. New data just out show entry-level job postings in the city are way down in the careers most exposed to AI. Below chart shows change in NYC since 2022: This is coming fast and we are doing almost nothing about it.
2
6
15
4,045
Expected Parrot (YC F25) retweeted
Should a swarm use majority-rule or unanimity to make decisions? I re-ran Guarnaschelli, McKelvey and Palfrey’s (2000) juror experiment. My jurors were GPT-5.6-terra. With majority rule, jurors essentially vote their signal. If the jury requires unanimity to convict, jurors vote to convict on an innocent signal 48 to 80% of the time. For Caltech students it was 36% and 48%. Under unanimity, a single vote for guilt only counts if all other jurors have voted guilty. And that’s exactly the situation where the innocent signal is likely incorrect, so a strategic juror disregards their own signal. So, unanimity made Caltech students less prone to convict the innocent. But it was worse than majority rule for both types of error for my model jurors: 48% of innocents convicted vs. 22%, when there were only three jurors. When asked what the probability that the defendant is guilty given their own private signal is, my AI jurors always answered exactly 70, or 30. This is Bayes’ rule. All of their strategic behavior was in the vote. You should collect what all the agents found out. And don’t count votes.
Follow-up to our @freesystemslab experiment this week: Rewarding AI agents for the swarm’s performance rather than their own makes the swarm less likely to converge on a false belief. But even with the right incentives, agents don’t always figure out how to avoid getting it wrong
1
1
8
906