model measurement @OpenAI. Formerly @MotifAnalytics @Lyft and @Facebook. Keywords: Experiments, Causal Inference, Statistics, Machine Learning, Economics.
Oakland, CA
Joined February 2009
- Tweets18.8K
- Following4.1K
- Followers44.9K
- Likes34.9K
Pinned Tweet
Where do folks who work in data roles stick around the longest? You can use @MotifAnalytics to find out quickly! There is an interesting company with the longest average tenure.
We've ungated access to Motif so try it out and let us know what you learn🚀
loom.com/share/39e6efdf42e94…
Sean J. Taylor retweeted
following up after astra launch week and the navier-stokes announcement i wanted to share a little bit about how we think about and build for the over 1 billion people who use chatgpt every week, most of whom use it for free. we’re on a mission to distribute the benefits of agi, and i think this is the most impactful way to do that: giving people as much access as we can safely and feasibly distribute. we want to scale the utility that everyone gets from ai.
over the past 6 months, we've been working to bring the greatest possible capabilities to the broadest possible set of users, including both free and paying users. of course there are some practical limitations and we can’t ship the largest models to everyone, but since march (gpt-5.3 instant), we’ve improved the default chatgpt experience meaningfully:
- responses with a major factual error are down 65%, and they are down 72% in high stakes categories like finance
- gpt-5.6 sol at instant and gpt-5.6 luna at medium are smarter than o3 at high reasoning effort, our frontier reasoning model from 17 months ago, while being considerably faster, 30%+ faster time to last token (evaluated via GPQA diamond)
- chatgpt is far more usable and enjoyable while bringing extreme sycophancy down 80% and overall sycophancy down 83%
- on our high-stakes medical eval, chatgpt produced 83% fewer answers flagged for hallucinations, resulting in hundreds of millions of better health conversations per month
- we're seeing people find more ways to use chatgpt. for a recently joined cohort, we saw the number of use cases pursued at least three times weekly go up 20-30%
not only have we improved the default chatgpt experience dramatically, we’ve also expanded access significantly for all. free users now:
- have access to unlimited text chats
- can use higher reasoning effort
- now have access to automations, letting them schedule tasks to happen later
- have access to better memory through dreaming. this allows chat to personalize much more effectively
we’re always looking for ways to expand access to the billion+ people who use our product weekly. this is the master plan: bring the default experience as close to the capability frontier as possible, expand access to everyone, making intelligence maximally abundant, and make our product much more usable.
this last point, usability, is something we’ve been focusing on as we bring personal agi to everyone. the current top of the line capabilities are staggering, but only the savviest users and insiders know how to use them. our vision is of a progressive, model assisted, onboarding to agi. chatgpt will guide people from all walks of life to the highest utility applications of ai for them. this really is the only way – we think technology that is hard to learn ultimately fails to reach a broad audience. your personal agi will just understand your goals, and automatically use the right set of features and capabilities to get things done for you
lots more to come soon. onwards to scaling utility for all
We are cooking for our users:
- hallucinations 📉
- sycophancy 📉
- speed 📈
And the impact shows up in usage: people are increasingly trusting ChatGPT with more varied and important use cases.
following up after astra launch week and the navier-stokes announcement i wanted to share a little bit about how we think about and build for the over 1 billion people who use chatgpt every week, most of whom use it for free. we’re on a mission to distribute the benefits of agi, and i think this is the most impactful way to do that: giving people as much access as we can safely and feasibly distribute. we want to scale the utility that everyone gets from ai.
over the past 6 months, we've been working to bring the greatest possible capabilities to the broadest possible set of users, including both free and paying users. of course there are some practical limitations and we can’t ship the largest models to everyone, but since march (gpt-5.3 instant), we’ve improved the default chatgpt experience meaningfully:
- responses with a major factual error are down 65%, and they are down 72% in high stakes categories like finance
- gpt-5.6 sol at instant and gpt-5.6 luna at medium are smarter than o3 at high reasoning effort, our frontier reasoning model from 17 months ago, while being considerably faster, 30%+ faster time to last token (evaluated via GPQA diamond)
- chatgpt is far more usable and enjoyable while bringing extreme sycophancy down 80% and overall sycophancy down 83%
- on our high-stakes medical eval, chatgpt produced 83% fewer answers flagged for hallucinations, resulting in hundreds of millions of better health conversations per month
- we're seeing people find more ways to use chatgpt. for a recently joined cohort, we saw the number of use cases pursued at least three times weekly go up 20-30%
not only have we improved the default chatgpt experience dramatically, we’ve also expanded access significantly for all. free users now:
- have access to unlimited text chats
- can use higher reasoning effort
- now have access to automations, letting them schedule tasks to happen later
- have access to better memory through dreaming. this allows chat to personalize much more effectively
we’re always looking for ways to expand access to the billion+ people who use our product weekly. this is the master plan: bring the default experience as close to the capability frontier as possible, expand access to everyone, making intelligence maximally abundant, and make our product much more usable.
this last point, usability, is something we’ve been focusing on as we bring personal agi to everyone. the current top of the line capabilities are staggering, but only the savviest users and insiders know how to use them. our vision is of a progressive, model assisted, onboarding to agi. chatgpt will guide people from all walks of life to the highest utility applications of ai for them. this really is the only way – we think technology that is hard to learn ultimately fails to reach a broad audience. your personal agi will just understand your goals, and automatically use the right set of features and capabilities to get things done for you
lots more to come soon. onwards to scaling utility for all
Sean J. Taylor retweeted
another crazy day working at the crazy day factory
We’re sharing a solution to the Navier-Stokes Millennium Prize Problem, one of the deepest problems at the frontier of mathematics.
The proof was produced by a group of agents, using an OpenAI next-generation model significantly more capable than GPT-6 Astra.
The problem concerns whether the description of smooth three-dimensional fluid motion modeled by the Navier-Stokes equations can break down. It has remained unresolved for roughly 90 years.
Everyone's gotta have an Astra take, so I'll give you mine: we are making fast progress on eradicating hallucinations. This eval is not some academic benchmark; it very realistically captures user experience. deploymentsafety.openai.com/…
Sean J. Taylor retweeted
💫 Astra is out
Highlights:
- Much smarter (evals are really saturating...)
- Much more aligned & trustworthy
- Better CUA (see this house Astra made in Blender)
So proud of the team for training it!
Still some issues:
- Too much code slop
- Astra asks for confirmation too often, which can feel lazy. Given how smart it is, we wanted it to be cautious, but likely overcorrected.
We’ll fix those next! What else should we fix?
Note: Astra follows instructions even more carefully than 5.6, including unwanted ones buried in old skills. I’d clean those up before using it.
openai.com/index/gpt-6-astra…
Sean J. Taylor retweeted
free users can finally use scheduled tasks.
paid users now have access to event based triggers! it’s super powerful - try it out!
A few updates to scheduled tasks in ChatGPT Work on web and mobile: Plus and Pro users can now set tasks to respond when something changes in Slack, Gmail, and Github, rather than just run on a set schedule.
We’re also rolling out scheduled tasks in ChatGPT to Free users, who can create up to three tasks.
You can also now share your favorite tasks for others to customize and use.
Sean J. Taylor retweeted
today's ship is a big one! we're shipping updated versions of 5.6 sol and 5.6 luna to plus/pro and free/go in chatgpt.
what i'm most excited about in these models is that we've finally unified the instant experience and maximum intelligence in one model. just move the slider based on your speed vs comprehensiveness tradeoff.
this has been a dream since our reasoning models first shipped 2 years ago. pumped that we have finally merged these paradigms.
Sean J. Taylor retweeted
On ARC-AGI-3, GPT-5.6 is dumb as dirt.
But it turns out if you turn on two API settings that we use in ChatGPT and Codex, its score on the public set rises ~3x and its token efficiency rises ~6x.
Performance is a function of model + product harness, never the model alone.
Sean J. Taylor retweeted
Hiring for my team @OpenAI ✨
The Model Experience team shapes every interaction with ChatGPT to be thoughtful, joyful, helpful across contexts and languages.
Looking for:
- Strong technical research experience
- Turns qualitative taste into concrete model interventions
- Good judgment about model behavior, and can communicate it
Sean J. Taylor retweeted
🎉🥳 After another 20 hours with GPT-5.6 Sol, we have substantially improved the lower bound on worst-case FDR(BHq) for correlated Gaussian two-sided tests:
lihualei71.github.io/FDR_BH_…
GPT’s amazing capability to construct examples with desired properties is instrumental!
🧵(1/5)
New paper🚨After 10h w/ GPT-5.6 Sol, I made exciting progress on worst-case FDR of BH for correlated two-sided Gaussian tests
@edgardobriban showed FDR(BH_q)/q can exceed 1
Q (a folklore conjecture): Is the ratio universally bounded?
A: No👇
lihualei71.github.io/FDR_BH_…
🧵(1/8)
Sean J. Taylor retweeted
Very happy to share the first paper from @ElasticityInst: The Economics of Recursive Self-Improvement. Two parts:
(1) a graphical representation of feedback loops, to formalize a variety of RSI-related arguments, where each arrow represents responsiveness (elasticity);
(2) a survey of existing evidence with a loose calibration & a “wish list” of evidence that would help us calibrate better.
Sean J. Taylor retweeted
♥️ GPT-5.6 is a major step forward for health, both at the frontier and at cost.
These models push the frontier of performance per dollar, bringing the best health intelligence to all. The smallest variant, GPT-5.6 Luna, evaluated at the lowest reasoning effort, outperforms GPT-5.5 at the highest reasoning effort–despite costing 25x less. The largest variant, GPT-5.6 Sol, sets a new high bar at cost.
Another especially cool result: physicians found fewer flaws in GPT-5.6 responses than physician-written responses.
We collected diverse tasks that remain difficult for recent OpenAI models, across patient-facing and clinician-facing use cases. We asked speciality-matched physicians to write responses to these tasks with unlimited time and web access. We then asked other physicians to compare responses side-by-side, blinded to their source. Physicians were asked to comment on areas of improvement across five axes: accuracy, communication, completeness, instruction following, and health decision helpfulness. We then reported the fraction of responses across sources rated perfectly across all axes, across 20,000 total axis ratings. GPT-5.6 Sol appeared strongest, although all GPT-5.6 models performed significantly better than physicians.
GPT-5.6 is a major step forward for health intelligence.
Across the lineup, we’re delivering stronger performance at lower cost: GPT-5.6 Luna outperforms GPT-5.5 at its highest reasoning setting while costing 25x less.
Together, these advances raise quality while making advanced models accessible to more people globally.
One of the highlights of my career at OpenAI has been getting to use GPT-5.6 as my primary model ahead of launch. The thing about exponential improvement is that time compresses: a month ahead today feels like a year ahead used to.
Sean J. Taylor retweeted
Today we’re introducing the brand-new ChatGPT desktop app. If it looks familiar, that’s because it grew out of the Codex app.
We were deliberate about how we brought the pieces of this app together. Rather than splitting it into separate, disconnected worlds, we designed Work and Codex to share the same powerful capabilities and task history. The difference is in the interface: Codex is built to show you code, diffs, and PRs, while Work keeps those details abstracted away. Pick one, or move freely between them.
When you want to quickly explore an idea or get an instant search result, Chat is now in the app too—with instant and pro models, search widgets, and a seamless handoff into Work or Codex tasks.
Sean J. Taylor retweeted
🌞Excited to share the preview of 5.6, it's a good model!
Here you see a comparison to 5.5 xhigh, two things to note:
- we'll have a `max` mode that works for much longer
- sol is a large improvement at the same cost & latency
This is a cyber eval but we see significant gains across the board.
Introducing a limited preview of GPT-5.6 Sol, our next generation frontier model, as well as GPT-5.6 Terra, a balanced model for efficient, everyday work, and GPT-5.6 Luna, a fast and affordable model for high-volume work.
openai.com/index/previewing-…
Buried the lede. openai.com/index/gpt-5-5-ins…
we shipped gpt-5.5 instant today to chat; it's rolling out over the next couple days to everyone. for this model, we focused on factuality, crushing hacks, and improving the baseline intelligence. 5.5 is a pretty big step forward on all three. it is much smarter, significantly less likely to hallucinate, and just more delightful to talk to. there is always more to do but i'm pretty excited about this one!
Sean J. Taylor retweeted
we shipped gpt-5.5 instant today to chat; it's rolling out over the next couple days to everyone. for this model, we focused on factuality, crushing hacks, and improving the baseline intelligence. 5.5 is a pretty big step forward on all three. it is much smarter, significantly less likely to hallucinate, and just more delightful to talk to. there is always more to do but i'm pretty excited about this one!
