Alec Radford (@AlecRad) header
Alec Radford (@AlecRad) profile picture
San Francisco, CA
Joined October 2012
Photo by Alec Radford (@AlecRad)Photo by Alec Radford (@AlecRad)Photo by Alec Radford (@AlecRad)Photo by Alec Radford (@AlecRad)Photo by Alec Radford (@AlecRad)Photo by Alec Radford (@AlecRad)Photo by Alec Radford (@AlecRad)Photo by Alec Radford (@AlecRad)Photo by Alec Radford (@AlecRad)Photo by Alec Radford (@AlecRad)
We are starting a new, nonprofit alignment organization, ⊢ Sequent Research, bringing together researchers previously on UK AISI’s Alignment Team, Timaeus, and elsewhere to research how to align superintelligence. We are hiring! 🧵
28
145
48
1,034
296,302
Alec Radford retweeted
New work with @AlecRad and @DavidDuvenaud: Have you ever dreamed of talking to someone from the past? Introducing talkie, a 13B model trained only on pre-1931 text. Vintage models should help us to understand how LMs generalize (e.g., can we teach talkie to code?). Thread:
178
395
173
3,195
1,243,284
Alec Radford retweeted
Announcing Talkie: a new, open-weight historical LLM! We trained and finetuned a 13B model on a newly-curated dataset of only pre-1930 data. Try it below! with @AlecRad and @status_effects 🧵
195
451
300
3,613
1,469,525
Alec Radford retweeted
We trained diffusion models on a billion LLM activations, and we want you to use them! New preprint: Learning a Generative Meta-Model of LLM Activations Joint work with @feng_jiahai, @trevordarrell, @AlecRad, @JacobSteinhardt. More in thread 🧵
32
191
22
1,442
227,048
Alec Radford retweeted
New paper, w/@AlecRad Models acquire a lot of capabilities during pretraining. We show that we can precisely shape what they learn simply by filtering their training data at the token level.
25
98
12
1,114
114,194
Alec Radford retweeted
Extremely excited to share work I've been doing at OpenAI the past few months: MuseNet, a neural net music generator. It's been a huge team effort pulling this all together!
Introducing MuseNet, a neural network which discovered how to generate music using many different instruments and styles. Listen & interact: openai.com/blog/musenet/ MuseNet will play an experimental concert today from 12–3pmPT on livestream: twitch.tv/openai
36
197
10
1,005
Alec Radford retweeted
Releasing some work today with @scottgray76 @AlecRad and @ilyasut. Contains some simple adaptations for Transformers that extend them to long sequences.
Releasing the Sparse Transformer, a network which sets records at predicting what comes next in a sequence — whether text, images, or sound. Improvements to neural 'attention' let it extract patterns from sequences 30x longer than possible previously: openai.com/blog/sparse-trans…
1
59
8
213
Alec Radford retweeted
One commonly cited argument about the difficulty of learning common-sense reasoning is that "no-one writes down common sense". A counter-argument is "well, the web is big": instructables.com/id/How-To-…
6
22
4
146
First, reproducibility is not about rerunning code to get the same results. Science must be more robust, as naive copying has many flaws. Second, reproducibility should never be above public safety. We must publish responsibility, with hope and kindness in our minds.
Don't the benefits of increased reproducibility and rigor on the part of the authors greatly outweigh any potential misuses of their work, at least for the vast majority of ICML/ICLR papers? I think the current shift towards empirical work puts a greater need on releasing code.
4
28
2
124
Alec Radford retweeted
I'd like to weigh in on the #GPT2 discussion. The decision not to release the trained model was carefully considered and important for norm-forming. Serving the public good requires us to draw lines on release somewhere: better long before catastrophe than after.
9
91
34
367
Alec Radford retweeted
Replying to @zeynep
It's interesting we're having this discussion upon releasing text models that _might_ have potential for misuse yet we never engaged as fully as a community when many of the technologies powering visual Deep Fakes were being released, including hard to make pretrained models.
2
5
1
38
Alec Radford retweeted
Shoutout to @katyanna_q who fed the system a curveball, which I always like to see. As you might expect by now after seeing AlphaStar, OpenAI 5 etc. etc., if you drag the system away from its training data and into weirder territory, it begins to wobble. theregister.co.uk/2019/02/14…
1
9
2
21
Announcing DepthFirstLearning.com! We are building a repository of study guides targeting consequential papers. Check it out, learn something in-depth, and help us build the next one. @avitaloliver @suryabhupa @kumarkagrawal @cinjoncin
6
152
16
388
Replying to @dennybritz
I’d like to move to a model where we talk about what we do as we do it, e.g. on Twitter. – at Malmo, Sweden
1
1
15
Today's paper shows that it is possible to implement John Von Neumann's claim: "With 4 parameters I can fit an elephant, and with 5 I can make him wiggle his trunk" Paper here: fermatslibrary.com/s/drawing…
12
542
74
1,458
Alec Radford retweeted
I've spent a year or so having the pleasure of thinking intently about all of the worst ways in which people may abuse or misuse AI, a research project that fits with my generally cheerful English personality. tl;dr: Dual-Use is gonna matter a lot. teapawt.net/OpenAI/status/96610092…
Preparing for Malicious Uses of AI: blog.openai.com/preparing-fo… Policy research on forecasting, prevention, and mitigation of AI-borne threats. With authors at @FHIOxford, @CSERCambridge, @EFF, @CNASdc, and many others.
8
53
2
197
Alec Radford retweeted
73
468
49
3,013
Alec Radford retweeted
The ML wish of 2018: may all your local minima be global, your variance bounded, your labelled data plentiful, and your compute massive!
24
652
45
1,897
Alec Radford retweeted
NIPS2017 Workshop on Machine Learning for Creativity & Design. Call for Papers & Artwork! Visit our site for details nips2017creativity.github.io…
5
117
11
256
For those of us who can't make it to @ArsElectronica, i made an online, ever-morphing hypnotic GAN portraits piece miketyka.com/projects/dreams…
7
33
6
96