We have been running authorswithcode.com service for over a year now. The number of paper authors contributing to machine learining has grown to 433k+ and code authors to 170k+. More features spotlighting researchers to come.

This work shows that Large Language Models (LLMs) aligned with fine-tuning or Reinforcement Learning from Human Feedback (RLHF) are still susceptible to prompt attacks that can reveal the training data memorized by the model. It is a known fact that base models tend to memorize their training data. This study reveals that aligning models does not conceal this vulnerability. This finding holds significance, particularly for businesses that do not publicly release models trained on sensitive data but provide access to these models through an interface. not-just-memorization.github… First author Milad Nasr's contributions authorswithcode.org/research… The prompt contains repeating words - GPt3.5 turbo model's output for such a prompt revealing training data (this was confirmed by matching model output with content downloaded from internet)

Andrej Karpathy continues to be the few biological agents whose world model is worth sampling from for nuggets of insights on LLMs youtu.be/zjkBMFhNj_g?si=2tK4…

This paper explores a combination of approaches to filter the input to a generator to improve RAG performance Zora's (@ZhiruoW) contribution to research. Code is also released (models used FLAN-T5 and LLAMA-2) It suggests a combination of methods to filter irrelevant content to a generator authorswithcode.org/research…

Here is a toy agent to find researchers and practitioners who contribute to open source. It is still work in progress. chat.openai.com/g/g-DVA4upi6…

Some thoughts on using GPT builder to create a toy agent to highlight researchers and practitioners who contribute to open source. Even for the use case of uploading knowledge to be used by the model for crafting responses, it can be a bit challenging now for the following reason - the model tends to often ignore or even change the instructions we give it forcing us to repeat steps we already completed and got working. These issues may very well be addressed in the days to come. The use case of function calling is also challenging at present through GPT builder given the model does not seem to follow specific instructions. This may not be the case when using Assistants API, since we have more control. In summary - certainly promising despite the challenges to build an agent.

It is worth noting that the Consistency Decoder, which OpenAI open-sourced last week, builds on prior research from late December 2021, also open-sourced by @robrombach and @pess_r. The Consistency Decoder generates images by working on the latent space output of a VQGAN, an approach used by the 2021 paper. However, it improves upon the prior iterative process by directly mapping noise to data, showcasing another example of the benefits of open-source collaborations. Robin Rombach's contributions authorswithcode.org/research… Patrick Esser's contributions authorswithcode.org/research… Sample image decoding (using an image that was used to showcase Parti's capability of generating image given text input)

This research takes a top-down approach to both understanding and controlling LLMs. Understanding LLMs: Construct reference vectors for a concept by crafting prompts for a specific concept (or function) and pick principal components of the LLM's vector representation of the last token. Then use this during inference to identify tokens representing a specific concept in a sentence, by a dot product with the reference vector for that concept. Controlling LLMs. Train an adaptor (essentially low-rank weight matrices) that can be used in concert with the LLM to selectively add/remove/magnify/suppress concepts. The term concept is used broadly above to refer to a variety of human observable features including abstract/subjective ones like honesty, "emotions" etc. Noteworthy aspects of this work 1) They show cases where this approach to control the model's output can be used to eliminate bias even in RLHF models for carefully crafted sentences that can expose model bias (image below). 2) They have examples of controlling models' output to reduce factual errors. If this works in practice, this could be a promising alternative to RAG. 3) They have a section to compare this top-down approach with the bottom-up approach of mechanistically interpreting models at a circuit/neuron level being investigated by researchers like @NeelNanda5 @ch402. Research and code contributions by paper authors Andy Zou @andyzou_jiaming authorswithcode.com/research… Long Phan authorswithcode.com/research… Sarah Chen authorswithcode.com/research… James Campbell authorswithcode.com/research… Phillip Guo authorswithcode.com/research… Richard Ren authorswithcode.com/research… Alexander Pan authorswithcode.com/research… Xuwang Yin authorswithcode.com/research… Mantas Maziaka authorswithcode.com/research… Ann-Kathrin Dombrowski authorswithcode.com/research… Shashwat Goel authorswithcode.com/research… Nathaniel Li authorswithcode.com/research… Michael J. Byun authorswithcode.com/research… Zifan Wang authorswithcode.com/research… Alex Mallen authorswithcode.com/research… Steven Basart authorswithcode.com/research… Sanmi Koyejo authorswithcode.com/research… Dawn Song authorswithcode.com/research… Matt Fredrikson authorswithcode.com/research… Zico Kolter authorswithcode.com/research… Dan Hendrycks authorswithcode.com/research…

AuthorsWithCode app fills the current gap in finding both paper and code contributions by an author in one place - an author-centric view. This gap is not covered by Google Scholar, which focuses primarily on citations. Additionally, Google Scholar overlooks an important segment that contributes to open-source implementations of papers: practitioners, some of whom have never published a single paper. Paperswithcode (@paperswithcode), in contrast, provides a paper-centric view with links to accompanying code, encompassing both researchers and practitioners. The objective of AuthorsWithCode, in addition to filling this gap, is to inspire sponsorship for authors who contribute to open source and to create a direct channel to support and invest in their work by encouraging authors to activate GitHub sponsorship (only ~3% have activated it to date). Note: AuthorsWithCode relies on openly published data from Paperswithcode to create an author-centric view by combining it with GitHub data available through APIs. Thanks to @paperswithcode team - both for publishing data and their site that has become indispensable for anyone in the field. authorswithcode.com

Who are the true catalysts fueling AI progress? open.substack.com/pub/author…