assistant professor of psychology/neuroscience @USC丨he/him丨semiprofessional dungeon master丨🍉
- Tweets2.5K
- Following2.6K
- Followers1.9K
- Likes18.5K
ALT Top: The title, authors, and journal of the paper. Title: "Reconciling symbols and neural networks: Evidence for a quasi-symbolic view." Authors: "Tom McCoy, Abi Tenenbaum, Zhang Enyan, Zhenghao Herbert Zhou." Journal: "Current Opinion in Behavioral Sciences" Bottom: A figure displaying how the vector representations learned by neural networks often show emergent symbolic structure. For instance, they can often be decomposed into meaningful constituents (illustrated in the figure by adding constituents to make a vector) or classified in vector space along symbolic dimensions (shown by an image of linear classifiers). However, these vectors are only quasi-symbolic: they deviate from pure symbolic structure in important ways - e.g., they can display graded similarity (“movie” is closer to “novel” than to “meal”), and they may capture graded context effects such as the word “good” affecting different nouns differently (the image shows such graded similarity among points in space
ALT Four panels show GammaNet progressively grouping the cued person in a photograph of two people at a restaurant. At T = 0, a red dot cues the man's arm. Yellow highlighting spreads across his arm, torso and head at T = 5, 10 and 15. The caption reads: The network progressively groups the cued person, shown in yellow. T indicates network processing steps.
ALT Overview slide of "A Unifying Perspective on Language Model Representations: From Filler-Role Structure to Mechanistic Interpretability". The title is on the top, followed by authors: Zhang Enyan and Tom McCoy Left: An elephant viewed from different perspectives, where each viewer think the elephant feels like a fan, snake, wall, and rope, respectively. This classic story is compared with a black-box LLM, where it is viewed as probes, SAEs, causal traces, and analogies. Center: explanation text "We unify a variety of interpretability methods under Tensor Product Representations (TPRs), and show that they can be constructed directly from TPRs with comparable performance ", followed by a schematic of how a Tensor Product Representation (TPR) is constructed. The schematic has arrows linking to four cells, each containing an interpretability method, demonstrating they can be viewed as applications of TPR operations
![Overview of the paper.
Title: The Emergent Symbolic Structure of Artificial Neural Networks
Authors: Tom McCoy, Paul Soulos, Tal Linzen, Paul Smolensky
Left: Neural networks encode information in vectors (there is then an image of a vector), yet they excel at tasks long thought to require symbolic structure (there is then an image of a symbolic representation, specifically a syntax tree). How do LLMs do it?
Right: We find that LLM representations can be closely approximated with symbolic structures. This approximation lets us edit the structure of an LLM’s output by editing the structure of its internal representations, as shown. There is then an image of two edits to LLMs. In the first one, the original input is 3 + 6 * 8, with an answer of 51. But if we swap the positions of the 3 and the 6, the output becomes 30. In the second one, the original input is a Python command repeating the list [Z, U] three times, producing [Z, U, Z, U, Z, U]. But if we edit the input in a way that adds](https://pbs.twimg.com/media/HRJ5Jr5W0AAQ-sN.jpg?name=orig)
ALT Overview of the paper. Title: The Emergent Symbolic Structure of Artificial Neural Networks Authors: Tom McCoy, Paul Soulos, Tal Linzen, Paul Smolensky Left: Neural networks encode information in vectors (there is then an image of a vector), yet they excel at tasks long thought to require symbolic structure (there is then an image of a symbolic representation, specifically a syntax tree). How do LLMs do it? Right: We find that LLM representations can be closely approximated with symbolic structures. This approximation lets us edit the structure of an LLM’s output by editing the structure of its internal representations, as shown. There is then an image of two edits to LLMs. In the first one, the original input is 3 + 6 * 8, with an answer of 51. But if we swap the positions of the 3 and the 6, the output becomes 30. In the second one, the original input is a Python command repeating the list [Z, U] three times, producing [Z, U, Z, U, Z, U]. But if we edit the input in a way that adds
