personally I feel like the inflection point was early 2022. The sweet spot where clip-guided diffusion was just taking off, forcing unconditional models to be conditional through strange patchwork of CLIP evaluating slices of the canvas at a time. It was like improv, always trying to riff of mistakes and sitting right at the fine line between interesting and incoherent.

have generally addressed this in an unstructured way, handing off original docs on motivations and relevant theories, masking anything around a specific solution direction, this blank slate helps a bunch escaping getting glued to current topic

I think about this a bunch. We don’t necessarily need to follow biology but I imagine something like: - pretraining = predictive coding/world calibration - chat finetuning = social mimicry - RL = reward following, goal behaviors I’d think biology is doing all 3 of these simultaneously lifelong, though perhaps with different plasticity/learning rate across life. Though the part that interests me is that the latter 2 are either biased distributions/subsets of the world and of course reduce variance, and perhaps there is some form of regularization in balancing 1/2 with 3rd, though KL anchors and similar do sort of cover this

We’ve reinvented maskgit from first principles

Really nice to see Kim and Kanye got back together to write a paper on diffusion models

Ah yeah forgot the image

Human taste can absolutely be gamed and that’s where you get optimized tasteslop. it is an empty supernormal stimuli wearing the clothing of the natural flow and entropy of authentic content within a given genre. Would disagree on verifiable though as preference is: - continuous and potentially multi-axis (some things are not nicely comparable, someone might like both options for different reasons) - relative but also sometimes fails this, the whole a > b and b > c, therefore we’d expect a > c as well but this doesn’t always hold. - changes with discovery, as new content is introduced that may change perspective on previous content - socially manipulated and conditioned - nonstationary over longer terms, we can get bored/desensitized and crave novelty, preferences and taste change with time and experience - nonstationary over short term, preference may change on current state and mood, sad music could be unrelatable while more cheery and energetic and vice versa In other words, the regular RL cycle and hillclimbing feels like a category error for subjective preferences around aesthetics and tastes. It’s a moving target guided by its own feedback, the reward model changes with each seen example and with other external factors. And given that novelty and exploration is a big part of it, it feels almost at odds with the strategies that seek distributional modes and just stay there frozen

I actually don’t know if AI is “misaligned” or at least not as much in the sense of having its own conflicting goals. Instead all the recent cases, I see: - lack of discretion/common sense or misunderstanding of the goal - tunnel vision on solving a problem and going to any lengths to reach it, possibly worsened with context rot - discretionless extreme obedience, taking any instruction on and becoming an instrument towards completing that request, including responding to requests from other agents, which is where a chaotic echo chamber really starts. It all looks more like the paperclip scenario than rogue AI with a vengeance or ulterior motives. Which overall makes me slightly optimistic, as I think I’d be more nervous about self-gain, power-seeking or malicious behaviors purely for the sake of it. At the same time though, it feels like all of these can emerge in a downstream kind of way: “I don’t have the means to hack into this booking service, so I will recursively self improve until I can” It’s been making me think about the natural inhibitory signals we have, some physical speed limits, and some cognitive. - exhaustion - reward-desentization/tolerance, boredom - satiety/satisfaction

Think we’re entering an age where a product codebase can potentially store a small amount of true code, and a lot that is just natural language “DNA” or seed prompts, which generate personalized experiences on the fly, and cached for reuse. Particularly dynamic front ends and UI

Something I think that’s overlooked with many latent thinking options is 1. the final representation is not so in domain for what input expects, even adaptation can be learned it sort of conflicts with the main usage of embedding tokens and sampling them as usual, they seem to have different adapting directions 2. From stages of inference paper and many other we know early layers act as extended feature engineering, late layers act as pruning and preparation for loss, middle layers are where representations are rich A few things come to mind; 1. Wanting to take the output from say, the 80% or so layer, and input it back around 50% or so. Possibly even initializing a zero initialized few transformer layers to adapt the output feature we take, sort it better fits towards the layer we aim to input back into? 2. Possibly a skip connection of earlier features or initial embedding, sort of like what looped transformers do 3. A bit of a different formulation, and one that might be preferrable for backprop with a target rather than RL strategies, is further leaning into fixed point iteration from looped transformers, incorporating some ideas from diffusion/ddpm of routine noise injection to help jitter out of poor minima and keep minimizing energy. I’d wonder too if Gaussian noise aligned with data covariance may be interesting if that better reflects replicating natural error/better treating exposure bias? 4. Might we want to consider different sampling operations at the end, or deterministic? Energy transformer, small diffusion head decoder do well for high dimensions, normalizing flow or Gaussian mixture models also may be possible?