
Love solving problems and βsimplifyingβ complexity. Talk to me about politics, tech and social issues. Engineer @Meta / Facebook | Opinions are my own.
Joined July 2011
- Tweets948
- Following465
- Followers103
- Likes2.4K
#til What associative memory is, in the context of LLMs
The way traditional memory works is when given a key, or memory address, the value is retrieved. Letβs say you want a blue box stored in rack number 10.
β¦
5
33
β¦ are conceptually similar. LLMs retrieve not based on exact matches as in the case of associative memory, but rather base it on similarity.
Associative memory needs exact matches, like βblue boxβ, while LLMs would also return a blue box, if your query was βsky coloured β¦
18
β¦
One other feature of associative memory is that the search is parallel across all data, making it faster than linear search on data in traditional memory.
There is specific hardware that supports associative memory.
LLMs however do not use this hardware but β¦
8
β¦It then does a pattern search to get to the expected result.
This obviously needs more memory and resources both when storing and retrieving.
β¦
12
β¦
You would input rack number 10 to get to the blue box in a racks filled with toys.
However, in associative memory, any of these inputs: βblue boxβ or βblue toyβ or βboxβ would lead you to the blue box.
This is because associative memory also holds what the data looks β¦
18
So I gave it a shot as well. This is fun !
Wordle 221 6/6
β¬β¬β¬β¬β¬
β¬β¬π©β¬β¬
π¨β¬π©π¨β¬
β¬π¨π¨β¬π¨
β¬β¬β¬β¬β¬
π©π©π©π©π©
1
2
Narrating a pretty bad experience with @HelloFreshUK
I forgot to cancel my order since they donβt even send a notification before dispatching an order or charging me. I called them up and asked them to cancel it so food is not wasted, their response - βnot possibleβ. #unethical
1
This new year, resolve to read βAnnihilation of Casteβ by Dr B R Ambedkar. It would take you a few hours but the understanding youβll gain is life changing.
bitsapien π³οΈβπ Ξ» | they/them retweeted
Everyone who hasnβt, should read Dr. B. R. Ambedkarβs #AnnhilationOfCaste.
Thatβs it. Thatβs the tweet.
Flew to London last week on an @airindiain flight. Faced some extremely rude behaviour and was lied to about their policies/infrastructure. I realised it was a lie because they switched their version once they knew I was flying business class. This is outright #discrimination !
1
Random Sunday Thought: Designing systems, starting from their interface makes more sense in my head than other strategies.
1
bitsapien π³οΈβπ Ξ» | they/them retweeted
Replying to @WakefitCo
@WakefitCo, horrible customer care service, I ordered a mattress one week ago, the delivery guy asked my address but didnot deliver,ever since I have been calling Wakefit and their delivery partner as well,they donβt know where my order is it is marked delivered. #horrible
#TIL Using `index` instead of an `id` as the key in #reactjs could be one of the reasons why your "infinity scroll" component is painfully slow.
Check this pretty interesting talk giving a gist of how React works internally
youtu.be/i793Qm6kv3U?t=1028
Discovered the IAM Policy Simulator today on the #AWS web console.
This has greatly simplified when trying to apply
the principle of least privilege", so that the IAM roles I create only have enough permissions to do what I ask them to do, nothing more and nothing less.
Q: What is the test for equality in #Clojure
#RichHickey: Equality means the same thing for everything in Clojure, it means - equal value .. Clojure definitely de-emphases identity
Wouldn't the world be a better place if this was THE test for equality - value and NOT identity?
1
bitsapien π³οΈβπ Ξ» | they/them retweeted
Central govt has assured Supreme Court to vaccinate the entire adult population by Dec end.
This requires 238 million doses every month from June.
Procure on a war footing.
Launch free universal mass vaccination drive NOW.
hindustantimes.com/india-newβ¦



