My opinions are not my own. They belong to the gnomes that live in my head
Poland
Joined October 2015
- Tweets399
- Following369
- Followers74
- Likes5.2K
Jakub Bartczuk retweeted
"I used AI to create Minecraft from scratch!"
"It is just a few hundred lines!"
pretty much sums the hype around AI replacing software devs.
Jakub Bartczuk retweeted
Don't miss Jo's talk + notes + slides etc
parlance-labs.com/education/…
Jakub Bartczuk retweeted
Once again, our application for observer status at WIPO, the @UN agency for intellectual property, was blocked by China.
Our presence at WIPO would help ensure that in the age of AI, copyright truly reflects the global and diverse needs of the internet ➡️ wikimediafoundation.org/news…
ALT Bay window of the WIPO Conference Hall. Punched into the sides of the structure, this window lets natural light flow into the building. Text says “For the fourth time, China blocked the Wikimedia Foundation as observer to the World Intellectual Property Organization”.
Jakub Bartczuk retweeted
people keep saying AI is moving so fast. some days I agree, but some days I'm not sure – so many papers published, but I don't feel like we're making that many fundamental breakthroughs.
to cap off 2023, here's a list of things we still don't know about language models:
- how can we reliably answer questions from long texts, like books?
- can we pretrain a world-class LM for less than a million dollars?
- can we prevent a model from making up facts?
- how can we "think more" for some inputs than others?
- how can we train on both private and public data?
- are our architectures (transformers) close to optimal?
- are our parameters (found via SGD) close to optimal? (how far can a 7b model go?)
- can we build systems that are resistant to jailbreaking and prompt inversion?
- how can we tell when a model is unsure of what it's saying?
- what's the most efficient way for two LMs to communicate with each other?
- can we detect LM-generated text?
- can we watermark LM-generated text for detection?
- what do we do when we run out of internet text data to train on? (some folks think we're quite close)
- how should we fairly evaluate trained models, if they've trained on all the text in the world?
- what's the best way to input text into models? (tokenization is a hack)
- how far can we shrink models down without losing performance? (lots of research in quantization but a long way to go)
- how can we understand what models are doing under the hood? (what are they thinking about, how do they store information, etc.)
- is next word prediction the best learning objective for AI?
- is there a natural way to combine text inputs with other types of data (eg images, video, audio)?
(...what did i miss?)
Jakub Bartczuk retweeted
Can LLMs generate mathematical proofs that can be rigorously checked?
We release LeanDojo (leandojo.org): an open-source playground consisting of toolkits, benchmarks, and models for LLMs to prove formal theorems in the Lean proof assistant.
Key features:
- Tools for data extraction and interacting with Lean.
- Fine-grained annotations of premises (e.g., existing lemmas) in proofs: where they are used and defined.
- LeanDojo Benchmark: 97K human-written theorems/proofs for developing machine learning models on theorem proving.
- ReProver (Retrieval-Augmented Prover): the first LLM-based prover augmented with retrieval for premise selection.
We open-source everything, providing the first set of open-source LLM-based theorem provers without any proprietary data, model, or code.
Jakub Bartczuk retweeted
In the first 7 months of 2023, German exports to Kyrgyzstan are up 1400% from the same period in 2019 (black), i.e. before COVID & Russia's invasion of Ukraine. That's powered by an eye-watering 6000% rise in cars and parts exports to Kyrgyzstan (blue). Vorsprung durch Technik...
Jakub Bartczuk retweeted
People are testing large language models (LLMs) on their "cognitive" abilities - theory of mind, causality, syllogistic reasoning, etc. Many (most?) of these evaluations are deeply flawed. To evaluate LLMs effectively, we need some principles from experimental psychology.🧵
Jakub Bartczuk retweeted
If you are worried about "AI causing mass unemployment" listen to economists who have studied the impact of technological revolutions on the labor market, like @erikbryn .
DO NOT listen to computer scientists who are concerned by the social consequences of their work.
Jakub Bartczuk retweeted
It looks like the stress is real. In my experience:
1. paradigms shift
2. numbers get higher
3. Yet, unsolved problems remain in big numbers.
Moreover, it looks like new discoveries create new opportunities. It is a bit hard when you are locked in with some topic that you chose for your thesis. Just finish your PhD (successfully) and move on. It is easier to adapt & change direction when you are not a grad student, believe me.
Overall, I think finishing projects is ultimately more important than chasing an new shiny target all the time. Yes, that next paper will not change the world, but ten finished papers on your CV can certainly change your life (or ten projects on your GitHub). Look, I started my journey even before ML became a thing and I am doing rather well.
Jakub Bartczuk retweeted
People are excited about using ChatGPT for learning. It's often very good. But the danger is that you can't tell when it's wrong unless you already know the answer. I tried some basic information security questions. In most cases the answers sounded plausible but were in fact BS.
ALT Q: Why are encryption algorithms kept secret? A: Encryption algorithms are often kept secret to protect them from being compromised by attackers. If the details of an encryption algorithm were widely known, it would be easier for someone to find a way to break the encryption and gain access to the protected information. Keeping the details of an encryption algorithm secret makes it more difficult for attackers to break the encryption and access the protected information. It also allows the creators of the encryption algorithm to constantly update and improve the algorithm to make it more secure.
Jakub Bartczuk retweeted
Germany 2015: "Greece must make painful economic sacrifices to receive bailout"
Germany 2022 " Germany will not make any economic sacrifices to stop a genocide"
Jakub Bartczuk retweeted
GPT-3 from @OpenAI got you interested in zero-shot and few-shot learning? You're lucky because our own @joeddav has just released a demo of zero-shot topic classification!
Test how the model can predict a topic it has NEVER been trained on: huggingface.co/zero-shot 🤯🤯🤯
Jakub Bartczuk retweeted
Incredibly proud to release this: a new deep learning library, incorporating lessons we've learned from @spacy_io, prodi.gy and @tiangolo's FastAPI.
Even if you're not looking for new tools, there's lots you might find interesting conceptually. Quick thread: 1/5
You’re unable to view this Post because this account owner limits who can view their Posts. Learn more