Werner Bogula retweeted
This is insane! 😱
You can now train a 100-billion parameters LLM on Google Colab.
Explanation and code below ⤵️⤵️⤵️
--- TL;DR ---
Distributed training over the Internet has become operational with the release of the new version of the PETALS distributed training package. Join forces with people from around the world and receive immense computing power in return for your use! 🌐⚡️
--- Intro ---
Distributed training over the internet has turned from a dream into a reality with the release of the new version of the Petals distributed training package. Upon connection, the package grants you access to immense computing power that allows you to do anything you can imagine. It doesn't even have to be your own computing power: Attaching below is a Colab notebook that trains a 65-billion-parameter model (and also supports 100 billion. Yes, models with a hundred billion parameters are accessible to you). 💻🚀
--- It surely takes years, doesn't it? ---
No. For example, with a 65-billion-parameter inference, you'll get a speed of 5 tokens per second. ⏱️⚡️
--- How does it work? ---
The concept is very simple. In LoRA-based training (My tweet about LoRA - teapawt.net/ItakGol/status/1…), the base model is the same for everyone and does not change until the end of training, so many people need it at the same time. With Petals, people load small parts of the model, and each executes their part in the computation of running the full model.
All the headache and complex software engineering required to make this madness run smoothly and quickly happens inside the package. 🤯🔧
--- How complex? ---
From your own perspective, simply run:
AutoDistributedModel.from_pretrained
--- Important! Privacy! ---
Obviously, this infrastructure sends your data outside to the world, so it's crucial to take that into account. Your code is never sent outside. 🌐🔒
--- Links ---
1. Article: Petals: Collaborative Inference and Fine-tuning of Large Models - aclanthology.org/2023.acl-de…
2. See the speed for yourself: Distributed Chatbot - chat.petals.ml/
3. Notebook for training a 65-billion-parameter model: colab.research.google.com/dr…
4. Code: github.com/bigscience-worksh…
5. Paper: arxiv.org/abs/2209.01188
6. Join the community: discord.com/invite/D9MwApKgW…
Werner Bogula retweeted
Das ARIC im Max Magazin:
Wenn eine Künstliche Intelligenz eine Weihnachtsgeschichte verfasst, endet das auch schon mal mit einem Axtmörder.
@Bogula erklärt im Interview mit @OsingTim, wie es dazu kommt und, wie #NLP funktioniert.
Jetzt am Kiosk!
#artificialintelligence #AI
Werner Bogula retweeted
Here’s your chance to win a free NFT, @PEEZMO_Art, because we are on 🔥 and we marked 200 followers.
- follow, like & RT
- tag 2 friends in your comment
- leave or dm your ETH -address
Winner announced on Sat, Aug 21th at 8PM CEST
#NFTGiveaway 🤟
opensea.io/assets/matic/0x92…
Werner Bogula retweeted
As someone who knew Frank Schirrmacher - and his views on Silicon Valley - rather well, I'm quite appalled that a prize in his honor has been given to Peter Thiel. What cosmic injustice is that?
Werner Bogula retweeted
Share of electricity that comes from fossil fuels:
🇿🇦 South Africa: 89%
🇦🇺 Australia: 75%
🇮🇳 India: 74%
🇯🇵 Japan: 69%
🇨🇳 China: 66%
🌍 World: 61%
🇺🇸 USA: 60%
🇩🇪 Germany: 44%
🇬🇧 UK: 41%
🇧🇷 Brazil: 14%
🇫🇷 France: 10%
🇸🇪 Sweden: 2%
Explore any country here: ourworldindata.org/grapher/p…
Werner Bogula retweeted
"Now the script has flipped and a new solar plant is almost three times cheaper than a new coal one. The price of electricity from solar declined by 89% between 2009 and 2019."
ourworldindata.org/battery-p…
Werner Bogula retweeted
MAGALL - Club - Berlin @cryptovoxels
Opening Party with @EclecticMethod
Supported by @nftsmart
Join here ➡️https://teapawt.net/t.co/9SRxgCggMO⬅️
Sun, May 9 2021
BERLIN, 10:00p / NEW YORK 4:00p / LOS ANGELES, 1:00p
NFTs Backstage Area:
@SkullsOnETH
@mamadousow221
@PEEZMO_Art
Werner Bogula retweeted
Great read: the effect of overdispersion and the consequences for effectively handling a pandemic like #COVID19.
theatlantic.com/health/archi…
Werner Bogula retweeted
💯
Werner Bogula retweeted
What an amazing and creative idea to fund businesses that struggle from Corona. This farm lets you add Alpacas to your next video conference against a fee to feed them. Surprise your colleagues! alpaca-calling.com/
Werner Bogula retweeted
“Guter Journalismus ist heute so wichtig wie noch nie”, sagt der SPIEGEL-Autor Juan Moreno. Auch er unterstützt die Forderung an die @EKD und @landesbischof, die #EJS zu erhalten. #ejsretten @RettetdieEJS – at Berlin, Germany
Werner Bogula retweeted
Ich habe dafür unterzeichnet, dass Produktionen von ARD und ZDF offen lizenziert für alle zur Verfügung stehen. Für gute Bildung für alle wäre das ein hilfreicher Schritt! #wasoffenseinsollte #twlz #twittercampus
Hier kannst du auch unterschreiben:
wikimedia.de/oeffentliches-g…
Werner Bogula retweeted
Our recent study discusses how tech can empower society: publikationen.ifa.de/out/wys… so we're happy to see the topic on the agenda at events around us:
progressives-zentrum.org/the…
@Bogula @DPZ_Berlin
Werner Bogula retweeted
I saw a guy at Starbucks today.
No iPhone.
No tablet.
No laptop.
He just sat there.
Drinking coffee.
Like a Psychopath.
Werner Bogula retweeted
As part of the #TechPlomacy🌐 initiative Denmark’s Foreign Minister @JeppeKofod together w/ Ministers @JY_LeDrian & @cedric_o signed a Joint French-Danish Vision for European Leadership in the Digital 21st Century. Important & timely message. Read it here: techamb.um.dk/~/media/Techam… – at Beijing, People's Republic of China
Werner Bogula retweeted
Thanks to all who joined us for 3 amazing days at #techfestival19.
We broke into 200+ key conversations, with 300+ digital minds. We came together to drive change into tech. And we are humbled to see so many like-minded humans in one place.
With ♥️
Your #techfestivalcph Team
Werner Bogula retweeted
2009: The blockchain will undermine central power!
2019: The blockchain will help Facebook to undermine the central power of the state by using the rhetoric of decentralization to actually centralize power in its own hands.
A decade of utter confusion.
Werner Bogula retweeted
TIL: For a student entering NYU today and taking out the full cost in loans, the interest payments (NOT paying down the principal) would be $1,000/month in perpetuity.
Werner Bogula retweeted
Replying to @the_claus
It's pretty amazing, how crowded the airspace over Germany is at any time.
(Btw. 40 Boeing 737 of 472 flights at 10:30)
😶
































