Werner Bogula (@Bogula) header
Werner Bogula (@Bogula) profile picture

True digital native and analogue lute player / currently AI enabler @ https://teapawt.net/t.co/HmWbJswrPd

Hamburg
Joined January 2009
Photo by Werner Bogula (@Bogula)Photo by Werner Bogula (@Bogula)Photo by Werner Bogula (@Bogula)Photo by Werner Bogula (@Bogula)Photo by Werner Bogula (@Bogula)Photo by Werner Bogula (@Bogula)Photo by Werner Bogula (@Bogula)Photo by Werner Bogula (@Bogula)Photo by Werner Bogula (@Bogula)Photo by Werner Bogula (@Bogula)
Werner Bogula retweeted
This is insane! 😱 You can now train a 100-billion parameters LLM on Google Colab. Explanation and code below ⤵️⤵️⤵️ --- TL;DR --- Distributed training over the Internet has become operational with the release of the new version of the PETALS distributed training package. Join forces with people from around the world and receive immense computing power in return for your use! 🌐⚡️ --- Intro --- Distributed training over the internet has turned from a dream into a reality with the release of the new version of the Petals distributed training package. Upon connection, the package grants you access to immense computing power that allows you to do anything you can imagine. It doesn't even have to be your own computing power: Attaching below is a Colab notebook that trains a 65-billion-parameter model (and also supports 100 billion. Yes, models with a hundred billion parameters are accessible to you). 💻🚀 --- It surely takes years, doesn't it? --- No. For example, with a 65-billion-parameter inference, you'll get a speed of 5 tokens per second. ⏱️⚡️ --- How does it work? --- The concept is very simple. In LoRA-based training (My tweet about LoRA - teapawt.net/ItakGol/status/1…), the base model is the same for everyone and does not change until the end of training, so many people need it at the same time. With Petals, people load small parts of the model, and each executes their part in the computation of running the full model. All the headache and complex software engineering required to make this madness run smoothly and quickly happens inside the package. 🤯🔧 --- How complex? --- From your own perspective, simply run: AutoDistributedModel.from_pretrained --- Important! Privacy! --- Obviously, this infrastructure sends your data outside to the world, so it's crucial to take that into account. Your code is never sent outside. 🌐🔒 --- Links --- 1. Article: Petals: Collaborative Inference and Fine-tuning of Large Models - aclanthology.org/2023.acl-de… 2. See the speed for yourself: Distributed Chatbot - chat.petals.ml/ 3. Notebook for training a 65-billion-parameter model: colab.research.google.com/dr… 4. Code: github.com/bigscience-worksh… 5. Paper: arxiv.org/abs/2209.01188 6. Join the community: discord.com/invite/D9MwApKgW…
How to Fine-Tune a Large Language Model (LLM) on a Single GPU or Google Colab? 😊 LoRA (Low-Rank Adaptation)! 🚀 Recently, LoRA has been a hot topic in the context of fine-tuning LLMs. But what does LoRA really mean? Let's dive in 📚 1 / 18
22
314
11
1,427
372,432
Werner Bogula retweeted
Das ARIC im Max Magazin: Wenn eine Künstliche Intelligenz eine Weihnachtsgeschichte verfasst, endet das auch schon mal mit einem Axtmörder. @Bogula erklärt im Interview mit @OsingTim, wie es dazu kommt und, wie #NLP funktioniert. Jetzt am Kiosk! #artificialintelligence #AI
1
4
Werner Bogula retweeted
Here’s your chance to win a free NFT, @PEEZMO_Art, because we are on 🔥 and we marked 200 followers. - follow, like & RT - tag 2 friends in your comment - leave or dm your ETH -address Winner announced on Sat, Aug 21th at 8PM CEST #NFTGiveaway 🤟 opensea.io/assets/matic/0x92…
10
12
1
20
As someone who knew Frank Schirrmacher - and his views on Silicon Valley - rather well, I'm quite appalled that a prize in his honor has been given to Peter Thiel. What cosmic injustice is that?
3
16
3
137
Share of electricity that comes from fossil fuels: 🇿🇦 South Africa: 89% 🇦🇺 Australia: 75% 🇮🇳 India: 74% 🇯🇵 Japan: 69% 🇨🇳 China: 66% 🌍 World: 61% 🇺🇸 USA: 60% 🇩🇪 Germany: 44% 🇬🇧 UK: 41% 🇧🇷 Brazil: 14% 🇫🇷 France: 10% 🇸🇪 Sweden: 2% Explore any country here: ourworldindata.org/grapher/p…
98
829
222
2,252
Werner Bogula retweeted
"Now the script has flipped and a new solar plant is almost three times cheaper than a new coal one. The price of electricity from solar declined by 89% between 2009 and 2019." ourworldindata.org/battery-p…
45
309
46
1,596
Werner Bogula retweeted
MAGALL - Club - Berlin @cryptovoxels Opening Party with @EclecticMethod Supported by @nftsmart Join here ➡️https://teapawt.net/t.co/9SRxgCggMO⬅️ Sun, May 9 2021 BERLIN, 10:00p / NEW YORK 4:00p / LOS ANGELES, 1:00p NFTs Backstage Area: @SkullsOnETH @mamadousow221 @PEEZMO_Art
5
15
1
26
Great read: the effect of overdispersion and the consequences for effectively handling a pandemic like #COVID19. theatlantic.com/health/archi…
1
1
1
3
💯
La meilleure définition d'une #startup c'est cette vidéo ! Are you ready ? 😎
1
2
What an amazing and creative idea to fund businesses that struggle from Corona. This farm lets you add Alpacas to your next video conference against a fee to feed them. Surprise your colleagues! alpaca-calling.com/
1
1
“Guter Journalismus ist heute so wichtig wie noch nie”, sagt der SPIEGEL-Autor Juan Moreno. Auch er unterstützt die Forderung an die @EKD und @landesbischof, die #EJS zu erhalten. #ejsretten @RettetdieEJS – at Berlin, Germany
1
16
2
36
Werner Bogula retweeted
Stanford Robotics Club shows off their many mini robos @stanfordrobotx
3
1
24
Werner Bogula retweeted
Ich habe dafür unterzeichnet, dass Produktionen von ARD und ZDF offen lizenziert für alle zur Verfügung stehen. Für gute Bildung für alle wäre das ein hilfreicher Schritt! #wasoffenseinsollte #twlz #twittercampus Hier kannst du auch unterschreiben: wikimedia.de/oeffentliches-g…
4
53
4
104
Our recent study discusses how tech can empower society: publikationen.ifa.de/out/wys… so we're happy to see the topic on the agenda at events around us: progressives-zentrum.org/the… @Bogula @DPZ_Berlin
2
5
Werner Bogula retweeted
I saw a guy at Starbucks today. No iPhone. No tablet. No laptop. He just sat there. Drinking coffee. Like a Psychopath.
5,793
131,733
8,587
844,523
Werner Bogula retweeted
As part of the #TechPlomacy🌐 initiative Denmark’s Foreign Minister @JeppeKofod together w/ Ministers @JY_LeDrian & @cedric_o signed a Joint French-Danish Vision for European Leadership in the Digital 21st Century. Important & timely message. Read it here: techamb.um.dk/~/media/Techam… – at Beijing, People's Republic of China
1
20
4
37
Thanks to all who joined us for 3 amazing days at #techfestival19. We broke into 200+ key conversations, with 300+ digital minds. We came together to drive change into tech. And we are humbled to see so many like-minded humans in one place. With ♥️ Your #techfestivalcph Team
1
7
12
52
2009: The blockchain will undermine central power! 2019: The blockchain will help Facebook to undermine the central power of the state by using the rhetoric of decentralization to actually centralize power in its own hands. A decade of utter confusion.
24
605
50
1,651
TIL: For a student entering NYU today and taking out the full cost in loans, the interest payments (NOT paying down the principal) would be $1,000/month in perpetuity.
11
64
11
304
Werner Bogula retweeted
Replying to @the_claus
It's pretty amazing, how crowded the airspace over Germany is at any time. (Btw. 40 Boeing 737 of 472 flights at 10:30) 😶
2
1