WHAT'S IMPORTANT
📍 New Year, New Look – The AI Consulting team from AMAI launched a new website. With a fresh look you can find live in-browser demos of text anonymization and visual segmentation. An updated careers page is accompanied by a newly started AI Expert developer blog.
AMAI Website
📍 Leaking Language Models – A collaboration between academic institutions and corporations highlights privacy concerns about large language models. In the "Extracting Training Data from Large Language Models" paper (arXiv) researchers from Google, OpenAI, Apple, Harvard, Stanford and others demonstrate a training data extraction attack. Although they only have black-box access (inputs and outputs, but no intermediate layers) to the model they are able to retrieve potentially sensitive information. One example is laid out below.
Google AI Blog
Read more about this on the Google AI Blog or more extensively in this video by Yannic Kilcher.
6 THINGS WE FOUND WORTH SHARING
👁 1. Miscellaneous – On a rather humorous note, this comic was drawn by XKCD two years ago. Given the paper above, this fits particularly well with the security concerns regarding large language model.
XKCD/2169
💡 2. Use Cases – In 2019 DeepMind introduced MuZero, which can achieve superhuman performance in tasks such as Chess, Go, and Atari Games, without knowing any underlying rules beforehand. After the recent publication in Nature, DeepMind's principal research scientist David Silver sat down with the BBC to discuss future applications. Now they have tasked MuZero with video compression, aiming to invent new ways to reduce the data footprint of video, which makes up the majority of internet traffic.
💭 3. Opinion – Yann LeCun, Chief AI Scientist at Facebook, reflects on the unrealistic expectations some people hold about large-scale language models such as GPT-3. In his Facebook post he writes:
"[Trying] to build intelligent machines by scaling up language models is like building a high-altitude airplanes to go to the moon. You might beat altitude records, but going to the moon will require a completely different approach."
🎓 4. Education – The interactive cheatsheet from Stanford's CS229 Machine Learning course is great for learners and those trying to understand ML terminology.
📖 5. Papers – Data-efficient image Transformers (DeiT) is a new method to train computer vision models that leverage Transformers. The method requires less data and far less computing resources to produce state-of-the-art image classification models. Read more in the Facebook AI Blog and the paper on arXiv.
Facebook AI Research
👩💻 6. Code – Animate the faces of other humans, Muppets or Nefertiti with your webcam in real-time. Using the First Order Motion Model, images of others can be matched to your movements. Try it right in your browser with this Colab notebook by Eyal Gruss. Read more on the First Order Motion Model in this Towards Data Science article.
Vladimir Alexeev, Towards Data Science
Do you want to bring AI to your company? 👉 Get in Touch
UPCOMING EVENTS
📆 January 5, 17:00 CET (online) – This afternoon Louis Dorard gives a Live demonstration of his Machine Learning Canvas, a framework used at AWS, BlaBlaCar and UCL. – Register on crowdcast.io.
📆 January 15-16 – Global AI Bootcamp is a free one-day event organized across the world by local communities. As of now 119 locations worldwide are organizing the Microsoft-sponsored event (most online) – Find your local community at globalai.community.
💙 January 21, 17:00 CET (online, German) – KI im Unternehmen: Wie und wo fange ich an? Our colleague Woldemar Metzler sheds light on the particular challenges German "Mittelständler" companies face when approaching AI. He shows first steps SMEs can take to increase competitiveness in their future with Artificial Intelligence applications. – Free admission here on digitalhub-nordschwarzwald.de.
