Published on

Diffusion LLMs

Presentation
Authors

The fifty-ninth MeetUp of the Machine Learning Singapore Group, was titled : "New Open Models and Techniques". This event covered a broad range of topics : So the title was basically a catch-all!

My Presentation

My talk was titled "Diffusion LLMs", and had the following outline:

  • Model Refresher :
    • BERT; GPT; + GPUs
    • Image diffusion
  • Diffusion for Text : Background
  • Diffusion LLMs :
    • Theory & Practice
  • Wrap-up & QR-code (the latter to reduce audience distractions)

To set the scene, I lead the audience through the basics of Transformers (since I needed to refer to both BERT and GPT-style models, and the on-the-ground facts are that many people who have joined the community since ChatGPT aren't particularly aware of Transformers overall). Then I covered the ideas from image diffusion models, so that the Diffusion over text/tokens would make some sense.

The talk was originally motivated by the launch of Inception Labs / Mercury Coder, and thus I briefly talked about papers that might be associated with the exciting technology that they are demonstrating now. For the in-person crowd, I went into a bit of a deep dive that isn't reflected in the slides...

The slides for my talk, which contain links to all of the reference materials and sources, are here :

Presentation Screenshot

If there are any questions about the presentation please ask below, or contact me using the details given on the slides themselves.

Presentation Content Example

Other Presentations

In his talk "Playing with Gemma 3", Sam Witteveen showed off the exciting new Gemma3 series of Open Source models from Google, including a demonstration of their multi-lingual and multi-modal features.

We were also pleased to welcome Tomasz Maszczyk for his talk "Social Signal Processing - tech4good". In this talk, Tomasz described how he spends his free time : Which included many interesting tech4good projects. He also demonstrated a number of different ways in which AI systems might be used in the context of senior care.

Acknowledgements

Many thanks to the Google team, who not only allowed us to use Google's Developer Space, but were also kind enough to provide Pizza for the attendees!