Published on

Neural Assets and World Models

Presentation
Authors

The sixty-fifth MeetUp of the Machine Learning Singapore Group, was titled : "Deep Agents, World Models and REFRAG" - and had talks covering a wide spectrum.

My Presentation

My talk was titled "Neural Assets and World Models", and had the following outline:

  • Intro
  • Image Models
    • DallE-v3 - power of annotation
    • Image changing Qwen
    • Neural Assets
  • World Models
    • Minecraft
    • GENIE evolution
    • TinyWorlds with Demo
  • Wrap-up & QR-code (the latter to reduce audience distractions)

One theme of the talk was the importance of creating captions/explanations automatically which can then be used to build generative models, i.e. training for:

  • A → Explaination? → B

allows us to build a model for:

  • A → Explaination → B?

I first covered some techniques that are (or at least appear to be) being used to train the latest generation of image models (including, speculatively, Nano Banana). This was followed up be an exploration of the ideas of World Models, first through MineRL and Dreamer-v4, and then on to the GENIE series from DeepMind. To round out the presentation, I gave a demonstration of the TinyWorlds model series, for which I had built a custom (shared) Colab notebook. The link is in the presentation.

The slides for my talk, which contain links to all of the reference materials and sources, are here :

Presentation Screenshot

If there are any questions about the presentation please ask below, or contact me using the details given on the slides themselves.

Presentation Content Example

Other Presentations

The MeetUp included two other speakers: Xiaoqiang Lin and Sam Witteveen:

  • Xiaoqiang's lightning talk, "REFRAG: Rethinking RAG based Decoding", covered the work he had done while at Meta Superintelligence Labs during his PhD.

  • Sam's (hour+ long) talk, "Building DeepAgents", covered Deep Agents, their core components and ways to make them run effectively for long-time horizon tasks and trajectories.

Acknowledgements

Many thanks to the Google team, who allowed us to use Google's Developer Space, and generously provided food for our audience.