Ph.D., Computer Science
Chief Scientist, Luma AI
Curriculum Vitae (outdated)


I am Chief Scientist in Luma AI, where we are working on next-generation multimodal foundation models.

During Luma AI, I led the training efforts on Dream Machine, a large-scale video generation model, and Genie, a text-to-3d generation model.

Prior to joining Luma AI, I was:

  • A research scientist in the Deep Imagination Research (DIR) group of NVIDIA Research. I worked on NVIDIA AI Foundations, specifically the Picasso project.
  • A research scientist in the Learning and Perception Research (LPR) group of NVIDIA Research. I was the first author of an ICLR 2023 and an ICML 2023 paper and also collaborated on various other papers.
  • A postdoc in Computer Science in Stanford University working with Stefano Ermon.
  • Ph.D. in Computer Science at Stanford University advised by Stefano as well.
  • Undergrad in the Department of Computer Science and Technology, Tsinghua University, where I worked with Jun Zhu and Lawrence Carin.

My “generative AI” qualifications:

Email: jiaming [dot] tsong [atgmaildotcom]


News
  • Organized the CVPR 2023 diffusion model tutorial. Slides are available.
  • Organizing the SIGGRAPH 2023 Course on Diffusion Models.
  • Talks at University of Edinburgh, DeepMind, Imperial College London, and Oxford. [slides]

Publications

2023

  1. [58]
    Jiaming Song, Qinsheng Zhang, Hongxu Yin, Morteza Mardani, Ming-Yu Liu, Yan Kautz, Arash Vahdat Loss-Guided Diffusion Models for Plug-and-Play Controllable Generation ICML 2023, In International Conference on Machine Learning.
  2. [59]
    Qinsheng Zhang, Jiaming Song, Xun Huang, Yongxin Chen, Ming-Yu Liu DiffCollage: Parallel Generation of Large Content with Diffusion Models CVPR 2023, In The IEEE/CVF Conference on Computer Vision and Pattern Recognition.
  3. [57]
    Jiaming Song, Arash Vahdat, Morteza Mardani, Jan Kautz Pseudoinverse-Guided Diffusion Models for Inverse Problems ICLR 2023, In International Conference on Learning Representations.
  4. [56]
    Xuan Su, Jiaming Song, Chenlin Meng, Stefano Ermon Dual Diffusion Implicit Bridges for Image-to-Image Translation ICLR 2023, In International Conference on Learning Representations. [website] [code] [colab]
  5. [55]
    Lantao Yu, Tianhe Yu, Jiaming Song, Willie Neiswanger, Stefano Ermon Offline Imitation Learning with Suboptimal Demonstrations via Relaxed Distribution Matching AAAI 2023, In AAAI Conference on Artificial Intelligence.

2022

  1. [54]
    Yogesh Balaji, Seungjun Nah, Xun Huang, Arash Vahdat, Jiaming Song, Karsten Kreis, Miika Aittala, Timo Aila, Samuli Laine, Bryan Catanzaro, Tero Karras, Ming-Yu Liu eDiff-I: Text-to-Image Diffusion Models with Ensemble of Expert Denoisers Preprint, arXiv preprint arXiv:2211.01324. [website]
  2. [53]
    Bahjat Kawar*, Jiaming Song*, Stefano Ermon, Michael Elad JPEG Artifact Correction using Denoising Diffusion Restoration Models NeurIPS 2022 SBM Workshop, In Neural Information Processing Systems (NeurIPS) Workshop on Score-Based Methods. [website] [code]
  3. [52]
    Bahjat Kawar, Michael Elad, Stefano Ermon, Jiaming Song Denoising Diffusion Restoration Models NeurIPS 2022, In Neural Information Processing Systems. [website] [code] Short version in ICLR 2022 Workshop on Deep Generative Models for Highly Structured Data (Oral presentation).
  4. [51]
    Chenlin Meng*, Kristy Choi*, Jiaming Song, Stefano Ermon Concrete Score Matching: Generalized Score Matching for Discrete Data NeurIPS 2022, In Neural Information Processing Systems.
  5. [50]
    Divyansh Garg, Sakanda Vaidyanath, Kuno Kim, Jiaming Song, Stefano Ermon LISA: Learning Interpretable Skill Abstractions from Language NeurIPS 2022, In Neural Information Processing Systems.
  6. [49]
    Jiaming Song*, Lantao Yu*, Willie Neiswanger, Stefano Ermon A General Recipe for Likelihood-free Bayesian Optimization ICML 2022, In International Conference on Machine Learning, Long oral presentation (Top 2.2%). [website] [code]
  7. [48]
    Samarth Sinha*, Jiaming Song*, Animesh Garg, Stefano Ermon Experience Replay with Likelihood-free Importance Weights L4DC 2022, In 4th Annual Conference on Learning for Dynamics and Control, Best paper award finalist.
  8. [47]
    Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, Stefano Ermon SDEdit: Image Synthesis and Editing with Stochastic Differential Equations ICLR 2022, In International Conference on Learning Representations. [website] [code] [colab]
  9. [46]
    Shengjia Zhao, Abhishek Sinha, Yutong He, Aidan Perreault, Jiaming Song, Stefano Ermon Comparing Distributions by Measuring Differences that Affect Decision Making ICLR 2022, In International Conference on Learning Representations, ICLR 2022 Outstanding Paper Award.
  10. [45]
    Chenlin Meng, Enci Liu, Willie Neiswanger, Jiaming Song, Marshall Burke, David Lobell, Stefano Ermon IS-COUNT: Large-scale Object Counting from Satellite Images with Covariate-based Importance Sampling AAAI 2022, In AAAI Conference on Artificial Intelligence. [website] [code] [colab]
  11. [44]
    Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, others Beyond the Imitation Game: Quantifying and Extrapolating the Capabilities of Language Models Preprint, arXiv preprint arXiv:2206.04615.

2021

  1. [41]
    Abhishek Sinha*, Jiaming Song*, Chenlin Meng, Stefano Ermon D2C: Diffusion-Denoising Models for Few-shot Conditional Generation NeurIPS 2021, In Neural Information Processing Systems. [website] [code] [colab]
  2. [40]
    Divyansh Garg, Shuvam Chakraborty, Chris Cundy, Jiaming Song, Stefano Ermon IQ-Learn: Inverse soft-Q Learning for Imitation NeurIPS 2021, In Neural Information Processing Systems, Spotlight presentation. [website] [code]
  3. [39]
    Lantao Yu, Jiaming Song, Yang Song, Stefano Ermon Pseudo-Spherical Contrastive Divergence NeurIPS 2021, In Neural Information Processing Systems.
  4. [38]
    Yusuke Tashiro, Jiaming Song, Yang Song, Stefano Ermon CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation NeurIPS 2021, In Neural Information Processing Systems. [code]
  5. [37]
    Jiayu Chen, Yuanxin Zhang, Yuanfan Xu, Huimin Ma, Huazhong Yang, Jiaming Song, Yu Wang, Yi Wu Variational Automatic Curriculum Learning for Sparse-Reward Cooperative Multi-Agent Problems NeurIPS 2021, In Neural Information Processing Systems. [website] [code]
  6. [36]
    Kuno Kim, Akshat Jindal, Yang Song, Jiaming Song, Yanan Sui, Stefano Ermon Imitation with Neural Density Models NeurIPS 2021, In Neural Information Processing Systems.
  7. [35]
    Jiaming Song, Chenlin Meng, Stefano Ermon Denoising Diffusion Implicit Models ICLR 2021, In International Conference on Learning Representations. [code]
  8. [34]
    Abhishek Sinha*, Ayush Kumar*, Jiaming Song*, Burak Ukzent, Hongxia Jin, Stefano Ermon Negative Data Augmentation ICLR 2021, In International Conference on Learning Representations. [code]
  9. [33]
    Chenlin Meng, Jiaming Song, Yang Song, Shengjia Zhao, Stefano Ermon Improved Autoregressive Modeling with Distribution Smoothing ICLR 2021, In International Conference on Learning Representations, Oral presentation. [code]

2020

  1. [32]
    Jiaming Song, Stefano Ermon Multi-label Contrastive Predictive Coding NeurIPS 2020, In Neural Information Processing Systems, Oral presentation.
  2. [31]
    Chenlin Meng, Lantao Yu, Yang Song, Jiaming Song, Stefano Ermon Autoregressive Score Matching NeurIPS 2020, In Neural Information Processing Systems.
  3. [30]
    Jonathan Kuck, Shuvam Chakraborty, Hao Tang, Rachel Luo, Jiaming Song, Ashish Sabharwal, Stefano Ermon Belief Propagation Neural Networks NeurIPS 2020, In Neural Information Processing Systems.
  4. [29]
    Jiaming Song, Michael Auli, Yann Dauphin, Tengyu Ma Robust and On-the-fly Dataset Denoising for Image Classification ECCV 2020, In European Conference on Computer Vision. [slides]
  5. [28]
    Chenhao Niu, Yang Song, Jiaming Song, Shengjia Zhao, Aditya Grover, Stefano Ermon Permutation Invariant Graph Generation via Score-Based Generative Modeling AISTATS 2020, In International Conference on Artificial Intelligence and Statistics. [code]
  6. [27]
    Chenlin Meng, Yang Song, Jiaming Song, Stefano Ermon Gaussianization Flows AISTATS 2020, In International Conference on Artificial Intelligence and Statistics. [code]
  7. [26]
    Lantao Yu, Yang Song, Jiaming Song, Stefano Ermon Training Deep Energy-Based Models with f-Divergence Minimization ICML 2020, In International Conference on Machine Learning.
  8. [25]
    Jiaming Song, Stefano Ermon Bridging the Gap Between f-GANs and Wasserstein GANs ICML 2020, In International Conference on Machine Learning. [slides] [code]
  9. [24]
    Kuno Kim, Yihong Gu, Jiaming Song, Shengjia Zhao, Stefano Ermon Domain Adaptive Imitation Learning ICML 2020, In International Conference on Machine Learning.
  10. [23]
    Jiaming Song, Stefano Ermon Understanding the Limitations of Variational Mutual Information Estimators ICLR 2020, In International Conference on Learning Representations. [slides] [code]
  11. [22]
    Yilun Xu, Shengjia Zhao, Jiaming Song, Russell Stewart, Stefano Ermon A Theory of Usable Information under Computational Constraints ICLR 2020, In International Conference on Learning Representations, Oral presentation.
  12. [21]
    Nate Gruver, Jiaming Song, Mykel J Kochenderfer, Stefano Ermon Multi-agent Adversarial Inverse Reinforcement Learning with Latent Variables AAMAS 2020, In International Conference on Autonomous Agents and MultiAgent Systems (extended abstract).
  13. [20]
    Rachel Luo, Shengjia Zhao, Jiaming Song, Jonathan Kuck, Stefano Ermon, Silvio Savarese Privacy Preserving Recalibration under Domain Shift Preprint, arXiv:2008.09643.

2019

  1. [19]
    Aditya Grover, Jiaming Song, Ashish Kapoor, Kenneth Tran, Alekh Agarwal, Eric J Horvitz, Stefano Ermon Bias Correction of Learned Generative Models using Likelihood-free Importance Weighting NeurIPS 2019, In Advances in Neural Information Processing Systems.
  2. [18]
    Ali Malik, Volodymyr Kuleshov, Jiaming Song, Danny Nemer, Harlan Seymour, Stefano Ermon Calibrated Model-based Deep Reinforcement Learning ICML 2019, In International Conference on Machine Learning. [code]
  3. [17]
    Lantao Yu, Jiaming Song, Stefano Ermon Multi-agent Adversarial Inverse Reinforcement Learning ICML 2019, In International Conference on Machine Learning. [code]
  4. [16]
    Shengjia Zhao, Jiaming Song, Stefano Ermon InfoVAE: Balancing Learning and Inference in Variational Autoencoders AAAI 2019, In AAAI Conference on Artificial Intelligence.
  5. [15]
    Jiaming Song, Pratyusha Kalluri, Aditya Grover, Shengjia Zhao, Stefano Ermon Learning Controllable Fair Representations AISTATS 2019, In International Conference on Artificial Intelligence and Statistics. [code]
  6. [14]
    Jiaming Song, Yang Song, Stefano Ermon Unsupervised Out-of-Distribution Detection with Batch Normalization Preprint, arXiv:1910.09115.

2018

  1. [13]
    Jiaming Song, Hongyu Ren, Dorsa Sadigh, Stefano Ermon Multi-Agent Generative Adversarial Imitation Learning NeurIPS 2018, In Advances in Neural Information Processing Systems. [code]
  2. [12]
    Shengjia Zhao, Hongyu Ren, Arianna Yuan, Jiaming Song, Noah Goodman, Stefano Ermon Bias and Generalization in Deep Generative Models: An Empirical Study NeurIPS 2018, In Advances in Neural Information Processing Systems, Spotlight presentation. [code]
  3. [11]
    Shengjia Zhao, Jiaming Song, Stefano Ermon The Information Autoencoding Family: A Lagrangian Perspective on Latent Variable Generative Models UAI 2018, In Conference on Uncertainty in Artificial Intelligence, Oral presentation. [code]
  4. [10]
    Yang Song, Jiaming Song, Stefano Ermon Accelerating Natural Gradient with Higher-Order Invariance ICML 2018, In International Conference on Machine Learning. [code]
  5. [9]
    Hongyu Ren, Russell Stewart, Jiaming Song, Volodymyr Kuleshov, Stefano Ermon Adversarial Constraint Learning for Structured Prediction IJCAI 2018, In International Joint Conference on Artificial Intelligence. [code]
  6. [8]
    Hongyu Ren, Russell Stewart, Jiaming Song, Volodymyr Kuleshov, Stefano Ermon Learning with weak supervision from physics and data-driven constraints AI Magazine, AI Magazine.

2017

  1. [7]
    Jiaming Song, Shengjia Zhao, Stefano Ermon A-NICE-MC: Adversarial training for MCMC NeurIPS 2017, In Advances in Neural Information Processing Systems. [slides] [code] [blog]
  2. [5]
    Yunzhu Li, Jiaming Song, Stefano Ermon InfoGAIL: Interpretable imitation learning from visual demonstrations NeurIPS 2017, In Advances in Neural Information Processing Systems. [code]
  3. [6]
    Shengjia Zhao, Jiaming Song, Stefano Ermon Learning Hierarchical Features from Deep Generative Models ICML 2017, In International Conference on Machine Learning. [code]
  4. [4]
    Shengjia Zhao, Jiaming Song, Stefano Ermon Towards deeper understanding of variational autoencoding models Preprint, arXiv:1702.08658.

2016

  1. [3]
    Jiaming Song, Zhe Gan, Lawrence Carin Factored Temporal Sigmoid Belief Networks for Sequence Learning ICML 2016, In International Conference on Machine Learning.
  2. [2]
    Bei Chen, Ning Chen, Jun Zhu, Jiaming Song, Bo Zhang Discriminative nonparametric latent feature relational models with data augmentation AAAI 2016, In AAAI Conference on Artificial Intelligence.
  3. [1]
    Jun Zhu, Jiaming Song, Bei Chen Max-margin Nonparametric Latent Feature Models for Link Prediction Preprint, arXiv:1602.07428.

Professional Services

Journal reviewer: IEEE TPAMI, JAIR, IEEE TIT, ACM TIST, JMLR

Conference reviewer / Program committee: ICML (2019, 2020, 2021), NeurIPS (2019, 2020, 2021), ICLR (2018, 2019, 2020, 2021), COLT (2019), UAI (2019, 2020, 2021), CVPR (2020, 2021), ECCV (2020), ICCV (2019, 2021), AAAI (2021), ACML (2018, 2019), WACV (2020)

Workshop organization:


Awards and Fellowships
  • ICLR 2022 Outstanding Paper Award
  • Qualcomm Innovation Fellowship (QInF 2018, 4.6%)
  • Stanford School of Engineering Fellowship (2016)
  • Google Excellence Scholarship (2015)
  • Outstanding Undergraduate, China Computer Federation (2015)
  • Outstanding Winner, Interdisciplinary Contest in Modeling (2015, 0.4%)
  • Zhong Shimo Scholarship (2013, 0.75%)

Teaching
  • CS228, Probablistic Graphical Models (Winter 2020, Head TA)
  • CS236, Deep Generative Models (Fall 2018, TA)

Acknowledgements: based on the al-folio template by Maruan Al-Shedivat.