PY 580 Machine Learning for Physicists. Spring 2023

 



This is the website for PY 580, Machine Learning for Physicists This website will be updated with HWs and suggested readings. In the fall, the class will be based on our review A high-bias, low-variance introduction to Machine Learning for physicsts. The review can be downloaded from Physics Reports or the arXiv if for some reason you prefer that formatting. The Jupyter Notebooks can be downloaded from Github. We will probably do first 11 chapters before pivoting to understand how stable diffusion models like DALL-E and large language models like ChatGPT3 work.


General course information: TTh 9:30-11:00. SCI B58.


Installing Python:

The easiest way to install Python is using the Anaconda distribution: Anaconda.
Please also install Jupyter Notebooks and/or Jupyter Lab: Jupyter. Alternatively, you can run the notebooks on Google Colab.


Useful Resources:

David Mackay's Information Theory, Inference, and Learning Algorithms.

Abu Mustafa's Online Course: Learning from Data

Michael Neilsen's Neural Networks and Deep Learning

Python Packages: Python scikit-learn Library , PyTorch , TensorFlow


Syllabus, Grading, and Course Information: pdf


Weeks 1-2: Chapter 1-4/ Notebook 1-2.

Other Viewing + Readings: Lectures 1-3 from Learning from Data. Chapters 1-3 of Information Theory, Inference, and Learning Algorithms. IPython Cookbook fourth feature recipe: Introduction to Machine Learning in Python with scikit-learn. Due date Feb. 7th.

Weeks 2-3: Chapter 5-7 Notebooks 3-4.

Other Viewing + Readings: IPython Cookbook fourth feature recipe: Introduction to Machine Learning in Python with scikit-learn. Due date Feb. 14th.

Weeks 10-11: Chapter 9-11 .

Other Viewing + Readings: Tutorials on Pytorch and Keras. Also lecture on common architectures. Please do this Homework. Due date April 7th.

Weeks 12-13: Diffusion Models .

Readings: Thursday April 6th: Please read this theory paper. It might also be useful to first read Radford Neal's paper on Annealed Importance Sampling. Tuesday April 11: Please read this blog post and this paper (summarized nicely in this blog posts) and annotated code from Huggin Face here.

Weeks 14: Word2Vec+Lecture on Attention.

Readings: Tuesday April 18th: Please read this papers Wor2Vec. It might also be useful to first read these notes, and code+tutorials in Pytorch and Tensorflow . Thursday April 20th: Lecture on Attention.

Weeks 14: Word2Vec+Lecture on Attention.

Readings: Tuesday April 25th: Please read the famous Attention is all you need as well as the Illustrated Transformer. Thursday April 30th. Please read the BERT paper.

Weeks 15: The GPT Papers.

Readings: Tuesday May 2nd: Please read the famous GPT papers (GPT2, GPT3), and the flawed but sociologically important scaling law paper.