An Introduction to running Large Language Models on Fox

This is an introductory course on running Large Language Models (LLMs) on the UiO High-Performance Computer Fox. LLMs, like ChatGPT, are trained to generate word sequences based on a context, i.e. continue word sequences. For instance, if we have the start of a sentence “The chicken crossed the road to get”, some possible completions are “to the other side” or “food”. To learn more about how LLMs are trained, we recommend the videos Intro to Large Language Models and Deep Dive into LLMs like ChatGPT by Andrej Karpathy.

In this course, we use Python programming to run LLMs. Therefore, the course requires basic programming skills. If you are unfamiliar with programming, we recommend doing a programming course before this course, for example the Software Carpentry course Plotting and Programming in Python.

This course consists of five chapters. In Getting Started, we log on to Fox. You will need a user account at the University of Oslo and a smart phone or other means of using two factor authentication (2FA).

In Installing Software, we install the software that we will need for this course.

In Querying LLMs (Chatbots), we learn to load and query basic LLMs.

In Summarization, we use LLMs for making shorter summaries of documents.

In Retrieval-Augmented Generation, we build an application for doing Retrieval-Augmented Generation based on a set of documents.