Course Information

Discover the exciting world of Generative AI with our comprehensive short course on Applied Generative AI Programming using LLaMA (Large Language Model Meta AI) and Python.

This full-day course is designed for enthusiasts and professionals eager to explore the capabilities of large language models. You’ll start with an introduction to Generative AI and its applications, followed by setting up your Python environment. Explore the architecture of LLaMA, learn data preparation techniques, and train your own LLaMA model. Fine-tune and optimise your model for specific tasks, and master the art of text generation. The course includes hands-on projects to apply your knowledge in real-world scenarios, along with discussions on ethical considerations and responsible AI use. By the end of this course, you’ll have a solid foundation in Generative AI and the skills to create innovative text generation applications.

Aims

The aim of this course is to equip participants with the knowledge and skills to effectively utilise LLaMA and Python for generative AI programming, enabling you to create innovative text generation applications.

Outcomes

By the end of this course, you should be able to:

• gain a comprehensive understanding of Generative AI and its applications
• consider ethical and responsible AI use and trustworthiness of results
• learn about the Large Language Model Meta AI (LLaMA) and its significance
• understand the architecture and components of LLaMA
• compare LLaMA with other generative models
• perform tokenisation and embedding
• create training and validation datasets
• configure training parameters and train the model on a dataset
• fine-tune the model for specific tasks
• generate text using the trained LLaMA model
• discuss retrieval-augmented generation (RAG) and ways to implement it
• control the output using temperature, top-k, and top-p sampling.

Content

1. Introduction to Generative AI

• Overview of Generative AI
• Applications and use cases
• Ethical and responsible AI use
• AI trustworthiness

2. Setting up the environment

• Introduction to Visual Studio Code
• Installing Python and necessary libraries
• Setting up a virtual environment

3. Understanding LLaMA

• Introduction to LLaMA
• Architecture and components of LLaMA
• How LLaMA works
• Comparison with other generative models

4. Data preparation

• Collecting and pre-processing data
• Tokenisation and embedding
• Creating training and validation datasets

5. Training LLaMA model

• Configuring the training parameters
• Training the model on a dataset
• Monitoring training progress and performance

6. Fine-Tuning and optimisation

• Fine-tuning the model for specific tasks
• Hyperparameter tuning
• Techniques for optimising performance
• Techniques for retrieval-augmented generation (RAG)

7. Generating text with LLaMA

• Using the trained model to generate text
• Controlling the output (temperature, top-k, top-p sampling)
• Evaluating the generated text

Intended audience

This course is suitable for data scientists, machine learning engineers, and software developers who want to enhance their skills in coding custom GenAI models. It requires a thorough understanding of the Python programming language and is not suited for beginner programmers.

Prerequisites

It is assumed you have completed the Python Programming 1B course or have equivalent knowledge in Python and Pandas programming.

Delivery style

This course is an interactive workshop which includes lectures, group exercises and discussion.

Delivery modes

• Face-to-face, presenter-taught training using your own device
• Online training via the platform Zoom

Materials

All course materials are provided electronically (via Dropbox). Printing services are not provided.

Before the course

Please download and install the following software prior to class:

• Visual Studio Code (or your preferred Python IDE like PyCharm or Spyder).

A Python extension is available through VS Code interface. No separate Python download required. Other required Python packages will be downloaded and installed in class.

Note – this course will rely on out-of-package, local large language models. No API/license for cloud based models (ChatGPT, etc) are required.

Recommended reading

Foster, D. (2019) Generative Deep Learning: Teaching Machines to Paint, Write, Compose, and Play. 1st ed. Sebastopol, CA: O’Reilly Media.

Mitchell, M. (2019) Artificial Intelligence: A Guide for Thinking Humans. 1st ed. London: Penguin Random House.

Chollet, F. (2018) Deep Learning with Python. 1st ed. Shelter Island, NY: Manning Publications.