Optimization Theory with Applications

Abstract

Abstract

This course explores optimization techniques with a focus on convex, non-convex optimization, Stochastic Gradient Markov Chain Monte Carlo (SG-MCMC) methods, evolutionary algorithms, and reinforcement learning. The curriculum combines theoretical lectures with practical laboratories to equip students with both the knowledge and skills needed to apply optimization techniques in various fields.

Teaching and Learning Methods:

The course will be divided into 7 lectures of 3 hours each, with a combination of theoretical discussions and hands-on laboratory exercises. Each lecture will be structured to provide a blend of foundational theory, advanced topics and practical applications.

Course Policies:

Attendance to lectures and exercise sessions is not mandatory but highly recommended.

Bibliography

None.

Requirements

Multi-variate calculus, Basics of Proabability

Description

Topics Breakdown:

Introduction to Optimization

Theory:
- Introduction of convex optimization.
- Common techniques for convex problems.
- Introduction to non-convex optimization: challenges.
Laboratory:
- Introduction to python optimization software and tools.
- Solving basic convex optimization problems using numerical methods. (This is easy but shows how when you go non-convex life is harder)

Stochastic Gradient Descent (SGD) and its Variants

Theory:
- Review of SGD.
- Continuous time perspective: SGD as a stochastic differential equation.
- Variants of SGD: momentum, Nesterov acceleration, and Adam.
Laboratory:
- Implementation of SGD and its variants.
- Empirical comparison of different optimization techniques on benchmark problems.

Bayesian Optimization and Stochastic Gradient MCMC (SG-MCMC) Methods

Theory:
- Fundamentals of Bayesian optimization.
- Continuous time Markov processes.
- Introduction to SG-MCMC methods.
- Applications of SG-MCMC in optimization.
Laboratory:
- Implementing SG-MCMC methods.
- Practical applications of Bayesian optimization.

Optimization "Through the Void" (Gradient-Free Methods)

Theory:
- Introduction to gradient-free optimization methods.
- Subgradients: SGD variants
- Evolutionary algorithms: principles and applications.
- Perturbation-based parameter updates.
Laboratory:
- Implementation of evolutionary algorithms.
- Case studies: optimizing functions without gradients.

Reinforcement Learing applications to Optimization

Theory:
- Lack of an available loss function
- Risks of optimizing for unintended outcomes.
- Methods to incorporate human feedback
Laboratory:
- Practical implementation of RL with human feedback. Analyzing outcomes and potential biases in feedback-driven systems.

Training vs. Testing Loss Landscapes in Machine Learning

Theory:
- Understanding the differences between training and testing loss landscapes.
- Generalization and overfitting: causes and consequences.
- Techniques to mitigate overfitting: regularization, dropout, and cross-validation.
- Case studies: analyzing loss landscapes in real-world machine learning models.
Laboratory:
- Visualization and analysis of training vs. testing loss landscapes.
- Implementing regularization techniques to improve generalization.
- Evaluating model performance on different datasets.

Learning Outcomes:

Distinguish between convex and non-convex problems, solving basic convex tasks via numerical tools.

- Implement and tune stochastic gradient descent (SGD) variants, interpreting them through a continuous-time lens.

- Deploy Bayesian optimization and SG-MCMC for black-box objectives, grasping theoretical foundations and applications.

- Apply evolutionary algorithms and other gradient-free methods to non-differentiable tasks, comparing them with subgradients

- Handle optimization scenarios lacking explicit loss functions in reinforcement learning, incorporating human feedback while mitigating unintended outcomes.

- Differentiate training from testing loss landscapes, relate sharpness/flatness to overfitting, and employ regularization or visualization for better generalization.

Nb hours: 21.00

Evaluation:

Exam (100% of the final grade);
Bonus points for questions during the course