Multiresolution in image restoration

Summary

Image restoration refers to the set of techniques aimed at recovering a clean and high–quality image from a degraded observation. In many real–world scenarios, images are affected by noise, blur, missing data, or distortions introduced by acquisition devices, transmission channels, or environmental conditions. The goal of restoration is to model these degradation processes and reconstruct the original image as faithfully as possible. This requires combining mathematical models, optimization methods, and prior knowledge about natural images. Image restoration plays a central role in applications such as medical imaging, remote sensing, photography, and computer vision, where visual quality and accurate reconstruction are essential.

In the course we will give an overview of the possible optimization frameworks for the solution of such problems: from the classical variational formulation as the sum of two possibly non smooth functions, to modern paradigms based on neural networks (plug and play and unrolled methods). In image restoration, it is essential to rely on convergent algorithms. We will therefore present an overview of the main approaches used in this field, ranging from standard smooth optimization methods to proximal algorithms and accelerated schemes. Particular attention will be given to their convergence analysis (sequence convergence, complexity, and convergence rates).

The course will also have a special focus on multiresolution. Since the foundations of wavelet theory were laid in the 1980’s, multiscale analysis has fruitfully irrigated all domains of signal and image processing. The goal of this course is to present the main recent research axes in image processing appropriately combining multiscale analysis and non smooth optimization schemes. A specific focus will be made on the benefit of multiscale approaches for the design of fast and efficient, possibly unrolled, optimization algorithms both for large scale image reconstruction.

At the end of the course, you will be able to:

• Formulate imagining reconstruction problems as optimization problems
• Analyze rigorously the convergence theory of proximal methods
• Identify opportunities to harness multiresolution
• Develop efficient implementations of these algorithms
• Analyze the performance of these algorithms at scale
• Design model-based neural network architectures and analyze their robustness
• Apply these algorithms to the solution of various important real-life problems

Organization

• Lecturers: Elisa Riccietti (ENS Lyon, LIP) and Nelly Pustelnik (ENS Lyon, CNRS, Physics department).
• Duration: 15 lectures of 2 hours.
• Evaluation: Practical python project guided by the reading of a research article, and practical home exercises between lectures (5 required).

Outline

1. Introduction to inverse problems in imaging and image restoration (motivation, key applications, types of degradations, overview of mathematical modeling, evolution of the domain, overview of the course and practicalities of the course), variational formulation of inverse problems (data fidelity terms, regularization principles), total Variation (TV)
2. Sparsity-based regularization models and non-smooth optimization basics (proximal operator, sub gradients, convexity)
3. Proximal methods (I) (FB) + convergence analysis
4. Conjugate, duality
5. Proximal methods (II) (DR, ADMM, primal-dual)
6. Acceleration techniques
7. Multiresolution analysis: wavelets
8. Plug-and-Play (PnP) Methods
9. Deep unrolled optimization networks and model-driven deep networks
10. Challenge of the dimension: possible approaches
11. Multilevel methods (1) (PDEs, smooth optimization)
12. Multilevel methods (2) (Coarse-to-fine strategies, hierarchical and pyramid-based optimization)
13. Multiscale deep learning approaches (U-Net architectures, wavelet-based neural networks, multilevel PnP)
14. Bonus (TBD)
15. Bonus (TBD)

Resources