Fundamental Numerical Methods for Model Parameter Estimation

A Gentle Hands-On Introduction

Block course at

The Sirindhorn International Thai-German Graduate School of Engineering,

February 3-7,2025.

Goto

Abstract

Numerical simulation is a widely used tool for understanding phenomena in
virtually all areas of science, engineering, economics, finance, ... Numerical
solutions for mathematical models of reality are predominantly implemented
as computer programs. In most cases, free parameters are introduced to
account for uncertainty. The latter results from abstraction, simplification and
approximation due to mathematical modeling and numerical algorithms.

The desired values of the free parameters are typically unknown. Given
estimates often suffer from substantial inaccuracy. Observations of reality
can be used to calibrate the free parameters with the objective of improved
representation of reality by its numerical simulation.

Numerical methods for parameter estimation aim to determine values for
the free parameters such that the discrepancy between the given observations
and the corresponding simulated results of the model is minimized. Both algebraic
and differential models are widely used. General-purpose nonlinear programming
as well as often more efficient custom regression methods can be employed.

This course presents a gentle introduction to fundamental numerical methods
for parameter estimation. Emphasis is on the thorough understanding of the concepts
presented. Hands-on implementation is expected to contribute positively
to the learning experience. The material is likely to exceed what can reasonably be
"digested" during the time allocated for the course. The remainder will be condensed
into an outlook.

A very small and simple subset of the programming language C++, referred to as SNC++
(Scripting for Numerics with C++), is used for reference implementations as well as
for numerous interactive hands-on exercise sessions. All concepts are introduced for
the scalar case allowing for geometric illustration and hence easier understanding.
Generalization for the non-scalar case follows quite naturally. A particular focus is on
differentiability of the numerical simulation programs to enable the use of derivative-based
nonlinear programming and regression methods as well as to facilitate robustness analysis
of the solutions obtained. Algorithmic differentiation (e.g., known from backpropagation
for training deep artificial neural networks) is introduced as the method of choice.

Prerequisites

All essential mathematical concepts will be explained.                                           

Participants should have access to a computer running VirtualBox
(www.virtualbox.org). An image of a simple Linux system with all required
software preinstalled is provided along with instructions on how to validate
correctness of the installation prior to the start of the course.

Contents

  • Introduction. Motivation. Story in a Nutshell
  • Essential Computer Arithmetic
  • Scalar Case
    • Models
    • Simulation
      • Explicit Euler Method
      • Implicit Euler Method
        • Bisection Method
        • Newton Method
    • Optimization
      • Bisection Method
      • Gradient Descent Method
        • Symbolic Differentiation
        • Finite Differences
        • Algorithmic Differentiation (with dco/c++)
      • Linear Regression Methods
        • Normal Equation Method
        • Householder Method
      • Newton Method
      • Nonlinear Regression Methods
        • Normal Equation Method
        • Householder Method
  • Vector Case (all of the above)

Material

  • slides
  • sample code (unpack in ~/TGGS_0225/; both TGGS_0225/thirdParty and TGGS:0225/set_BASE_DIR.sh should be there already)

Getting Started

  1. Make sure that you have approx. 10GB of disc space available.
  2. Install and start VirtualBox (see www.virtualbox.org for guidance).
  3. Download the compressed VirtualBox image (stce-teaching.zip).
    This may take a while (due to size of 3.1GB) depending on your Internet connection.
  4. Unpack to get the two files stce-teaching.vbox and stce-teaching.vdi.
  5. Open the file stce-teaching.vbox with VirtualBox (double-click).
  6. Start the virtual machine "stce-teaching".
    • Your user name is "stce", so is your password.
    • Click on the "Terminal Emulator" icon to open a Linux shell.
    • Type "cd ../TGGS_0225" at the command line.
    • Type ". set_BASE_DIR.sh" (mind the dot) at the command line.
    • Type "make" at the command line. Numerous programs
      should be built and run without errors. This may take several minutes. 
    • Type "make clean" to remove all files generated by the build process.
    • Click on the "Web Browser" icon to run the chromium Web browser.
      Open, for example, this page to validate Internet access.
    • Familiarize with a text editor of your choice (gedit, nano, vi are preinstalled).
      You will need one of them to complete the exercises during the course.
    • Optionally, you may want to take a look at some basic Linux commands, e.g. here.
  7. Any problems? Contact: info@stce.rwth-aachen.de

Scripting for Numerics with C++