⚠️ Warning

This repository is currently part of a pending patent process. As a result, we cannot release the full code at this moment. We will continuously update the code with the portions we are allowed to share. Once the patent is filed and we are permitted to release the full codebase, it will be made available here as open-source.

At this point we recommend users to wait for the full release inside an exemplery simulator set up, but if wanted we provide instructions for users to include their own MPC formulation and vehicle simulator in the following.

Diff-WMPC

Diff-WMPC is a framework for the development of Differentiable Nonlinear Model Predictive Controllers (Diff-NMPC) for Autonomous Vehicle Motion Control.

This respository acommpanies the RA-L submission: "Differentiable Weights-Varying Nonlinear MPC via Gradient-based Policy Learning: An Autonomous Vehicle Guidance Example"

MPC Implementations

Diff-WMPC includes the following MPC implementations:

• Differentiable NMPC - Diff-MPC (Single Weight Set): A differentiable approach with a fixed weight set, that updates once per lap.
• Differentiable Weights-Varying NMPC - Diff-WMPC: A novel differentiable approach with dynamic weight adaptation.

📋 Workflow

1. Load MPC parameters.
2. Load reference trajectory.
3. Initialize the vehicle state and simulator.
4. Configure the MPC solver.
5. Start the simulation loop:
   • Update the reference trajectory.
   • Solve the MPC problem and log the solution.
   • Learn and update the MPC cost weights.
   • Perform a simulation step and update the vehicle state.

🔧 Installation

System Requirements

• Tested Operating System: Ubuntu 22.04 LTS
• Python Version: Python 3.10.12

The stack is tested on a desktop with an AMD CPU, as well as a laptop with an Intel CPU. The runtime experiments are conducted on the AMD Ryzen 7950X CPU.

Installation Steps

1. Set up a Conda environment:

   conda create --name diff-wmpc python=3.10
   conda activate diff-wmpc

2. Clone the repository:

   Download and unpack the Repository ZIP from: https://anonymous.4open.science/api/repo/Diff-WMPC/zip

3. Install Python dependencies:

   cd Diff-WMPC/
   pip install -r requirements.txt

4. Install Acados:

   cd ..
   git clone https://github.com/acados/acados.git
   cd acados
   git submodule update --recursive --init
   mkdir -p build
   cd build
   cmake -DACADOS_WITH_QPOASES=ON ..
   make install -j4

5. Install and link the Acados Python interface:

   cd $HOME/acados
   pip install -e interfaces/acados_template
   export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:"/home/$USER/acados/lib"
   export ACADOS_SOURCE_DIR="/home/$USER/acados"

   To avoid exporting the paths in every new terminal, it is possible to add them to the .bashrc file:

   echo 'export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:"/home/$USER/acados/lib"' >> ~/.bashrc && echo 'export ACADOS_SOURCE_DIR="/home/$USER/acados"' >> ~/.bashrc

🔌 Integrating Your Own MPC

Overview

To integrate your own MPC formulation into the Diff-WMPC learning framework, you need three high-level components:

1. MPC Controller: Solves the Optimal Control Problem (OCP) at each timestep
2. Vehicle Simulator: Calculates vehicle states and provides the next timestep
3. Reference Trajectory: Defines the desired path to follow

High-Level Structure

• MPC Controller: Takes the current state and reference trajectory, solves the OCP, and returns optimal control inputs and predictions
• Vehicle Simulator: Takes control inputs, simulates vehicle dynamics, and returns the next state
• Reference Trajectory: Provides position, velocity, curvature, and other reference values along the desired path

Required Modifications

To use your own components, modify the following sections:

1. Configuration Files

Diff-MPC.py (Lines 60-67) | Diff-WMPC.py (Lines 60-67)

# Add your simulation and MPC configuration files
sim_main_params_file = config_path + "<your_vehicle>/sim_main_params.yaml"
MPC_params_file = "<your_vehicle>/MPC_params.yaml"

Action Required: Replace <your_vehicle> with your vehicle configuration folder path.

2. Simulation Parameters

Diff-MPC.py (Lines 70-72) | Diff-WMPC.py (Lines 70-72)

# Set the simulation time and starting index
sim_main_params['T'] = 1800
sim_main_params['idx_ref_start'] = -150

Action Required: Adjust simulation time T and starting index idx_ref_start for your scenario.

3. Vehicle Simulator Initialization

Diff-MPC.py (Lines 144-146) | Diff-WMPC.py (Lines 95-97)

# Add your vehicle model here
Passenger_Vehicle = PassengerVehicleSimulator(
    config_path, sim.sim_main_params, sim.Ts, sim_stm)

Action Required: Replace with your own vehicle simulator class that implements:

• State initialization
• Dynamics simulation
• Control input handling

4. MPC Solver Initialization

Diff-MPC.py (Lines 165-167) | Diff-WMPC.py (Lines 146-148)

# Create the MPC controller
MPC = Model_Predictive_Controller(
    config_path, MPC_params, sim.sim_main_params, sim.X0_MPC, pred_stm)

Action Required: Replace with your own MPC controller that implements:

• OCP formulation
• ACADOS solver interface
• Sensitivity extraction (see below)

5. Reference Trajectory

Diff-MPC.py (Lines 226-229) | Diff-WMPC.py (Lines 177-180)

# Add your trajectory file here
segments_file_path = "Trajectories/<your_trajectory>.json"
with open(segments_file_path, "r") as file:
    track_segments = json.load(file)

Action Required: Replace <your_trajectory> with your trajectory file path.

6. Planner Emulator

Diff-MPC.py (Lines 231-233) | Diff-WMPC.py (Lines 182-184)

# Add your own planner emulator here
PlannerEmulator = PlannerEmulatorClass(track_segments, sim.sim_main_params)

Action Required: Implement a planner that extracts the local reference trajectory for the MPC horizon.

7. Reference Trajectory Extraction

Diff-MPC.py (Lines 272-278) | Diff-WMPC.py (Lines 223-229)

# Set up your planner emulator to extract the reference trajectory
current_ref_idx, current_ref_traj = PlannerEmulator(
    sim.ref_traj_set, sim.current_pose, sim.N+1, sim.Tp, loop_circuit=True)

Action Required: Ensure your planner returns the reference trajectory in the expected format.

8. MPC Solve Call

Diff-MPC.py (Lines 286-293)

# Add your own MPC solve call here
u0, pred_X, MPC_stats, MPC_sensitivities, sensitivity_time, adjoint_time = MPC.solve(
    current_ref_traj, sens_shooting_node=sens_shooting_node, diffmode=diffmode)

Diff-WMPC.py (Lines 237-244)

# Add your own MPC solve call here
u0, pred_X, MPC_stats, MPC_sensitivities, sensitivity_time, adjoint_time = MPC.solve(
    current_ref_traj, sens_shooting_node=sens_shooting_node, diffmode=diffmode)

Action Required: Implement the solve() method to return:

• u0: Optimal control input
• pred_X: Predicted state trajectory
• MPC_stats: Solver status
• MPC_sensitivities: Sensitivities (see ACADOS requirements below)
• sensitivity_time, adjoint_time: Timing metrics

9. MPC Reinitialization

Diff-MPC.py (Lines 305-307)

# Add your own MPC reinitialization and reset call
MPC.reintialize_solver(x_next)
MPC.reset(x_next)

Diff-WMPC.py (Lines 256-258)

# Add your own MPC reinitialization and reset call
MPC.reintialize_solver(x_next)
MPC.reset(x_next)

Action Required: Implement solver reinitialization for failure recovery.

Interface Requirements Summary

Your MPC controller must:

• Accept current state and reference trajectory
• Solve the OCP using ACADOS with forward sensitivity calculation enabled
• Return optimal controls, predictions, solver status, and sensitivities
• Support reinitialization for failure recovery

Your vehicle simulator must:

• Accept control inputs and current state
• Simulate vehicle dynamics for one timestep
• Return next state

Your reference trajectory must:

• Provide position, velocity, curvature, and other required references
• Support extraction of local horizon for MPC

🔬 ACADOS Sensitivity Extraction

For the learning framework to compute gradients, your MPC solver must provide forward sensitivities at a specified shooting node. This is critical for backpropagation through the MPC.

Prerequisites

Before extracting sensitivities, your ACADOS OCP formulation must satisfy the following requirements:

1. External Cost Function

ocp.cost.cost_type = 'EXTERNAL'    # Cost type at intermediate shooting nodes (1 to N-1)
ocp.cost.cost_type_e = 'EXTERNAL'  # Cost type at terminal shooting node (N)

2. Discrete Dynamics

ocp.solver_options.integrator_type = "DISCRETE"

3. Exact Hessian Approximation

ocp.solver_options.hessian_approx = 'EXACT'
ocp.solver_options.exact_hess_constr = 1
ocp.solver_options.exact_hess_dyn = 1

4. GG-Exponent Constraint

⚠️ ACADOS Bug Warning: ACADOS will only output sensitivities if the gg_exponent in your dynamics file is set to 1.0 or 2.0. Ensure your model definition uses one of these values.

Enabling Sensitivity Computation

Once the prerequisites are satisfied, enable sensitivity extraction with:

ocp.solver_options.with_solution_sens_wrt_params = True

Using Global Parameters

Critical: ACADOS only outputs sensitivities for global consistent parameters. This means:

• Parameters cannot change over the prediction horizon
• Parameters must be stored in ocp.model.p_global (not ocp.model.p)
• Cost weights should be defined as global parameters in your model

Extracting Sensitivities During Solve

In your MPC solve() method, extract sensitivities using:

sensitivity_dict = self.acados_solver.eval_solution_sensitivity(
    sens_shooting_node,      # Shooting node index (1 in our case)
    "p_global",              # Parameter type
    return_sens_x=True,      # Return state sensitivities
    return_sens_u=True,      # Return control sensitivities
    return_sens_pi=False,    # Skip dual variables
    return_sens_lam=False,   # Skip dual variables
    return_sens_su=False,    # Skip slack variables
    return_sens_sl=False,    # Skip slack variables
    sanity_checks=True       # Enable validation checks
)

Weight Dimension Note

⚠️ Important: Our current setup assumes 4 state weights and 2 control weights (6 total). If your MPC formulation uses a different number of weights, you must:

1. Adjust the p_global dimension in your OCP model
2. Update the loss function to handle your weight structure
3. Modify the policy network output dimension if using Diff-WMPC

🚀 Diff-MPC and Diff-WMPC

Running Simulations

1. Navigate to the project directory:

   cd Diff-WMPC/

2. Run one of the main scripts based on the MPC formulation:

   • Single-Weight Set Differentiable NMPC:

     python Diff-MPC.py

   • Weights-Varying Differentiable NMPC:

     python Diff-WMPC.py

---

📊 Appendix A: Implementation Details

Parameter	Value/Setting
Learning Optimization
Optimizer	Adam
Learning Rate	2.9 × 10⁻⁵
Betas	(0.5, 0.99)
Batch Size (N_batch)	10 timesteps
Gradient Clipping	[-0.1, +0.1]
Loss Function Weights	α=2.5e-5, β=2.25, γ=2.5e-7, δ=9e-3
Policy Architecture
Network Type	Fully Connected
Input Features	Future Velocity ($v_{k+1 \dots k+5}$), Curvature ($c_{k+1 \dots k+5}$)
Hidden Layers	2
Neurons	128 per layer
Activation Function	Softplus (Output Only)
Output Dimension	6 (MPC cost weights q, r)
Initial Output Weights	q₀=[2.5, 2.9, 2.0, 5.0], r₀=[4.3, 6.8]
MPC Solver Settings
Cost Type	External Cost
Dynamics Model	Discrete Dynamics (RK4)
Implementation	C-Code Generation & Compilation
NLP Solver	SQP_RTI
QP Solver	FULL_CONDENSING_HPIPM
QP/KKT Tolerance	1e-6
Hessian Approximation	Exact
Sensitivity Calculation	Forward Sensitivities
Sensitivity Shooting Node	1
Control & Horizon Parameters
Prediction Horizon (T_p)	2.55 s
Discretization Time (T_s)	0.075 s
Shooting Nodes (N)	34