Cost expression refactor, parameter broadcasting, and PF headroom slack porting by acostarelli · Pull Request #112 · Sienna-Platform/PowerOperationsModels.jl

acostarelli · 2026-05-06T19:33:30Z

Sienna-Platform/PowerSimulations.jl#1564
Sienna-Platform/PowerSimulations.jl#1609
Sienna-Platform/PowerSimulations.jl#1566

Requires IOM on Sienna-Platform/InfrastructureOptimizationModels.jl#97

Co-Authored-By: Claude Opus 4.7 <[email protected]>

Copilot

Pull request overview

This PR ports PowerSimulations.jl cost-expression refactors, parameter broadcasting fixes, and power-flow-in-the-loop (PFitL) functionality into PowerOperationsModels.jl, including headroom-proportional slack participation and expanded cost-expression reporting.

Changes:

Refactors device constructors to register cost expressions via a new add_cost_expressions! bundle and expands exported cost-expression types.
Reworks time-series parameter population/multiplier handling (including a regression fix for parallel-branch multipliers) using IOM fast-path setters.
Implements PFitL in PowerFlowsExt (input mapping, data update, solve/aux readback, headroom slack) and adds new regression/coverage tests.

Reviewed changes

Copilot reviewed 24 out of 25 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
test/test_power_flow_in_the_loop.jl	Adds PFitL tests including headroom-proportional slack coverage and baseline PFitL scenarios.
test/test_parallel_branch_parameter_multipliers.jl	Adds regression test ensuring per-branch multiplier rows are populated (no NaNs) for parallel branches.
test/test_model_decision.jl	Updates expected expression counts and realized output export assertions for the expanded cost-expression set.
test/test_device_thermal_generation_constructors.jl	Expands thermal cost tests to validate constituent decomposition and startup-cost behavior.
test/test_device_renewable_generation_constructors.jl	Adds nonnegativity test for `CurtailmentCostExpression`.
test/Project.toml	Pins test dependency rev of InfrastructureOptimizationModels to the required branch.
src/static_injector_models/thermalgeneration_constructor.jl	Switches from individual `add_expressions!` calls to `add_cost_expressions!` for thermals.
src/static_injector_models/source_constructor.jl	Switches sources to `add_cost_expressions!`.
src/static_injector_models/renewablegeneration_constructor.jl	Switches renewables to `add_cost_expressions!`.
src/static_injector_models/renewable_generation.jl	Adds curtailment-cost hook in renewable objective building.
src/static_injector_models/load_constructor.jl	Switches loads to `add_cost_expressions!`.
src/static_injector_models/hydro_generation.jl	Uses IOM internal setters for faster parameter/multiplier writes in hydro parameter population.
src/PowerOperationsModels.jl	Adds explicit imports/exports for cost-expression bindings and includes new objective helper file.
src/core/interfaces.jl	Improves `add_power_flow_data!` fallback behavior and error message when PowerFlows extension isn’t loaded.
src/common_models/objective_function.jl	Introduces renewable curtailment-cost expression generation.
src/common_models/market_bid_plumbing.jl	Routes VOM costs into `VOMCostExpression` via updated IOM helper signature.
src/common_models/add_to_expression.jl	Refactors fuel-consumption expression construction (dispatch-based methods) and compact-formulation handling.
src/common_models/add_parameters.jl	Reworks time-series parameter/multiplier population via IOM fast setters and fixes parallel-branch multiplier axis bug.
src/common_models/add_expressions.jl	Adds `add_cost_expressions!` bundles and fuel-curve predicates for container setup decisions.
Project.toml	Pins main dependency rev of InfrastructureOptimizationModels to the required branch.
ext/PowerFlowsExt/PowerFlowsExt.jl	Sets up the PowerFlows extension module and includes PFitL implementation files.
ext/PowerFlowsExt/pf_input_mapping.jl	Implements PF input key mapping and aux-var container creation for PFitL.
ext/PowerFlowsExt/pf_data_update.jl	Implements writing optimization results into PF data/system for PFitL.
ext/PowerFlowsExt/pf_headroom.jl	Implements headroom-proportional slack participation factor recomputation.
ext/PowerFlowsExt/pf_solve_and_aux.jl	Implements PF solve dispatch and aux-variable readback from PF results.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

acostarelli · 2026-05-07T18:59:29Z

+    for t in get_time_steps(container)
+        offer_max = _renewable_offer_max(container, component, name, t)
+        dispatch = dispatch_vars[name, t]
+        curtailment_cost = proportional_term_per_unit * dt * (offer_max - dispatch)


@m-bossart This was also missing from your PR, is this intentional?

github-actions · 2026-05-06T20:07:51Z

Performance Results

Version	Precompile Time
Main	2.968320492
This Branch	2.961960465

Version	Build Time
Main-Build Time Precompile	102.570013626
Main-Build Time Postcompile	13.586854927
This Branch-Build Time Precompile	102.880881618
This Branch-Build Time Postcompile	13.640438019

Version	Solve Time
Main-Solve Time Precompile	412.175637142
Main-Solve Time Postcompile	377.694636645
This Branch-Solve Time Precompile	172.60790733
This Branch-Solve Time Postcompile	136.232628488

acostarelli · 2026-05-07T14:17:16Z

Closes #106 and #109

luke-kiernan

Big picture things:

Just generally: before adding code to POM, ask yourself "is there device specific handling here?" If the answer is "no," it probably belongs better in IOM. And there may be a helper for it there already.
All the expression stuff is currently in POM, not in IOM, whether device-specific or generic. That's purely due to copy-pasting from PSI, not due to intentional organization choices. We should probably change that at some point and move the generic implementations into IOM.
Test coverage. I ran the tests locally and they pass, but with the level of complexity here, I wouldn't be surprised if some amount of the code isn't actually getting run. Lots of different multiple dispatch cases: e.g. _set_parameter_at with HDF5 storage.

luke-kiernan · 2026-05-07T17:43:41Z

    reduced_branch_tracker = get_reduced_branch_tracker(network_model)
    all_branch_maps_by_type = PNM.get_all_branch_maps_by_type(net_reduction_data)

    # TODO: Temporary workaround to get the name where we assume all the names are the same across devices.


Ick we should open an issue for this

luke-kiernan · 2026-05-07T17:56:25Z

+        if sos_status == SOSStatusVariable.NO_VARIABLE
+            JuMP.add_to_expression!(expression[name, t], P_min * prop_term_per_unit * dt)
+        elseif sos_status == SOSStatusVariable.PARAMETER
+            param = get_default_on_parameter(d)
+            bin = get_parameter(container, param, V).parameter_array[name, t]
+            JuMP.add_to_expression!(
+                expression[name, t], P_min * prop_term_per_unit * dt, bin)
+        elseif sos_status == SOSStatusVariable.VARIABLE
+            var = get_default_on_variable(d)
+            bin = get_variable(container, var, V)[name, t]
+            JuMP.add_to_expression!(
+                expression[name, t], P_min * prop_term_per_unit * dt, bin)
+        else
+            @assert false
        end


Call _determine_bin_lhs in IOM instead: up to hard-coded var = OnVariable there vs var = get_default_on_variable(d) here, this is

bin = _determine_bin_lhs(container, sos_status, V, name, t) JuMP.add_to_expression!(expression[name, t], P_min * prop_term_per_unit * dt, bin)

luke-kiernan · 2026-05-07T18:07:09Z

+# rather than a runtime branch.
+
+# LinearCurve fuel: linear in the dispatch variable.
+function _add_fuel_consumption_term!(


Identical to IOM's add_proportional_cost_invariant!, except we skip the final step where we add to the objective function and to the ProductionCostExpression. Refactor?

- Rename solve_powerflow! → solve_power_flow! to match PowerFlows naming (kiernan). - Fix aux-var readback guard to compare AuxVarKey entry-type, not the whole key, against branch_aux_vars / bus_aux_vars; add a regression test covering an unsupported PowerFlowAuxVariable key. - Rename _get_variable_if_exists → _get_cost_if_exists so it no longer reads as an optimization-variable accessor. - Route LinearCurve fuel cost and renewable curtailment cost through the new IOM helpers add_cost_term_to_expression! and add_cost_term_invariant!; route compact-fuel SOS dispatch through IOM._determine_bin_lhs to drop the duplicated three-way branch. - Replace Ref(x) with singleton tuples (x,) in broadcast calls for stack allocation; drop unnecessary Float64(multiplier) casts. - Tighten devices argument on _add_time_series_parameters! to Union{Vector{D}, IS.FlattenIteratorWrapper{D}}. Co-Authored-By: Claude Opus 4.7 <[email protected]>

`findlast` returns `nothing` when no `PowerFlowEvaluationData` entry has solved, which previously bubbled up as an opaque `MethodError` from `datas[nothing]`. Raise an explicit error and mark the broader handling as a FIXME pending the PF-failure design discussion. Addresses kiernan's review comment on PR #112. Co-Authored-By: Claude Opus 4.7 <[email protected]>

luke-kiernan

Looks good. I'm noticing the following sequence of calls in a couple different places

power_units = PSY.get_power_units(var_cost)
proportional_term = PSY.get_proportional_term(value_curve)
prop_term_per_unit = get_proportional_cost_per_system_unit(
    proportional_term, power_units, base_power, device_base_power)
rate = prop_term_per_unit * dt # sometimes also * multipler.

but that's something we can address later

Anthony Costarelli and others added 8 commits May 5, 2026 15:48

claude port

0ff1f94

formatting

70a4967

claude port

98ae53b

claude port

27cb611

ported more pf tests

060bbac

Merge ac/cost-expression-refactor

0ab2bf4

Merge ac/headroom-slack (incl. ac/fix-parameter-broadcasting)

9369016

Point IOM source to consolidated branch for combined PR

e517c2e

Co-Authored-By: Claude Opus 4.7 <[email protected]>

acostarelli requested a review from Copilot May 6, 2026 19:33

Copilot started reviewing on behalf of acostarelli May 6, 2026 19:34 View session

Copilot AI reviewed May 6, 2026

View reviewed changes

luke-kiernan reviewed May 7, 2026

View reviewed changes

Anthony Costarelli and others added 2 commits May 11, 2026 10:58

acostarelli requested review from jd-lara and luke-kiernan and removed request for jd-lara and luke-kiernan May 11, 2026 16:50

luke-kiernan approved these changes May 11, 2026

View reviewed changes

move bustype check inside loop

0ba657d

luke-kiernan mentioned this pull request May 12, 2026

Lk/flow sign trait #117

Open

This was linked to issues May 12, 2026

Move over PSI PR #1609 #106

Open

Implement power flow in-the-loop #109

Open

Conversation

acostarelli commented May 6, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

acostarelli May 7, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

acostarelli commented May 7, 2026

Uh oh!

luke-kiernan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

luke-kiernan May 7, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

luke-kiernan May 7, 2026

Choose a reason for hiding this comment

Uh oh!

luke-kiernan May 7, 2026

Choose a reason for hiding this comment

Uh oh!

luke-kiernan left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

github-actions Bot commented May 6, 2026 •

edited

Loading