feat: v1 semantic convention

[FBumann]

The strict v1 semantic convention for linopy — predictable coordinate alignment and NaN handling.

This is the master PR for the new semantic convention in linopy. It starts with our Design & transitioning goals, which is carried out in our New Semantics spec. Both files are tracked in this branch. WHat you read is the current state.

Both might change until this PR is merged

Scope

The convention ships behind linopy.options["semantics"] — v1 opt-in, legacy the default. This PR carries the design, spec, tests and implementation; documentation notebooks follow separately.

Testing

All tests in linopy will be executed for both semantics.
Differing behaviour will be tested using pytest.markers.

This will increase ci time temporarily until v1 is released.

Defered to a follow up PR

Docs: Documentation, Migration guide, Release notes

[FBumann]

Coming next, in order:

• Slice F — §11 auxiliary-coordinate conflicts. Raise on non-dimension-coord conflict during alignment (covers Auxiliary non-dimension coordinates leak into expressions and break alignment #295). Small.
• Slice G — §13 reductions audit. Most reductions already use skipna=True; need a focused audit + tests for sum / mean / groupby / resample / coarsen and the objective. Likely mostly tests.
• Slice P — linopy/piecewise.py and linopy/sos_reformulation.py. Internal callers build expressions by .isel(...)-slicing along a piece dim and comparing the two slices (delta_hi <= delta_lo); the slices share the dim with different coords, which v1 §8 rejects. Until this lands, test_piecewise_constraints.py + test_piecewise_feasibility.py carry a module-level pytestmark = pytest.mark.legacy and two SOS2 tests are method-marked legacy. Fixing piecewise removes those marks.

Then a final pass on docs (user-facing migration / rollout — deferred so far).

[FBumann]

@MaykThewessen
Feel free to PR a linopy.alignment_trace() context manager. Although if its not to complex i would prefer to have it as a documented helper in the docs instead of inside the codebase. But only if thats possible...

Also feel free to PR a context manager like 'with linopy.alignment(join="override")', although i would argue that you should wait until this PR is reviewed and merged.

I would defer performance polishes like the proposed short-circuits until we merge this, maybe even until we drop the legacy convention. THis will keep the convention more focused and readable as long as its not 100% proven.

[MaykThewessen]

Good call — docs recipe is tighter scope than a public linopy.alignment_trace() symbol. Less API surface to maintain, no docstring/signature compat to preserve across releases, users can copy-and-tweak for their own warning-filter needs.

Planned shape: a short notebook under doc/ (next to existing examples) titled "Migrating from legacy to v1 semantics" with three sections:

1. Set up a trace context. ~20 lines: warnings.catch_warnings() + simplefilter("always", LinopySemanticsWarning) + a record list, wrapped in a contextlib.contextmanager.
2. Parse warnings into a DataFrame. Regex extracts divergence_class from the canonical message prefix ("Coordinate mismatch" / "treated as 0" / "divisor.*treated as 1" / "'…'.*fillna"), plus stacklevel-walk for (file, line). ~30 lines.
3. Report patterns. df.groupby(["file", "line", "divergence_class"]).size() collapsing 50k repeats to a punch list.

End result reads as a recipe a user runs once during opt-in, gets a punch list, fixes top-N sites, re-runs until empty. I'll wait for the spec-drift items from the line review to settle so the regex catalogue matches the final message texts, then PR. Sound good?

[MaykThewessen]

Thanks @FBumann — that ordering works for me.

Plan:

Hold alignment(join="override") ctx mgr and the short-circuits until #717 merges (and revisit short-circuits post legacy removal).
For alignment_trace: try a pure-stdlib docs snippet first, leaning on the existing LinopySemanticsWarning channel — no new public API needed. Sketch below. If reviewers think it deserves to live in linopy/semantics.py after all (e.g. for stacklevel correctness across the call sites), happy to promote it then.
Proposed docs snippet (drop into the migration guide, not the package):

import warnings
from collections import Counter
from contextlib import contextmanager

from linopy.semantics import LinopySemanticsWarning


@contextmanager
def alignment_trace(*, summary: bool = True):
    """Capture every LinopySemanticsWarning raised inside the block.

    Use during migration to surface where legacy alignment/NaN behaviour
    is being relied on, without touching the codebase under test.
    """
    with warnings.catch_warnings(record=True) as caught:
        warnings.simplefilter("always", LinopySemanticsWarning)
        yield caught

    if summary:
        hits = Counter(
            (w.filename, w.lineno, str(w.message).splitlines()[0])
            for w in caught
            if issubclass(w.category, LinopySemanticsWarning)
        )
        for (fname, lineno, msg), n in hits.most_common():
            print(f"{n:>4}× {fname}:{lineno}  {msg}")

Usage in migration docs:

with alignment_trace() as warns:
    m = build_model()      # legacy semantics still default
    m.solve()

→ ranked list of call sites still relying on legacy NaN/align behaviour

Lets users sweep a real PyPSA-Eur / linopy workflow once, get a ranked punch-list of sites to fix, then flip linopy.options["semantics"] = "v1". Zero surface area in the library.

I'll open the docs PR after #717 lands so the import paths and warning class are stable.

[FBumann]

Blocked by #732

[FBumann]

For reference: Needed changes in flixopt to pass under the new v1 convention. With those changes, the code works under both v1 and legacy

[FabianHofmann]

@FBumann while working on #732 and trying to align the multiindex handling with the new convention, a gap in the convention revealed: The alignment on coords and dims does not only count for two linopy objects but for also for operations with non-linopy objects. IE. when performing a arithmetic operation between a linopy object and a non-linopy array-object, all expectations and conventions should be applied on the non-linopy object too. likely worth defining the object scope.

[FBumann]

@FBumann while working on #732 and trying to align the multiindex handling with the new convention, a gap in the convention revealed: The alignment on coords and dims does not only count for two linopy objects but for also for operations with non-linopy objects. IE. when performing a arithmetic operation between a linopy object and a non-linopy array-object, all expectations and conventions should be applied on the non-linopy object too. likely worth defining the object scope.

@FabianHofmann I think this should currently hold, as linopy owns the arithmetics. Did you find a case where this doesnt?

[FabianHofmann]

@FBumann while working on #732 and trying to align the multiindex handling with the new convention, a gap in the convention revealed: The alignment on coords and dims does not only count for two linopy objects but for also for operations with non-linopy objects. IE. when performing a arithmetic operation between a linopy object and a non-linopy array-object, all expectations and conventions should be applied on the non-linopy object too. likely worth defining the object scope.

@FabianHofmann I think this should currently hold, as linopy owns the arithmetics. Did you find a case where this doesnt?

it's true, just explicitly naming that in the conventions would be beneficial. other objects have to first go via as_dataarray and then arithmetics are performed. that is as_dataarray must be fully aligned with the convention in order to get object-agnostic behavior, ie. operations between expr and arr must behave exactly the same as operations between expr and arr_expr where arr_expr is a linear expression which only holds constants of the same shape as arr. this needs to hold for commutativity, associativity, distributivity.