Affine Units Functionality

[Deduction42]

This adds a new type of Dimension (AffineDimension) that can be used to describe units with offsets (like Celsius and Fahrenheit). Like most other packages, operations on AffineDimensions are forbidden except subtraction. This is done by throwing an error on convert if the offset is zero; operations are allowed if the offset is zero (by using promote/convert to SI units).

[MilesCranmer]

Thanks! Can you walk me through the implementation a bit? How does it work, what are the fields used for, etc?

[MilesCranmer]

And FYI I would like to get things to 100% unit-test coverage if possible (particularly important as I'm unfamiliar with the code). My expectation is that there will be many unexpected errors that will pop out when going through this process.

[MilesCranmer]

I think the Aqua and JET tests will likely take a bit of work to get passing but are also important

[Deduction42]

Sure! As for how it works. Almost all of the changes are in "affine_dimensions.jl", which somewhat mirrors the patterns found in "symbolic_dimensions.jl" (or as much as reasonably possible). The basic structure is the AffineDimension <: AbstractAffineDimension <: AbstractDimension which has the following fields:

@kwdef struct AffineDimensions{R} <: AbstractAffineDimensions{R}
    scale::Float64 = 1.0
    offset::Float64 = 0.0
    basedim::Dimensions{R}
    symbol::Symbol = :nothing
end

The scale and offset parameters are used to convert this unit to its SI equivalent. So the SI quantitiy is basically

si_quantity = affine_quantity*scale + offset

This is essentially what uexpand(affine_quantity) does when it converts things to SI dimensions. The symbol field is optional and is for displaying units. If no symbol is present, it will merely display (scalse+offset basedim). For example:

kelvin  = AffineDimensions(scale=1.0, offset=0.0, basedim=u"K")
rankine = AffineDimensions(scale=5/9, offset=0.0, basedim=kelvin)
fahrenheit = AffineDimensions(scale=1.0, offset=459.67, basedim=rankine)
>> (0.5555555555555556+255.37222222222223 K)

But if we used

fahrenheit = AffineDimensions(scale=1.0, offset=459.67, basedim=rankine, symbol=:°F)
>>°F

[Deduction42]

Generally speaking, it's not encouraged to do mathematical operations in affine units because of potential ambiguities present in the offset. Promotion rules will attempt to convert affine quantities into basic dimensional (SI) quantities. However, the function "convert" will fail if the offset is not zero (I allow typical multiplication/division operations for affine units with zero offsets in order to make affine unit macros work with compound units like m/s).

The affine unit marcro is "ua" and I registered the Celsius and Fahrenheit units. So you could write

 uconvert(ua"°C", 0ua"°F")
>> -17.77777777777777 °C

Becuase AffineDimensions are more general than Dimensions and SymbolicDimensions, registering a unit will also make it available to the "ua" macro, and I registered all items in UNIT_SYMBOLS and UNIT_VALUES off the bat, so you could write:

velocity = ua"mm/s"
>>1.0 (0.001 m s⁻¹)

This generality is so that you could make a single Type-Stable container (like a dictionary or vector) that represents both Affine and Non-Affine units and easily convert them all to SI or Symbolic units. Note however, compound non-affine units break, becuase offsets cannot be multiplied:

ua"°C/K"
ERROR: AssertionError: AffineDimensions °C has a non-zero offset, implicit conversion is not allowed due to ambiguity. Use uexpand(x) to explicitly convert
Stacktrace:

[Deduction42]

Finally, there is a special macro for registering AffineDimensions only, because the regular registration macro doesn't handle offsets. Registration will apply to anything that can be evaluated as a dimension or quantity. For example, you could do:

@register_unit psi 6.89476us"kPa" #Regular registration method for PSI
@register_affine_unit psig AffineDimensions(offset=u"Constants.atm", basedim=u"psi") #Gauge-pressure subtracts atmospheric

uconvert(ua"psig", u"Constants.atm")
>> 0.0 psig

Generally, when building AffineDimensions, you want to be explicit about your "offset" units. If you use a numeric value, it will assume the same units as "basedim" which could be a quantity.

AffineDimensions(offset=1, basedim=0.5u"m") #Offset units are assumed to be half-meters "0.5m"
>>(0.5+0.5 m)

This may seem odd until we account for the fact that we could register half-meters so

@register_unit half_meter 0.5u"m"
AffineDimensions(offset=1, basedim=u"half_meter")
>>(0.5+0.5 m)

In such a case, it would be "least astonishing" that in the absence of units, the offset is in the same units as the base dimensions because no other units are mentioned. Building AffineDimesnions from other AffineDimensions will use the basedim's offset as a starting point, so you could do something like this:

kelvin  = AffineDimensions(scale=1.0, offset=0.0, basedim=u"K")
rankine = AffineDimensions(scale=5/9, offset=0.0, basedim=kelvin)
fahrenheit = AffineDimensions(scale=1.0, offset=459.67, basedim=rankine)
celsius = AffineDimensions(scale=9/5, offset=32, basedim=fahrenheit)

uconvert(Quantity(1.0, fahrenheit), Quantity(-40.0, celsius))
>>-39.99999999999996 °F

[MilesCranmer]

Quick question - why do affine dimensions need to support algebraic operations on non-offset affine dimensions? If it's non-offset, why not just force the user to immediately convert to SI units if they want to use such operations?

[Deduction42]

Anyway, I'll start writing the tests for this on Monday, I just wanted to make sure you were onboard with adding this before I put the effort into it. I was pleasantly surprised that this took me less than a week to do; your package has some fairly elegant ways to plug in this feature (although these plug-in points are a bit hard to find). To reiterate, almost all of the changes are in the new "affine_dimensions.jl" file I added. I also had to make some changes to register_units.jl to add automatic registering of any units to the AffineUnits registry, and make a separate macro for registering AffineDimensions only.

I do have some questions:

1. In the "test coverage" tools, is there any way to identify what lines are missing from the coverage? Or is this a game of educated whack-a-mole?
2. I noticed in "register_units.jl" that you use a SpinLock(), is there any reason you're using this rather than the more common (and more recommended) ReentrantLock()?
3. Last time I contributed, I was able to run all the tests by just running "run_tests.jl", but this time, there's a bunch of test-related dependencies. Is there a standard way of loading this environment before doing the tests, or do I have to just install all the test dependencies myself?

[Deduction42]

Quick question - why do affine dimensions need to support algebraic operations on non-offset affine dimensions? If it's non-offset, why not just force the user to immediately convert to SI units if they want to use such operations?

The only reason I do this is so that I can parse units like "ua"m/s"" as an Affine Unit. However, I could just as well do an automatic promotion to SI units and then evaluate the compound unit as Affine in the end if that's something you would prefer.

[MilesCranmer]

This is the command I usually run at the root of this repository:

julia --startup-file=no --depwarn=yes --threads=auto --code-coverage=user --project=. -e 'using Pkg; Pkg.test(coverage=true)'
julia --startup-file=no --depwarn=yes --threads=auto coverage.jl

Then I use coverage gutters in VSCode https://marketplace.visualstudio.com/items?itemName=ryanluker.vscode-coverage-gutters which highlights the missing lines inside the IDE.

[MilesCranmer]

• I noticed in "register_units.jl" that you use a SpinLock(), is there any reason you're using this rather than the more common (and more recommended) ReentrantLock()?

ReentrantLock lets the same thread acquire a lock multiple times and I didn't see any use of that in this context. Also I expect the SpinLock to basically never get hit, it's just for the case where someone actually tries to register two units simultaneously, which I would expect to be exceedingly rare.

[Deduction42]

Excellent, thanks!

[MilesCranmer]

There are still some issues related to the other mathematical operators. For example, mod does not error

[MilesCranmer]

Please use the getters (affine_scale, etc.). I also added one for the symbol

[MilesCranmer]

As a heads up I'm making some quick changes as this is taking a long time to go back-and-forth. I removed the definition of == and also removed the - behaviour when the dimensions are the same. The implementation of affine dimensions is already really complex, so I want to avoid these kinds of convenience functions and value-dependent branches.

[MilesCranmer]

Please use single spaces between infix operators. So 0 * psig

[MilesCranmer]

Please avoid these kinds of stylistic patterns with multiple spaces to line up the ==

[MilesCranmer]

I am worried we are playing whack-a-mole with these issues and I am concerned with how many reviews I have had to do of this PR which is supposed to be for a fairly simple purpose. Maybe the current design of the AffineDimensions is just not compatible with DQ, if we need to keep monkey patching stuff like this?

For example, what if a user has their own function that they define for quantities, and, as done throughout the entirety of DynamicQuantities, they simply do a dimension(q) == dimension(q2) check before doing an op(ustrip(q), ustrip(q2)) on the units and computing the result. If someone were to pass AffineDimensions into that function, it would give incorrect result, and be completely silent about it. It seems like (at the moment) the solution is to just patch stuff manually so that it works for AffineDimensions.

I'm not sure how to solve this though. It seems really difficult...

[MilesCranmer]

The clearest example of this is mod. We shouldn't need to patch mod within the affine_dimensions.jl code. It should just work with the existing code. If it doesn't work I worry there is a deeper incompatibility, or maybe a redesign is needed.

Like maybe a different approach could be that ua"degC" returns a special quantity which is NOT a subtype of AbstractQuantity, and is only defined for multiplication, at which point it immediately starts using Dimensions. So you would be forced to write something like 22ua"degC" - but the result would be in Dimensions, not AffineDimensions. So we wouldn't need to worry about this other stuff.

The downside is that a raw ua"degC" wouldn't be a quantity. But maybe that's precisely what we want.

[MilesCranmer]

Or maybe just have ustrip return the scaled value, and dimension return the base dimension? Then I guess stuff would work out-of-the-box?

[Deduction42]

For example, what if a user has their own function that they define for quantities, and, as done throughout the entirety of DynamicQuantities, they simply do a dimension(q) == dimension(q2) check before doing an op(ustrip(q), ustrip(q2)) on the units and computing the result. If someone were to pass AffineDimensions into that function, it would give incorrect result, and be completely silent about it. It seems like (at the moment) the solution is to just patch stuff manually so that it works for AffineDimensions.

The most "Julian" solution to this is to properly constrain the parameters and define the operation on those dimensions

function my_op(q1::Quantity, q2::Quantity)
    return Quantity(my_op(ustrip(q1), ustrip(q2)), my_op(dimension(q1), dimension(q2)))
end
function my_op(d1::AbstractScaledDimension, d2::AbstractScaledDimension)
   return d1==d2 ? d1 : throw(DimensionError(l, r))
end

As long as the op isn't defined for an AffineDimension, it should throw an error if AffineDimensions are used. Defining a function for AbstractDimension is questionable, because you don't know what kind of whak-a-doodle dimensions you can get (think decibels), so you need to be really sure about its generality. At least defining an abstract type AbstractScaledDimension will let the user know they need to think along the lines of scale. This is what Unitful.jl does.

[Deduction42]

You don't even have to redefine "my_op" for dimensions. You could also simply have a function called scaled_dim_check

function my_op(q1::Quantity, q2::Quantity)
    return Quantity(my_op(ustrip(q1), ustrip(q2)), scaled_dim_check(dimension(q1), dimension(q2)))
end
scaled_dim_check(d1::AbstractScaledDimension, d2::AbstractScaledDimension) =  (d1==d2) ? d1 : throw(DimensionError(l, r))

This will also throw errors on AffineDimensions. Or you could just constrain the the quantity dimension types

function my_op(q1::Quantity{<:Number, <:AbstractScaledDimension}, q2::Quantity{<:Number, <:AbstractScaledDimension)
    (d1==d2) || throw(DimensionError(l, r))
    return Quantity(my_op(ustrip(q1), ustrip(q2)), d1)
end

The key is using the correct level of abstraction. It's just that your current definition of AbstractDimension assumes the dimension is scaled which isn't necessarily the case if we want to handle affine dimensions. So either we create a more general AbstractDimension such as AbstractScaledOrAffine or we subtype AbstractDimension so that it has AbstractScaledDimension and AbstractAffineDimension.

[Deduction42]

I think in this case, it's best to define AbstractDimension as a subtype of AbstractScaledOrAffine and just not export the latter. That way, people are less likely to fall into the affine trap. If people want to write their own functions, warn them to make sure they use Quantity{<:Number, <:AbstractDimension}. As it stands now, as long as we keep the internal set of operators constrained, whack-a-mole is a legitimate strategy until we properly subtype the rest of the function definitions because some of them only parameterize on UnionAbstractQuantity instead of UnionAbstractQuantity{<:Any, <:AbstractDimension}

[MilesCranmer]

Thanks for the detailed thoughts. Given that DynamicQuantities is already at v1 with a stable API, we can’t ask users to restrict their function definitions to Quantity{<:Number, <:AbstractDimension} without breaking the API contract. For me this is a non-option.

I appreciate the convenience of treating AffineDimensions like a typical AbstractDimension and incrementally patching uncovered methods. However, I'm concerned this approach would silently introduce correctness issues—especially in cases where users assume compatibility based on the current API.

A safer route would be to either ensure AffineDimensions reliably integrate with the existing dimension/ustrip API by simply throwing errors when those methods are used (with custom affine_dimension or affine_ustrip to be explicit), or use ua"degC" to produce a specialized, restricted type which is completely distinct from AbstractQuantity. Those ideas would prevent some of the more sinister correctness errors.

[Deduction42]

At this point, I'm not sure what kind of problems this would fix because how you implemented SymbolicDimensions and various quantities is rather confusing to me. I'm also not sure how to validate if these changes fix your problems. You're welcome to attempt this but I'm not sure I can help you here.

[MilesCranmer]

Basically what I am saying is that needing to patch unrelated functions like mod inside affine_dimensions.jl is a code smell which indicates the current AffineDimensions design isn’t entirely compatible with the DQ API. I gave some ideas for how to get around this above.

[Deduction42]

I did give some ideas on how to make the API more generic, but it would break the API and require a 2.x release. Maybe this isn't going to work and I'll have to create yet another units package.

[MilesCranmer]

I'm confused. Why wouldn't the ideas above work? These ones:

• Throwing errors when ustrip or dimension are used; instead requiring affine_dimension or affine_ustrip to be called.
  • This automatically errors for any functions with ustrip(x), which is exactly the part of the API that would cause silent bugs with affine dimensions. So this would I think basically solve the issue entirely.
• Have ua"degC" by a specialized, restricted type which is completely distinct from AbstractQuantity, and only define / and * which would convert to a regular Quantity{_, <:Dimensions}

[Deduction42]

It would technically work but would require a whole separate API for affine units, where the API in Unitful.jl is pretty straightforward. I'm already trying to use this version of the API in some of my codebases with local versions of this repo and having to make separate functions for affine units will just make this package a headache to use. I'm now having to play whack-a-mole to break an API that makes perfect sense.

[Deduction42]

So from my end:

1. I need to be able to have generic Quantity{T, <:AffineDimensions} and ua"degC" needs to return that. I already have an more general affine unit registry that can contain both symbolic and affine units so that I can have type-stable objects containing both affine and non-affine units. Messing with this behaviour is a non-starter.
2. Breaking dimension(q::Quantity{T, <:AffineDimensions} is drastic, and is also a non-starter. Grabbing AffineDimensions from a quantity is a reasonable operation. I'd sooner break "==" on two affine dimensions.
3. I wholeheartedly agree with breaking ustrip and creating affine_ustrip because ustrip on an affine dimension is dangerous operation anyway. I think this will solve all the real problems we're having.

[MilesCranmer]

Ok maybe let’s try 3? Maybe that will help fix some of the issues

[Deduction42]

Okay, I'll give it a shot.

[Deduction42]

Welp, breaking ustrip really broke a lot of things. I'm at the point where I don't know if I can fix the remaining issues. I managed to get 1.11 working, but I'm having a hard time with the Julia 1.10 implementation, the problem seems to be in utils.jl file around line 358, I'm not sure why Julia 1.10 is breaking here. This part of the code is hard for me to understand so I'm not sure if it's safe to patch. Are you able to look into this one?