Affine Units Functionality

[Deduction42]

This adds a new type of Dimension (AffineDimension) that can be used to describe units with offsets (like Celsius and Fahrenheit). Like most other packages, operations on AffineDimensions are forbidden except subtraction. This is done by throwing an error on convert if the offset is zero; operations are allowed if the offset is zero (by using promote/convert to SI units).

[MilesCranmer]

Thanks! Can you walk me through the implementation a bit? How does it work, what are the fields used for, etc?

[MilesCranmer]

And FYI I would like to get things to 100% unit-test coverage if possible (particularly important as I'm unfamiliar with the code). My expectation is that there will be many unexpected errors that will pop out when going through this process.

[MilesCranmer]

I think the Aqua and JET tests will likely take a bit of work to get passing but are also important

[Deduction42]

Sure! As for how it works. Almost all of the changes are in "affine_dimensions.jl", which somewhat mirrors the patterns found in "symbolic_dimensions.jl" (or as much as reasonably possible). The basic structure is the AffineDimension <: AbstractAffineDimension <: AbstractDimension which has the following fields:

@kwdef struct AffineDimensions{R} <: AbstractAffineDimensions{R}
    scale::Float64 = 1.0
    offset::Float64 = 0.0
    basedim::Dimensions{R}
    symbol::Symbol = :nothing
end

The scale and offset parameters are used to convert this unit to its SI equivalent. So the SI quantitiy is basically

si_quantity = affine_quantity*scale + offset

This is essentially what uexpand(affine_quantity) does when it converts things to SI dimensions. The symbol field is optional and is for displaying units. If no symbol is present, it will merely display (scalse+offset basedim). For example:

kelvin  = AffineDimensions(scale=1.0, offset=0.0, basedim=u"K")
rankine = AffineDimensions(scale=5/9, offset=0.0, basedim=kelvin)
fahrenheit = AffineDimensions(scale=1.0, offset=459.67, basedim=rankine)
>> (0.5555555555555556+255.37222222222223 K)

But if we used

fahrenheit = AffineDimensions(scale=1.0, offset=459.67, basedim=rankine, symbol=:°F)
>>°F

[Deduction42]

Generally speaking, it's not encouraged to do mathematical operations in affine units because of potential ambiguities present in the offset. Promotion rules will attempt to convert affine quantities into basic dimensional (SI) quantities. However, the function "convert" will fail if the offset is not zero (I allow typical multiplication/division operations for affine units with zero offsets in order to make affine unit macros work with compound units like m/s).

The affine unit marcro is "ua" and I registered the Celsius and Fahrenheit units. So you could write

 uconvert(ua"°C", 0ua"°F")
>> -17.77777777777777 °C

Becuase AffineDimensions are more general than Dimensions and SymbolicDimensions, registering a unit will also make it available to the "ua" macro, and I registered all items in UNIT_SYMBOLS and UNIT_VALUES off the bat, so you could write:

velocity = ua"mm/s"
>>1.0 (0.001 m s⁻¹)

This generality is so that you could make a single Type-Stable container (like a dictionary or vector) that represents both Affine and Non-Affine units and easily convert them all to SI or Symbolic units. Note however, compound non-affine units break, becuase offsets cannot be multiplied:

ua"°C/K"
ERROR: AssertionError: AffineDimensions °C has a non-zero offset, implicit conversion is not allowed due to ambiguity. Use uexpand(x) to explicitly convert
Stacktrace:

[Deduction42]

Finally, there is a special macro for registering AffineDimensions only, because the regular registration macro doesn't handle offsets. Registration will apply to anything that can be evaluated as a dimension or quantity. For example, you could do:

@register_unit psi 6.89476us"kPa" #Regular registration method for PSI
@register_affine_unit psig AffineDimensions(offset=u"Constants.atm", basedim=u"psi") #Gauge-pressure subtracts atmospheric

uconvert(ua"psig", u"Constants.atm")
>> 0.0 psig

Generally, when building AffineDimensions, you want to be explicit about your "offset" units. If you use a numeric value, it will assume the same units as "basedim" which could be a quantity.

AffineDimensions(offset=1, basedim=0.5u"m") #Offset units are assumed to be half-meters "0.5m"
>>(0.5+0.5 m)

This may seem odd until we account for the fact that we could register half-meters so

@register_unit half_meter 0.5u"m"
AffineDimensions(offset=1, basedim=u"half_meter")
>>(0.5+0.5 m)

In such a case, it would be "least astonishing" that in the absence of units, the offset is in the same units as the base dimensions because no other units are mentioned. Building AffineDimesnions from other AffineDimensions will use the basedim's offset as a starting point, so you could do something like this:

kelvin  = AffineDimensions(scale=1.0, offset=0.0, basedim=u"K")
rankine = AffineDimensions(scale=5/9, offset=0.0, basedim=kelvin)
fahrenheit = AffineDimensions(scale=1.0, offset=459.67, basedim=rankine)
celsius = AffineDimensions(scale=9/5, offset=32, basedim=fahrenheit)

uconvert(Quantity(1.0, fahrenheit), Quantity(-40.0, celsius))
>>-39.99999999999996 °F

[MilesCranmer]

Quick question - why do affine dimensions need to support algebraic operations on non-offset affine dimensions? If it's non-offset, why not just force the user to immediately convert to SI units if they want to use such operations?

[Deduction42]

Anyway, I'll start writing the tests for this on Monday, I just wanted to make sure you were onboard with adding this before I put the effort into it. I was pleasantly surprised that this took me less than a week to do; your package has some fairly elegant ways to plug in this feature (although these plug-in points are a bit hard to find). To reiterate, almost all of the changes are in the new "affine_dimensions.jl" file I added. I also had to make some changes to register_units.jl to add automatic registering of any units to the AffineUnits registry, and make a separate macro for registering AffineDimensions only.

I do have some questions:

1. In the "test coverage" tools, is there any way to identify what lines are missing from the coverage? Or is this a game of educated whack-a-mole?
2. I noticed in "register_units.jl" that you use a SpinLock(), is there any reason you're using this rather than the more common (and more recommended) ReentrantLock()?
3. Last time I contributed, I was able to run all the tests by just running "run_tests.jl", but this time, there's a bunch of test-related dependencies. Is there a standard way of loading this environment before doing the tests, or do I have to just install all the test dependencies myself?

[Deduction42]

Quick question - why do affine dimensions need to support algebraic operations on non-offset affine dimensions? If it's non-offset, why not just force the user to immediately convert to SI units if they want to use such operations?

The only reason I do this is so that I can parse units like "ua"m/s"" as an Affine Unit. However, I could just as well do an automatic promotion to SI units and then evaluate the compound unit as Affine in the end if that's something you would prefer.

[MilesCranmer]

This is the command I usually run at the root of this repository:

julia --startup-file=no --depwarn=yes --threads=auto --code-coverage=user --project=. -e 'using Pkg; Pkg.test(coverage=true)'
julia --startup-file=no --depwarn=yes --threads=auto coverage.jl

Then I use coverage gutters in VSCode https://marketplace.visualstudio.com/items?itemName=ryanluker.vscode-coverage-gutters which highlights the missing lines inside the IDE.

[MilesCranmer]

• I noticed in "register_units.jl" that you use a SpinLock(), is there any reason you're using this rather than the more common (and more recommended) ReentrantLock()?

ReentrantLock lets the same thread acquire a lock multiple times and I didn't see any use of that in this context. Also I expect the SpinLock to basically never get hit, it's just for the case where someone actually tries to register two units simultaneously, which I would expect to be exceedingly rare.

[Deduction42]

Excellent, thanks!

[MilesCranmer]

Basically what I am saying is that needing to patch unrelated functions like mod inside affine_dimensions.jl is a code smell which indicates the current AffineDimensions design isn’t entirely compatible with the DQ API. I gave some ideas for how to get around this above.

[Deduction42]

I did give some ideas on how to make the API more generic, but it would break the API and require a 2.x release. Maybe this isn't going to work and I'll have to create yet another units package.

[MilesCranmer]

I'm confused. Why wouldn't the ideas above work? These ones:

• Throwing errors when ustrip or dimension are used; instead requiring affine_dimension or affine_ustrip to be called.
  • This automatically errors for any functions with ustrip(x), which is exactly the part of the API that would cause silent bugs with affine dimensions. So this would I think basically solve the issue entirely.
• Have ua"degC" by a specialized, restricted type which is completely distinct from AbstractQuantity, and only define / and * which would convert to a regular Quantity{_, <:Dimensions}

[Deduction42]

It would technically work but would require a whole separate API for affine units, where the API in Unitful.jl is pretty straightforward. I'm already trying to use this version of the API in some of my codebases with local versions of this repo and having to make separate functions for affine units will just make this package a headache to use. I'm now having to play whack-a-mole to break an API that makes perfect sense.

[Deduction42]

So from my end:

1. I need to be able to have generic Quantity{T, <:AffineDimensions} and ua"degC" needs to return that. I already have an more general affine unit registry that can contain both symbolic and affine units so that I can have type-stable objects containing both affine and non-affine units. Messing with this behaviour is a non-starter.
2. Breaking dimension(q::Quantity{T, <:AffineDimensions} is drastic, and is also a non-starter. Grabbing AffineDimensions from a quantity is a reasonable operation. I'd sooner break "==" on two affine dimensions.
3. I wholeheartedly agree with breaking ustrip and creating affine_ustrip because ustrip on an affine dimension is dangerous operation anyway. I think this will solve all the real problems we're having.

[MilesCranmer]

Ok maybe let’s try 3? Maybe that will help fix some of the issues

[Deduction42]

Okay, I'll give it a shot.

[Deduction42]

Welp, breaking ustrip really broke a lot of things. I'm at the point where I don't know if I can fix the remaining issues. I managed to get 1.11 working, but I'm having a hard time with the Julia 1.10 implementation, the problem seems to be in utils.jl file around line 358, I'm not sure why Julia 1.10 is breaking here. This part of the code is hard for me to understand so I'm not sure if it's safe to patch. Are you able to look into this one?

[MilesCranmer]

I can take a look. Before that can you try to fix your merged changes from edits? I think some of my edits were erased; stuff like adding affine_symbol and PLACEHOLDER_SYMBOL

[Deduction42]

Shoot, I thought I force-pulled from your changes. I'm not sure how to fix that.

[Deduction42]

I'm probably going to be hands-off this for a while. You can safely edit at will.

[MilesCranmer]

Before handing over the reins, can you put back 31c8bfe? I don’t know how you merged the changes but that commit seems to be gone and also affine_symbol. Also some of the other commits around that time look to be overwritten by your merge

[Deduction42]

Okay, I did the best I could; there were a lot of merge conflicts.

[Deduction42]

I'm having second thoughts on this though, breaking ustrip really broke a lot of things and it required a lot of code to be copied over, so I'm worried about shotgun surgery. Maybe this package just isn't ready for affine units. I still think the proper way is to define some constant like

const ScalarQuantity{T} = Quantity{T, <:Union{Dimensions, SymbolicDimensions}}

And make sure users dispatch their functions on that. This is a big feature and would probably require a 2.0 release, if we're not ready for that, we shouldn't force such a kludgy solution.

[MilesCranmer]

Breaking ustrip is kind of correct though, no? Since the interpretation is essentially ambiguous when we are talking about affine units.

What does Unitful do for stripping units in an affine context?

[Deduction42]

In Unitful.jl, it defines ustrip(q::Quantity)=q.val so it would work for all units, because there are a lot of valid times to use ustrip on affine quantities (a lot more than I previously thought).

For numerical functions, Unitful.jl defines a couple of useful derived classes:

const AffineQuantity{T,D,U} = AbstractQuantity{T,D,U} where U<:AffineUnits
const ScalarQuantity{T,D,U} = AbstractQuantity{T,D,U} where U<:ScalarUnits

Users can therefore define operations that only work on ScalarQuantity, which I think is the correct approach for this.

[MilesCranmer]

What is the practical use of defining functions for affine quantities? I thought the main objective is to move stuff to SI quantities and then operate there. I mean there’s only degC and degF; what’s the main benefit here. I don’t want to shoot it down, but like, why does it justify so much complexity? I think Unitful is overly complex at the moment, and they can’t go back on that; they are stuck with increased complexity.

[MilesCranmer]

If I scroll through https://github.com/search?q=/AffineQuantity/%20language:julia&type=code, it looks like most appearances of AffineQuantity in libraries is just to convert it to regular quantities or throw errors…

[MilesCranmer]

Actually it looks even less frequently used than I thought. Do any libraries use affine quantities internally for computing things? Most uses seem like just compatibility layers for handling all possible Unitful inputs

[Deduction42]

That's correct, it's basically an add-on. The thing is though, since every unit is it's own distinct type in Unitful, adding AffineUnits is pretty easy because you don't really treat it any different from any other type except restricting some operations. However, with DynamicQuantities, we want to make sure we can represent all reasonable units with a single type. However, you can't represent an AffineDimension with a SymbolicDimension (but fortunately, AffineDimensions can represent SymbolicDimensions). This means that if I'm going to read a spreadsheet of units from a technician, it's best if I read it with AffineDimensions because all units will work even if there's the odd Fahrenheit in there. This also means that I now need to be able to parse raw input as AffineDimensions, in fact, I will probably always use ua"..." and aff_uparse("...") because it will work for everything, and then convert to SI at the first chance I get (which will run all calculations with reasonable speed).

[Deduction42]

As far as end users go, you only ever do two things with affine units: (1) Take differences, (2) Convert them to scalar units. Affine units are usually only temperatures (sometimes gauge pressure), and people who do calculations with temperature are sufficiently acquainted with thermodynamics to know that you only do operations in Celsius/Fahrenheit for temperature differences, otherwise you use absolute temperatures. If a function does actual math with Celsius/Fahrenheit, it's for an empirical model that's been trained on those units. In such cases, you do a ustrip, plug it into the equation, and apply the correct units to the equation's result (this is done a lot for enthalpy changes over large temperature differences).

[MilesCranmer]

Sorry for a maybe dumb question, but why not handle this dataset with a regular expression? What does your dataset look like?

[Deduction42]

Anyway, I guess what I'm saying is that users should know how to use affine units, so I'm not sure we have to worry too much about users defining functions and silently returning bad results, because if there are calculations, affine units will break pretty quickly (you won't get very far without +-*./) even without breaking ustrip (I know, because it happens to me with Unitful and Pint (python)).

[MilesCranmer]

Alternative implementation here: #168. Just leaves the absolute basics

[MilesCranmer]

Moving to #168