Integer dtypes and units can cause a footgun

[jokasimr]

When converting units the dtype is conserved, this can create some surprising behaviors:

 sc.scalar(3, unit='deg').to(unit='rad')
# <scipp.Variable> ()      int64            [rad]  0
 sc.scalar(3.0, unit='deg').to(unit='rad')
# <scipp.Variable> ()    float64            [rad]  0.0523599

To work around it we can pass dtype='float64' as an extra argument in the unit conversion:

 sc.scalar(3, unit='deg').to(unit='rad', dtype='float64')

We are doing lots of unit conversions like this in our workflows, to make them able to handle user inputs having different units, and it's likely that they fail silently as above if the user passes an integer argument instead of a float.
There are some cases when you might want to preserve dtype after a unit conversion, mainly when converting counts, for example converting counts/s to counts/min (although in the opposite direction you do want a change of dtype!).

The example above uses deg and rad because that's a common case, but the same problem can happen with other units.

If we don't want users having to be aware of this we could consider either:

• Making all scipp variables be dtype float64 by default unless the user explicitly specifies dtype.
  • This is what I'm leaning towards. It won't solve the underlying problem but will probably reduce the number of times this causes bugs.
  • It's different than how numpy behaves, but they don't have units.
• Making unit conversions automatically convert the dtype to float64 if the input is an integer type.
  • Not appropriate for some count related unit conversions.

What are your thoughts about it?

[SimonHeybrock]

Deviating from NumPy behavior is usually a bad idea, since it will be surprising for the majority of users and will require a lot of extra explanation and references in the docs. If you need the behavior you describe I suggest you create wrapper functions.

[jokasimr]

Then what is your opinion on the second option, making unit conversions automatically convert the dtype to float?
I think that is almost always the correct behavior, only rarely will the result of a unit conversion be an integer, even if the input is.

[SimonHeybrock]

We could consider auto-converting in unit conversions when the factor is less than 1? And maybe add preserve_dtype: bool = False argument? Should have a wider discussion around this, and see if there are any pitfalls/downsides.

[jokasimr]

We could consider auto-converting in unit conversions when the factor is less than 1?

The problem is still there if the conversion factor is larger than 1. Consider converting foo to bar with a conversion factor of 1.5.
3 foo is currently converted to floor(1.5 * 3) = 4 bar, but of course the result should be 4.5 bar.

We could avoid converting the unit when the scaling factor is an integer.

[SimonHeybrock]

👍 for integer scale factors.

[YooSunYoung]

My vote:

• convert dtype to float64 of integer variables when we operate unit conversion by default.
• add preserve_dtype=False argument so that users can explicitly keep the dtype, so that they don't have to do to(unit='otherunit', dtype=foo.dtype)

[jokasimr]

A concrete example where this problem occurs, see here and here.

If simulation.distance or distance_resolution are integer dtype and unit mm or cm, then they will be rounded to the closest m after conversion if distance_unit is m.

There are multiple other examples involving angels in the essreflectometry package, for example here.