Skip to content Skip to sidebar Skip to footer
Home / Python / Python Typing / Type Hinting

How To Hint At Number *types* (i.e. Subclasses Of Number) - Not Numbers Themselves?

Post a Comment
Assuming I want to write a function that accepts any type of number in Python, I can annotate it as follows: from numbers import Number def foo(bar: Number): print(bar) Takin

Solution 1:

In general, how do we hint classes, rather than instances of classes?


In general, if we want to tell a type-checker that any instance of a certain class (or any instance of a subclass of that class) should be accepted as an argument to a function, we do it like so:

def accepts_int_instances(x: int) -> None:
    pass


class IntSubclass(int):
    pass


accepts_int_instances(42) # passes MyPy (an instance of `int`)
accepts_int_instances(IntSubclass(666)) # passes MyPy (an instance of a subclass of `int`)
accepts_int_instances(3.14) # fails MyPy (an instance of `float` — `float` is not a subclass of `int`)

Try it on MyPy playground here!

If, on the other hand, we have a class C, and we want to hint that the class C itself (or a subclass of C) should be passed as an argument to a function, we use type[C] rather than C. (In Python <= 3.8, you will need to use typing.Type instead of the builtin type function, but as of Python 3.9 and PEP 585, we can parameterise type directly.)

def accepts_int_and_subclasses(x: type[int]) -> None:
    pass


class IntSubclass(int):
    pass


accepts_int_and_subclasses(int) # passes MyPy 
accepts_int_and_subclasses(float) # fails Mypy (not a subclass of `int`)
accepts_int_and_subclasses(IntSubclass) # passes MyPy

How can we annotate a function to say that any numeric class should be accepted for a certain parameter?

int, float, and all the numpy numeric types are all subclasses of numbers.Number, so we should be able to just use type[Number] if we want to say that all numeric classes are permitted!

At least, Python says that float and int are subclasses of Number:

>>> from numbers import Number 
>>> issubclass(int, Number)
True
>>> issubclass(float, Number)
True

And if we're using a runtime type-checking library such as typeguard, using type[Number] seems to work fine:

>>> from typeguard import typechecked
>>> from fractions import Fraction
>>> from decimal import Decimal
>>> import numpy as np
>>>
>>> @typechecked
... def foo(bar: type[Number]) -> None:
...     pass
... 
>>> foo(str)
Traceback (most recent call last):
TypeError: argument "bar" must be a subclass of numbers.Number; got str instead
>>> foo(int)
>>> foo(float)
>>> foo(complex)
>>> foo(Decimal)
>>> foo(Fraction)
>>> foo(np.int64)
>>> foo(np.float32)
>>> foo(np.ulonglong)
>>> # etc.

But wait! If we try using type[Number] with a static type-checker, it doesn't seem to work. If we run the following snippet through MyPy, it raises an error for each class except for fractions.Fraction:

from numbers import Number
from fractions import Fraction
from decimal import Decimal


NumberType = type[Number]


def foo(bar: NumberType) -> None:
    pass


foo(float)  # fails 
foo(int)  # fails 
foo(Fraction)  # succeeds!
foo(Decimal)  # fails

Try it on MyPy playground here!

Surely Python wouldn't lie to us about float and int being subclasses of Number. What's going on?



Why type[Number] doesn't work as a static type hint for numeric classes


While issubclass(float, Number) and issubclass(int, Number) both evaluate to True, neither float nor int is, in fact, a "strict" subclass of numbers.Number. numbers.Number is an Abstract Base Class, and int and float are both registered as "virtual subclasses" of Number. This causes Python at runtime to recognise float and int as "subclasses" of Number, even though Number is not in the method resolution order of either of them.

See this StackOverflow question for an explanation of what a class's "method resolution order", or "mro", is.

>>> # All classes have `object` in their mro
>>> class Foo: pass
>>> Foo.__mro__
(<class '__main__.Foo'>, <class 'object'>)
>>>
>>> # Subclasses of a class have that class in their mro
>>> class IntSubclass(int): pass
>>> IntSubclass.__mro__
(<class '__main__.IntSubclass'>, <class 'int'>, <class 'object'>)
>>> issubclass(IntSubclass, int)
True
>>>
>>> # But `Number` is not in the mro of `int`...
>>> int.__mro__
(<class 'int'>, <class 'object'>)
>>> # ...Yet `int` still pretends to be a subclass of `Number`!
>>> from numbers import Number 
>>> issubclass(int, Number)
True
>>> #?!?!!??

What's an Abstract Base Class? Why is numbers.Number an Abstract Base Class? What's "virtual subclassing"?

The issue is that MyPy does not understand the "virtual subclassing" mechanism that ABCs use (and, perhaps never will).

MyPy does understand some ABCs in the standard library. For instance, MyPy knows that list is a subtype of collections.abc.MutableSequence, even though MutableSequence is an ABC, and list is only a virtual subclass of MutableSequence. However, MyPy only understands list to be a subtype of MutableSequence because we've been lying to MyPy about the method resolution order for list.

MyPy, along with all other major type-checkers, uses the stubs found in the typeshed repository for its static analysis of the classes and modules found in the standard library. If you look at the stub for list in typeshed, you'll see that list is given as being a direct subclass of collections.abc.MutableSequence. That's not true at all — MutableSequence is written in pure Python, whereas list is an optimised data structure written in C. But for static analysis, it's useful for MyPy to think that this is true. Other collections classes in the standard library (for example, tuple, set, and dict) are special-cased by typeshed in much the same way, but numeric types such as int and float are not.


If we lie to MyPy about collections classes, why don't we also lie to MyPy about numeric classes?

A lot of people (including me!) think we should, and discussions have been ongoing for a long time about whether this change should be made (e.g. typeshed proposal, MyPy issue). However, there are various complications to doing so.

Credit goes to @chepner in the comments for finding the link to the MyPy issue.



Possible solution: use duck-typing


One possible (albeit slightly icky) solution here might be to use typing.SupportsFloat.


SupportsFloat is a runtime-checkable protocol that has a single abstractmethod, __float__. This means that any classes that have a __float__ method are recognised — both at runtime and by static type-checkers — as being subtypes of SupportsFloat, even if SupportsFloat is not in the class's method resolution order.

What's a protocol? What's duck-typing? How do protocols do what they do? Why are some protocols, but not all protocols, checkable at runtime?

Note: although user-defined protocols are only available in Python >= 3.8, SupportsFloat has been in the typing module since the module's addition to the standard library in Python 3.5.

Advantages of this solution

  1. Comprehensive support* for all major numeric types: fractions.Fraction, decimal.Decimal, int, float, np.int32, np.int16, np.int8, np.int64, np.int0, np.float16, np.float32, np.float64, np.float128, np.intc, np.uintc, np.int_, np.uint, np.longlong, np.ulonglong, np.half, np.single, np.double, np.longdouble, np.csingle, np.cdouble, and np.clongdouble all have a __float__ method.

  2. If we annotate a function argument as being of type[SupportsFloat], MyPy correctly accepts* the types that conform to the protocol, and correctly rejects the types that do not conform to the protocol.

  3. It is a fairly general solution — you do not need to explicitly enumerate all possible types that you wish to accept.

  4. Works with both static type-checkers and runtime type-checking libraries such as typeguard.

Disadvantages of this solution

  1. It feels like (and is) a hack. Having a __float__ method isn't anybody's reasonable idea of what defines a "number" in the abstract.

  2. Mypy does not recognise complex as a subtype of SupportsFloat. complex does, in fact, have a __float__ method in Python <= 3.9. However, it does not have a __float__ method in the typeshed stub for complex. Since MyPy (along with all other major type-checkers) uses typeshed stubs for its static analysis, this means it is unaware that complex has this method. complex.__float__ is likely omitted from the typeshed stub due to the fact that the method always raises TypeError; for this reason, the __float__ method has in fact been removed from the complex class in Python 3.10.

  3. Any user-defined class, even if it is not a numeric class, could potentially define __float__. In fact, there are even several non-numeric classes in the standard library that define __float__. For example, although the str type in Python (which is written in C) does not have a __float__ method, collections.UserString (which is written in pure Python) does. (The source code for str is here, and the source code for collections.UserString is here.)

Example usage

This passes MyPy for all numeric types I tested it with, except for complex:

from typing import SupportsFloat


NumberType = type[SupportsFloat]


def foo(bar: NumberType) -> None:
    pass

Try it on MyPy playground here!

If we want complex to be accepted as well, a naive tweak to this solution would just be to use the following snippet instead, special-casing complex. This satisfies MyPy for every numeric type I could think of. I've also thrown type[Number] into the type hint, as it could be useful to catch a hypothetical class that does directly inherit from numbers.Number and does not have a __float__ method. I don't know why anybody would write such a class, but there are some classes that directly inherit from numbers.Number (e.g. fractions.Fraction), and it certainly would be theoretically possible to create a direct subclass of Number without a __float__ method. Number itself is an empty class that has no methods — it exists solely to provide a "virtual base class" for other numeric classes in the standard library.

from typing import SupportsFloat, Union 
from numbers import Number


NumberType = Union[type[SupportsFloat], type[complex], type[Number]]

# You can also write this more succinctly as:
# NumberType = type[Union[SupportsFloat, complex, Number]]
# The two are equivalent.

# In Python >= 3.10, we can even write it like this:
# NumberType = type[SupportsFloat | complex | Number]
# See PEP 604: https://www.python.org/dev/peps/pep-0604/


def foo(bar: NumberType) -> None:
    pass

Try it on MyPy playground here!

Translated into English, NumberType here is equivalent to:

Any class, if and only if:

  1. It has a __float__ method;
  2. AND/OR it is complex;
  3. AND/OR it is a subclass of complex;
  4. AND/OR it is numbers.Number;
  5. AND/OR it is a "strict" (non-virtual) subclass of numbers.Number.

I don't see this as a "solution" to the problem with complex — it's more of a workaround. The issue with complex is illustrative of the dangers of this approach in general. There may be other unusual numeric types in third-party libraries, for example, that do not directly subclass numbers.Number or have a __float__ method. It would be exceedingly difficult to know what they might look like in advance, and special-case them all.



Addenda


Why SupportsFloat instead of typing.SupportsInt?

fractions.Fraction has a __float__ method (inherited from numbers.Rational) but does not have an __int__ method.


Why SupportsFloat instead of SupportsAbs?

Even complex has an __abs__ method, so typing.SupportsAbs looks like a promising alternative at first glance! However, there are several other classes in the standard library that have an __abs__ method and do not have a __float__ method, and it would be a stretch to argue that they are all numeric classes. (datetime.timedelta doesn't feel very number-like to me.) If you used SupportsAbs rather than SupportsFloat, you would risk casting your net too wide and allowing all sorts of non-numeric classes.


Why SupportsFloat instead of SupportsRound?

As an alternative to SupportsFloat, you could also consider using typing.SupportsRound, which accepts all classes that have a __round__ method. This is just as comprehensive as SupportsFloat (it covers all major numeric types other than complex). It also has the advantage that collection.UserString has no __round__ method whereas, as discussed above, it does have a __float__ method. Lastly, it seems less likely that third-party non-numeric classes would include a __round__ method.

However, if you went for SupportsRound rather than SupportsFloat, you would, in my opinion, run a greater risk of excluding valid third-party numeric classes that, for whatever reason, do not define __round__.

"Having a __float__ method" and "having a __round__ method" are both pretty poor definitions of what it means for a class to be a "number". However, the former feels much closer to the "true" definition than the latter. As such, it feels safer to count on third-party numeric classes having a __float__ method than it does to count on them having a __round__ method.

If you wanted to be "extra safe" when it comes to ensuring a valid third-party numeric type is accepted by your function, I can't see any particular harm in extending NumberType even further with SupportsRound:

from typing import SupportsFloat, SupportsRound, Union
from numbers import Number

NumberType = Union[type[SupportsFloat], type[SupportsRound], type[complex], type[Number]]

However, I would question whether it's really necessary to include SupportsRound, given that any type that has a __round__ method is very likely to have a __float__ method as well.


Solution 2:

There is no general way to do this. Numbers are not strictly related to begin with and their types are even less.

While numbers.Number might seem like "the type of numbers" it is not universal. For example, decimal.Decimal is explicitly not a numbers.Number as either subclass, subtype or virtual subclass. Specifically for typing, numbers.Number is not endorsed by PEP 484 -- Type Hints.

In order to meaningfully type hint "numbers", one has to explicitly define what numbers are in that context. This might be a pre-existing numeric type set such as int <: float <: complex, a typing.Union/TypeVar of numeric types, a typing.Protocol to define operations and algebraic structure, or similar.

from typing import TypeVar
from decimal import Decimal
from fractions import Fraction

#: typevar of rational numbers if we squint real hard
Q = TypeVar("Q", float, Decimal, Fraction)

All that said, "the type of the type of numbers" is even less meaningful. Even the specific numbers.Number has practically no features at all: it cannot be converted to a concrete type, nor can it be instantiated to a meaningful number.

Instead, use "the type of some type of numbers":

from typing import Type

def zero(t: Type[Q]) -> Q:
    return t()  # all Type[Q]s can be instantiated without arguments

print(zero(Fraction))

If the only goal of the Type is to create instances, it can be better to request a Callable instead. This covers types as well as factory functions.

def one(t: Callable[[int], Q]) -> Q:
    return t(1)

Solution 3:

This is not quite an answer to the original question. (Alex Waygood's answer is properly selected as responsive.) However, I have attempted to generalize my own work-arounds for the sharp edges between numbers and typing in Python. Those work-arounds now live in numerary (having extracted it via c-section from dyce, where it was conceived).

I didn't spend a lot of time on naming, in the hopes it will be short-lived. Docs are online. It should be considered experimental, but it is rapidly approaching stability. Feedback, suggestions, and contributions are desperately appreciated.


Post a Comment for "How To Hint At Number *types* (i.e. Subclasses Of Number) - Not Numbers Themselves?"