Repository navigation
math.isfinite() can raise exception when called on a number #91277
Description
| BPO | 47121 |
|---|---|
| Nosy | @tim-one, @rhettinger, @mdickinson, @stevendaprano, @serhiy-storchaka, @nathanielmanistaatgoogle |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2022-03-25.17:28:43.481>
labels = ['3.11', 'type-feature', 'docs']
title = 'math.isfinite() can raise exception when called on a number'
updated_at = <Date 2022-04-08.10:09:13.165>
user = 'https://bugs.python.org/tfish2'bugs.python.org fields:
activity = <Date 2022-04-08.10:09:13.165>
actor = 'tfish2'
assignee = 'docs@python'
closed = False
closed_date = None
closer = None
components = ['Documentation']
creation = <Date 2022-03-25.17:28:43.481>
creator = 'tfish2'
dependencies = []
files = []
hgrepos = []
issue_num = 47121
keywords = []
message_count = 8.0
messages = ['416010', '416011', '416018', '416040', '416840', '416842', '416843', '416969']
nosy_count = 8.0
nosy_names = ['tim.peters', 'rhettinger', 'mark.dickinson', 'steven.daprano', 'docs@python', 'serhiy.storchaka', 'Nathaniel Manista', 'tfish2']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue47121'
versions = ['Python 3.11']
Activity
tfish2 commentedon Mar 25, 2022
>>> help(math.isfinite) isfinite(x, /) Return True if x is neither an infinity nor a NaN, and False otherwise.So, one would expect the following expression to return
TrueorFalse. We instead observe:(There likewise is a corresponding issue with other, similar, functions).
This especially hurts since PEP-484 states that having a Sequence[float]
xsdoes not allow us to infer thatall(issubclass(type(x), float) for x in xs)actually holds - since a PEP-484 "float" actually does also include "int" (and still, issubclass(int, float) == False).Now, strictly speaking,
help(math)states thatDESCRIPTION
This module provides access to the mathematical functions
defined by the C standard.
...but according to "man 3 isfinite", the math.h "isfinite" is a macro and not a function - and the man page does not show type information for that reason.
rhettinger commentedon Mar 25, 2022
The math.isfinite() docs could be changed to something like, "coerces x to a float if possible and then returns True if x is neither an infinity nor a NaN, and False otherwise." Or there could be a general note about which functions (most of them) coerce to float (which can fail).
With respect to typing and PEP-484, I don't see a bug or documentation issue. Types relationships are useful for verifying which methods are available, but they don't make promises about the range of valid values. For example math.sqrt(float) -> float promises which types are acceptable but doesn't promise that negative inputs won't raise an exception. Likewise, "n: int=10; len(range(n))" is type correct but will raise an OverflowError for "n = 10**100".
tfish2 commentedon Mar 25, 2022
The problem with PEP-484 is that if one wants to use static type analysis, neither of these options are good:
out expectations in docstrings. Do note that the two types places
where "float" is mentioned here refer to different concepts.
This looks as if there were duplication, but there actually
isn't, since the claims are different. This is confusing as hell.
Args:
x: float, the foobar
Returns:
float, the foofoo"""
The floats in the docstring give me a guarantee: "If I feed in a float, I am guaranteed to receive back a float". The floats in the static type annotation merely say "yeah, can be float or int, and I'd call it ok in these cases" - that's a very different statement.
from docstrings, and accept that we lose the ability to
stringently reason about the behavior of code.
With respect to this latter option, I think we can wait for "losing the ability to stringently reason about the behavior of code" to cause major security headaches. That's basically opening up the door to many problems at the level of "I can crash the webserver by requesting the url http://lpt1".
serhiy-storchaka commentedon Mar 26, 2022
Most (but not all) functions in the math module implicitly convert its arguments to float. Here we can get an OverflowError. Do we want to add a note to every function that does it? Or add a general note at the top of the file and add exclusion notes to functions which do not convert arguments to float?
BTW, there is a general note: "Except when explicitly
noted otherwise, all return values are floats."
stevendaprano commentedon Apr 6, 2022
Isn't this just a quality of implementation issue?
math.isfinite should return True for all ints, since all ints are finite. (There are no int infinities or NANs). There is no need to coerce them to float to know that they are finite.
Likewise for Fractions. If they overflow, that could be caught and True returned.
Decimal infinities convert to float infinities, so the only way you can get an overflow error is if the Decimal is finite but too big to convert. So again, isfinite could (should?) catch the overflow error and return True.
Any numeric type that has an infinity which does not coerce to float infinity, but overflows instead, is buggy, and its not isfinite's responsibility to protect against that.
So here is my suggestion:
isfinite() should return True for all ints, without needing to coerce them to float. For other numeric types, it should try to coerce them to float, and catch OverflowError and raise True. This should be documented, so that other numeric types know what contract they are required to follow (infinity coerces to float infinity).
I'm going to change this to an enhancement for 3.11 (or 3.12).
rhettinger commentedon Apr 6, 2022
Whoa there. You're venturing into changing what those math functions were all about and the core approach to how they operate.
A zigzag to this new direction would need discussion with all the math folks. Many tools in the math module are thin wrapper around libmath and we never intended to be universal handlers of all numeric types. We have a lot of functions that start by coercing their input to a float.
My two cents is that this would open an endless can of worms and make everyone pay a time and complexity cost for a feature that almost no one needs. AFAICT is more of a purity issue than an actual practical need that would warrant separate code paths (for example, we recently closed an issue where someone wanted math.log() to have a separate code path for Fraction inputs where it return log(f.numerator) - log(f.denominator) with the goal of handling fractional values smaller than float_min).
tim-one commentedon Apr 6, 2022
I'll testify that I won't volunteer one second of my time pursuing these abstract "purity" crusades ;-)
isfinite()et alia were added to supply functions defined by current standards to work on IEEE floating-point values. People working with floats have reasonable expectations that Python will supply workalikes for industry-stand float functions.If was had to cater to all possible "numberish" types, we'd never add anything new again :-( Doing that in a principled way requires dedicated new dunder methods, and that's a high bar.
That I said, I extended math.log() decades ago to work on giant integers. It was useful for real projects I was working on at the time - I was scratching my own itches.
tfish2 commentedon Apr 8, 2022
Tim, the problem may well be simply due to the documentation of math.isfinite() being off here.
This is what we currently have:
https://docs.python.org/3/library/math.html#math.isfinite
===
New in version 3.2.
===
If this were re-worded as follows (and corresponding changes were made to other such functions), everyone would know what the expectations and behavior are:
===
If
xis afloatinstance, this evaluates toTrueifxisneither a float infinity nor a NaN, and
Falseotherwise.If
xis not afloatinstance, this is evaluates tomath.isfinite(float(x)).New in version 3.2.
===
This would be an accurate defining description of the actual behavior. Note that, "thanks to PEP-484", this abbreviation would currently be ambiguous though:
===
If
xis a float, this evaluates toTrueifxisneither a float infinity nor a NaN, and
Falseotherwise.If
xis not a float, this is evaluates tomath.isfinite(float(x)).New in version 3.2.
===
("ambiguous" since "float" means different things as a static type and as a numbers class - and it is not clear what would be referred to here).
Changing/generalizing the behavior might potentially be an interesting other proposal, but I would argue that then one would want to change the behavior of quite a few other functions here as well, and all this should then perhaps go into some other
xmath(or so) module - bit like it is withcmath.However, since the Python philosophy is to not rely on bureaucracy to enforce contracts (as C++, Java, etc. do it), but instead to rely on people's ability to define their own contracts, making the math.isfinite() contract more accurate w.r.t. actual behavior in the CPython implementation via extra clarification looks like a good thing to do, no?
nathanielmanistaatgoogle commentedon Apr 18, 2022
I came to this enhancement request wanting to contribute support for it, but these kinds of statements are the obstacle over which I'm currently stumbling. I want it to be the case that given the
intinteger type built into the language and the standard library module namedmaththat the two just work smoothly and intuitively together, producing correct results rather than raising a boundary-of-what-was-forecast-to-be-needed-was-reachedOverflowError. I want it to be the case that where there need to be sharp corners, special cases, caveats, and limits, that users have to deliberately opt into them by "reaching further" for the harder-to-use, less-readily-at-hand sharp-cornered system -import floatmathorimport ieee754mathorfrom c_standard import mathlib_functionsor something like that. But obviously there's history here of which I hadn't before been aware, and I also can't pretend thatThis module provides access to the mathematical functions defined by the C standard.isn't currently the first sentence of themathmodule's documentation.Well, yeah. This issue definitely makes me wonder if that discussion would be worth having, and curious to what extent it may have been had before. (I wonder if you meant "the math folks" or "the
mathfolks"? 😛 Assuming for the moment the latter...) That the module is in the standard library and has the unqualified name "math" places it "readily at hand" for all programmers and invites general use for general use cases. Given that, has anyone before asked "IsThis module provides access to the mathematical functions defined by the C standard.really the right 'mission statement' for the module to continue to have?"?Agreed - but I'd hypothesize that "people working with floats" is likely only a subset of those who would add
import mathto their code. Just from its appearance aloneimport mathlooks like it can be expected to work for all kinds of math stuff, right? 🙂Agreed, and I suspect there's an opportunity here to let the perfect be the enemy of the good, and we oughtn't take advantage of that opportunity. I'd like to advocate that the
ints built into the language work correctly with themathin the standard library, and I believe there's space to campaign for that without also arguing that all numeric types and all mathematical functions in all libraries and in all codebases additionally be made to behave well together.serhiy-storchaka commentedon Apr 18, 2022
A reasonable expectation is that if
math.isfinite(x)does not fail,math.atan(x)should not fail either. If you makemath.isfinite()not failing for10**1000, you will need to add similar changes in all functions which implicitly convert arguments tofloat. Otherwise you can break user's code.There are also technical problems. The only general way of supporting arbitrary numbers in
math.isfinite()is to introduce a special method__isfinite__. You will end with adding special methods for allmathfunctions:__sqrt__,__sin__, etc. And different set of special methods forcmathfunctions:__csqrt__,__csin__, etc. It will negatively affect performance, because of additional lookup in dictionary and additional calls.rhettinger commentedon May 1, 2022
I concur with Tim that this is just an abstract purity nit and also with Serhiy that this doesn't make sense in light of how the other functions work. I'll add that I don't see that any real user issue is being addresses here. Mostly, this seems to be an "it bugs me" request rather than a "people need this" request.
Thanks for the suggestion, but we will decline.
fischbacher commentedon May 16, 2022
Right now, the
mathdocumentation states:===
https://docs.python.org/3/library/math.html#math.isfinite
Return True if x is neither an infinity nor a NaN, and False otherwise. (Note that 0.0 is considered finite.)
New in version 3.2.
Current behavior does not align with the documentation. This is by definition a bug.
There are multiple options for resolving this, one of them is keeping the current behavior and adjusting the documentation.
But a bug it is.
serhiy-storchaka commentedon May 16, 2022
Does it return True for an infinity or a NaN? Does it return False for any x which is neither an infinity nor a NaN?
fischbacher commentedon May 16, 2022
Does it show any behavior other than returning True or False on type-wise valid numerical input? Yes.
fischbacher commentedon May 16, 2022
Perhaps I need to explain this more slowly.
https://docs.python.org/3/library/math.html#math.isfinite clearly says:
Return True if x is neither an infinity nor a NaN, and False otherwise. (Note that 0.0 is considered finite.)
The opposite of "Responds with True on X" is not "Responds with False on X", but "Fails to respond with True on X".
So, in small steps:
We can ignore the redundant. "Note that 0.0 is considered finite" statement.
The specification is logically equivalent to:
"If x is an infinity or NaN, returns False. Otherwise returns True."
"Fails to follow this specification" is here not logically equivalent to:
"Returns True if x is an infinity or NaN, OR returns False on other input."
(as the comment above apparently thinks it would).
(I feel a bit silly even having to explain this.)
Rather, "fails to follow this specification" is logically equivalent to:
"Fails to return True if x is an infinity or NaN, OR fails to return False on other input."
Is 10**1000 a real number? Pretty much so. PEP-484 even thinks this is a "float"(!)
Does math.isfinite() fail to return False on this number? That also holds.
So, does math.isfinite() violate its own documentation?
How can there be even a discussion about that?
There is of course some sort-of open issue around "what is valid input here"?
The very fact that this is not clarified is in itself a problem in the documentation that needs rectifying.
However, note that the beginning of https://docs.python.org/3/library/math.html states:
"These functions cannot be used with complex numbers."
This is a clear case of "exceptio probat regulam in casibus non exceptis" - the "complex numbers" are here mentioned as an exception to the not explicitly stated rule that these functions can process not-complex, i.e. real, input.