Inspecting Python functions
I have a piece of C++ code that calls user-defined functions implemented in Python. Instead of requiring all functions to have the same signature with 6 arguments, the C++ code inspects the function signature and passes only the arguments function accepts - 1, 3, or all 6 of them. I use the inspect
module and getargspec
function for that but it feels a bit wrong and bloated. So let’s see how we can get the work done without the inspect
module.
For extra fun, I will look at Python 3.8 that has PEP 570 about positional-only arguments implemented.
A function definition using all features looks like this:
def f(pos1, pos2, /, pos_or_kwd, *, kwd1, kwd2):
Arguments pos1
and pos2
before the /
are positional-only arguments. You can call the following function like f(1, 2)
but not like f(a=1, b=2)
or you will get a TypeError
:
>>> def f(a, b, /): pass
...
>>> f(1, 2)
>>> f(a=1, b=2)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: f() got some positional-only arguments passed as keyword arguments: 'a, b'
Arguments kwd1
and kwd2
after the *
are keyword-only arguments. You can call the following function like f(a=1, b=2)
and f(b=2, a=1)
but not like f(1, 2)
or you will get a TypeError
:
>>> def f(*, a, b): pass
...
>>> f(a=1, b=2)
>>> f(b=2, a=1)
>>> f(1, 2)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: f() takes 0 positional arguments but 2 were given
And of course, without the positional-only and keyword-only arguments, you can call the function however you like. f(a=1, b=2)
or f(b=2, a=1)
or f(1, 2)
will work:
>>> def f(a, b): pass
...
>>> f(a=1, b=2)
>>> f(b=2, a=1)
>>> f(1, 2)
Let’s look how to inspect the following function containing most of the function argument features:
>>> def f(pos1, pos2=2, /, pos_or_kwd=3, *, kwd=4, kwd2=5, kwd3=6): pass
...
How to retrieve the number of arguments this function accepts? Python has a lot of dunder attributes containing interesting bits. First, under __code__
there are three interesting attributes:
>>> f.__code__.co_argcount
3
>>> f.__code__.co_kwonlyargcount
3
>>> f.__code__.co_posonlyargcount
2
__code__.co_argcount
will hold the value3
which is the number of arguments not including keyword-only arguments__code__.co_kwonlyargcount
will hold the value3
which is the number of keyword-only arguments__code__.co_posonlyargcount
will hold the value2
which is the number of positional-only arguments.
To get the number of function arguments you have to sum __code__.co_argcount
and __code__.co_kwonlyargcount
. However, __code__.co_posonlyargcount
is already included in __code__.co_argcount
.
To find the names of arguments, even the positional-only ones (!), we have to look at __code__.co_varnames
. It contains both the function arguments and local variables and we have to take only the first elements matching the argument count:
>>> f.__code__.co_varnames[:f.__code__.co_argcount+f.__code__.co_kwonlyargcount]
('pos1', 'pos2', 'pos_or_kwd', 'kwd', 'kwd2', 'kwd3')
What about the default values of the arguments? They live on the function object itself:
>>> f.__defaults__
(2, 3)
>>> f.__kwdefaults__
{'kwd': 4, 'kwd2': 5, 'kwd3': 6}
__defaults__
contains atuple
with default values orNone
if there are none. It has the values for all except keyword-only arguments.__kwdefaults__
contains adict
with the default values of keyword-only arguments.
For our function argument count (__code__.co_argcount
) is 3 but the __defaults__
contains just 2 default values. Since default arguments must follow non-default arguments, to match defaults with the actual argument names we can do something like this:
>>> dict(zip(
... f.__code__.co_varnames[: f.__code__.co_argcount][-len(f.__defaults__) :],
... f.__defaults__,
... ))
{'pos2': 2, 'pos_or_kwd': 3}
But what about *args
and **kwargs
for arbitrary positional and keyword arguments? Those are special:
>>> def f(*args, **kwargs): pass
...
>>> f.__code__.co_argcount
0
>>> bin(f.__code__.co_flags)
'0b1001111'
They do not affect the argument count (__code__.co_argcount
) and it will be 0
. Instead, the fact that the function accepts arbitrary arguments is reflected as bits inside flags (__code__.co_flags
). The third bit (or 4) indicates that the function accepts arbitrary positional arguments and the fourth bit (or 8) indicates that the function accepts arbitrary keyword arguments.
These arguments do not have to be named args
and kwargs
, it is just a naming convention. The real argument names can be found in __code__.co_varnames
but after all function arguments:
>>> def f(pos1, pos2=2, /, pos_or_kwd=3, *args, kwd=4, kwd2=5, kwd3=6, **kwargs): pass
...
>>> f.__code__.co_varnames
('pos1', 'pos2', 'pos_or_kwd', 'kwd', 'kwd2', 'kwd3', 'args', 'kwargs')
Long story short, I replaced the use of inspect
module with a few lines of direct pybind11
C++ code:
auto &&func = server.attr(svcinfo->name);
auto &&code = func.attr("__code__");
long argcount = (code.attr("co_argcount") + code.attr("co_kwonlyargcount"))
.cast<py::int_>();
auto &&args = code.attr("co_varnames")[py::slice(0, argcount, 1)];