Skip to content

Commit 5414529

Browse files
committed
Support Callable function parameters (GH-1424)
Signed-off-by: Eric Shi <ershi@nvidia.com>
1 parent bd91b99 commit 5414529

7 files changed

Lines changed: 612 additions & 13 deletions

File tree

CHANGELOG.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,8 @@
2929
([GH-1402](https://github.com/NVIDIA/warp/issues/1402)).
3030
- Add `wp.copysign(x, y)`: returns a value with the magnitude of `x` and the
3131
sign of `y`, matching C `copysign` ([GH-1444](https://github.com/NVIDIA/warp/issues/1444)).
32+
- Support passing user-defined Warp functions to `Callable` parameters in `@wp.func` functions
33+
([GH-1424](https://github.com/NVIDIA/warp/issues/1424)).
3234

3335
### Removed
3436

design/callable-func-parameters.md

Lines changed: 144 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,144 @@
1+
# Callable Function Parameters
2+
3+
**Status**: Implemented
4+
5+
**Issue**: [GH-1424](https://github.com/NVIDIA/warp/issues/1424)
6+
7+
## Motivation
8+
9+
Warp users can write higher-order Python helpers that accept a callable and
10+
apply it to values, but the same pattern did not work inside user-defined
11+
`@wp.func` code. A function parameter annotated as `Callable` was not matched
12+
consistently during Warp overload resolution, especially across the two standard
13+
library import paths:
14+
15+
```python
16+
from typing import Callable as TypingCallable
17+
from collections.abc import Callable as AbcCallable
18+
```
19+
20+
The aliases are illustrative; Warp treats both origins as the same type-erased
21+
`Callable` marker. This design implements the user-defined function portion of
22+
GH-1424 as a first pass: support direct inline calls such as
23+
`apply(double_it, 3.0)` from a kernel or another `@wp.func`, without depending
24+
on local function-variable assignment behavior.
25+
26+
## Requirements
27+
28+
| ID | Requirement | Priority | Notes |
29+
| --- | ----------- | -------- | ----- |
30+
| R1 | Recognize `typing.Callable` and `collections.abc.Callable` as callable annotations. | Must | Includes bare and parameterized forms. |
31+
| R2 | Allow user-defined `@wp.func` objects to match `Callable` parameters. | Must | Callable values are type-erased. |
32+
| R3 | Preserve the existing `"c"` type code for callable annotations. | Must | Used by module hashing and overload keys. |
33+
| R4 | Keep annotation recognition compatible with Python 3.10 through 3.14. | Must | Avoid private `typing` internals. |
34+
| R5 | Include callable argument and default targets in module hashes and dependencies. | Must | Prevent stale compiled modules when callable targets change. |
35+
| R6 | Reject unsupported callable specializations explicitly. | Must | Built-ins are deferred; custom grad/replay functions are first-pass non-goals. |
36+
37+
**Non-goals**:
38+
39+
- Validate parameterized callable signatures such as
40+
`Callable[[float], float]`.
41+
- Implement runtime dispatch for arbitrary Python callables.
42+
- Support kernel-local function variable assignment.
43+
- Support built-in Warp functions such as `wp.sin` or `wp.add` as callable
44+
arguments in this first pass. They are rejected until built-in callable
45+
identity can participate safely in specialization hashing and dependency
46+
tracking.
47+
- Support callable-specialized functions that have custom gradient or replay
48+
functions.
49+
50+
Parameterized callable annotations are recognized as callable markers, but
51+
their argument and return types remain unchecked.
52+
53+
## Design
54+
55+
### Approach
56+
57+
The implementation treats `Callable` parameters as compile-time function
58+
references. A callable parameter never becomes a runtime C++ function pointer or
59+
kernel parameter. Instead, code generation specializes the user-defined function
60+
for the concrete user-defined Warp function passed at each call site.
61+
62+
This means:
63+
64+
- `apply(double_it, x)` and `apply(triple_it, x)` produce separate specialized
65+
native functions.
66+
- Callable parameters are bound into the specialized function's codegen symbol
67+
table.
68+
- Callable parameters are omitted from the emitted forward and reverse C++
69+
function signatures.
70+
- Runtime calls pass only the non-callable arguments.
71+
72+
The callable annotation predicate lives in `warp/_src/types.py` as
73+
`is_callable_annotation(annotation)`. It returns true for:
74+
75+
- bare `typing.Callable`,
76+
- bare `collections.abc.Callable`,
77+
- `typing.Callable[...]`,
78+
- `collections.abc.Callable[...]`.
79+
80+
The helper uses `typing.get_origin()` plus the canonical
81+
`collections.abc.Callable` object, which covers the supported Python versions
82+
without relying on private implementation details.
83+
84+
### Alternatives Considered
85+
86+
One option was to normalize annotations globally during `@wp.func` registration.
87+
That would make all `Callable` forms identical up front, but it would also
88+
change the raw annotations stored on every function and could affect unrelated
89+
signature handling. A small predicate keeps the behavior localized.
90+
91+
Another option was to support built-in Warp functions as callable values.
92+
GH-1424 includes built-ins, but this first pass narrows support to user-defined
93+
targets. Built-ins require hashing and dependency behavior for callable
94+
identity, so they are rejected explicitly to avoid partial support that could
95+
reuse stale cached modules.
96+
97+
### Key Implementation Details
98+
99+
`get_type_code()` returns `"c"` for callable annotations before the generic
100+
type branches. This keeps module hash type-code behavior stable for bare and
101+
parameterized `Callable` forms.
102+
103+
`func_match_args()` treats a `warp._src.context.Function` value as compatible
104+
with any callable annotation. Non-function values continue to fail normal
105+
overload resolution.
106+
107+
During `Adjoint.add_call()`, default arguments are applied first. Callable
108+
arguments and callable defaults are then collected. Built-in function values are
109+
rejected, and user-defined function values trigger creation of a specialized
110+
`Function` clone. The clone receives:
111+
112+
- a hash-suffixed native function name,
113+
- a fresh `Adjoint` with the original annotations, including the return
114+
annotation,
115+
- `callable_arg_values` used to bind callable parameter names during codegen.
116+
117+
Module hashing and dependency discovery both inspect callable arguments and
118+
callable defaults. User-defined callable targets are included in the referenced
119+
function set for hashes and in module references/dependents for invalidation.
120+
121+
Callable-specialized functions with custom gradients or custom replay functions
122+
are rejected. Their custom functions are tied to the unspecialized native
123+
function signature, while callable specialization removes callable runtime
124+
parameters from the emitted signature.
125+
126+
## Testing Strategy
127+
128+
`warp/tests/test_func.py` covers:
129+
130+
- bare `typing.Callable` and `collections.abc.Callable` runtime calls,
131+
- parameterized `Callable[[float], float]` runtime calls,
132+
- generic user functions that combine `Callable` with `Any`,
133+
- default callable arguments,
134+
- keyword callable arguments,
135+
- nested user-defined function calls,
136+
- callable targets affecting module hashes,
137+
- callable targets updating cross-module dependents,
138+
- return annotation preservation on specialized functions,
139+
- explicit rejection for built-in callable targets,
140+
- explicit rejection for callable-specialized functions with custom grad or
141+
replay functions.
142+
143+
Local verification should include the focused `TestFunc` suite and pre-commit
144+
over the changed files. CI provides the full Python 3.10 through 3.14 matrix.

docs/user_guide/limitations.rst

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,11 @@ Kernels and User Functions
3535
(e.g., ``wp.float64(wp.PI)`` or ``wp.int64(large_value)``).
3636
* Python ``IntFlag`` values behave like raw integers in Warp kernels: bitwise negation (``~``)
3737
produces the integer negation, not a masked combination of flags as in standard Python ``IntFlag`` behavior.
38+
* ``Callable`` parameters in user functions only support direct inline calls with user-defined
39+
:func:`@wp.func <warp.func>` targets.
40+
Arbitrary Python callables, built-in Warp functions such as ``wp.sin`` or ``wp.add``,
41+
kernel-local function variable assignment, and callable-specialized functions with custom gradient
42+
or replay functions are not supported.
3843

3944
A limitation of Warp is that each dimension of the grid used to launch a kernel must be representable as a 32-bit
4045
signed integer. Therefore, no single dimension of a grid should exceed :math:`2^{31}-1`.

0 commit comments

Comments
 (0)