Option for arg/return type hints and correct typing for std::filesystem::path (pybind#5450)

* Added arg/return type handling.

* Added support for nested arg/return type in py::typing::List

* Added support for arg/return type in stl/filesystem

* Added tests for arg/return type in stl/filesystem and py::typing::List

* Added arg/return name to more py::typing classes

* Added arg/return type to Callable[...]

* Added tests for typing container classes (also nested)

* Changed typing classes to avoid using C++14 auto return type deduction.

* Fixed clang-tidy errors.

* Changed Enable to SFINAE

* Added test for Tuple[T, ...]

* Added RealNumber with custom caster for testing typing classes.

* Added tests for Set, Iterable, Iterator, Union, and Optional

* Added tests for Callable

* Fixed Callable with ellipsis test

* Changed TypeGuard/TypeIs to use return type (being the narrower type) + Tests

* Added test for use of fallback type name with stl vector

* Updated documentation.

* Fixed unnecessary constructor call in test.

* Fixed reference counting in example type caster.

* Fixed clang-tidy issues.

* Fix for clang-tidy

* Updated cast method to use pybind11 API rather than Python C API in custom caster example

* Updated load to use pybind11 API rather than Python C API in custom caster example

* Changed test of arg/return name to use pybind11 API instead of Python C API

* Updated code in adcanced/cast example and improved documentation text

* Fixed references in custom type caster docs

* Fixed wrong logical and operator in test

* Fixed wrong logical operator in doc example

* Added comment to test about `float` vs `float | int`

* Updated std::filesystem::path docs in cast/overview section

* Remove one stray dot.

---------

Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>

Author: timohl

docs/advanced/cast/custom.rst

@@ -1,35 +1,53 @@
 Custom type casters
 ===================
 
-In very rare cases, applications may require custom type casters that cannot be
-expressed using the abstractions provided by pybind11, thus requiring raw
-Python C API calls. This is fairly advanced usage and should only be pursued by
-experts who are familiar with the intricacies of Python reference counting.
-
-The following snippets demonstrate how this works for a very simple ``inty``
-type that that should be convertible from Python types that provide a
-``__int__(self)`` method.
+Some applications may prefer custom type casters that convert between existing
+Python types and C++ types, similar to the ``list`` ↔ ``std::vector``
+and ``dict`` ↔ ``std::map`` conversions which are built into pybind11.
+Implementing custom type casters is fairly advanced usage.
+While it is recommended to use the pybind11 API as much as possible, more complex examples may
+require familiarity with the intricacies of the Python C API.
+You can refer to the `Python/C API Reference Manual <https://docs.python.org/3/c-api/index.html>`_
+for more information.
+
+The following snippets demonstrate how this works for a very simple ``Point2D`` type.
+We want this type to be convertible to C++ from Python types implementing the
+``Sequence`` protocol and having two elements of type ``float``.
+When returned from C++ to Python, it should be converted to a Python ``tuple[float, float]``.
+For this type we could provide Python bindings for different arithmetic functions implemented
+in C++ (here demonstrated by a simple ``negate`` function).
+
+..
+    PLEASE KEEP THE CODE BLOCKS IN SYNC WITH
+        tests/test_docs_advanced_cast_custom.cpp
+        tests/test_docs_advanced_cast_custom.py
+    Ideally, change the test, run pre-commit (incl. clang-format),
+    then copy the changed code back here.
+    Also use TEST_SUBMODULE in tests, but PYBIND11_MODULE in docs.
 
 .. code-block:: cpp
 
-    struct inty { long long_value; };
+    namespace user_space {
 
-    void print(inty s) {
-        std::cout << s.long_value << std::endl;
-    }
+    struct Point2D {
+        double x;
+        double y;
+    };
 
-The following Python snippet demonstrates the intended usage from the Python side:
+    Point2D negate(const Point2D &point) { return Point2D{-point.x, -point.y}; }
 
-.. code-block:: python
+    } // namespace user_space
 
-    class A:
-        def __int__(self):
-            return 123
 
+The following Python snippet demonstrates the intended usage of ``negate`` from the Python side:
+
+.. code-block:: python
 
-    from example import print
+    from my_math_module import docs_advanced_cast_custom as m
 
-    print(A())
+    point1 = [1.0, -1.0]
+    point2 = m.negate(point1)
+    assert point2 == (-1.0, 1.0)
 
 To register the necessary conversion routines, it is necessary to add an
 instantiation of the ``pybind11::detail::type_caster<T>`` template.
@@ -38,56 +56,82 @@ type is explicitly allowed.
 
 .. code-block:: cpp
 
-    namespace PYBIND11_NAMESPACE { namespace detail {
-        template <> struct type_caster<inty> {
-        public:
-            /**
-             * This macro establishes the name 'inty' in
-             * function signatures and declares a local variable
-             * 'value' of type inty
-             */
-            PYBIND11_TYPE_CASTER(inty, const_name("inty"));
-
-            /**
-             * Conversion part 1 (Python->C++): convert a PyObject into a inty
-             * instance or return false upon failure. The second argument
-             * indicates whether implicit conversions should be applied.
-             */
-            bool load(handle src, bool) {
-                /* Extract PyObject from handle */
-                PyObject *source = src.ptr();
-                /* Try converting into a Python integer value */
-                PyObject *tmp = PyNumber_Long(source);
-                if (!tmp)
+    namespace pybind11 {
+    namespace detail {
+
+    template <>
+    struct type_caster<user_space::Point2D> {
+        // This macro inserts a lot of boilerplate code and sets the default type hint to `tuple`
+        PYBIND11_TYPE_CASTER(user_space::Point2D, const_name("tuple"));
+        // `arg_name` and `return_name` may optionally be used to specify type hints separately for
+        // arguments and return values.
+        // The signature of our negate function would then look like:
+        // `negate(Sequence[float]) -> tuple[float, float]`
+        static constexpr auto arg_name = const_name("Sequence[float]");
+        static constexpr auto return_name = const_name("tuple[float, float]");
+
+        // C++ -> Python: convert `Point2D` to `tuple[float, float]`. The second and third arguments
+        // are used to indicate the return value policy and parent object (for
+        // return_value_policy::reference_internal) and are often ignored by custom casters.
+        // The return value should reflect the type hint specified by `return_name`.
+        static handle
+        cast(const user_space::Point2D &number, return_value_policy /*policy*/, handle /*parent*/) {
+            return py::make_tuple(number.x, number.y).release();
+        }
+
+        // Python -> C++: convert a `PyObject` into a `Point2D` and return false upon failure. The
+        // second argument indicates whether implicit conversions should be allowed.
+        // The accepted types should reflect the type hint specified by `arg_name`.
+        bool load(handle src, bool /*convert*/) {
+            // Check if handle is a Sequence
+            if (!py::isinstance<py::sequence>(src)) {
+                return false;
+            }
+            auto seq = py::reinterpret_borrow<py::sequence>(src);
+            // Check if exactly two values are in the Sequence
+            if (seq.size() != 2) {
+                return false;
+            }
+            // Check if each element is either a float or an int
+            for (auto item : seq) {
+                if (!py::isinstance<py::float_>(item) && !py::isinstance<py::int_>(item)) {
                     return false;
-                /* Now try to convert into a C++ int */
-                value.long_value = PyLong_AsLong(tmp);
-                Py_DECREF(tmp);
-                /* Ensure return code was OK (to avoid out-of-range errors etc) */
-                return !(value.long_value == -1 && !PyErr_Occurred());
+                }
             }
+            value.x = seq[0].cast<double>();
+            value.y = seq[1].cast<double>();
+            return true;
+        }
+    };
 
-            /**
-             * Conversion part 2 (C++ -> Python): convert an inty instance into
-             * a Python object. The second and third arguments are used to
-             * indicate the return value policy and parent object (for
-             * ``return_value_policy::reference_internal``) and are generally
-             * ignored by implicit casters.
-             */
-            static handle cast(inty src, return_value_policy /* policy */, handle /* parent */) {
-                return PyLong_FromLong(src.long_value);
-            }
-        };
-    }} // namespace PYBIND11_NAMESPACE::detail
+    } // namespace detail
+    } // namespace pybind11
+
+    // Bind the negate function
+    PYBIND11_MODULE(docs_advanced_cast_custom, m) { m.def("negate", user_space::negate); }
 
 .. note::
 
     A ``type_caster<T>`` defined with ``PYBIND11_TYPE_CASTER(T, ...)`` requires
     that ``T`` is default-constructible (``value`` is first default constructed
     and then ``load()`` assigns to it).
 
+.. note::
+    For further information on the ``return_value_policy`` argument of ``cast`` refer to :ref:`return_value_policies`.
+    To learn about the ``convert`` argument of ``load`` see :ref:`nonconverting_arguments`.
+
 .. warning::
 
     When using custom type casters, it's important to declare them consistently
-    in every compilation unit of the Python extension module. Otherwise,
+    in every compilation unit of the Python extension module to satisfy the C++ One Definition Rule
+    (`ODR <https://en.cppreference.com/w/cpp/language/definition>`_). Otherwise,
     undefined behavior can ensue.
+
+.. note::
+
+    Using the type hint ``Sequence[float]`` signals to static type checkers, that not only tuples may be
+    passed, but any type implementing the Sequence protocol, e.g., ``list[float]``.
+    Unfortunately, that loses the length information ``tuple[float, float]`` provides.
+    One way of still providing some length information in type hints is using ``typing.Annotated``, e.g.,
+    ``Annotated[Sequence[float], 2]``, or further add libraries like
+    `annotated-types <https://github.com/annotated-types/annotated-types>`_.

docs/advanced/cast/overview.rst

@@ -151,7 +151,7 @@ as arguments and return values, refer to the section on binding :ref:`classes`.
 +------------------------------------+---------------------------+-----------------------------------+
 | ``std::variant<...>``              | Type-safe union (C++17)   | :file:`pybind11/stl.h`            |
 +------------------------------------+---------------------------+-----------------------------------+
-| ``std::filesystem::path<T>``       | STL path (C++17) [#]_     | :file:`pybind11/stl/filesystem.h` |
+| ``std::filesystem::path``          | STL path (C++17) [#]_     | :file:`pybind11/stl/filesystem.h` |
 +------------------------------------+---------------------------+-----------------------------------+
 | ``std::function<...>``             | STL polymorphic function  | :file:`pybind11/functional.h`     |
 +------------------------------------+---------------------------+-----------------------------------+
@@ -167,4 +167,4 @@ as arguments and return values, refer to the section on binding :ref:`classes`.
 +------------------------------------+---------------------------+-----------------------------------+
 
 .. [#] ``std::filesystem::path`` is converted to ``pathlib.Path`` and
-   ``os.PathLike`` is converted to ``std::filesystem::path``.
+   can be loaded from ``os.PathLike``, ``str``, and ``bytes``.

include/pybind11/cast.h

@@ -34,6 +34,39 @@ PYBIND11_WARNING_DISABLE_MSVC(4127)
 
 PYBIND11_NAMESPACE_BEGIN(detail)
 
+// Type trait checker for `descr`
+template <typename>
+struct is_descr : std::false_type {};
+
+template <size_t N, typename... Ts>
+struct is_descr<descr<N, Ts...>> : std::true_type {};
+
+template <size_t N, typename... Ts>
+struct is_descr<const descr<N, Ts...>> : std::true_type {};
+
+// Use arg_name instead of name when available
+template <typename T, typename SFINAE = void>
+struct as_arg_type {
+    static constexpr auto name = T::name;
+};
+
+template <typename T>
+struct as_arg_type<T, typename std::enable_if<is_descr<decltype(T::arg_name)>::value>::type> {
+    static constexpr auto name = T::arg_name;
+};
+
+// Use return_name instead of name when available
+template <typename T, typename SFINAE = void>
+struct as_return_type {
+    static constexpr auto name = T::name;
+};
+
+template <typename T>
+struct as_return_type<T,
+                      typename std::enable_if<is_descr<decltype(T::return_name)>::value>::type> {
+    static constexpr auto name = T::return_name;
+};
+
 template <typename type, typename SFINAE = void>
 class type_caster : public type_caster_base<type> {};
 template <typename type>
@@ -1080,6 +1113,8 @@ struct pyobject_caster {
         return src.inc_ref();
     }
     PYBIND11_TYPE_CASTER(type, handle_type_name<type>::name);
+    static constexpr auto arg_name = as_arg_type<handle_type_name<type>>::name;
+    static constexpr auto return_name = as_return_type<handle_type_name<type>>::name;
 };
 
 template <typename T>
@@ -1608,7 +1643,7 @@ class argument_loader {
                   "py::args cannot be specified more than once");
 
     static constexpr auto arg_names
-        = ::pybind11::detail::concat(type_descr(make_caster<Args>::name)...);
+        = ::pybind11::detail::concat(type_descr(as_arg_type<make_caster<Args>>::name)...);
 
     bool load_args(function_call &call) { return load_impl_sequence(call, indices{}); }
 

include/pybind11/pybind11.h

@@ -336,8 +336,8 @@ class cpp_function : public function {
 
         /* Generate a readable signature describing the function's arguments and return
            value types */
-        static constexpr auto signature
-            = const_name("(") + cast_in::arg_names + const_name(") -> ") + cast_out::name;
+        static constexpr auto signature = const_name("(") + cast_in::arg_names
+                                          + const_name(") -> ") + as_return_type<cast_out>::name;
         PYBIND11_DESCR_CONSTEXPR auto types = decltype(signature)::types();
 
         /* Register the function with Python from generic (non-templated) code */

include/pybind11/stl/filesystem.h

@@ -107,6 +107,8 @@ struct path_caster {
     }
 
     PYBIND11_TYPE_CASTER(T, const_name("os.PathLike"));
+    static constexpr auto arg_name = const_name("Union[os.PathLike, str, bytes]");
+    static constexpr auto return_name = const_name("Path");
 };
 
 #endif // PYBIND11_HAS_FILESYSTEM || defined(PYBIND11_HAS_EXPERIMENTAL_FILESYSTEM)