From 4d31b9ae2968773e7c01d9b5627c8ff35351ac25 Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Fri, 28 Nov 2025 18:11:04 +0100 Subject: [PATCH 1/8] New first chapter of a C API tutorial --- Doc/c-api/intro.rst | 40 ++ Doc/extending/first-extension-module.rst | 431 ++++++++++++++++++++ Doc/extending/index.rst | 49 ++- Doc/includes/capi-extension/spammodule-01.c | 73 ++++ Doc/includes/tutorial-new-api.rst | 9 + 5 files changed, 593 insertions(+), 9 deletions(-) create mode 100644 Doc/extending/first-extension-module.rst create mode 100644 Doc/includes/capi-extension/spammodule-01.c create mode 100644 Doc/includes/tutorial-new-api.rst diff --git a/Doc/c-api/intro.rst b/Doc/c-api/intro.rst index bb94bcb86a7899..5e90d9b7bc91ed 100644 --- a/Doc/c-api/intro.rst +++ b/Doc/c-api/intro.rst @@ -107,6 +107,46 @@ header files properly declare the entry points to be ``extern "C"``. As a result there is no need to do anything special to use the API from C++. +.. _capi-system-includes: + +System includes +--------------- + + :file:`Python.h` includes several standard header files. + C extensions should include the standard headers that they use, + and should not rely on these implicit includes. + The implicit includes are: + + * ```` + * ```` (on Windows) + * ```` + * ```` + * ```` + * ```` + * ```` + * ```` (if present) + + The following are included for backwards compatibility, unless using + :ref:`Limited API ` 3.13 or newer: + + * ```` + * ```` (on POSIX) + + The following are included for backwards compatibility, unless using + :ref:`Limited API ` 3.11 or newer: + + * ```` + * ```` + * ```` + * ```` + +.. note:: + + Since Python may define some pre-processor definitions which affect the standard + headers on some systems, you *must* include :file:`Python.h` before any standard + headers are included. + + Useful macros ============= diff --git a/Doc/extending/first-extension-module.rst b/Doc/extending/first-extension-module.rst new file mode 100644 index 00000000000000..d8f6628f716fe2 --- /dev/null +++ b/Doc/extending/first-extension-module.rst @@ -0,0 +1,431 @@ +.. highlight:: c + + +********************************* +Your first C API extension module +********************************* + +This tutorial will take you, line by line, through creating a simple +Python extension module written in C or C++. + +This document assumes basic knowledge about Python: you should be able to +define functions in Python code before writing them in another language. +See :ref:`tutorial-index` for an introduction to Python itself. + +It also assumes a working knowledge of the C language. +We will use several concepts that a C beginner would not be expected to know, +like ``static`` functions or linkage declarations, but the tutorial should +be useful for anyone who can write a basic C library. + +As a word warning before we begin: the compilation of an extension module can +be tricky, as it depends on how your system is set up, and on how your Python +is installed. +It is generally best to use a third-party tool to handle the details. +This is covered in later chapters. + +The tutorial assumes that you use a Unix system (including macOS) or Windows. + +.. include:: ../includes/tutorial-new-api.rst + + +What we'll do +============= + +Let's create an extension module called ``spam`` (the favorite food of Monty +Python fans...) and let's say we want to create a Python interface to the C +standard library function :c:func:`system`. +This function takes a C string as argument, runs the argument as a system +command, and returns a result value as an integer. +In documentation, it might be summarized as:: + + #include + + int system(const char *command); + +Note that like many functions in the C standard library, +this function is already exposed in Python, as :py:func:`os.system`. +We'll make our own "wrapper". + +We want this function to be callable from Python as follows: + +.. code-block:: pycon + + >>> import spam + >>> status = spam.system("whoami") + User Name + >>> status + 0 + +.. note:: + + The system command ``whoami`` prints out your username. + It's useful in tutorials like this one because it has the same name on + both Unix and Windows. + + +.. _extending-spammodule: + +The ``spam`` module +=================== + + +Begin by creating a file :file:`spammodule.c`. + +.. note:: + + Historically, if a module is called ``spam``, the C file containing its + implementation is called :file:`spammodule.c`. + But the naming is entirely up to you. + If you use a different name, remember to adjust the commands later, + when you build the extension. + +You can add code to this file as you go through the tutorial. +Alternatively, if you'd like to copy the final result to the file now, +get it :ref:`below ` and come back here. + +The first line of this file (after any comment describing the purpose of +the module, copyright notices, and the like) will pull in the Python API: + +.. literalinclude:: ../includes/capi-extension/spammodule-01.c + :start-at: + :end-at: + +Next, we bring in the declaration of the C library function we want to call. +Documentation of the :c:func:`system` function tells us that we need +``stdlib.h``: + +.. literalinclude:: ../includes/capi-extension/spammodule-01.c + :start-at: + :end-at: + +Be sure to put this, and any other standard library includes, *after* +:file:`Python.h`, since Python may define some pre-processor definitions +which affect the standard headers on some systems. + +.. tip:: + + The ```` include is technically not necessary. + :file:`Python.h` :ref:`includes several standard header files ` + for its own use and for backwards compatibility, + and ``stdlib`` is one of them. + However, it is good practice to explicitly include what you need. + +The next thing we add to our module file is the C function that will be called +when the Python expression ``spam.system(string)`` is evaluated. +We'll see shortly how it ends up being called. +The function should look like this: + +.. literalinclude:: ../includes/capi-extension/spammodule-01.c + :start-after: /// Implementation of spam.system + :end-before: /// + +Let's go through this line by line. + +This function will not be needed outside the current ``.c`` file, so we will +define it as ``static``. +(This can prevent name conflicts with similarly named functions elsewhere.) +It will return a pointer to a Python object -- the result that Python code +should receive when it calls ``spam.system``: + +.. literalinclude:: ../includes/capi-extension/spammodule-01.c + :start-at: static PyObject * + :end-at: static PyObject * + +We'll name the function ``spam_system``. +Combining the module name and the function name like this is standard practice, +and avoids clashes with other uses of ``system``. + +The Python function we are defining will take a single argument. +Its C implementation takes two arguments, here named *self* and *arg*, +both pointers to a Python object: + +.. literalinclude:: ../includes/capi-extension/spammodule-01.c + :start-at: spam_system(PyObject *self, PyObject *arg) + :end-at: { + +Since this is a module-level function, the *self* argument will point to +a module object. +For a method, *self* would point to the object instance, like a *self* +argument in Python. + +The *arg* will be a pointer to a Python object that the Python function +received as its single argument. + +In order for our C code to use the information in a Python object, we will need +to convert it to a C value --- in this case, a C string (``char *``). +The :c:func:`PyUnicode_AsUTF8` function does just that: it decodes a Python +string to a C one: + +.. literalinclude:: ../includes/capi-extension/spammodule-01.c + :start-at: const char *command = PyUnicode_AsUTF8(arg); + :end-at: const char *command = PyUnicode_AsUTF8(arg); + +If :c:func:`PyUnicode_AsUTF8` was successful, the string value of the argument +has been stored in the local variable *command*. +This is a C string, that is, a pointer to an array of bytes. +You are not supposed to modify the string to which it points, +which is why the variable is declared as ``const char *command``. + +.. note:: + + As its name implies, :c:func:`PyUnicode_AsUTF8` uses the UTF-8 encoding + --- the same that :py:meth:`str.decode` uses by default. + This is not always the correct encoding to use for system commands, but + it will do for our purposes. + +If :c:func:`PyUnicode_AsUTF8` was *not* successful, it returns a ``NULL`` +pointer. +When using the Python C API, we always need to handle such error cases. +Let's study this one in a bit more detail. + +Errors and exceptions +--------------------- + +An important convention throughout the Python interpreter is the following: +when a function fails, it should do two things: ensure an exception is set, +and return an error value. + +The error value is usually ``-1`` for functions that return :c:type:`!int`, +and ``NULL`` for functions that return a pointer. +However, this convention is not used in *all* cases. +You should always check the documentation for the functions you use. + +"Setting an exception" means that an exception object is stored in the +interpreter's thread state. +(If you are not familiar with threads, think of it like a global variable.) + +In our example, :c:func:`PyUnicode_AsUTF8` follows the convention: +on failure, it sets an exception and returns ``NULL``. + +Our function needs to detect this, and also return the ``NULL`` +error indicator: + +.. literalinclude:: ../includes/capi-extension/spammodule-01.c + :start-at: if (command == NULL) { + :end-at: } + +Our function does not neeed to set an exception itself, since +:c:func:`PyUnicode_AsUTF8` has already set it in the error case. + +In the non-error case, the ``return`` statement is skipped and execution +continues. + +.. note:: + + If we were calling a function that does *not* follow the Python convention + (for example, :c:func:`malloc`), we would need to set an exception using, + for example, :c:func:`PyErr_SetString`. + + +Back to the example +------------------- + +The next statement is a call to the standard C function :c:func:`system`, +passing it the C string we just got, and storing its result --- a C +integer: + +.. literalinclude:: ../includes/capi-extension/spammodule-01.c + :start-at: int status = system(command); + :end-at: int status = system(command); + +Our function must then convert the C integer into a Python integer. +This is done using the function :c:func:`PyLong_FromLong`: + +.. literalinclude:: ../includes/capi-extension/spammodule-01.c + :start-at: return PyLong_FromLong(status); + :end-at: return PyLong_FromLong(status); + +In this case, it will return a Python :py:class:`int` object. +(Yes, even integers are objects on the heap in Python!) +We return this object directly, as the result of our function. + +Note that when :c:func:`PyLong_FromLong` fails, it sets an exception and +returns ``NULL``, just like :c:func:`PyUnicode_AsUTF8` we used before. +In this case, our function should --- also like before --- return ``NULL`` +as well. +In both cases --- on success and on error --- we are returning +:c:func:`PyLong_FromLong`'s result, so we do not need an ``if`` here. + +.. note:: + + The names used in the C API are peculiar: + + * ``PyUnicode`` is used for Python :py:type:`str` + * ``PyLong`` is used for Python :py:type:`int` + + This convention dates back to the time when Python strings were implemented + by what is now :py:type:`bytes`, and integers were restricted to 64 bits + or so. + Unicode strings (then named :py:class:`!unicode`) and arbitrarily large + integers (then named :py:class:`!long`) were added later, + and in time, were renamed to :py:type:`str` and :py:type:`int` we know + today. + + However, the C API retains the old naming for backwards compatibility. + + Thus, the :c:func:`PyLong_FromLong` function creates a ``PyLong`` + (Python :py:class:`int`) object from a C :c:type:`!long`. + (C converts from C :c:type:`!int` to C :c:type:`!long` automatically.) + And the :c:func:`PyUnicode_AsUTF8` function represents a ``PyUnicode`` + (Python :py:class:`str`) object as a UTF-8-encoded, NUL-terminatedC string. + +That is it for the C implementation of the ``spam.function``! + +.. literalinclude:: ../includes/capi-extension/spammodule-01.c + :start-at: } + :end-at: } + +But, we are not done yet: we still need to define how the implementation +is called from Python code. + + +The module's method table +========================= + +To make our function available to Python programs, +we need to list it in a "method table" -- an array of :c:type:`PyMethodDef` +structures with a zero-filled *sentinel* +marking the end of the array. + +.. note:: Why not "PyFunctionDef"? + + The C API uses a common mechanism to define both functions of modules + and methods of classes. + You might say that, in the C API, module-level functions act like methods + of a module object. + +Since we are defining a single function, the array will have a single entry +before the sentinel: + +.. literalinclude:: ../includes/capi-extension/spammodule-01.c + :start-after: /// Module method table + :end-before: /// + +The :c:member:`!PyMethodDef.ml_name` and :c:member:`!PyMethodDef.ml_meth` +fields contain the Python name of the function, and the address of +the implementation. + +The :c:member:`!PyMethodDef.ml_flags` field tells the interpreter +the calling convention to be used for the C function. +In particular, :c:data:`METH_O` specifies that this function takes a single +unnamed (positional-only) argument. +This is among the simplest, but least powerful, calling conventions. +Perfect for your first function! + +Finally, :c:member:`!PyMethodDef.ml_doc` gives the docstring. +If this is left out (or ``NULL``), the function will not have a docstring. +The :c:func:`PyDoc_STR` macro is used to save a bit of memory when building +without docstrings. + + +ABI information +=============== + +Before defining the module itself, we'll add one more piece of boilerplate: +a variable with ABI compatibility information. +This is a variable that CPython checks to avoid loading incompatible modules +--- it allows raising an exception rather than crashing (or worse) +if we, for example, load the extension on an incompatible Python version. + +The ABI information defined using a macro that collects the relevant +build-time settings: + +.. literalinclude:: ../includes/capi-extension/spammodule-01.c + :start-after: /// ABI information + :end-before: /// + +This macro expands to a variable definition like +``static PyABIInfo abi_info = { ... data ... };`` + + +The slot table and export hook +============================== + +Now, let's fit all the pieces of our module together. + +The method table we just defined, ``spam_methods``, +must be referenced in the module definition table: +an array of :c:type:`PyModuleDef_Slot` structures. +Like with the method table, a zero-filled *sentinel* marks the end. + +Besides the method table, this "slot table" will contain the module's +top-level information: the name, a docstring, and the +ABI compatibility slot we defined earlier: + +.. literalinclude:: ../includes/capi-extension/spammodule-01.c + :start-after: /// Module slot table + :end-before: /// + +This structure, in turn, must be passed to the interpreter in the module's +:ref:`export hook ` function. +The hook must be named :c:func:`!PyModExport_name`, where *name* is the name +of the module, and it should be the only non-\ ``static`` item defined in the +module file. + +Several modern C compilers emit warnings when you define a public function +without a prototype declaration. +Normally, prototypes go in a header file so they can be used from other +C files. +A function without a prototype is allowed in C, but is normally a strong +hint that something is wrong -- for example, the name is misspelled. + +In our case, Python will load our function dynamically, so a prototype +isn't needed. +We'll only add one to avoid well-meaning compiler warnings: + +.. literalinclude:: ../includes/capi-extension/spammodule-01.c + :start-after: /// Export hook prototype + :end-before: /// + +The :c:macro:`PyMODEXPORT_FUNC` macro declares the function's +``PyModuleDef_Slot *`` return type, and adds any special linkage +declarations required by the platform. +For C++, it declares the function as ``extern "C"``. + +Just after the prototype, we'll add the function itself. +Its only job is to return the slot array, which in turn contains +all the information needed to create our module object: + +.. literalinclude:: ../includes/capi-extension/spammodule-01.c + :start-after: /// Module export hook + :end-before: /// + + +Legacy initialization function +============================== + +Lastly, we will add a function that serves no purpose but to detect +installation mistakes, and to avoid errors with older versions of some common +build tools. +You may want to try building the extension without this, and only add it +if you encounter errors involving ``PyInit``: + +.. literalinclude:: ../includes/capi-extension/spammodule-01.c + :start-after: /// Legacy initialization function + +This is an old-style :ref:`initialization function ` +that was required in extension modules for CPython 3.14 and below. +Current CPython will not call it, but some build tools may still assume that +all extension modules need to define it. + +When called, it raises a :py:exc:`SystemError` exception using +:c:func:`PyErr_SetString`, and returns the error indicator, ``NULL``. +Thus, if this extension does end up loaded on Python 3.14, the user will +get a proper error message. + + +End of file +=========== + +That's it! +You have written a simple Python C API extension module. +The result is repeated below. + +Now, you'll need to compile it. +We'll see how to do this in :ref:`building`. + +Here is the entire source file, for your convenience: + +.. _extending-spammodule-source: + +.. literalinclude:: ../includes/capi-extension/spammodule-01.c diff --git a/Doc/extending/index.rst b/Doc/extending/index.rst index 4cc2c96d8d5b47..e89fdf55903219 100644 --- a/Doc/extending/index.rst +++ b/Doc/extending/index.rst @@ -4,16 +4,18 @@ Extending and Embedding the Python Interpreter ################################################## -This document describes how to write modules in C or C++ to extend the Python -interpreter with new modules. Those modules can not only define new functions -but also new object types and their methods. The document also describes how +This tutorial describes how to write modules in C or C++ to extend the Python +interpreter with new modules. Those modules can do what Python code does -- +define functions, object types and methods -- but also interact with native +libraries or achieve better performance by avoiding the overhead of an +interpreter. The document also describes how to embed the Python interpreter in another application, for use as an extension language. Finally, it shows how to compile and link extension modules so that they can be loaded dynamically (at run time) into the interpreter, if the underlying operating system supports this feature. -This document assumes basic knowledge about Python. For an informal -introduction to the language, see :ref:`tutorial-index`. :ref:`reference-index` +This document assumes basic knowledge about C and Python. For an informal +introduction to Python, see :ref:`tutorial-index`. :ref:`reference-index` gives a more formal definition of the language. :ref:`library-index` documents the existing object types, functions and modules (both built-in and written in Python) that give the language its wide application range. @@ -21,6 +23,24 @@ Python) that give the language its wide application range. For a detailed description of the whole Python/C API, see the separate :ref:`c-api-index`. +To support extensions, Python's C API (Application Programmers Interface) +defines a set of functions, macros and variables that provide access to most +aspects of the Python run-time system. The Python API is incorporated in a C +source file by including the header ``"Python.h"``. + +.. note:: + + The C extension interface is specific to CPython, and extension modules do + not work on other Python implementations. In many cases, it is possible to + avoid writing C extensions and preserve portability to other implementations. + For example, if your use case is calling C library functions or system calls, + you should consider using the :mod:`ctypes` module or the `cffi + `_ library rather than writing + custom C code. + These modules let you write Python code to interface with C code and are more + portable between implementations of Python than writing and compiling a C + extension module. + Recommended third party tools ============================= @@ -31,6 +51,21 @@ as part of this version of CPython. Some :ref:`third party tools C and C++ extensions for Python. +C API Tutorial +============== + +This tutorial describes how to write a simple module in C or C++, +using the Python C API -- that is, using the basic tools provided +as part of this version of CPython. + + +.. toctree:: + :maxdepth: 2 + :numbered: + + first-extension-module.rst + + Creating extensions without third party tools ============================================= @@ -39,10 +74,6 @@ assistance from third party tools. It is intended primarily for creators of those tools, rather than being a recommended way to create your own C extensions. -.. seealso:: - - :pep:`489` -- Multi-phase extension module initialization - .. toctree:: :maxdepth: 2 :numbered: diff --git a/Doc/includes/capi-extension/spammodule-01.c b/Doc/includes/capi-extension/spammodule-01.c new file mode 100644 index 00000000000000..7d9449a566f1ff --- /dev/null +++ b/Doc/includes/capi-extension/spammodule-01.c @@ -0,0 +1,73 @@ + +/// Includes + +#include +#include + +/// Implementation of spam.system + +static PyObject * +spam_system(PyObject *self, PyObject *arg) +{ + const char *command = PyUnicode_AsUTF8(arg); + if (command == NULL) { + return NULL; + } + int status = system(command); + return PyLong_FromLong(status); +} + + +/// Module method table + +static PyMethodDef spam_methods[] = { + { + .ml_name="system", + .ml_meth=spam_system, + .ml_flags=METH_O, + .ml_doc=PyDoc_STR("Execute a shell command."), + }, + {NULL, NULL, 0, NULL} /* Sentinel */ +}; + +/// ABI information + +PyABIInfo_VAR(abi_info); + + +/// Module slot table + +static PyModuleDef_Slot spam_module[] = { + {Py_mod_abi, &abi_info}, + {Py_mod_name, "spam"}, + {Py_mod_doc, PyDoc_STR("A wonderful module with an example function")}, + {Py_mod_methods, spam_methods}, + {0, NULL} /* Sentinel */ +}; + + +/// Export hook prototype + +PyMODEXPORT_FUNC PyModExport_spam(void); + +/// Module export hook + +PyMODEXPORT_FUNC +PyModExport_spam(void) +{ + return spam_module; +} + + +/// Legacy initialization function + +PyMODINIT_FUNC PyInit_spam(void); + +PyMODINIT_FUNC +PyInit_spam(void) +{ + PyErr_SetString( + PyExc_SystemError, + "incompatible extension, need Python 3.15 or higher"); + return NULL; +} diff --git a/Doc/includes/tutorial-new-api.rst b/Doc/includes/tutorial-new-api.rst new file mode 100644 index 00000000000000..709eb49149f0b5 --- /dev/null +++ b/Doc/includes/tutorial-new-api.rst @@ -0,0 +1,9 @@ +.. note:: + + This tutorial may use API that was added in CPython |version|. + To create an extension compatible with earlier versions of CPython, please + follow an earlier version of this documentation. + + This tutorial uses some syntax added in C11 and C++20. + If your extension needs to be compatible with earlier standards, + please follow an earlier version of this documentation. From 4c2b9825c663732964998d4b795e44b2235b483b Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Mon, 1 Dec 2025 17:45:03 +0100 Subject: [PATCH 2/8] Reword more --- Doc/extending/first-extension-module.rst | 40 +++++++++------- Doc/extending/index.rst | 52 ++++++++++++--------- Doc/includes/capi-extension/spammodule-01.c | 2 +- 3 files changed, 53 insertions(+), 41 deletions(-) diff --git a/Doc/extending/first-extension-module.rst b/Doc/extending/first-extension-module.rst index d8f6628f716fe2..f6f7b670e646b8 100644 --- a/Doc/extending/first-extension-module.rst +++ b/Doc/extending/first-extension-module.rst @@ -1,6 +1,8 @@ .. highlight:: c +.. _first-extension-module: + ********************************* Your first C API extension module ********************************* @@ -8,22 +10,22 @@ Your first C API extension module This tutorial will take you, line by line, through creating a simple Python extension module written in C or C++. -This document assumes basic knowledge about Python: you should be able to -define functions in Python code before writing them in another language. +The tutorial assumes basic knowledge about Python: you should be able to +define functions in Python code before starting to write them in C. See :ref:`tutorial-index` for an introduction to Python itself. -It also assumes a working knowledge of the C language. -We will use several concepts that a C beginner would not be expected to know, -like ``static`` functions or linkage declarations, but the tutorial should -be useful for anyone who can write a basic C library. +The tutorial should be useful for anyone who can write a basic C library. +While we will mention several concepts that a C beginner would not be expected +to know, like ``static`` functions or linkage declarations, understanding these +is not necessary for success. -As a word warning before we begin: the compilation of an extension module can -be tricky, as it depends on how your system is set up, and on how your Python -is installed. +As a word warning before we begin: after the code is written, you will need to +compile it with the right tools and settings for your system. It is generally best to use a third-party tool to handle the details. -This is covered in later chapters. +This is covered in later chapters, not in this tutorial. -The tutorial assumes that you use a Unix system (including macOS) or Windows. +The tutorial assumes that you use a Unix-like system (including macOS and +Linux) or Windows. .. include:: ../includes/tutorial-new-api.rst @@ -338,8 +340,8 @@ This macro expands to a variable definition like ``static PyABIInfo abi_info = { ... data ... };`` -The slot table and export hook -============================== +The slot table +============== Now, let's fit all the pieces of our module together. @@ -349,15 +351,19 @@ an array of :c:type:`PyModuleDef_Slot` structures. Like with the method table, a zero-filled *sentinel* marks the end. Besides the method table, this "slot table" will contain the module's -top-level information: the name, a docstring, and the -ABI compatibility slot we defined earlier: +top-level information: the name, a docstring, and the ABI compatibility +information we've just defined: .. literalinclude:: ../includes/capi-extension/spammodule-01.c :start-after: /// Module slot table :end-before: /// -This structure, in turn, must be passed to the interpreter in the module's -:ref:`export hook ` function. + +Module export hook +================== + +The :c:type:`PyModuleDef_Slot` array must be passed to the interpreter in the +module's :ref:`export hook ` function. The hook must be named :c:func:`!PyModExport_name`, where *name* is the name of the module, and it should be the only non-\ ``static`` item defined in the module file. diff --git a/Doc/extending/index.rst b/Doc/extending/index.rst index e89fdf55903219..43e53414317c2e 100644 --- a/Doc/extending/index.rst +++ b/Doc/extending/index.rst @@ -4,7 +4,7 @@ Extending and Embedding the Python Interpreter ################################################## -This tutorial describes how to write modules in C or C++ to extend the Python +This document describes how to write modules in C or C++ to extend the Python interpreter with new modules. Those modules can do what Python code does -- define functions, object types and methods -- but also interact with native libraries or achieve better performance by avoiding the overhead of an @@ -42,14 +42,32 @@ source file by including the header ``"Python.h"``. extension module. +.. toctree:: + :maxdepth: 2 + :numbered: + + first-extension-module.rst + extending.rst + newtypes_tutorial.rst + newtypes.rst + building.rst + windows.rst + embedding.rst + + Recommended third party tools ============================= -This guide only covers the basic tools for creating extensions provided +This document only covers the basic tools for creating extensions provided as part of this version of CPython. Some :ref:`third party tools ` offer both simpler and more sophisticated approaches to creating C and C++ extensions for Python. +While this document is aimed at extension authors, it should also be helpful to +the authors of such tools. +For example, the tutorial module can serve as a simple test case for a build +tool or sample expected output of a code generator. + C API Tutorial ============== @@ -59,30 +77,22 @@ using the Python C API -- that is, using the basic tools provided as part of this version of CPython. -.. toctree:: - :maxdepth: 2 - :numbered: - - first-extension-module.rst +#. :ref:`first-extension-module` -Creating extensions without third party tools -============================================= +Guides for intermediate topics +============================== This section of the guide covers creating C and C++ extensions without assistance from third party tools. It is intended primarily for creators of those tools, rather than being a recommended way to create your own C extensions. -.. toctree:: - :maxdepth: 2 - :numbered: - - extending.rst - newtypes_tutorial.rst - newtypes.rst - building.rst - windows.rst +* :ref:`extending-intro` +* :ref:`defining-new-types` +* :ref:`new-types-topics` +* :ref:`building` +* :ref:`building-on-windows` Embedding the CPython runtime in a larger application ===================================================== @@ -92,8 +102,4 @@ interpreter as the main application, it is desirable to instead embed the CPython runtime inside a larger application. This section covers some of the details involved in doing that successfully. -.. toctree:: - :maxdepth: 2 - :numbered: - - embedding.rst +* :ref:`embedding` diff --git a/Doc/includes/capi-extension/spammodule-01.c b/Doc/includes/capi-extension/spammodule-01.c index 7d9449a566f1ff..785bddc6ce8bae 100644 --- a/Doc/includes/capi-extension/spammodule-01.c +++ b/Doc/includes/capi-extension/spammodule-01.c @@ -42,7 +42,7 @@ static PyModuleDef_Slot spam_module[] = { {Py_mod_name, "spam"}, {Py_mod_doc, PyDoc_STR("A wonderful module with an example function")}, {Py_mod_methods, spam_methods}, - {0, NULL} /* Sentinel */ + {0, NULL} }; From 1525795df2cd7fe112fe8aa96ce3ebbbc1da3b93 Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Wed, 3 Dec 2025 08:50:03 +0100 Subject: [PATCH 3/8] Start from the beginning --- Doc/extending/first-extension-module.rst | 130 ++++++++++++++++++++--- Doc/includes/tutorial-new-api.rst | 9 -- 2 files changed, 115 insertions(+), 24 deletions(-) delete mode 100644 Doc/includes/tutorial-new-api.rst diff --git a/Doc/extending/first-extension-module.rst b/Doc/extending/first-extension-module.rst index f6f7b670e646b8..bd600075692c19 100644 --- a/Doc/extending/first-extension-module.rst +++ b/Doc/extending/first-extension-module.rst @@ -7,7 +7,7 @@ Your first C API extension module ********************************* -This tutorial will take you, line by line, through creating a simple +This tutorial will take you through creating a simple Python extension module written in C or C++. The tutorial assumes basic knowledge about Python: you should be able to @@ -19,15 +19,30 @@ While we will mention several concepts that a C beginner would not be expected to know, like ``static`` functions or linkage declarations, understanding these is not necessary for success. -As a word warning before we begin: after the code is written, you will need to +As a word warning before we begin: as the code is written, you will need to compile it with the right tools and settings for your system. It is generally best to use a third-party tool to handle the details. This is covered in later chapters, not in this tutorial. The tutorial assumes that you use a Unix-like system (including macOS and Linux) or Windows. +If you don't have a preference, currently you're most likely to get the best +experience on Linux. -.. include:: ../includes/tutorial-new-api.rst +If you learn better by having the full context at the beginning, +skip to the end to see :ref:`the resulting source `, +then return here. +The tutorial will explain every line, although in a different order. + +.. note:: + + This tutorial uses API that was added in CPython 3.15. + To create an extension that's compatible with earlier versions of CPython, + please follow an earlier version of this documentation. + + This tutorial uses some syntax added in C11 and C++20. + If your extension needs to be compatible with earlier standards, + please follow tutorials in documentation for Python 3.14 or below. What we'll do @@ -65,25 +80,108 @@ We want this function to be callable from Python as follows: both Unix and Windows. -.. _extending-spammodule: +Warming up your build tool: an empty module +=========================================== -The ``spam`` module -=================== +Begin by creating a file named :file:`spammodule.c`. +The name is entirely up to you, though some tools can be picky about +the ``.c`` extension. +(Some people would just use :file:`spam.c` to implement a module +named ``spam``, for example. +If you do this, you'll need to adjust some instructions below.) + +Now, while the file is emptly, we'll compile it, so that you can make +and test incremental changes as you follow the rest of the tutorial. +Choose a build tool such as Setuptools or Meson, and follow its instructions +to compile and install the empty :file:`spammodule.c` as if it was a +C extension module. -Begin by creating a file :file:`spammodule.c`. +.. note:: Workaround for missing ``PyInit`` + + If your build tool output complains about missing ``PyInit_spam``, + add the following function to your module for now: + + .. code-block:: c + + // A workaround + void *PyInit_spam(void) { return NULL; } + + This is a shim for an old-style :ref:`initialization function `, + which was required in extension modules for CPython 3.14 and below. + Current CPython will not call it, but some build tools may still assume that + all extension modules need to define it. + + If you use this workaround, you will get the exception + ``SystemError: initialization of spam failed without raising an exception`` + instead of an :py:exc:`ImportError` in the next step. .. note:: - Historically, if a module is called ``spam``, the C file containing its - implementation is called :file:`spammodule.c`. - But the naming is entirely up to you. - If you use a different name, remember to adjust the commands later, - when you build the extension. + Using a third-party build tool is heavily recommended, as in will take + care of various details of your platform and Python installation, + naming the resulting extension, and, later, distributing your work. + + If you don't want to use a tool, you can try to run your compiler directly. + The following command should work for many flavors of Linux, and generate + a ``spam.so`` file that you need to put in a directory + in :py:attr:`sys.path`: + + .. code-block:: shell + + gcc --shared spammodule.c -o spam.so + +When your extension is compiled and installed, start Python and try to import +your extension. +This should fail with the following exception: + +.. code-block:: pycon + + >>> import spam + Traceback (most recent call last): + File "", line 1, in + import spam + ImportError: dynamic module does not define module export function (PyModExport_spam or PyInit_spam) + + +Including the Header +==================== + + +Now, put the first line in the file: include :file:`Python.h` to pull in +all definitions of the Python C API: + +.. literalinclude:: ../includes/capi-extension/spammodule-01.c + :start-at: + :end-at: + +This header contains all of the Python C API. + +Next, bring in the declaration of the C library function we want to call. +Documentation of the :c:func:`system` function tells us that we need +``stdlib.h``: + +.. literalinclude:: ../includes/capi-extension/spammodule-01.c + :start-at: + :end-at: + +Be sure to put this, and any other standard library includes, *after* +:file:`Python.h`, since Python may define some pre-processor definitions +which affect the standard headers on some systems. + +.. tip:: + + The ```` include is technically not necessary. + :file:`Python.h` :ref:`includes several standard header files ` + for its own use and for backwards compatibility, + and ``stdlib`` is one of them. + However, it is good practice to explicitly include what you need. + + + +The ``spam`` module +=================== -You can add code to this file as you go through the tutorial. -Alternatively, if you'd like to copy the final result to the file now, -get it :ref:`below ` and come back here. The first line of this file (after any comment describing the purpose of the module, copyright notices, and the like) will pull in the Python API: @@ -420,6 +518,8 @@ Thus, if this extension does end up loaded on Python 3.14, the user will get a proper error message. +.. _first-extension-result: + End of file =========== diff --git a/Doc/includes/tutorial-new-api.rst b/Doc/includes/tutorial-new-api.rst deleted file mode 100644 index 709eb49149f0b5..00000000000000 --- a/Doc/includes/tutorial-new-api.rst +++ /dev/null @@ -1,9 +0,0 @@ -.. note:: - - This tutorial may use API that was added in CPython |version|. - To create an extension compatible with earlier versions of CPython, please - follow an earlier version of this documentation. - - This tutorial uses some syntax added in C11 and C++20. - If your extension needs to be compatible with earlier standards, - please follow an earlier version of this documentation. From dc844ddb6611822548cc1bd7735b108678f66145 Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Wed, 3 Dec 2025 16:12:32 +0100 Subject: [PATCH 4/8] TMP --- Doc/extending/first-extension-module.rst | 181 +++++++++++++++++++- Doc/includes/capi-extension/spammodule-01.c | 5 +- 2 files changed, 180 insertions(+), 6 deletions(-) diff --git a/Doc/extending/first-extension-module.rst b/Doc/extending/first-extension-module.rst index bd600075692c19..c962ab53997cef 100644 --- a/Doc/extending/first-extension-module.rst +++ b/Doc/extending/first-extension-module.rst @@ -149,7 +149,7 @@ Including the Header Now, put the first line in the file: include :file:`Python.h` to pull in -all definitions of the Python C API: +all declarations of the Python C API: .. literalinclude:: ../includes/capi-extension/spammodule-01.c :start-at: @@ -158,8 +158,8 @@ all definitions of the Python C API: This header contains all of the Python C API. Next, bring in the declaration of the C library function we want to call. -Documentation of the :c:func:`system` function tells us that we need -``stdlib.h``: +Documentation of the :c:func:`system` function tells us that teh necessary +header is: .. literalinclude:: ../includes/capi-extension/spammodule-01.c :start-at: @@ -177,6 +177,181 @@ which affect the standard headers on some systems. and ``stdlib`` is one of them. However, it is good practice to explicitly include what you need. +With the includes in place, compile and import the extension again. +You should get the same exception as with the empty file. + +.. note:: + + Third-party build tools should handle pointing the compiler to + the CPython headers and libraries, and setting appropriate options. + + If you are running the compiler directly, you will need to do this yourself. + If your installation of Python comes with a corresponding ``python-config`` + command, you can run something like: + + .. code-block:: shell + + gcc --shared $(python-config --cflags --ldflags) spammodule.c -o spam.so + + +Module export hook +================== + +The exception you should be getting tells you that Python is looking for an +module :ref:`export hook ` function. +Let's define one. + +First, let's add a prototype: + +.. literalinclude:: ../includes/capi-extension/spammodule-01.c + :start-after: /// Export hook prototype + :end-before: /// + +The :c:macro:`PyMODEXPORT_FUNC` macro declares the function's +return type, and adds any special linkage declarations required by the platform +to make the compiled extension export the function. +For C++, it declares the function as ``extern "C"``. + +.. tip:: + The prototype is not strictly necessary, but some modern C compilers emit + warnings when a public, exported function has no prototype declaration. + It's better to add a prototype than disable the warning, so that in other + code you're notified in common cases of forgetting a header or misspelling + a name. + +After the prototype, add the function itself. +For now, make it return ``NULL``: + +.. code-block:: c + + PyMODEXPORT_FUNC + PyModExport_spam(void) + { + return NULL; + } + +Compile and load the module again. +You should get a different error this time: + +.. code-block:: pycon + + >>> import spam + Traceback (most recent call last): + File "", line 1, in + import spam + SystemError: module export hook for module 'spam' failed without setting an exception + +Many functions in the Python C API, including export hooks, are expected to +do two things to signal that they have failed: return ``NULL``, and +set an exception. +Here, Python assumes that you only did half of this. + +.. note:: + + This is one of a few situation where CPython checks this situation and + emits a "nice" error message. + Elsewhere, returning ``NULL`` without setting an exception can + trigger undefined behavior. + + +The slot table +============== + +Rather than ``NULL``, the export hook should return the information needed to +create a module, as a ``static`` array of :c:type:`PyModuleDef_Slot` entries. +Define this array just before your export hook: + +.. code-block:: c + + static PyModuleDef_Slot spam_slots[] = { + {Py_mod_name, "spam"}, + {Py_mod_doc, PyDoc_STR("A wonderful module with an example function")}, + {0, NULL} + }; + +The array contains: + +* the module name (as a NUL-terminated UTF-8 encoded C string), +* the docstring (similarly encoded), and +* a zero-filled *sentinel* marking the end of the array. + +.. tip:: + + The :c:func:`PyDoc_STR` macro can, in a special build mode, omit the + docstring to save a bit of memory. + +Return this array from your export hook, instead of ``NULL``: + +.. code-block:: c + :emphasize-lines: 4 + + PyMODEXPORT_FUNC + PyModExport_spam(void) + { + return spam_slots; + } + +Now, recompile and try it out: + +.. code-block:: pycon + + >>> import spam + >>> print(spam) + + +You now have a extension module! +Try ``help(spam)`` to see the docstring. + +The next step will be adding a function to it. + + +Adding a function +================= + +To expose a C function to Python, you need three things: + +* The *implementation*: a C function that does what you need, and +* a *name* to use in Python code. + +You can also add a *dosctring*. +OK, *amongst* the things you need to expose a C function are + + + + +The :c:type:`PyModuleDef_Slot` array must be passed to the interpreter in the +module's :ref:`export hook ` function. +The hook must be named :c:func:`!PyModExport_name`, where *name* is the name +of the module, and it should be the only non-\ ``static`` item defined in the +module file. + +Several modern C compilers emit warnings when you define a public function +without a prototype declaration. +Normally, prototypes go in a header file so they can be used from other +C files. +A function without a prototype is allowed in C, but is normally a strong +hint that something is wrong -- for example, the name is misspelled. + +In our case, Python will load our function dynamically, so a prototype +isn't needed. +We'll only add one to avoid well-meaning compiler warnings: + +.. literalinclude:: ../includes/capi-extension/spammodule-01.c + :start-after: /// Export hook prototype + :end-before: /// + +The :c:macro:`PyMODEXPORT_FUNC` macro declares the function's +``PyModuleDef_Slot *`` return type, and adds any special linkage +declarations required by the platform. +For C++, it declares the function as ``extern "C"``. + +Just after the prototype, we'll add the function itself. +Its only job is to return the slot array, which in turn contains +all the information needed to create our module object: + +.. literalinclude:: ../includes/capi-extension/spammodule-01.c + :start-after: /// Module export hook + :end-before: /// The ``spam`` module diff --git a/Doc/includes/capi-extension/spammodule-01.c b/Doc/includes/capi-extension/spammodule-01.c index 785bddc6ce8bae..43c4b10b189c71 100644 --- a/Doc/includes/capi-extension/spammodule-01.c +++ b/Doc/includes/capi-extension/spammodule-01.c @@ -37,8 +37,7 @@ PyABIInfo_VAR(abi_info); /// Module slot table -static PyModuleDef_Slot spam_module[] = { - {Py_mod_abi, &abi_info}, +static PyModuleDef_Slot spam_slots[] = { {Py_mod_name, "spam"}, {Py_mod_doc, PyDoc_STR("A wonderful module with an example function")}, {Py_mod_methods, spam_methods}, @@ -55,7 +54,7 @@ PyMODEXPORT_FUNC PyModExport_spam(void); PyMODEXPORT_FUNC PyModExport_spam(void) { - return spam_module; + return spam_slots; } From 32ae62033cc12ca5b4f766863a7c908ff48aedf0 Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Thu, 4 Dec 2025 16:22:45 +0100 Subject: [PATCH 5/8] Full draft --- Doc/extending/first-extension-module.rst | 320 ++++++++++++++++++-- Doc/includes/capi-extension/spammodule-01.c | 4 +- 2 files changed, 291 insertions(+), 33 deletions(-) diff --git a/Doc/extending/first-extension-module.rst b/Doc/extending/first-extension-module.rst index c962ab53997cef..3cfb2cce37cdbc 100644 --- a/Doc/extending/first-extension-module.rst +++ b/Doc/extending/first-extension-module.rst @@ -10,7 +10,7 @@ Your first C API extension module This tutorial will take you through creating a simple Python extension module written in C or C++. -The tutorial assumes basic knowledge about Python: you should be able to +It assumes basic knowledge about Python: you should be able to define functions in Python code before starting to write them in C. See :ref:`tutorial-index` for an introduction to Python itself. @@ -19,20 +19,18 @@ While we will mention several concepts that a C beginner would not be expected to know, like ``static`` functions or linkage declarations, understanding these is not necessary for success. -As a word warning before we begin: as the code is written, you will need to -compile it with the right tools and settings for your system. -It is generally best to use a third-party tool to handle the details. -This is covered in later chapters, not in this tutorial. +We will focus on giving you a "feel" of what Python's C API is like. +It will not teach you important concepts, like error handling +and reference counting, which are covered in later chapters. -The tutorial assumes that you use a Unix-like system (including macOS and -Linux) or Windows. -If you don't have a preference, currently you're most likely to get the best -experience on Linux. +As you write the code, you will need to compile it. +Prepare to spend some time choosing, installing and configuring a build tool, +since CPython itself does not include one. -If you learn better by having the full context at the beginning, -skip to the end to see :ref:`the resulting source `, -then return here. -The tutorial will explain every line, although in a different order. +We will assume that you use a Unix-like system (including macOS and +Linux), or Windows. +On other systems, you might need to adjust some details -- for example, +a system command name. .. note:: @@ -48,20 +46,21 @@ The tutorial will explain every line, although in a different order. What we'll do ============= -Let's create an extension module called ``spam`` (the favorite food of Monty -Python fans...) and let's say we want to create a Python interface to the C +Let's create an extension module called ``spam`` [#why-spam]_, +which will include a Python interface to the C standard library function :c:func:`system`. -This function takes a C string as argument, runs the argument as a system +This function is defined in ``stdlib.h``. +It takes a C string as argument, runs the argument as a system command, and returns a result value as an integer. -In documentation, it might be summarized as:: +A manual page for ``system`` might summarize it this way:: #include - int system(const char *command); Note that like many functions in the C standard library, -this function is already exposed in Python, as :py:func:`os.system`. -We'll make our own "wrapper". +this function is already exposed in Python. +In production, use :py:func:`os.system` or :py:func:`subprocess.run` +rather than the module you'll write here. We want this function to be callable from Python as follows: @@ -80,8 +79,8 @@ We want this function to be callable from Python as follows: both Unix and Windows. -Warming up your build tool: an empty module -=========================================== +Warming up your build tool +========================== Begin by creating a file named :file:`spammodule.c`. The name is entirely up to you, though some tools can be picky about @@ -90,7 +89,7 @@ the ``.c`` extension. named ``spam``, for example. If you do this, you'll need to adjust some instructions below.) -Now, while the file is emptly, we'll compile it, so that you can make +Now, while the file is empty, we'll compile it, so that you can make and test incremental changes as you follow the rest of the tutorial. Choose a build tool such as Setuptools or Meson, and follow its instructions @@ -305,17 +304,274 @@ Try ``help(spam)`` to see the docstring. The next step will be adding a function to it. -Adding a function -================= +Exposing a function +=================== + +To add a function to the module, you will need a C function to call. +Add a minimal one now:: + + static PyObject * + spam_system(PyObject *self, PyObject *arg) + { + Py_RETURN_NONE; + } + +This function is defined as ``static``. +This will be a common theme: usually, the only non-``static`` function +in a Python extension module is the export hook. +Data declarations are typically ``static`` as well. + +As a Python function, our ``spam_system`` returns a Python object. +In the C API, this is represented as a pointer to the ``PyObject`` structure. + +This function also takes two pointers to Python objects as arguments. +The "real" argument, as passed in from Python code, will be *arg*. +Meanwhile, *self* will be set to the module object -- in the C API, +module-level functions behave a bit like "methods" of the module object. + +The macro :c:macro:`Py_RETURN_NONE`, which we use as a body for now, +properly ``return``\s a Python :py:data:`None` object. + +Recompile your extension to make sure you don't have syntax errors. +You might get a warning that ``spam_system`` is unused. +This is true: we haven't yet added the function to the module. + + +Method definitions +------------------ + +To expose the C function to Python, you will need to provide several pieces of +information. +These are collected in a structure called :c:type:`PyMethodDef`, +They are: + +* ``ml_name``: the name of the function to use in Python code; +* ``ml_meth``: the *implementation* -- the C function that does what + you need; +* ``ml_flags``: a set of flags describing details like how Python arguments are + passed to the C function; and +* ``ml_doc``: a docstring. -To expose a C function to Python, you need three things: +The name and docstring are encoded as for module. +For ``ml_meth``, we'll use the ``spam_system`` function you just defined. -* The *implementation*: a C function that does what you need, and -* a *name* to use in Python code. +For ``ml_flags``, we'll use :c:data:`METH_O` -- a flag that specifies that a +C function with two ``PyObject*`` arguments (your ``spam_system``) will be +exposed as a Python function taking a single unnamed argument. +(Other flags exist to handle named arguments, but they are harder to use.) + +Because modules typically create several functions, these definitions +need to be collected in an array: + +.. literalinclude:: ../includes/capi-extension/spammodule-01.c + :start-after: /// Module method table + :end-before: /// + +As with module slots, a zero-filled sentinel marks the end of the array. + +Add the array to your module, between the ``spam_system`` function and +the module slots. + +.. note:: + + Why :c:type:`!PyMethodDef` and not :c:type:`!PyFunctionDef`? + + In the C API, module-level functions behave a bit like "methods" + of a module object: they take an extra *self* argument, and + you need to provide the same information to create. + +To add the method(s) defined in a :c:type:`PyMethodDef` array, +add a :c:data:`Py_mod_methods` slot: + +.. literalinclude:: ../includes/capi-extension/spammodule-01.c + :start-after: /// Module slot table + :end-before: /// + :emphasize-lines: 6 + +Recompile your extension again, and test it. +You should now be able to call the function, and get ``None`` from it: + +.. code-block:: pycon + + >>> import spam + >>> print(spam.system) + + >>> print(spam.system('whoami')) + None + + +Returning an integer +==================== + +Your ``spam_system`` function currently returns ``None``. +We want it to return a number -- that is, a Python :py:type:`int` object. +Ultimately, this will be the exit code of a system command, +but let's start with a fixed value: ``3``. + +The Python C API provides a function to create a Python :py:type:`int` object +from a C ``int`` values: :c:func:`PyLong_FromLong`. +(The name might not seem obvious to you; it'll get better as you get used to +C API naming conventions.) + +Let's call it: + +.. this could be a one-liner, we use 3 lines to show the data types. + +.. code-block:: c + :emphasize-lines: 4-6 + + static PyObject * + spam_system(PyObject *self, PyObject *arg) + { + int status = 3; + PyObject *result = PyLong_FromLong(status); + return result; + } + + +Recompile and run again, and check that you get a 3: + +.. code-block:: pycon + + >>> import spam + >>> spam.system('whoami') + 3 + + +Accepting a string +================== -You can also add a *dosctring*. -OK, *amongst* the things you need to expose a C function are +Your ``spam_system`` function should accepts one argument, +which is passed in as ``PyObject *arg``. +In order to use the information in it, we will need +to convert it to a C value --- in this case, a C string (``const char *``). +The argument should be a Python string. +There's a slight type mismatch here: Python's :c:type:`str` objects store +Unicode text, but C strings are arrays of bytes. +So, we'll need to *encode* the data. +In our example, we'll use the UTF-8 encoding. +It might not always be correct for system commands, but it's what +:py:meth:`str.decode` uses by default, +and the C API has special support for it. + +The function to decode into a UTF-8 buffer is named :c:func:`PyUnicode_AsUTF8`. +Call it like this: + +.. code-block:: c + :emphasize-lines: 4 + + static PyObject * + spam_system(PyObject *self, PyObject *arg) + { + const char *command = PyUnicode_AsUTF8(arg); + int status = 3; + PyObject *result = PyLong_FromLong(status); + return result; + } + +If :c:func:`PyUnicode_AsUTF8` is successful, *command* will point to the +resulting array of bytes. +This buffer is managed by the *arg* object, +which means limited: + +* You should only use the buffer inside the ``spam_system`` function. + When ``spam_system`` returns, *arg* and the array it manages might be + garbage-collected. +* You must not modify it. This is why we use ``const``. + +Both are fine for our use. + +If :c:func:`PyUnicode_AsUTF8` was *not* successful, it returns a ``NULL`` +pointer. +When using the Python C API, we always need to handle such error cases. +Here, the correct thing to do is returning ``NULL`` also from ``spam_system``: + + +.. code-block:: c + :emphasize-lines: 5-7 + + static PyObject * + spam_system(PyObject *self, PyObject *arg) + { + const char *command = PyUnicode_AsUTF8(arg); + if (command == NULL) { + return NULL; + } + int status = 3; + PyObject *result = PyLong_FromLong(status); + return result; + } + +That's it for the setup. +Now, all that is left is calling C library function ``system`` with +the ``char *`` buffer, and using its result instead of the ``3``: + +.. code-block:: c + :emphasize-lines: 8 + + static PyObject * + spam_system(PyObject *self, PyObject *arg) + { + const char *command = PyUnicode_AsUTF8(arg); + if (command == NULL) { + return NULL; + } + int status = system(command); + PyObject *result = PyLong_FromLong(status); + return result; + } + +Compile it, and test: + +.. code-block:: pycon + + >>> import spam + >>> result = spam.system('whoami') + User Name + >>> result + 0 + +You might also want to test error cases: + +.. code-block:: pycon + + >>> import spam + >>> result = spam.system('nonexistent-command') + sh: line 1: nonexistent-command: command not found + >>> result + 32512 + + >>> spam.system(3) + Traceback (most recent call last): + File "", line 1, in + spam.system(3) + ~~~~~~~~~~~^^^ + TypeError: bad argument type for built-in operation + + +The result +========== + + +Congratulations! +You have written a complete Python C API extension module, +and completed this tutorial! + +Here is the entire source file, for your convenience: + +.. _extending-spammodule-source: + +.. literalinclude:: ../includes/capi-extension/spammodule-01.c + + +.. rubric:: Footnotes + +.. [#why-spam] ``spam`` is the favorite food of Monty Python fans... + + +Accepting a string +================== @@ -507,8 +763,8 @@ Our function must then convert the C integer into a Python integer. This is done using the function :c:func:`PyLong_FromLong`: .. literalinclude:: ../includes/capi-extension/spammodule-01.c - :start-at: return PyLong_FromLong(status); - :end-at: return PyLong_FromLong(status); + :start-at: PyLong_FromLong + :end-at: return In this case, it will return a Python :py:class:`int` object. (Yes, even integers are objects on the heap in Python!) diff --git a/Doc/includes/capi-extension/spammodule-01.c b/Doc/includes/capi-extension/spammodule-01.c index 43c4b10b189c71..3db7ef1d5faafb 100644 --- a/Doc/includes/capi-extension/spammodule-01.c +++ b/Doc/includes/capi-extension/spammodule-01.c @@ -14,7 +14,8 @@ spam_system(PyObject *self, PyObject *arg) return NULL; } int status = system(command); - return PyLong_FromLong(status); + PyObject *result = PyLong_FromLong(status); + return result; } @@ -38,6 +39,7 @@ PyABIInfo_VAR(abi_info); /// Module slot table static PyModuleDef_Slot spam_slots[] = { + {Py_mod_abi, &abi_info}, {Py_mod_name, "spam"}, {Py_mod_doc, PyDoc_STR("A wonderful module with an example function")}, {Py_mod_methods, spam_methods}, From 13d5efaa04c4d3ad3cb17a87dd0e9129b053dd26 Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Thu, 4 Dec 2025 17:00:01 +0100 Subject: [PATCH 6/8] Ruthlessly miminise explanation See: https://diataxis.fr/tutorials/#ruthlessly-minimise-explanation --- Doc/extending/extending.rst | 421 ++---------- Doc/extending/first-extension-module.rst | 683 ++++---------------- Doc/extending/index.rst | 3 +- Doc/includes/capi-extension/spammodule-01.c | 45 +- 4 files changed, 217 insertions(+), 935 deletions(-) diff --git a/Doc/extending/extending.rst b/Doc/extending/extending.rst index f9b65643dfe888..05c8f65b83bc5e 100644 --- a/Doc/extending/extending.rst +++ b/Doc/extending/extending.rst @@ -3,154 +3,20 @@ .. _extending-intro: -****************************** -Extending Python with C or C++ -****************************** +******************************** +Using the C API: Assorted topics +******************************** -It is quite easy to add new built-in modules to Python, if you know how to -program in C. Such :dfn:`extension modules` can do two things that can't be -done directly in Python: they can implement new built-in object types, and they -can call C library functions and system calls. - -To support extensions, the Python API (Application Programmers Interface) -defines a set of functions, macros and variables that provide access to most -aspects of the Python run-time system. The Python API is incorporated in a C -source file by including the header ``"Python.h"``. - -The compilation of an extension module depends on its intended use as well as on -your system setup; details are given in later chapters. - -.. note:: - - The C extension interface is specific to CPython, and extension modules do - not work on other Python implementations. In many cases, it is possible to - avoid writing C extensions and preserve portability to other implementations. - For example, if your use case is calling C library functions or system calls, - you should consider using the :mod:`ctypes` module or the `cffi - `_ library rather than writing - custom C code. - These modules let you write Python code to interface with C code and are more - portable between implementations of Python than writing and compiling a C - extension module. - - -.. _extending-simpleexample: - -A Simple Example -================ - -Let's create an extension module called ``spam`` (the favorite food of Monty -Python fans...) and let's say we want to create a Python interface to the C -library function :c:func:`system` [#]_. This function takes a null-terminated -character string as argument and returns an integer. We want this function to -be callable from Python as follows: - -.. code-block:: pycon - - >>> import spam - >>> status = spam.system("ls -l") - -Begin by creating a file :file:`spammodule.c`. (Historically, if a module is -called ``spam``, the C file containing its implementation is called -:file:`spammodule.c`; if the module name is very long, like ``spammify``, the -module name can be just :file:`spammify.c`.) - -The first two lines of our file can be:: - - #define PY_SSIZE_T_CLEAN - #include - -which pulls in the Python API (you can add a comment describing the purpose of -the module and a copyright notice if you like). - -.. note:: - - Since Python may define some pre-processor definitions which affect the standard - headers on some systems, you *must* include :file:`Python.h` before any standard - headers are included. - - ``#define PY_SSIZE_T_CLEAN`` was used to indicate that ``Py_ssize_t`` should be - used in some APIs instead of ``int``. - It is not necessary since Python 3.13, but we keep it here for backward compatibility. - See :ref:`arg-parsing-string-and-buffers` for a description of this macro. - -All user-visible symbols defined by :file:`Python.h` have a prefix of ``Py`` or -``PY``, except those defined in standard header files. - -.. tip:: - - For backward compatibility, :file:`Python.h` includes several standard header files. - C extensions should include the standard headers that they use, - and should not rely on these implicit includes. - If using the limited C API version 3.13 or newer, the implicit includes are: - - * ```` - * ```` (on Windows) - * ```` - * ```` - * ```` - * ```` - * ```` - * ```` (if present) - - If :c:macro:`Py_LIMITED_API` is not defined, or is set to version 3.12 or older, - the headers below are also included: - - * ```` - * ```` (on POSIX) - - If :c:macro:`Py_LIMITED_API` is not defined, or is set to version 3.10 or older, - the headers below are also included: - - * ```` - * ```` - * ```` - * ```` - -The next thing we add to our module file is the C function that will be called -when the Python expression ``spam.system(string)`` is evaluated (we'll see -shortly how it ends up being called):: - - static PyObject * - spam_system(PyObject *self, PyObject *args) - { - const char *command; - int sts; - - if (!PyArg_ParseTuple(args, "s", &command)) - return NULL; - sts = system(command); - return PyLong_FromLong(sts); - } - -There is a straightforward translation from the argument list in Python (for -example, the single expression ``"ls -l"``) to the arguments passed to the C -function. The C function always has two arguments, conventionally named *self* -and *args*. - -The *self* argument points to the module object for module-level functions; -for a method it would point to the object instance. - -The *args* argument will be a pointer to a Python tuple object containing the -arguments. Each item of the tuple corresponds to an argument in the call's -argument list. The arguments are Python objects --- in order to do anything -with them in our C function we have to convert them to C values. The function -:c:func:`PyArg_ParseTuple` in the Python API checks the argument types and -converts them to C values. It uses a template string to determine the required -types of the arguments as well as the types of the C variables into which to -store the converted values. More about this later. - -:c:func:`PyArg_ParseTuple` returns true (nonzero) if all arguments have the right -type and its components have been stored in the variables whose addresses are -passed. It returns false (zero) if an invalid argument list was passed. In the -latter case it also raises an appropriate exception so the calling function can -return ``NULL`` immediately (as we saw in the example). +The :ref:`tutorial ` walked you through +creating a C API extension module, but left many areas unexplained. +This document looks at several concepts that you'll need to learn +in order to write more complex extensions. .. _extending-errors: -Intermezzo: Errors and Exceptions -================================= +Errors and Exceptions +===================== An important convention throughout the Python interpreter is the following: when a function fails, it should set an exception condition and return an error value @@ -321,194 +187,14 @@ call to :c:func:`PyErr_SetString` as shown below:: } -.. _backtoexample: - -Back to the Example -=================== - -Going back to our example function, you should now be able to understand this -statement:: - - if (!PyArg_ParseTuple(args, "s", &command)) - return NULL; - -It returns ``NULL`` (the error indicator for functions returning object pointers) -if an error is detected in the argument list, relying on the exception set by -:c:func:`PyArg_ParseTuple`. Otherwise the string value of the argument has been -copied to the local variable :c:data:`!command`. This is a pointer assignment and -you are not supposed to modify the string to which it points (so in Standard C, -the variable :c:data:`!command` should properly be declared as ``const char -*command``). - -The next statement is a call to the Unix function :c:func:`system`, passing it -the string we just got from :c:func:`PyArg_ParseTuple`:: - - sts = system(command); - -Our :func:`!spam.system` function must return the value of :c:data:`!sts` as a -Python object. This is done using the function :c:func:`PyLong_FromLong`. :: - - return PyLong_FromLong(sts); - -In this case, it will return an integer object. (Yes, even integers are objects -on the heap in Python!) - -If you have a C function that returns no useful argument (a function returning -:c:expr:`void`), the corresponding Python function must return ``None``. You -need this idiom to do so (which is implemented by the :c:macro:`Py_RETURN_NONE` -macro):: - - Py_INCREF(Py_None); - return Py_None; - -:c:data:`Py_None` is the C name for the special Python object ``None``. It is a -genuine Python object rather than a ``NULL`` pointer, which means "error" in most -contexts, as we have seen. - - -.. _methodtable: - -The Module's Method Table and Initialization Function -===================================================== - -I promised to show how :c:func:`!spam_system` is called from Python programs. -First, we need to list its name and address in a "method table":: - - static PyMethodDef spam_methods[] = { - ... - {"system", spam_system, METH_VARARGS, - "Execute a shell command."}, - ... - {NULL, NULL, 0, NULL} /* Sentinel */ - }; - -Note the third entry (``METH_VARARGS``). This is a flag telling the interpreter -the calling convention to be used for the C function. It should normally always -be ``METH_VARARGS`` or ``METH_VARARGS | METH_KEYWORDS``; a value of ``0`` means -that an obsolete variant of :c:func:`PyArg_ParseTuple` is used. - -When using only ``METH_VARARGS``, the function should expect the Python-level -parameters to be passed in as a tuple acceptable for parsing via -:c:func:`PyArg_ParseTuple`; more information on this function is provided below. - -The :c:macro:`METH_KEYWORDS` bit may be set in the third field if keyword -arguments should be passed to the function. In this case, the C function should -accept a third ``PyObject *`` parameter which will be a dictionary of keywords. -Use :c:func:`PyArg_ParseTupleAndKeywords` to parse the arguments to such a -function. - -The method table must be referenced in the module definition structure:: - - static struct PyModuleDef spam_module = { - ... - .m_methods = spam_methods, - ... - }; - -This structure, in turn, must be passed to the interpreter in the module's -initialization function. The initialization function must be named -:c:func:`!PyInit_name`, where *name* is the name of the module, and should be the -only non-\ ``static`` item defined in the module file:: - - PyMODINIT_FUNC - PyInit_spam(void) - { - return PyModuleDef_Init(&spam_module); - } - -Note that :c:macro:`PyMODINIT_FUNC` declares the function as ``PyObject *`` return type, -declares any special linkage declarations required by the platform, and for C++ -declares the function as ``extern "C"``. - -:c:func:`!PyInit_spam` is called when each interpreter imports its module -:mod:`!spam` for the first time. (See below for comments about embedding Python.) -A pointer to the module definition must be returned via :c:func:`PyModuleDef_Init`, -so that the import machinery can create the module and store it in ``sys.modules``. - -When embedding Python, the :c:func:`!PyInit_spam` function is not called -automatically unless there's an entry in the :c:data:`PyImport_Inittab` table. -To add the module to the initialization table, use :c:func:`PyImport_AppendInittab`, -optionally followed by an import of the module:: - - #define PY_SSIZE_T_CLEAN - #include - - int - main(int argc, char *argv[]) - { - PyStatus status; - PyConfig config; - PyConfig_InitPythonConfig(&config); - - /* Add a built-in module, before Py_Initialize */ - if (PyImport_AppendInittab("spam", PyInit_spam) == -1) { - fprintf(stderr, "Error: could not extend in-built modules table\n"); - exit(1); - } - - /* Pass argv[0] to the Python interpreter */ - status = PyConfig_SetBytesString(&config, &config.program_name, argv[0]); - if (PyStatus_Exception(status)) { - goto exception; - } - - /* Initialize the Python interpreter. Required. - If this step fails, it will be a fatal error. */ - status = Py_InitializeFromConfig(&config); - if (PyStatus_Exception(status)) { - goto exception; - } - PyConfig_Clear(&config); - - /* Optionally import the module; alternatively, - import can be deferred until the embedded script - imports it. */ - PyObject *pmodule = PyImport_ImportModule("spam"); - if (!pmodule) { - PyErr_Print(); - fprintf(stderr, "Error: could not import module 'spam'\n"); - } - - // ... use Python C API here ... - - return 0; - - exception: - PyConfig_Clear(&config); - Py_ExitStatusException(status); - } - -.. note:: - - If you declare a global variable or a local static one, the module may - experience unintended side-effects on re-initialisation, for example when - removing entries from ``sys.modules`` or importing compiled modules into - multiple interpreters within a process - (or following a :c:func:`fork` without an intervening :c:func:`exec`). - If module state is not yet fully :ref:`isolated `, - authors should consider marking the module as having no support for subinterpreters - (via :c:macro:`Py_MOD_MULTIPLE_INTERPRETERS_NOT_SUPPORTED`). - -A more substantial example module is included in the Python source distribution -as :file:`Modules/xxlimited.c`. This file may be used as a template or simply -read as an example. - - .. _compilation: -Compilation and Linkage -======================= +Embedding an extension +====================== -There are two more things to do before you can use your new extension: compiling -and linking it with the Python system. If you use dynamic loading, the details -may depend on the style of dynamic loading your system uses; see the chapters -about building extension modules (chapter :ref:`building`) and additional -information that pertains only to building on Windows (chapter -:ref:`building-on-windows`) for more information about this. - -If you can't use dynamic loading, or if you want to make your module a permanent +If you want to make your module a permanent part of the Python interpreter, you will have to change the configuration setup -and rebuild the interpreter. Luckily, this is very simple on Unix: just place +and rebuild the interpreter. On Unix, place your file (:file:`spammodule.c` for example) in the :file:`Modules/` directory of an unpacked source distribution, add a line to the file :file:`Modules/Setup.local` describing your file: @@ -536,7 +222,7 @@ on the line in the configuration file as well, for instance: Calling Python Functions from C =============================== -So far we have concentrated on making C functions callable from Python. The +The tutorial concentrated on making C functions callable from Python. The reverse is also useful: calling Python functions from C. This is especially the case for libraries that support so-called "callback" functions. If a C interface makes use of callbacks, the equivalent Python often needs to provide a @@ -581,7 +267,7 @@ be part of a module definition:: } This function must be registered with the interpreter using the -:c:macro:`METH_VARARGS` flag; this is described in section :ref:`methodtable`. The +:c:macro:`METH_VARARGS` flag in :c:type:`PyMethodDef.ml_flags`. The :c:func:`PyArg_ParseTuple` function and its arguments are documented in section :ref:`parsetuple`. @@ -676,14 +362,21 @@ the above example, we use :c:func:`Py_BuildValue` to construct the dictionary. : Py_DECREF(result); +.. index:: single: PyArg_ParseTuple (C function) + .. _parsetuple: Extracting Parameters in Extension Functions ============================================ -.. index:: single: PyArg_ParseTuple (C function) +The :ref:`tutorial ` uses a ":c:data:`METH_O`" +function, which is limited to a single Python argument. +If you want more, you can use :c:data:`METH_VARARGS` instead. +With this flag, the C function will receive a *tuple* of arguments +instead of a single object. -The :c:func:`PyArg_ParseTuple` function is declared as follows:: +For unpacking the tuple, CPython provides the :c:func:`PyArg_ParseTuple` +function, declared as follows:: int PyArg_ParseTuple(PyObject *arg, const char *format, ...); @@ -693,6 +386,19 @@ whose syntax is explained in :ref:`arg-parsing` in the Python/C API Reference Manual. The remaining arguments must be addresses of variables whose type is determined by the format string. +For example, to receive a single Python :py:class:`str` object and turn it +into a C buffer, you would use ``"s"`` as the format string:: + + const char *command; + if (!PyArg_ParseTuple(args, "s", &command)) { + return NULL; + } + +If an error is detected in the argument list, :c:func:`!PyArg_ParseTuple` +returns ``NULL`` (the error indicator for functions returning object pointers); +your function may return ``NULL``, relying on the exception set by +:c:func:`PyArg_ParseTuple`. + Note that while :c:func:`PyArg_ParseTuple` checks that the Python arguments have the required types, it cannot check the validity of the addresses of C variables passed to the call: if you make mistakes there, your code will probably crash or @@ -703,7 +409,6 @@ Note that any Python object references which are provided to the caller are Some example calls:: - #define PY_SSIZE_T_CLEAN #include :: @@ -773,6 +478,17 @@ Some example calls:: Keyword Parameters for Extension Functions ========================================== +If you also want your function to accest *keyword* arguments, +use the :c:data:`METH_KEYWORDS` flag in combination with +:c:data:`METH_VARARGS`. +(It can also be used with other flags; see its documentation for the allowed +combinations.) + +In this case, the C function should accept a third ``PyObject *`` parameter +which will be a dictionary of keywords. +Use :c:func:`PyArg_ParseTupleAndKeywords` to parse the arguments to such a +function. + .. index:: single: PyArg_ParseTupleAndKeywords (C function) The :c:func:`PyArg_ParseTupleAndKeywords` function is declared as follows:: @@ -833,19 +549,6 @@ Philbrick (philbrick@hks.com):: {NULL, NULL, 0, NULL} /* sentinel */ }; - static struct PyModuleDef keywdarg_module = { - .m_base = PyModuleDef_HEAD_INIT, - .m_name = "keywdarg", - .m_size = 0, - .m_methods = keywdarg_methods, - }; - - PyMODINIT_FUNC - PyInit_keywdarg(void) - { - return PyModuleDef_Init(&keywdarg_module); - } - .. _buildvalue: @@ -986,11 +689,11 @@ needed. Ownership of a reference can be transferred. There are three ways to dispose of an owned reference: pass it on, store it, or call :c:func:`Py_DECREF`. Forgetting to dispose of an owned reference creates a memory leak. -It is also possible to :dfn:`borrow` [#]_ a reference to an object. The +It is also possible to :dfn:`borrow` [#borrow]_ a reference to an object. The borrower of a reference should not call :c:func:`Py_DECREF`. The borrower must not hold on to the object longer than the owner from which it was borrowed. Using a borrowed reference after the owner has disposed of it risks using freed -memory and should be avoided completely [#]_. +memory and should be avoided completely [#dont-check-refcount]_. The advantage of borrowing over owning a reference is that you don't need to take care of disposing of the reference on all possible paths through the code @@ -1169,7 +872,7 @@ checking. The C function calling mechanism guarantees that the argument list passed to C functions (``args`` in the examples) is never ``NULL`` --- in fact it guarantees -that it is always a tuple [#]_. +that it is always a tuple [#old-calling-convention]_. It is a severe error to ever let a ``NULL`` pointer "escape" to the Python user. @@ -1226,8 +929,8 @@ the module whose functions one wishes to call might not have been loaded yet! Portability therefore requires not to make any assumptions about symbol visibility. This means that all symbols in extension modules should be declared ``static``, except for the module's initialization function, in order to -avoid name clashes with other extension modules (as discussed in section -:ref:`methodtable`). And it means that symbols that *should* be accessible from +avoid name clashes with other extension modules. And it means that symbols +that *should* be accessible from other extension modules must be exported in a different way. Python provides a special mechanism to pass C-level information (pointers) from @@ -1269,8 +972,9 @@ file corresponding to the module provides a macro that takes care of importing the module and retrieving its C API pointers; client modules only have to call this macro before accessing the C API. -The exporting module is a modification of the :mod:`!spam` module from section -:ref:`extending-simpleexample`. The function :func:`!spam.system` does not call +The exporting module is a modification of the :mod:`!spam` module from the +:ref:`tutorial `. +The function :func:`!spam.system` does not call the C library function :c:func:`system` directly, but a function :c:func:`!PySpam_System`, which would of course do something more complicated in reality (such as adding "spam" to every command). This function @@ -1412,15 +1116,14 @@ code distribution). .. rubric:: Footnotes -.. [#] An interface for this function already exists in the standard module :mod:`os` - --- it was chosen as a simple and straightforward example. - -.. [#] The metaphor of "borrowing" a reference is not completely correct: the owner - still has a copy of the reference. +.. [#borrow] The metaphor of "borrowing" a reference is not completely correct: + the owner still has a copy of the reference. -.. [#] Checking that the reference count is at least 1 **does not work** --- the +.. [#dont-check-refcount] Checking that the reference count is at least 1 + **does not work** --- the reference count itself could be in freed memory and may thus be reused for another object! -.. [#] These guarantees don't hold when you use the "old" style calling convention --- +.. [#old-calling-convention] These guarantees don't hold when you use the + "old" style calling convention --- this is still found in much existing code. diff --git a/Doc/extending/first-extension-module.rst b/Doc/extending/first-extension-module.rst index 3cfb2cce37cdbc..17d6fe361aa467 100644 --- a/Doc/extending/first-extension-module.rst +++ b/Doc/extending/first-extension-module.rst @@ -32,6 +32,7 @@ Linux), or Windows. On other systems, you might need to adjust some details -- for example, a system command name. + .. note:: This tutorial uses API that was added in CPython 3.15. @@ -82,19 +83,15 @@ We want this function to be callable from Python as follows: Warming up your build tool ========================== -Begin by creating a file named :file:`spammodule.c`. -The name is entirely up to you, though some tools can be picky about -the ``.c`` extension. -(Some people would just use :file:`spam.c` to implement a module -named ``spam``, for example. -If you do this, you'll need to adjust some instructions below.) +Begin by creating a file named :file:`spammodule.c`. [#why-spammodule]_ -Now, while the file is empty, we'll compile it, so that you can make -and test incremental changes as you follow the rest of the tutorial. +Now, while the file is empty, we'll compile it. Choose a build tool such as Setuptools or Meson, and follow its instructions -to compile and install the empty :file:`spammodule.c` as if it was a -C extension module. +to compile and install the empty :file:`spammodule.c` as a C extension module. + +This will ensure that your build tool works, so that you can make +and test incremental changes as you follow the rest of the text. .. note:: Workaround for missing ``PyInit`` @@ -119,14 +116,14 @@ C extension module. Using a third-party build tool is heavily recommended, as in will take care of various details of your platform and Python installation, - naming the resulting extension, and, later, distributing your work. + of naming the resulting extension, and, later, of distributing your work. If you don't want to use a tool, you can try to run your compiler directly. The following command should work for many flavors of Linux, and generate a ``spam.so`` file that you need to put in a directory in :py:attr:`sys.path`: - .. code-block:: shell + .. code-block:: sh gcc --shared spammodule.c -o spam.so @@ -138,35 +135,30 @@ This should fail with the following exception: >>> import spam Traceback (most recent call last): - File "", line 1, in - import spam + ... ImportError: dynamic module does not define module export function (PyModExport_spam or PyInit_spam) Including the Header ==================== - -Now, put the first line in the file: include :file:`Python.h` to pull in +Now, add the first line to your file: include :file:`Python.h` to pull in all declarations of the Python C API: .. literalinclude:: ../includes/capi-extension/spammodule-01.c :start-at: :end-at: -This header contains all of the Python C API. - -Next, bring in the declaration of the C library function we want to call. -Documentation of the :c:func:`system` function tells us that teh necessary -header is: +Next, include the header for the :c:func:`system` function: .. literalinclude:: ../includes/capi-extension/spammodule-01.c :start-at: :end-at: Be sure to put this, and any other standard library includes, *after* -:file:`Python.h`, since Python may define some pre-processor definitions -which affect the standard headers on some systems. +:file:`Python.h`. +On some systems, Python may define some pre-processor definitions +that affect the standard headers. .. tip:: @@ -196,27 +188,25 @@ You should get the same exception as with the empty file. Module export hook ================== -The exception you should be getting tells you that Python is looking for an -module :ref:`export hook ` function. +The exception you got when you tried to import the module told you that Python +is looking for a "module export function", also known as a +:ref:`module export hook `. Let's define one. -First, let's add a prototype: +First, add a prototype below the ``#include`` lines: .. literalinclude:: ../includes/capi-extension/spammodule-01.c :start-after: /// Export hook prototype :end-before: /// -The :c:macro:`PyMODEXPORT_FUNC` macro declares the function's -return type, and adds any special linkage declarations required by the platform -to make the compiled extension export the function. -For C++, it declares the function as ``extern "C"``. - .. tip:: - The prototype is not strictly necessary, but some modern C compilers emit - warnings when a public, exported function has no prototype declaration. - It's better to add a prototype than disable the warning, so that in other - code you're notified in common cases of forgetting a header or misspelling - a name. + The prototype is not strictly necessary, but some modern compilers emit + warnings without it. + It's generally better to add the prototype than to disable the warning. + +The :c:macro:`PyMODEXPORT_FUNC` macro declares the function's +return type, and adds any special linkage declarations needed +to make the function visible and usable when CPython loads it. After the prototype, add the function itself. For now, make it return ``NULL``: @@ -230,56 +220,53 @@ For now, make it return ``NULL``: } Compile and load the module again. -You should get a different error this time: +You should get a different error this time. .. code-block:: pycon >>> import spam Traceback (most recent call last): - File "", line 1, in - import spam + ... SystemError: module export hook for module 'spam' failed without setting an exception -Many functions in the Python C API, including export hooks, are expected to -do two things to signal that they have failed: return ``NULL``, and -set an exception. -Here, Python assumes that you only did half of this. - -.. note:: - - This is one of a few situation where CPython checks this situation and - emits a "nice" error message. - Elsewhere, returning ``NULL`` without setting an exception can - trigger undefined behavior. +Simply returning ``NULL`` is *not* correct behavior for an export hook, +and CPython complains about it. +That's good -- it means that CPython found the function! +Let's now make it do something useful. The slot table ============== Rather than ``NULL``, the export hook should return the information needed to -create a module, as a ``static`` array of :c:type:`PyModuleDef_Slot` entries. +create a module. +Let's with the basics: the name and docstring. + +The information should de defined in as ``static`` array of +:c:type:`PyModuleDef_Slot` entries, which are essentially key-value pairs. Define this array just before your export hook: .. code-block:: c static PyModuleDef_Slot spam_slots[] = { {Py_mod_name, "spam"}, - {Py_mod_doc, PyDoc_STR("A wonderful module with an example function")}, + {Py_mod_doc, "A wonderful module with an example function"}, {0, NULL} }; -The array contains: +For both name and docstring, the values are C strings -- that is, +NUL-terminated UTF-8 encoded byte arrays. -* the module name (as a NUL-terminated UTF-8 encoded C string), -* the docstring (similarly encoded), and -* a zero-filled *sentinel* marking the end of the array. +Note the zero-filled sentinel entry at the end. +If you forget it, you'll trigger undefined behavior. -.. tip:: - - The :c:func:`PyDoc_STR` macro can, in a special build mode, omit the - docstring to save a bit of memory. +The array is defined as ``static`` -- not visible outside this ``.c`` file. +This will be a common theme. +CPython only needs to access the export hook; all global variables +and all other functions should generally be ``static``, so that they don't +clash with other extensions. -Return this array from your export hook, instead of ``NULL``: +Return this array from your export hook instead of ``NULL``: .. code-block:: c :emphasize-lines: 4 @@ -298,17 +285,27 @@ Now, recompile and try it out: >>> print(spam) -You now have a extension module! +You have a extension module! Try ``help(spam)`` to see the docstring. -The next step will be adding a function to it. +The next step will be adding a function. +.. _backtoexample: + Exposing a function =================== -To add a function to the module, you will need a C function to call. -Add a minimal one now:: +To expose the ``system`` C function directly to Python, +we'll need to write a layer of glue code to convert arguments from Python +objects to C values, and the C return value back to Python. + +One of the simplest glue code is a ":c:data:`METH_O`" function, +which takes two Python objects and returns one. +All Pyton objects -- regardless of the Python type -- are represented in C +as pointers to the ``PyObject`` structure. + +Add such a function above the slots array:: static PyObject * spam_system(PyObject *self, PyObject *arg) @@ -316,52 +313,32 @@ Add a minimal one now:: Py_RETURN_NONE; } -This function is defined as ``static``. -This will be a common theme: usually, the only non-``static`` function -in a Python extension module is the export hook. -Data declarations are typically ``static`` as well. - -As a Python function, our ``spam_system`` returns a Python object. -In the C API, this is represented as a pointer to the ``PyObject`` structure. - -This function also takes two pointers to Python objects as arguments. -The "real" argument, as passed in from Python code, will be *arg*. -Meanwhile, *self* will be set to the module object -- in the C API, -module-level functions behave a bit like "methods" of the module object. - -The macro :c:macro:`Py_RETURN_NONE`, which we use as a body for now, -properly ``return``\s a Python :py:data:`None` object. +For now, we'll ignor the arguments, and use the :c:macro:`Py_RETURN_NONE` +macro to properly ``return`` a Python :py:data:`None` object. Recompile your extension to make sure you don't have syntax errors. -You might get a warning that ``spam_system`` is unused. -This is true: we haven't yet added the function to the module. +We haven't yet added ``spam_system`` to the module, so you might get a +warning that ``spam_system`` is unused. Method definitions ------------------ To expose the C function to Python, you will need to provide several pieces of -information. -These are collected in a structure called :c:type:`PyMethodDef`, -They are: +information in a structure called +:c:type:`PyMethodDef` [#why-pymethoddef]_: -* ``ml_name``: the name of the function to use in Python code; -* ``ml_meth``: the *implementation* -- the C function that does what - you need; +* ``ml_name``: the name of the Python function; +* ``ml_doc``: a docstring; +* ``ml_meth``: the C function to be called; and * ``ml_flags``: a set of flags describing details like how Python arguments are - passed to the C function; and -* ``ml_doc``: a docstring. - -The name and docstring are encoded as for module. -For ``ml_meth``, we'll use the ``spam_system`` function you just defined. - -For ``ml_flags``, we'll use :c:data:`METH_O` -- a flag that specifies that a -C function with two ``PyObject*`` arguments (your ``spam_system``) will be -exposed as a Python function taking a single unnamed argument. -(Other flags exist to handle named arguments, but they are harder to use.) + passed to the C function. + We'll use :c:data:`METH_O` here -- the flag that matches our + ``spam_system`` function's signature. Because modules typically create several functions, these definitions -need to be collected in an array: +need to be collected in an array, with a zero-filled sentinel at the end. +Add this array just below the ``spam_system`` function: .. literalinclude:: ../includes/capi-extension/spammodule-01.c :start-after: /// Module method table @@ -369,19 +346,8 @@ need to be collected in an array: As with module slots, a zero-filled sentinel marks the end of the array. -Add the array to your module, between the ``spam_system`` function and -the module slots. - -.. note:: - - Why :c:type:`!PyMethodDef` and not :c:type:`!PyFunctionDef`? - - In the C API, module-level functions behave a bit like "methods" - of a module object: they take an extra *self* argument, and - you need to provide the same information to create. - -To add the method(s) defined in a :c:type:`PyMethodDef` array, -add a :c:data:`Py_mod_methods` slot: +Next, we'll add the method to the module. +Add a :c:data:`Py_mod_methods` slot to your a :c:type:`PyMethodDef` array: .. literalinclude:: ../includes/capi-extension/spammodule-01.c :start-after: /// Module slot table @@ -389,7 +355,7 @@ add a :c:data:`Py_mod_methods` slot: :emphasize-lines: 6 Recompile your extension again, and test it. -You should now be able to call the function, and get ``None`` from it: +You should now be able to call the function, and get ``None`` back: .. code-block:: pycon @@ -403,19 +369,18 @@ You should now be able to call the function, and get ``None`` from it: Returning an integer ==================== -Your ``spam_system`` function currently returns ``None``. -We want it to return a number -- that is, a Python :py:type:`int` object. -Ultimately, this will be the exit code of a system command, -but let's start with a fixed value: ``3``. +Now, let's take a look at the return value. +Instead of ``None``, we'll want ``spam.system`` to return a number -- that is, +a Python :py:type:`int` object. +Eventually this will be the exit code of a system command, +but let's start with a fixed value, say, ``3``. The Python C API provides a function to create a Python :py:type:`int` object -from a C ``int`` values: :c:func:`PyLong_FromLong`. -(The name might not seem obvious to you; it'll get better as you get used to -C API naming conventions.) +from a C ``int`` values: :c:func:`PyLong_FromLong`. [#why-pylongfromlong]_ -Let's call it: +To call it, replace the ``Py_RETURN_NONE`` with the following 3 lines: -.. this could be a one-liner, we use 3 lines to show the data types. +.. this could be a one-liner, but we want to how the data types here. .. code-block:: c :emphasize-lines: 4-6 @@ -429,7 +394,7 @@ Let's call it: } -Recompile and run again, and check that you get a 3: +Recompile and run again, and check that the function now returns 3: .. code-block:: pycon @@ -441,21 +406,28 @@ Recompile and run again, and check that you get a 3: Accepting a string ================== -Your ``spam_system`` function should accepts one argument, -which is passed in as ``PyObject *arg``. +Finally, let's handle the function argument. + +Our C function, :c:func:`!spam_system`, takes two arguments. +The first one, ``PyObject *self``, will be set to the ``spam`` module +object. +This isn't useful in our case, so we'll ignore it. + +The other one, ``PyObject *arg``, will be set to the object that the user +called the Python with. +We expect that it should be a Python string. In order to use the information in it, we will need to convert it to a C value --- in this case, a C string (``const char *``). -The argument should be a Python string. There's a slight type mismatch here: Python's :c:type:`str` objects store Unicode text, but C strings are arrays of bytes. -So, we'll need to *encode* the data. -In our example, we'll use the UTF-8 encoding. -It might not always be correct for system commands, but it's what +So, we'll need to *encode* the data, and we'll use the UTF-8 encoding for it. +(UTF-8 might not always be correct for system commands, but it's what :py:meth:`str.decode` uses by default, -and the C API has special support for it. +and the C API has special support for it.) -The function to decode into a UTF-8 buffer is named :c:func:`PyUnicode_AsUTF8`. +The function to decode a Python string into a UTF-8 buffer is named +:c:func:`PyUnicode_AsUTF8` [#why-pyunicodeasutf8]_. Call it like this: .. code-block:: c @@ -472,20 +444,24 @@ Call it like this: If :c:func:`PyUnicode_AsUTF8` is successful, *command* will point to the resulting array of bytes. -This buffer is managed by the *arg* object, -which means limited: +This buffer is managed by the *arg* object, which means we don't need to free +it, but we must follow some rules: -* You should only use the buffer inside the ``spam_system`` function. - When ``spam_system`` returns, *arg* and the array it manages might be +* We should only use the buffer inside the ``spam_system`` function. + When ``spam_system`` returns, *arg* and the buffer it manages might be garbage-collected. -* You must not modify it. This is why we use ``const``. - -Both are fine for our use. +* We must not modify it. This is why we use ``const``. If :c:func:`PyUnicode_AsUTF8` was *not* successful, it returns a ``NULL`` pointer. -When using the Python C API, we always need to handle such error cases. -Here, the correct thing to do is returning ``NULL`` also from ``spam_system``: +When calling *any* Python C API, we always need to handle such error cases. +The way to do this in general is left for later chapters of this documentation. +For now, be assured that we are already handling errors from +:c:func:`PyLong_FromLong` correctly. + +For the :c:func:`PyUnicode_AsUTF8` call, the correct way to handle errors is +returning ``NULL`` from ``spam_system``. +Add an ``if`` block for this: .. code-block:: c @@ -522,7 +498,7 @@ the ``char *`` buffer, and using its result instead of the ``3``: return result; } -Compile it, and test: +Compile your module, and test: .. code-block:: pycon @@ -544,11 +520,14 @@ You might also want to test error cases: >>> spam.system(3) Traceback (most recent call last): - File "", line 1, in - spam.system(3) - ~~~~~~~~~~~^^^ + ... TypeError: bad argument type for built-in operation + >>> print(spam.system('too', 'many', 'arguments')) + Traceback (most recent call last): + ... + TypeError: spam.system() takes exactly one argument (3 given) + The result ========== @@ -563,406 +542,26 @@ Here is the entire source file, for your convenience: .. _extending-spammodule-source: .. literalinclude:: ../includes/capi-extension/spammodule-01.c + :start-at: /// .. rubric:: Footnotes .. [#why-spam] ``spam`` is the favorite food of Monty Python fans... - - -Accepting a string -================== - - - -The :c:type:`PyModuleDef_Slot` array must be passed to the interpreter in the -module's :ref:`export hook ` function. -The hook must be named :c:func:`!PyModExport_name`, where *name* is the name -of the module, and it should be the only non-\ ``static`` item defined in the -module file. - -Several modern C compilers emit warnings when you define a public function -without a prototype declaration. -Normally, prototypes go in a header file so they can be used from other -C files. -A function without a prototype is allowed in C, but is normally a strong -hint that something is wrong -- for example, the name is misspelled. - -In our case, Python will load our function dynamically, so a prototype -isn't needed. -We'll only add one to avoid well-meaning compiler warnings: - -.. literalinclude:: ../includes/capi-extension/spammodule-01.c - :start-after: /// Export hook prototype - :end-before: /// - -The :c:macro:`PyMODEXPORT_FUNC` macro declares the function's -``PyModuleDef_Slot *`` return type, and adds any special linkage -declarations required by the platform. -For C++, it declares the function as ``extern "C"``. - -Just after the prototype, we'll add the function itself. -Its only job is to return the slot array, which in turn contains -all the information needed to create our module object: - -.. literalinclude:: ../includes/capi-extension/spammodule-01.c - :start-after: /// Module export hook - :end-before: /// - - -The ``spam`` module -=================== - - -The first line of this file (after any comment describing the purpose of -the module, copyright notices, and the like) will pull in the Python API: - -.. literalinclude:: ../includes/capi-extension/spammodule-01.c - :start-at: - :end-at: - -Next, we bring in the declaration of the C library function we want to call. -Documentation of the :c:func:`system` function tells us that we need -``stdlib.h``: - -.. literalinclude:: ../includes/capi-extension/spammodule-01.c - :start-at: - :end-at: - -Be sure to put this, and any other standard library includes, *after* -:file:`Python.h`, since Python may define some pre-processor definitions -which affect the standard headers on some systems. - -.. tip:: - - The ```` include is technically not necessary. - :file:`Python.h` :ref:`includes several standard header files ` - for its own use and for backwards compatibility, - and ``stdlib`` is one of them. - However, it is good practice to explicitly include what you need. - -The next thing we add to our module file is the C function that will be called -when the Python expression ``spam.system(string)`` is evaluated. -We'll see shortly how it ends up being called. -The function should look like this: - -.. literalinclude:: ../includes/capi-extension/spammodule-01.c - :start-after: /// Implementation of spam.system - :end-before: /// - -Let's go through this line by line. - -This function will not be needed outside the current ``.c`` file, so we will -define it as ``static``. -(This can prevent name conflicts with similarly named functions elsewhere.) -It will return a pointer to a Python object -- the result that Python code -should receive when it calls ``spam.system``: - -.. literalinclude:: ../includes/capi-extension/spammodule-01.c - :start-at: static PyObject * - :end-at: static PyObject * - -We'll name the function ``spam_system``. -Combining the module name and the function name like this is standard practice, -and avoids clashes with other uses of ``system``. - -The Python function we are defining will take a single argument. -Its C implementation takes two arguments, here named *self* and *arg*, -both pointers to a Python object: - -.. literalinclude:: ../includes/capi-extension/spammodule-01.c - :start-at: spam_system(PyObject *self, PyObject *arg) - :end-at: { - -Since this is a module-level function, the *self* argument will point to -a module object. -For a method, *self* would point to the object instance, like a *self* -argument in Python. - -The *arg* will be a pointer to a Python object that the Python function -received as its single argument. - -In order for our C code to use the information in a Python object, we will need -to convert it to a C value --- in this case, a C string (``char *``). -The :c:func:`PyUnicode_AsUTF8` function does just that: it decodes a Python -string to a C one: - -.. literalinclude:: ../includes/capi-extension/spammodule-01.c - :start-at: const char *command = PyUnicode_AsUTF8(arg); - :end-at: const char *command = PyUnicode_AsUTF8(arg); - -If :c:func:`PyUnicode_AsUTF8` was successful, the string value of the argument -has been stored in the local variable *command*. -This is a C string, that is, a pointer to an array of bytes. -You are not supposed to modify the string to which it points, -which is why the variable is declared as ``const char *command``. - -.. note:: - - As its name implies, :c:func:`PyUnicode_AsUTF8` uses the UTF-8 encoding - --- the same that :py:meth:`str.decode` uses by default. - This is not always the correct encoding to use for system commands, but - it will do for our purposes. - -If :c:func:`PyUnicode_AsUTF8` was *not* successful, it returns a ``NULL`` -pointer. -When using the Python C API, we always need to handle such error cases. -Let's study this one in a bit more detail. - -Errors and exceptions ---------------------- - -An important convention throughout the Python interpreter is the following: -when a function fails, it should do two things: ensure an exception is set, -and return an error value. - -The error value is usually ``-1`` for functions that return :c:type:`!int`, -and ``NULL`` for functions that return a pointer. -However, this convention is not used in *all* cases. -You should always check the documentation for the functions you use. - -"Setting an exception" means that an exception object is stored in the -interpreter's thread state. -(If you are not familiar with threads, think of it like a global variable.) - -In our example, :c:func:`PyUnicode_AsUTF8` follows the convention: -on failure, it sets an exception and returns ``NULL``. - -Our function needs to detect this, and also return the ``NULL`` -error indicator: - -.. literalinclude:: ../includes/capi-extension/spammodule-01.c - :start-at: if (command == NULL) { - :end-at: } - -Our function does not neeed to set an exception itself, since -:c:func:`PyUnicode_AsUTF8` has already set it in the error case. - -In the non-error case, the ``return`` statement is skipped and execution -continues. - -.. note:: - - If we were calling a function that does *not* follow the Python convention - (for example, :c:func:`malloc`), we would need to set an exception using, - for example, :c:func:`PyErr_SetString`. - - -Back to the example -------------------- - -The next statement is a call to the standard C function :c:func:`system`, -passing it the C string we just got, and storing its result --- a C -integer: - -.. literalinclude:: ../includes/capi-extension/spammodule-01.c - :start-at: int status = system(command); - :end-at: int status = system(command); - -Our function must then convert the C integer into a Python integer. -This is done using the function :c:func:`PyLong_FromLong`: - -.. literalinclude:: ../includes/capi-extension/spammodule-01.c - :start-at: PyLong_FromLong - :end-at: return - -In this case, it will return a Python :py:class:`int` object. -(Yes, even integers are objects on the heap in Python!) -We return this object directly, as the result of our function. - -Note that when :c:func:`PyLong_FromLong` fails, it sets an exception and -returns ``NULL``, just like :c:func:`PyUnicode_AsUTF8` we used before. -In this case, our function should --- also like before --- return ``NULL`` -as well. -In both cases --- on success and on error --- we are returning -:c:func:`PyLong_FromLong`'s result, so we do not need an ``if`` here. - -.. note:: - - The names used in the C API are peculiar: - - * ``PyUnicode`` is used for Python :py:type:`str` - * ``PyLong`` is used for Python :py:type:`int` - - This convention dates back to the time when Python strings were implemented - by what is now :py:type:`bytes`, and integers were restricted to 64 bits - or so. - Unicode strings (then named :py:class:`!unicode`) and arbitrarily large - integers (then named :py:class:`!long`) were added later, - and in time, were renamed to :py:type:`str` and :py:type:`int` we know - today. - - However, the C API retains the old naming for backwards compatibility. - - Thus, the :c:func:`PyLong_FromLong` function creates a ``PyLong`` - (Python :py:class:`int`) object from a C :c:type:`!long`. - (C converts from C :c:type:`!int` to C :c:type:`!long` automatically.) - And the :c:func:`PyUnicode_AsUTF8` function represents a ``PyUnicode`` - (Python :py:class:`str`) object as a UTF-8-encoded, NUL-terminatedC string. - -That is it for the C implementation of the ``spam.function``! - -.. literalinclude:: ../includes/capi-extension/spammodule-01.c - :start-at: } - :end-at: } - -But, we are not done yet: we still need to define how the implementation -is called from Python code. - - -The module's method table -========================= - -To make our function available to Python programs, -we need to list it in a "method table" -- an array of :c:type:`PyMethodDef` -structures with a zero-filled *sentinel* -marking the end of the array. - -.. note:: Why not "PyFunctionDef"? - - The C API uses a common mechanism to define both functions of modules - and methods of classes. - You might say that, in the C API, module-level functions act like methods - of a module object. - -Since we are defining a single function, the array will have a single entry -before the sentinel: - -.. literalinclude:: ../includes/capi-extension/spammodule-01.c - :start-after: /// Module method table - :end-before: /// - -The :c:member:`!PyMethodDef.ml_name` and :c:member:`!PyMethodDef.ml_meth` -fields contain the Python name of the function, and the address of -the implementation. - -The :c:member:`!PyMethodDef.ml_flags` field tells the interpreter -the calling convention to be used for the C function. -In particular, :c:data:`METH_O` specifies that this function takes a single -unnamed (positional-only) argument. -This is among the simplest, but least powerful, calling conventions. -Perfect for your first function! - -Finally, :c:member:`!PyMethodDef.ml_doc` gives the docstring. -If this is left out (or ``NULL``), the function will not have a docstring. -The :c:func:`PyDoc_STR` macro is used to save a bit of memory when building -without docstrings. - - -ABI information -=============== - -Before defining the module itself, we'll add one more piece of boilerplate: -a variable with ABI compatibility information. -This is a variable that CPython checks to avoid loading incompatible modules ---- it allows raising an exception rather than crashing (or worse) -if we, for example, load the extension on an incompatible Python version. - -The ABI information defined using a macro that collects the relevant -build-time settings: - -.. literalinclude:: ../includes/capi-extension/spammodule-01.c - :start-after: /// ABI information - :end-before: /// - -This macro expands to a variable definition like -``static PyABIInfo abi_info = { ... data ... };`` - - -The slot table -============== - -Now, let's fit all the pieces of our module together. - -The method table we just defined, ``spam_methods``, -must be referenced in the module definition table: -an array of :c:type:`PyModuleDef_Slot` structures. -Like with the method table, a zero-filled *sentinel* marks the end. - -Besides the method table, this "slot table" will contain the module's -top-level information: the name, a docstring, and the ABI compatibility -information we've just defined: - -.. literalinclude:: ../includes/capi-extension/spammodule-01.c - :start-after: /// Module slot table - :end-before: /// - - -Module export hook -================== - -The :c:type:`PyModuleDef_Slot` array must be passed to the interpreter in the -module's :ref:`export hook ` function. -The hook must be named :c:func:`!PyModExport_name`, where *name* is the name -of the module, and it should be the only non-\ ``static`` item defined in the -module file. - -Several modern C compilers emit warnings when you define a public function -without a prototype declaration. -Normally, prototypes go in a header file so they can be used from other -C files. -A function without a prototype is allowed in C, but is normally a strong -hint that something is wrong -- for example, the name is misspelled. - -In our case, Python will load our function dynamically, so a prototype -isn't needed. -We'll only add one to avoid well-meaning compiler warnings: - -.. literalinclude:: ../includes/capi-extension/spammodule-01.c - :start-after: /// Export hook prototype - :end-before: /// - -The :c:macro:`PyMODEXPORT_FUNC` macro declares the function's -``PyModuleDef_Slot *`` return type, and adds any special linkage -declarations required by the platform. -For C++, it declares the function as ``extern "C"``. - -Just after the prototype, we'll add the function itself. -Its only job is to return the slot array, which in turn contains -all the information needed to create our module object: - -.. literalinclude:: ../includes/capi-extension/spammodule-01.c - :start-after: /// Module export hook - :end-before: /// - - -Legacy initialization function -============================== - -Lastly, we will add a function that serves no purpose but to detect -installation mistakes, and to avoid errors with older versions of some common -build tools. -You may want to try building the extension without this, and only add it -if you encounter errors involving ``PyInit``: - -.. literalinclude:: ../includes/capi-extension/spammodule-01.c - :start-after: /// Legacy initialization function - -This is an old-style :ref:`initialization function ` -that was required in extension modules for CPython 3.14 and below. -Current CPython will not call it, but some build tools may still assume that -all extension modules need to define it. - -When called, it raises a :py:exc:`SystemError` exception using -:c:func:`PyErr_SetString`, and returns the error indicator, ``NULL``. -Thus, if this extension does end up loaded on Python 3.14, the user will -get a proper error message. - - -.. _first-extension-result: - -End of file -=========== - -That's it! -You have written a simple Python C API extension module. -The result is repeated below. - -Now, you'll need to compile it. -We'll see how to do this in :ref:`building`. - -Here is the entire source file, for your convenience: - -.. _extending-spammodule-source: - -.. literalinclude:: ../includes/capi-extension/spammodule-01.c +.. [#why-spammodule] The source file name is entirely up to you, + though some tools can be picky about the ``.c`` extension. + This tutorial uses the traditional ``*module.c`` suffix. + Some people would just use :file:`spam.c` to implement a module + named ``spam``, + projects where Python isn't the primary language might use ``py_spam.c``, + and so on. +.. [#why-pymethoddef] The :c:type:`!PyMethodDef` structure is also used + to create methods of classes, so there's no separate + ":c:type:`!PyFunctionDef`". +.. [#why-pylongfromlong] The name :c:func:`PyLong_FromLong` + might not seem obvious. + ``PyLong`` refers to a the Python :py:class:`int`, which was originally + called ``long``; the ``FromLong`` refers to the C ``long`` (or ``long int``) + type. +.. [#why-pyunicodeasutf8] Here, ``PyUnicode`` refers to the original name of + the Python :py:class:`str` class: ``unicode``. diff --git a/Doc/extending/index.rst b/Doc/extending/index.rst index 43e53414317c2e..c0c494c3059d99 100644 --- a/Doc/extending/index.rst +++ b/Doc/extending/index.rst @@ -43,8 +43,7 @@ source file by including the header ``"Python.h"``. .. toctree:: - :maxdepth: 2 - :numbered: + :hidden: first-extension-module.rst extending.rst diff --git a/Doc/includes/capi-extension/spammodule-01.c b/Doc/includes/capi-extension/spammodule-01.c index 3db7ef1d5faafb..45b2c6c9d26784 100644 --- a/Doc/includes/capi-extension/spammodule-01.c +++ b/Doc/includes/capi-extension/spammodule-01.c @@ -1,3 +1,6 @@ +/* This file needs to be kept in sync with the tutorial + * at Doc/extending/first-extension-module.rst + */ /// Includes @@ -9,16 +12,15 @@ static PyObject * spam_system(PyObject *self, PyObject *arg) { - const char *command = PyUnicode_AsUTF8(arg); - if (command == NULL) { - return NULL; - } - int status = system(command); - PyObject *result = PyLong_FromLong(status); - return result; + const char *command = PyUnicode_AsUTF8(arg); + if (command == NULL) { + return NULL; + } + int status = system(command); + PyObject *result = PyLong_FromLong(status); + return result; } - /// Module method table static PyMethodDef spam_methods[] = { @@ -26,27 +28,20 @@ static PyMethodDef spam_methods[] = { .ml_name="system", .ml_meth=spam_system, .ml_flags=METH_O, - .ml_doc=PyDoc_STR("Execute a shell command."), + .ml_doc="Execute a shell command.", }, {NULL, NULL, 0, NULL} /* Sentinel */ }; -/// ABI information - -PyABIInfo_VAR(abi_info); - - /// Module slot table static PyModuleDef_Slot spam_slots[] = { - {Py_mod_abi, &abi_info}, {Py_mod_name, "spam"}, - {Py_mod_doc, PyDoc_STR("A wonderful module with an example function")}, + {Py_mod_doc, "A wonderful module with an example function"}, {Py_mod_methods, spam_methods}, {0, NULL} }; - /// Export hook prototype PyMODEXPORT_FUNC PyModExport_spam(void); @@ -56,19 +51,5 @@ PyMODEXPORT_FUNC PyModExport_spam(void); PyMODEXPORT_FUNC PyModExport_spam(void) { - return spam_slots; -} - - -/// Legacy initialization function - -PyMODINIT_FUNC PyInit_spam(void); - -PyMODINIT_FUNC -PyInit_spam(void) -{ - PyErr_SetString( - PyExc_SystemError, - "incompatible extension, need Python 3.15 or higher"); - return NULL; + return spam_slots; } From 7f1014e8d25234e200b8885c7cb3e35fab23e725 Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Fri, 5 Dec 2025 17:25:20 +0100 Subject: [PATCH 7/8] Re-add some reference tragets --- Doc/extending/first-extension-module.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/Doc/extending/first-extension-module.rst b/Doc/extending/first-extension-module.rst index 17d6fe361aa467..e95cd3f5ea394b 100644 --- a/Doc/extending/first-extension-module.rst +++ b/Doc/extending/first-extension-module.rst @@ -1,6 +1,7 @@ .. highlight:: c +.. _extending-simpleexample: .. _first-extension-module: ********************************* @@ -320,6 +321,7 @@ Recompile your extension to make sure you don't have syntax errors. We haven't yet added ``spam_system`` to the module, so you might get a warning that ``spam_system`` is unused. +.. _methodtable: Method definitions ------------------ From 4c61bf48cf0ea255dbf8fdd82d1b332f481bc55d Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Fri, 5 Dec 2025 18:19:04 +0100 Subject: [PATCH 8/8] Typo fix --- Doc/extending/extending.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/extending/extending.rst b/Doc/extending/extending.rst index 05c8f65b83bc5e..b7ae6eeb9ca792 100644 --- a/Doc/extending/extending.rst +++ b/Doc/extending/extending.rst @@ -478,7 +478,7 @@ Some example calls:: Keyword Parameters for Extension Functions ========================================== -If you also want your function to accest *keyword* arguments, +If you also want your function to accept *keyword* arguments, use the :c:data:`METH_KEYWORDS` flag in combination with :c:data:`METH_VARARGS`. (It can also be used with other flags; see its documentation for the allowed