Skip to content

robotpy-build meson rewrite #252

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 11 commits into
base: next
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
75 changes: 36 additions & 39 deletions .github/workflows/dist.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,14 +50,14 @@ jobs:
with:
submodules: true

- uses: actions/setup-python@v4
with:
python-version: 3.8
- name: Sphinx
run: |
pip --disable-pip-version-check install -e .
pip --disable-pip-version-check install -r docs/requirements.txt
cd docs && make clean html SPHINXOPTS="-W --keep-going"
# - uses: actions/setup-python@v5
# with:
# python-version: 3.8
# - name: Sphinx
# run: |
# pip --disable-pip-version-check install -e .
# pip --disable-pip-version-check install -r docs/requirements.txt
# cd docs && make clean html SPHINXOPTS="-W --keep-going"

build:
runs-on: ubuntu-22.04
Expand All @@ -67,7 +67,7 @@ jobs:
submodules: recursive
fetch-depth: 0

- uses: actions/setup-python@v4
- uses: actions/setup-python@v5
with:
python-version: 3.8

Expand Down Expand Up @@ -97,12 +97,12 @@ jobs:
- '3.11'
- '3.12'
- '3.13'
architecture: [x86, x64]
exclude:
- os: macos-13
architecture: x86
- os: ubuntu-22.04
architecture: x86
architecture: [x64]
# exclude:
# - os: macos-13
# architecture: x86
# - os: ubuntu-22.04
# architecture: x86
include:
- os: macos-14
python_version: 3.9
Expand All @@ -116,7 +116,7 @@ jobs:
with:
submodules: true

- uses: actions/setup-python@v4
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python_version }}
architecture: ${{ matrix.architecture }}
Expand All @@ -133,25 +133,32 @@ jobs:
key: ${{ matrix.os }}-${{ matrix.architecture }}-${{ matrix.python_version }}
variant: ccache

- name: Setup MSVC
uses: bus1/cabuild/action/msdevshell@e22aba57d6e74891d059d66501b6b5aed8123c4d # v1
with:
architecture: x64
if: runner.os == 'Windows'

- name: Configure ccache for MSVC
shell: bash
# https://github.com/mesonbuild/meson/issues/10423
run: |
echo "CC=ccache.exe cl" >> $GITHUB_ENV
echo "CXX=ccache.exe cl" >> $GITHUB_ENV
if: runner.os == 'Windows'

- name: Install
shell: bash
working-directory: dist
run: python -m pip --disable-pip-version-check install *.whl

- name: Install test dependencies
shell: bash
working-directory: tests
run: python -m pip --disable-pip-version-check install -r requirements.txt

- name: Test wheel
shell: bash
env:
RPYBUILD_PARALLEL: 1
RPYBUILD_STRIP_LIBPYTHON: 1
RPYBUILD_CC_LAUNCHER: ccache
working-directory: tests
run: |
python run_tests.py
run: python run_tests.py

cross-build:
runs-on: ubuntu-latest
Expand All @@ -160,8 +167,8 @@ jobs:
max-parallel: ${{ fromJSON(needs.setup_concurrency.outputs.max-parallel).v }}
matrix:
container:
- wpilib/roborio-cross-ubuntu:2024-22.04-py312
- wpilib/raspbian-cross-ubuntu:bullseye-22.04-py312
- wpilib/roborio-cross-ubuntu:2025-22.04-py313
- wpilib/raspbian-cross-ubuntu:2025-bookworm-22.04-py313

container:
image: "${{ matrix.container }}"
Expand All @@ -183,33 +190,23 @@ jobs:
name: dist
path: dist

- name: Install setuptools
run: |
/build/venv/bin/build-pip --disable-pip-version-check install setuptools
/build/venv/bin/cross-pip --disable-pip-version-check install setuptools
- name: Setup cross environment
uses: robotpy/build-actions/setup-cross-meson@semiwrap

- name: Install
working-directory: dist
run: |
# install to both cross and build in case things get mixed up
/build/venv/bin/build-pip --disable-pip-version-check install *.whl
/build/venv/bin/cross-pip --disable-pip-version-check install *.whl

- name: Install test dependencies
shell: bash
working-directory: tests
run: |
# install to both cross and build in case things get mixed up
/build/venv/bin/build-pip --disable-pip-version-check install -r requirements.txt
/build/venv/bin/cross-pip --disable-pip-version-check install -r requirements.txt

- name: Build cross wheel
env:
RPYBUILD_PARALLEL: 1
RPYBUILD_STRIP_LIBPYTHON: 1
RPYBUILD_CC_LAUNCHER: ccache
working-directory: tests/cpp
run: /build/venv/bin/cross-python run_install.py wheel
run: /build/venv/bin/cross-python run_install.py --config-settings=setup-args=--cross-file=cross.txt

publish:
runs-on: ubuntu-latest
Expand Down
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,4 @@ __pycache__
/dist
/build

/robotpy_build/version.py
/src/semiwrap/version.py
4 changes: 2 additions & 2 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
[submodule "robotpy_build/pybind11"]
path = robotpy_build/pybind11
[submodule "src/semiwrap/pybind11"]
path = src/semiwrap/pybind11
url = https://github.com/pybind/pybind11.git
2 changes: 0 additions & 2 deletions MANIFEST.in

This file was deleted.

25 changes: 18 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,18 @@
robotpy-build
=============
semiwrap
========

This is a build tool intended to be generally useful for any python project
that has binary dependencies. It is especially designed to meet the needs
of RobotPy's various wrapper libraries, chiefly around:
that wants to use pybind11 to create a Python API around C/C++ binary dependencies.



This is primarily intended to be used with hatchling and meson to build

This tool parses C/C++ headers


It is especially designed to meet the needs of RobotPy's various wrapper
libraries, chiefly around:

* Managing upstream binary dependencies
* Autogenerating pybind11 wrappers around those dependencies
Expand All @@ -14,13 +23,15 @@ Requires Python 3.8+
Documentation
-------------

Documentation can be found at https://robotpy-build.readthedocs.io/
Documentation can be found at https://semiwrap.readthedocs.io/

Author
------

Dustin Spicuzza is the primary author of robotpy-build, but it is the
Dustin Spicuzza is the primary author of semiwrap.

Semiwrap is a direct decendant of the robotpy-build project, and is
culmination of many years of experimentation with automated wrapper
generation by members of the RobotPy community.

robotpy-build is available under the BSD 3-clause license.
semiwrap is available under the BSD 3-clause license.
19 changes: 10 additions & 9 deletions docs/autowrap.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
Automated C++ header wrapping
=============================

robotpy-build can be told to parse C/C++ headers and automatically generate
semiwrap can be told to parse C/C++ headers and automatically generate
:std:doc:`pybind11 <pybind11:basics>` wrappers around the functions
and objects found in that header.

Expand All @@ -15,9 +15,9 @@ and objects found in that header.
C++ Features
------------

robotpy-build uses a pure python C++ parser and macro processor to attempt to
semiwrap uses a pure python C++ parser and macro processor to attempt to
parse header files. As a result, a full AST of the header files is not created.
This means particularly opaque code might confuse the parser, as robotpy-build
This means particularly opaque code might confuse the parser, as semiwrap
only receives the names, not the actual type information.

However, most basic features typically work without needing to coerce the
Expand All @@ -34,6 +34,7 @@ generator into working correctly, including:
* final classes/methods - cannot be overridden from Python code
* Enumerations
* Global variables
* Many many more weird edge cases too

Additionally, the following features are supported, but require some manual
intervention:
Expand All @@ -46,22 +47,22 @@ to your package in ``pyproject.toml``:

.. code-block:: toml

[tool.robotpy-build.wrappers."MYPACKAGE".autogen_headers]
[tool.semiwrap.extension_modules."PACKAGE.NAME".headers]
demo = "demo.h"

That causes ``demo.h`` to be parsed and wrapped.

.. note:: If you're importing a large number of headers, the
``robotpy-build scan-headers`` tool can generate this for you
``semiwrap scan-headers`` tool can generate this list for you
automatically.

Documentation
-------------

robotpy-build will find doxygen documentation comments on many types of elements
and use sphinxify to translate them into python docstrings. All elements that
support documentation strings can have their docstrings set explicitly using
a ``doc`` value in the YAML file.
semiwrap will find doxygen documentation comments on many types of elements
and use sphinxify to translate them into python docstrings. If this is not
sufficient, all elements that support documentation strings can have their
docstrings set explicitly using a ``doc`` value in the YAML file.

.. code-block:: yaml

Expand Down
5 changes: 3 additions & 2 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@

os.environ["GENERATING_DOCUMENTATION"] = "True"

__version__ = pkg_resources.get_distribution("robotpy-build").version
__version__ = pkg_resources.get_distribution("semiwrap").version


# -- RTD configuration ------------------------------------------------
Expand All @@ -25,7 +25,7 @@

# -- Project information -----------------------------------------------------

project = "robotpy-build"
project = "semiwrap"
copyright = "2020, RobotPy Development Team"
author = "RobotPy Development Team"

Expand Down Expand Up @@ -72,3 +72,4 @@
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = []

56 changes: 29 additions & 27 deletions docs/config/autowrap.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,66 +4,68 @@
Generator Customization
=======================

Because robotpy-build's code generation is intended to be a semi-automated
Because semiwrap's code generation is intended to be a semi-automated
process (except for simple C++ code), a rich set of per-{class/function/parameter}
configuration options can be specified in per-file YAML configurations.

Additionally, some headers are too complex for the autogenerator to
completely process, so when this occurs you must manually specify required
information in the YAML file.

Most files generated by robotpy-build are customizable.
Most files generated by semiwrap are customizable.

.. note:: robotpy-build is designed for the RobotPy project and may contain
.. note:: semiwrap is designed for the RobotPy project and may contain
defaults that aren't appropriate for all projects. If you find that
you need more customization, file an issue on github and let's talk
about it!

Location of customization file
------------------------------

In your ``pyproject.toml``, you can specify either a single YAML file with
customizations, or you can specify a directory that robotpy-build will search
for YAML files.

Single file:

.. code-block:: toml

[tool.robotpy-build.wrappers."PACKAGENAME"]
generation_data = "gen/data.yml"

Multiple files:
By default customization files are located in the ``wrapcfg`` directory. The name
of the YAML file is the ``key`` in the extension module headers table:

.. code-block:: toml

[tool.robotpy-build.wrappers."PACKAGENAME"]
generation_data = "gen"

When a directory is specified, pybind11 will search for YAML files in the
directory based on the header filename. In the above example, customization
data for ``header.h`` could be specified in ``gen/header.yml``.
[tool.semiwrap.extension_modules."PACKAGE.NAME".headers]
# yaml file is `wrapcfg/demo.yml`
demo = "include/demo.h"


Autogeneration
--------------

The default values for these YAML files can be generated via the robotpy-build
command line tool:
The default values for these YAML files should be generated via the semiwrap command
line tool:

.. code-block:: sh

robotpy-build create-gen --write
semiwrap create-gen --write

This can be a good way to get the boilerplate out of the way when you need to
provide customizations.

Manual pybind11 APIs
--------------------

You can write your own custom .cpp files and add them to the python extensions
generated by semiwrap. In your ``meson.build`` after you include the generated
``wrapcfg`` subdir, add files to ``NAME_sources``.

.. code-block:: meson

subdir('wrapcfg')

my_module_sources += files(
'src/my_module/main.cpp',
)


Reference
---------

The following strctures describe the dictionaries that are read from the YAML
file. The toplevel structure is :class:`.AutowrapConfigYaml`.
files. The toplevel structure is :class:`.AutowrapConfigYaml`.

.. automodule:: robotpy_build.config.autowrap_yml
.. automodule:: semiwrap.config.autowrap_yml
:members:
:undoc-members:
Loading
Loading