Skip to content

tests: Run gunittest-based tests with pytest that don’t require gunittest #5584

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 24 commits into from
May 7, 2025

Conversation

echoix
Copy link
Member

@echoix echoix commented Apr 23, 2025

Some tests are being run by gunittest, which doesn’t bring much value over pytest. Some of them don’t even require features brought by gunittest, and don’t require too much special grass setup to run.

Thus, this PR starts the work of making pytest run some of the tests in testsuite directories. This first subset are only the ones that have all of their tests passing, on all three platforms.

  • This is the continuation of my recent preparatory PRs that made test collection work across our platforms.
  • Code coverage is enabled correctly for Linux, and lots more files are « discovered » and counted for. This is great improvement.
  • This doesn’t remove these tests to run in gunittest yet. To see the equivalence, I’d like to have both running for at least two weeks. (A failure would occur in pytest and stop before starting gunittest on macOS and Windows, but are separate on Ubuntu)
  • The tests running here only account for a couple seconds of the total gunittest run time, so no real advantage except easier debugging, test selection, and enforcement.
  • Some tests should not even be attempted to be ran, as it crashes in the setup phase. These are the files that I added with --deselect in a list. I just couldn’t find the way to not have the Python interpreter just crash when calling a grass C-library functions through ctypes (like G_mapset_path(), or any setup-like functions). Since it isn’t run through a grass --exec, there’s a difference.
  • The file lib/init/testsuite/test_grass_tmp_mapset.py is ignored on Windows, I have another PR for that, but it’s not simple or obvious what the good solution would be, so I’ll be getting this through first without having it work on Windows.
  • This is the fruit of long and long work, split up with what I’m confident to have merged now.

@github-actions github-actions bot added windows Microsoft Windows specific macOS macOS specific CI Continuous integration Python Related code is in Python labels Apr 23, 2025
@echoix echoix requested a review from wenzeslaus April 23, 2025 22:58
@echoix echoix force-pushed the gunittest-pytest-already-working branch from 4c4efbc to 11a7f5b Compare April 23, 2025 22:59
@echoix
Copy link
Member Author

echoix commented May 2, 2025

@wenzeslaus If you have the chance, can you take a look before the weekend, so I can continue off of it after?

@echoix
Copy link
Member Author

echoix commented May 2, 2025

There's a new error since I merged main that I didn't saw yet before.

wenzeslaus
wenzeslaus previously approved these changes May 2, 2025
Copy link
Member

@wenzeslaus wenzeslaus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have time to re-review this in depth right now, but I already looked at it earlier and didn't see any red flags then and I don't see them now.

wenzeslaus
wenzeslaus previously approved these changes May 2, 2025
@echoix
Copy link
Member Author

echoix commented May 2, 2025

I'll have to find what changed in the last 8 days or so that make the collection fail now.. The cause is not what I was expecting, so I'll need to dig deeper, and will need a new approval then.

@echoix
Copy link
Member Author

echoix commented May 4, 2025

Ok, I've taken a look. It is not a problem from the commits we did since: trying different revisions end up failing too, and even re-running a passing job from 2 weeks ago (April 23 2025), now fails.

I compared an old copy of the grass85 OSGeo4W install folder I had from March 8, 2025, and I indeed see that some functions are not in the python/grass/lib/raster.py file anymore:

  • Rast_legal_semantic_label
  • Rast_mask_status
  • Rast_mask_is_present
    image
    image

Looking at a rerun of a previously passing job, I now see in the compilation logs the following, that explains why the function isn't there:

ERROR: Function "Rast_legal_semantic_label" depends on an unknown typedef "bool". Function "Rast_legal_semantic_label" will not be output

image
It can also be observed here on main:
https://github.com/OSGeo/grass/actions/runs/14820561314/job/41606915501#step:7:3510
And that test also fails on main for Windows with pure Gunittest: https://github.com/OSGeo/grass/actions/runs/14820561314/job/41606915501#step:13:182

Running .\lib\raster\testsuite\test_raster_metadata.py...
========================================================================
Traceback (most recent call last):

  File "lib\raster\testsuite\test_raster_metadata.py", line 23, in <module>

    from grass.lib.raster import (

ImportError: cannot import name 'Rast_legal_semantic_label' from 'grass.lib.raster' (etc\python\grass\lib\raster.py). Did you mean: 'Rast_read_semantic_label'?

========================================================================

Since gunittest on Windows doesn't enforce 100% success in CI, it got unnoticed, as the success (file-wise) was 96.3 and the threshold was 96.

So what changed that made the new errors appear in the compilation process to create the ctypes bindings?
I looked at the msys installation logs, and compared the packages listed to install, and found it. GCC 15 (15.1) was released recently, and msys already changed. So that's probably the cause. Now, how do we fix this?

@echoix
Copy link
Member Author

echoix commented May 4, 2025

The first appearance of the error in our CI logs was in a PR run on April 27, 2025, https://github.com/OSGeo/grass/actions/runs/14692952416/job/41230768339#step:7:3522
For commits on main, it is between

@echoix
Copy link
Member Author

echoix commented May 4, 2025

A nice hint is given in the page "Porting to GCC 15", section "New keywords"
https://gcc.gnu.org/gcc-15/porting_to.html#c23-new-keywords

New keywords
C23 added various new keywords, including bool, true, false, nullptr, and thread_local. Code that uses these for identifiers will need changing. For example typedef int bool; will fail with:

<source>:1:13: error: 'bool' cannot be defined via 'typedef'
    1 | typedef int bool;
      |             ^~~~
<source>:1:13: note: 'bool' is a keyword with '-std=c23' onwards
<source>:1:1: warning: useless type name in empty declaration
    1 | typedef int bool;
      | ^~~~~~~

In C99 and later you can use #include <stdbool.h> which provides definitions of bool, true, and false compatible with C23.

Note that the bool type is not the same as int at ABI level, and so care may be needed porting declarations that appear at an ABI boundary (or serialized to the filesystem).

So, as announced, GCC15 defaults to C23. Seems we don't work correctly now, and maybe rebuilds of grass84 on OSGeo4W could be broken if it doesn't keep the old default. For grass 8.5, we've had some work done to fix many issues, but we forgot that part it seems.

@nilason Do you have ideas where to start to find a fix for compiling correctly with GCC15? In the meantime, setting -std=c17 or -std=gnu17 might get it back to work.

@echoix
Copy link
Member Author

echoix commented May 4, 2025

There's also the top of this article that mentions something interesting: "ell bool, true and false are new keywords. And specifically false is not equals to 0 or NULL."

@marisn
Copy link
Contributor

marisn commented May 6, 2025

See also #5585 for bool issues.

wenzeslaus
wenzeslaus previously approved these changes May 6, 2025
@echoix
Copy link
Member Author

echoix commented May 6, 2025

Let's see if #5618 was enough. I hope so.

@echoix
Copy link
Member Author

echoix commented May 6, 2025

Unfortunately, #5618 wasn't enough, the ctypes binding still don't emit Rast_legal_semantic_label.
image

@echoix
Copy link
Member Author

echoix commented May 6, 2025

If we would be able to have our builds working with Ubuntu 24.04, I had work started to be able to run the equivalent of the gcc17 standard check job, but with newer compiler versions. But, some packages are missing (at least 2). For example, since pdal isn't in the package repository anymore, the current package list fails.

I would expect to see some of the gcc15 issues there, if it is caused only by gcc15, and not ctypesgen, that would need to be updated here from upstream, and upstream is not really active.

@echoix
Copy link
Member Author

echoix commented May 7, 2025

Update this PR again after #5621 is merged. Remove the contents added in the last commit.

Keeping it running to show it works

@echoix echoix requested a review from wenzeslaus May 7, 2025 14:22
@echoix
Copy link
Member Author

echoix commented May 7, 2025

Ready to merge :)

@echoix echoix merged commit 0b3b2eb into OSGeo:main May 7, 2025
29 checks passed
@github-actions github-actions bot added this to the 8.5.0 milestone May 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI Continuous integration libraries macOS macOS specific Python Related code is in Python windows Microsoft Windows specific
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants