Skip to content

[Bug] t.register crash using file option #5585

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Dankotchev opened this issue Apr 23, 2025 · 17 comments · Fixed by #5618
Closed

[Bug] t.register crash using file option #5585

Dankotchev opened this issue Apr 23, 2025 · 17 comments · Fixed by #5618
Labels
bug Something isn't working
Milestone

Comments

@Dankotchev
Copy link

Describe the bug

When I run the t.register command using the file option, an error occurs and the following message is displayed.

To reproduce

Steps to reproduce the behavior:

  1. run the command t.register --overwrite input=region@PERMANENT file=file.txt
  2. Observe the following error:
Process Process-2:
Traceback (most recent call last):
  File "/usr/lib64/python3.13/multiprocessing/process.py",
line 313, in _bootstrap
    self.run()
    ~~~~~~~~^^
  File "/usr/lib64/python3.13/multiprocessing/process.py",
line 108, in run
    self._target(*self._args, **self._kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/grass84/etc/python/grass/temporal/c_libra
ries_interface.py", line 1275, in c_library_server
    functions[data[0]](lock, conn, data)
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/grass84/etc/python/grass/temporal/c_libra
ries_interface.py", line 589, in _write_semantic_label
    if libraster.Rast_legal_semantic_label(semantic_label)
is False:
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'grass.lib.raster' has no attribute
'Rast_legal_semantic_label'. Did you mean:
'Rast_read_semantic_label'?
ERROR: Exception raised: [Errno 104] Connection reset by peer Message: write_raster_timestamp
WARNING:root:Needed to restart the libgis server, caller:
Server check thread

Expected behavior

Rasters should be registered correctly without error.

System description

  • Operating System: Fedora 42 with Kernel 6.14.2-300.fc42.x86_64
  • GRASS GIS version:
version=8.4.1
date=2025
revision=45ca3179ab
build_date=2025-02-24
build_platform=x86_64-redhat-linux-gnu
build_off_t_size=8
libgis_revision=45ca3179ab
libgis_date=2025-02-24T10:57:24+00:00
proj=9.5.1
gdal=3.10.2
geos=3.13.0
sqlite=3.47.2
  • Python: 3.13.3 (main, Apr 9 2025, 00:00:00) [GCC 15.0.1 20250410 (Red Hat 15.0.1-0)]
  • wxPython: 4.2.2 gtk3 (phoenix) wxWidgets 3.2.6

Additional context

@Dankotchev Dankotchev added the bug Something isn't working label Apr 23, 2025
@petrasovaa
Copy link
Contributor

Hmm, as far as I can tell, grass.lib.raster has Rast_legal_semantic_label, so I am not sure what's wrong. Did you compile it? Maybe some issue during compilation?

@Dankotchev
Copy link
Author

No, I didn't compile GRASS. I installed it right from the Fedora repository. I installed it when I was in Fedora 41 and now I'm in Fedora 42 (if this information is relevant). I will leave below part of the file that I am using and that worked previously

x0240719_B01@PERMANENT|2024-07-19 00:00:00.0000|B01
x0240719_B02@PERMANENT|2024-07-19 00:00:00.0000|B02
x0240719_B03@PERMANENT|2024-07-19 00:00:00.0000|B03
x0240719_B04@PERMANENT|2024-07-19 00:00:00.0000|B04
...

@neteler
Copy link
Member

neteler commented Apr 25, 2025

I installed it right from the Fedora repository

Fedora packager here. Can you please provide a reproducible example using e.g. the North Carolina dataset?

@Dankotchev
Copy link
Author

I run the following (NC dataset, PERMANENT location):

t.create output=test_temporal semantictype=mean title="Test Temporal" description=Description

t.list

test_temporal@PERMANENT

Space time raster datasets with absolute time available in
mapset :

t.register input=test_temporal@PERMANENT file=/home/user/grassdata/timestamps.txt

Process Process-2:
Traceback (most recent call last):
File "/usr/lib64/python3.13/multiprocessing/process.py",
line 313, in _bootstrap
self.run()
~~~~~~~~^^
File "/usr/lib64/python3.13/multiprocessing/process.py",
line 108, in run
self._target(*self._args, **self._kwargs)
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/grass84/etc/python/grass/temporal/c_libra
ries_interface.py", line 1275, in c_library_server
functions[data[0]](lock, conn, data)
~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
File "/usr/lib64/grass84/etc/python/grass/temporal/c_libra
ries_interface.py", line 589, in _write_semantic_label
if libraster.Rast_legal_semantic_label(semantic_label)
is False:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'grass.lib.raster' has no attribute
'Rast_legal_semantic_label'. Did you mean:
'Rast_read_semantic_label'?
ERROR: Exception raised: [Errno 104] Connection reset by peer Message: write_raster_timestamp

t.rast.list input=test_temporal@PERMANENT

ERROR: Nothing found in the database for space time dataset <test_temporal@PERMANENT> (type: strds): Dataset is empty

timestamp.txt

lsat7_2002_10|2002-05-24 00:00:00.0000|B10
lsat7_2002_20|2002-05-24 00:00:00.0000|B20
lsat7_2002_30|2002-05-24 00:00:00.0000|B30
lsat7_2002_40|2002-05-24 00:00:00.0000|B40
lsat7_2002_50|2002-05-24 00:00:00.0000|B50
lsat7_2002_61|2002-05-24 00:00:00.0000|B61
lsat7_2002_62|2002-05-24 00:00:00.0000|B62
lsat7_2002_70|2002-05-24 00:00:00.0000|B70
lsat7_2002_80|2002-05-24 00:00:00.0000|B80

@neteler
Copy link
Member

neteler commented Apr 29, 2025

Indeed, I can reproduce it with podman/docker.
Here the detailed steps in case someone else wants to try and debug:

docker pull fedora:42
docker run -it --rm --volume="$(pwd)/:/data" fedora:42
dnf update -y
dnf install wget unzip vim -y
dnf install grass -y
cd /tmp/
wget https://grass.osgeo.org/sampledata/north_carolina/nc_spm_08_grass7.zip
unzip  nc_spm_08_grass7.zip

# data from above
vim /tmp/timestamps.txt

grass nc_spm_08_grass7/PERMANENT/
GRASS nc_spm_08_grass7/PERMANENT:tmp > 

t.create output=test_temporal semantictype=mean title="Test Temporal" description=Description

t.register input=test_temporal@PERMANENT file=/tmp/timestamps.txt
Process Process-2:
Traceback (most recent call last):
  File "/usr/lib64/python3.13/multiprocessing/process.py", line 313, in _bootstrap
    self.run()
    ~~~~~~~~^^
  File "/usr/lib64/python3.13/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/grass84/etc/python/grass/temporal/c_libraries_interface.py", line 1275, in c_library_server
    functions[data[0]](lock, conn, data)
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/grass84/etc/python/grass/temporal/c_libraries_interface.py", line 589, in _write_semantic_label
    if libraster.Rast_legal_semantic_label(semantic_label) is False:
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'grass.lib.raster' has no attribute 'Rast_legal_semantic_label'. Did you mean: 'Rast_read_semantic_label'?
ERROR: Exception raised: [Errno 104] Connection reset by peer Message:
       write_raster_timestamp

--> here I needed to press Ctrl-C to get out of it

^CProcess Process-1:
Process Process-3:
Traceback (most recent call last):
  File "/usr/lib64/python3.13/multiprocessing/process.py", line 313, in _bootstrap
    self.run()
    ~~~~~~~~^^
  File "/usr/lib64/python3.13/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/grass84/etc/python/grass/pygrass/messages/__init__.py", line 66, in message_server
    conn.poll(None)
    ~~~~~~~~~^^^^^^
  File "/usr/lib64/python3.13/multiprocessing/connection.py", line 257, in poll
    return self._poll(timeout)
           ~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/multiprocessing/connection.py", line 440, in _poll
    r = wait([self], timeout)
  File "/usr/lib64/python3.13/multiprocessing/connection.py", line 1148, in wait
    ready = selector.select(timeout)
  File "/usr/lib64/python3.13/selectors.py", line 398, in select
    fd_event_list = self._selector.poll(timeout)
KeyboardInterrupt
Traceback (most recent call last):
Traceback (most recent call last):
  File "/usr/lib64/python3.13/multiprocessing/process.py", line 313, in _bootstrap
    self.run()
    ~~~~~~~~^^
  File "/usr/lib64/python3.13/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/grass84/etc/python/grass/temporal/c_libraries_interface.py", line 1272, in c_library_server
    conn.poll(None)
    ~~~~~~~~~^^^^^^
  File "/usr/lib64/python3.13/multiprocessing/connection.py", line 257, in poll
    return self._poll(timeout)
           ~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/multiprocessing/connection.py", line 440, in _poll
    r = wait([self], timeout)
  File "/usr/lib64/python3.13/multiprocessing/connection.py", line 1148, in wait
    ready = selector.select(timeout)
  File "/usr/lib64/python3.13/selectors.py", line 398, in select
    fd_event_list = self._selector.poll(timeout)
KeyboardInterrupt
  File "/usr/lib64/grass84/etc/python/grass/pygrass/rpc/base.py", line 169, in safe_receive
    ret = self.client_conn.recv()
  File "/usr/lib64/python3.13/multiprocessing/connection.py", line 250, in recv
    buf = self._recv_bytes()
  File "/usr/lib64/python3.13/multiprocessing/connection.py", line 430, in _recv_bytes
    buf = self._recv(4)
  File "/usr/lib64/python3.13/multiprocessing/connection.py", line 395, in _recv
    chunk = read(handle, remaining)
ConnectionResetError: [Errno 104] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib64/grass84/scripts/t.register", line 153, in <module>
    tgis.profile_function(main)
    ~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/usr/lib64/grass84/etc/python/grass/temporal/core.py", line 79, in profile_function
    func()
    ~~~~^^
  File "/usr/lib64/grass84/scripts/t.register", line 129, in main
    tgis.register_maps_in_space_time_dataset(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        type=type,
        ^^^^^^^^^^
    ...<9 lines>...
        fs=separator,
        ^^^^^^^^^^^^^
    )
    ^
  File "/usr/lib64/grass84/etc/python/grass/temporal/register.py", line 416, in register_maps_in_space_time_dataset
    statement += map_object.insert(dbif=dbif, execute=False)
                 ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/grass84/etc/python/grass/temporal/abstract_map_dataset.py", line 364, in insert
    self.write_timestamp_to_grass()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/usr/lib64/grass84/etc/python/grass/temporal/space_time_datasets.py", line 312, in write_timestamp_to_grass
    check = self.ciface.write_raster_timestamp(
        self.get_name(), self.get_mapset(), self._convert_timestamp()
    )
  File "/usr/lib64/grass84/etc/python/grass/temporal/c_libraries_interface.py", line 1649, in write_raster_timestamp
    return self.safe_receive("write_raster_timestamp")
           ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/grass84/etc/python/grass/pygrass/rpc/base.py", line 176, in safe_receive
    raise FatalError("Exception raised: " + str(e) + " Message: " + message)
grass.exceptions.FatalError: Exception raised: [Errno 104] Connection reset by peer Message: write_raster_timestamp

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib64/grass84/scripts/t.register", line 155, in <module>
    grass.fatal(e)
    ~~~~~~~~~~~^^^
  File "/usr/lib64/grass84/etc/python/grass/script/core.py", line 794, in fatal
    error(msg, env=env)
    ~~~~~^^^^^^^^^^^^^^
  File "/usr/lib64/grass84/etc/python/grass/script/core.py", line 777, in error
    message(msg, flag="e", env=env)
    ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/grass84/etc/python/grass/script/core.py", line 696, in message
    run_command("g.message", flags=flag, message=msg, errors="ignore", env=env)
    ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/grass84/etc/python/grass/script/core.py", line 476, in run_command
    ps = start_command(*args, **kwargs)
  File "/usr/lib64/grass84/etc/python/grass/script/core.py", line 431, in start_command
    return Popen(args, **popts)
  File "/usr/lib64/grass84/etc/python/grass/script/core.py", line 72, in __init__
    subprocess.Popen.__init__(self, args, **kwargs)
    ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/subprocess.py", line 1039, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
    ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                        pass_fds, cwd, env,
                        ^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
                        gid, gids, uid, umask,
                        ^^^^^^^^^^^^^^^^^^^^^^
                        start_new_session, process_group)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/subprocess.py", line 1922, in _execute_child
    part = os.read(errpipe_read, 50000)
KeyboardInterrupt
WARNING:root:Needed to restart the libgis server, caller: Server check thread

Might it be Python 3.13 related?

(as a Fedora packager, I "simply" take the released source code tarball and let Fedora's build system do the job)

@petrasovaa
Copy link
Contributor

Are there any compilation logs somewhere? Or maybe try to compile it in that environment?

@neteler
Copy link
Member

neteler commented Apr 29, 2025

@petrasovaa
Copy link
Contributor

ERROR: Function "Rast_legal_semantic_label" depends on an unknown typedef "bool". Function "Rast_legal_semantic_label" will not be output

@marisn
Copy link
Contributor

marisn commented Apr 30, 2025

Could be related to ctypesgen/ctypesgen#133 ? @nilason ? Should we abandon bool in our C code due to issues with ctypesgen?

@neteler
Copy link
Member

neteler commented Apr 30, 2025

Yes, the log is here: kojipkgs.fedoraproject.org//packages/grass/8.4.1/1.fc42/data/logs/x86_64/build.log

(Un)related, there are additionally similar errors in the log file:

ERROR: Function "Vect_line_to_wkt" depends on an unknown typedef "bool". Function "Vect_line_to_wkt" will not be output
ERROR: Function "Vect_line_to_wkt2" depends on an unknown typedef "bool". Function "Vect_line_to_wkt2" will not be output
ERROR: Function "Vect_line_to_wkt2" depends on an unknown typedef "bool". Function "Vect_line_to_wkt2" will not be output
ERROR: Function "Vect_read_area_to_wkt2" depends on an unknown typedef "bool". Function "Vect_read_area_to_wkt2" will not be output

While at it, from the log (searching for "will not be output"):

WARNING: Could not parse macro "#define serialize_int32_le(buf,x) do { ( buf ) [ 0 ] = ( ( x ) >> 0 ) & 0xFF ; ( buf ) [ 1 ] = ( ( x ) >> 8 ) & 0xFF ; ( buf ) [ 2 ] = ( ( x ) >> 16 ) & 0xFF ; ( buf ) [ 3 ] = ( ( x ) >> 24 ) & 0xFF ; } while ( 0 )"
WARNING: Could not parse macro "#define serialize_int32_be(buf,x) do { ( buf ) [ 0 ] = ( ( x ) >> 24 ) & 0xFF ; ( buf ) [ 1 ] = ( ( x ) >> 16 ) & 0xFF ; ( buf ) [ 2 ] = ( ( x ) >> 8 ) & 0xFF ; ( buf ) [ 3 ] = ( ( x ) >> 0 ) & 0xFF ; } while ( 0 )"
WARNING: Macro "TRUE" depends on an unknown identifier "true". Macro "TRUE" will not be output
WARNING: Macro "FALSE" depends on an unknown identifier "false". Macro "FALSE" will not be output

and

/usr/bin/install -c -m 644 OBJ.x86_64-redhat-linux-gnu/vedit.py /builddir/build/BUILD/grass-8.4.1-build/grass-8.4.1/dist.x86_64-redhat-linux-gnu/etc/python/grass/lib/vedit.py
python3 -m py_compile /builddir/build/BUILD/grass-8.4.1-build/grass-8.4.1/dist.x86_64-redhat-linux-gnu/etc/python/grass/lib/vedit.py
WARNING: Macro "GS_NEAR_EQUAL" depends on an unknown identifier "GS_BETWEEN". Macro "GS_NEAR_EQUAL" will not be output
WARNING: Macro "GS_NEAR_EQUAL" depends on an unknown identifier "GS_BETWEEN". Macro "GS_NEAR_EQUAL" will not be output

@nilason
Copy link
Contributor

nilason commented Apr 30, 2025

Might it be Python 3.13 related?

Not likely, just tested and it works fine.

Could be related to ctypesgen/ctypesgen#133 ? @nilason ? Should we abandon bool in our C code due to issues with ctypesgen?

No. Ctypesgen works fine with bool.

I think it might be related to stricter compiler settings somehow. bool is defined in <stdbool.h> which must be included. We have been bit lazy on that part (relying on its inclusion in gis.h).

Could you try the following patch:

diff --git a/include/grass/defs/raster.h b/include/grass/defs/raster.h
index f81445a374..9630862bb4 100644
--- a/include/grass/defs/raster.h
+++ b/include/grass/defs/raster.h
@@ -1,6 +1,8 @@
 #ifndef GRASS_RASTERDEFS_H
 #define GRASS_RASTERDEFS_H
 
+#include <stdbool.h>
+
 #include <grass/gis.h>
 
 /* --- ANSI prototypes for the lib/raster functions --- */
diff --git a/include/grass/defs/vector.h b/include/grass/defs/vector.h
index 42ed74c532..6a67209767 100644
--- a/include/grass/defs/vector.h
+++ b/include/grass/defs/vector.h
@@ -1,6 +1,8 @@
 #ifndef GRASS_VECTORDEFS_H
 #define GRASS_VECTORDEFS_H
 
+#include <stdbool.h>
+
 /*
  * "Public" functions, for use in modules
  */
diff --git a/include/grass/gis.h b/include/grass/gis.h
index ad0687ffa6..ee944de4e3 100644
--- a/include/grass/gis.h
+++ b/include/grass/gis.h
@@ -76,11 +76,11 @@ static const char *GRASS_copyright UNUSED = "GRASS GNU GPL licensed Software";
  */
 /* and 'false' For historical reasons 'TRUE' and 'FALSE' are still valid. */
 #ifndef TRUE
-#define TRUE true
+#define TRUE 1
 #endif
 
 #ifndef FALSE
-#define FALSE false
+#define FALSE 0
 #endif
 
 /*! \brief Cross-platform Newline Character */

@nilason
Copy link
Contributor

nilason commented Apr 30, 2025

WARNING: Macro "TRUE" depends on an unknown identifier "true". Macro "TRUE" will not be output

This one is curious, stdbool.h is included in gis.h. ...

@nilason
Copy link
Contributor

nilason commented Apr 30, 2025

WARNING: Macro "GS_NEAR_EQUAL" depends on an unknown identifier "GS_BETWEEN". Macro "GS_NEAR_EQUAL" will not be output

This is unrelated:

#define BETWEEN(x, a, b) (((x) > (a) && (x) < (b)) || ((x) > (b) && (x) < (a)))
#define GS_NEAR_EQUAL(x, y, ratio) \
((x) == (y) || \
((x) == 0.0 ? GS_BETWEEN((x), (y) + (y) * (ratio), (y) - (y) * (ratio)) \
: GS_BETWEEN((y), (x) + (x) * (ratio), (x) - (x) * (ratio))))

GS_BETWEEN should probably be BETWEEN, but as GS_NEAR_EQUAL is never used in code this doesn't cause build failure. Ctypesgen probably always complained about this.

@nilason
Copy link
Contributor

nilason commented Apr 30, 2025

WARNING: Macro "TRUE" depends on an unknown identifier "true". Macro "TRUE" will not be output

This one is curious, stdbool.h is included in gis.h. ...

Maybe related to C23, something with re-define keywords. I vaguely remember seeing compiler complains on this.

@neteler
Copy link
Member

neteler commented Apr 30, 2025

Could you try the following patch:

Nice, the compilation error is gone.

@marisn
Copy link
Contributor

marisn commented Apr 30, 2025

I think it might be related to stricter compiler settings somehow. bool is defined in <stdbool.h> which must be included. We have been bit lazy on that part (relying on its inclusion in gis.h).

Which was a reasonable assumption as per https://en.cppreference.com/w/c/preprocessor/include:

When a file is included, it is processed by translation phases 1-4, which may include, recursively, expansion of the nested #include directives, up to an implementation-defined nesting limit.

@marisn
Copy link
Contributor

marisn commented May 6, 2025

See also #5584 for issues with bool.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants