Output Struct Overhaul

.flake8

@@ -16,6 +16,8 @@ ignore =
     N815
     # Docstring in imperative mood. This should *not* be the case for @property's, but can't ignore them atm.
     D401
+    # Module shadowing a builtin
+    A005
 max-line-length = 88
 max-complexity = 70
 docstring-convention=numpy

.github/workflows/test_build_macos.yaml

@@ -20,7 +20,7 @@ jobs:
     env:
       PYTHON: ${{ matrix.python-version }}
       CC: gcc
-    name: Testing
+    name: Test MacOS Build
     runs-on: ${{ matrix.os }}
     strategy:
       fail-fast: false
@@ -46,7 +46,6 @@ jobs:
         uses: conda-incubator/setup-miniconda@v3
         with:
           # auto-update-conda: true
-          mamba-version: "*"
           channels: conda-forge,defaults
           python-version: ${{ matrix.python-version }}
           environment-file: ci/macos-latest-env.yml

.github/workflows/test_suite.yaml

@@ -46,7 +46,6 @@ jobs:
         uses: conda-incubator/setup-miniconda@v3
         with:
           # auto-update-conda: true
-          mamba-version: "*"
           channels: conda-forge,defaults
           python-version: ${{ matrix.python-version }}
           environment-file: ci/${{ matrix.os }}-env.yml

.github/workflows/test_suite_nointegration.yaml

@@ -20,7 +20,7 @@ jobs:
       PYTHON: ${{ matrix.python-version }}
       OS: ${{ matrix.os }}
       CC: gcc
-    name: Testing
+    name: Sans Integration Tests
     runs-on: ${{ matrix.os }}
     strategy:
       fail-fast: false
@@ -45,7 +45,6 @@ jobs:
         uses: conda-incubator/setup-miniconda@v3
         with:
           # auto-update-conda: true
-          mamba-version: "*"
           channels: conda-forge,defaults
           python-version: ${{ matrix.python-version }}
           environment-file: ci/${{ matrix.os }}-env.yml

devel/TestFindHalos.py

@@ -12,8 +12,6 @@
 from py21cmfast.c_21cmfast import ffi, lib
 from py21cmfast.wrapper._utils import StructInstanceWrapper
 
-global_params = StructInstanceWrapper(lib.global_params, ffi)
-
 user_params = UserParams(
     DIM=150,
     HII_DIM=50,

docs/conf.py

@@ -35,7 +35,7 @@ def __getattr__(cls, name):
 out = subprocess.run(["python", "setup.py", "--version"], capture_output=True)
 
 try:
-    from py21cmfast import cache_tools
+    from py21cmfast.io import cache_tools
 except ImportError:
     raise 
 

docs/faqs/misc.rst

@@ -41,35 +41,14 @@ To make the current configuration permanent, simply use the ``write`` method::
     >>> p21.config['direc'] = 'my_own_cache'
     >>> p21.config.write()
 
-Global Parameters
------------------
-There are a bunch of "global" parameters that are used throughout the C code. These are
-parameters that are deemed to be constant enough to not expose them through the
-regularly-used input structs, but nevertheless may necessitate modification from
-time-to-time. These are accessed through the ``global_params`` object::
-
-    >>> from py21cmfast import global_params
-
-Help on the attributes can be obtained via ``help(global_params)`` or
-`in the docs <../reference/_autosummary/py21cmfast.inputs.html>`_. Setting the
-attributes (which affects them everywhere throughout the code) is as simple as, eg::
-
-    >>> global_params.Z_HEAT_MAX = 30.0
-
-If you wish to use a certain parameter for a fixed portion of your code (eg. for a single
-run), it is encouraged to use the context manager, eg.::
-
-    >>> with global_params.use(Z_HEAT_MAX=10):
-    >>>    run_lightcone(...)
-
 How can I read a Coeval object from disk?
 -----------------------------------------
 
 The simplest way to read a :class:`py21cmfast.outputs.Coeval` object that has been
 written to disk is by doing::
 
     import py21cmfast as p21c
-    coeval = p21c.Coeval.read("my_coeval.h5")
+    coeval = p21c.Coeval.from_file("my_coeval.h5")
 
 However, you may want to read parts of the data, or read the data using a different
 language or environment. You can do this as long as you have the HDF5 library (i.e.
@@ -84,9 +63,6 @@ structure of the file yourself interactively. But here is an example using h5py:
     # the CosmoParams, FlagOptions and AstroParams are accessed the same way.
     print(dict(fl['user_params'].attrs))
 
-    # print a dict of all globals used for the coeval
-    print(dict(fl['_globals'].attrs))
-
     # Get the redshift and random seed of the coeval box
     redshift = fl.attrs['redshift']
     seed = fl.attrs['random_seed']
@@ -122,3 +98,19 @@ while the globally averaged quantities are in the ``global_quantities`` group::
     redshifts = fl['node_redshifts']
 
     plt.plot(redshifts, global_Tb)
+
+Can I instantiate my own OutputStruct objects?
+-------------------------------------------
+Usually, you create instances of an :class:`py21cmfast.wrapper.outputs.OutputStruct`
+object by running either :func:`py21cmfast.run_coeval` or some lower-level function,
+like :func:`py21cmfast.compute_initial_conditions`. However, it's possible you want to
+switch out a simulation step from ``21cmFAST`` and insert your own, but then go on using
+that box in further ``21cmFAST`` simulation components. The way to do this is as follows,
+using the ``InitialConditions`` as an example::
+
+    ics = p21c.InitialConditions.new(inputs=p21c.InputParameters())
+    ics.set('lowres_density', my_computed_value)
+
+You would use this ``.set()`` method on each of the fields you needed to set. Now this
+data should be properly shared with the backend C-code, and the object can be used
+in subsequent steps within ``21cmFAST``.

docs/notes_for_developers.rst

@@ -88,12 +88,6 @@ can be handled by initalising the array as a 1D numpy array, but then setting it
 attribute (after creation) to the appropriate n-dimensional shape (see the
 ``_init_arrays`` method for the ``InitialConditions`` class for examples of this).
 
-Modifying the ``global_params`` struct should be relatively straightforward, and no
-changes in the Python are necessary. However, you may want to consider adding the new
-parameter to relevant ``_filter_params`` lists for the output struct wrapping classes in
-the wrapper. These lists control which global parameters affect which output structs,
-and merely provide for more accurate caching mechanisms.
-
 C Function Standards
 ~~~~~~~~~~~~~~~~~~~~
 The C-level functions are split into two groups -- low-level "private" functions, and
@@ -222,10 +216,10 @@ necessary to write them to disk to relieve memory pressure, and load them back i
 That means that any time, a given array in a C-based class may have one of several different "states":
 
 1. Completely Uninitialized
-1. Allocated an initialized in memory
-1. Computed (i.e. filled with the values defining that array after computation in C)
-1. Stored on disk
-1. Stored *and* in memory.
+2. Allocated an initialized in memory
+3. Computed (i.e. filled with the values defining that array after computation in C)
+4. Stored on disk
+5. Stored *and* in memory.
 
 It's important to keep track of these states, because when passing the struct to the ``compute()``
 function of another struct (as input), we go and check if the array exists in memory, and