Update deps incl cbc (#744)

* Update dependency pinning to have as few upper bounds as feasible 
* Move to recommending CBC as the main open-source solver
* Use conda forge GLPK link to mitigate markdown link check errors

Author: brynpickering

.github/workflows/pr-ci.yml

@@ -52,24 +52,14 @@ jobs:
           -f requirements/dev.txt
           curl
           python=3.${{ matrix.py3version }}
+          coin-or-cbc
           gurobi::gurobi
         post-cleanup: all
         cache-environment: true
 
     - name: Install package
       run: pip install --no-dependencies -e .
 
-    - name: install CBC (Windows)
-      if: matrix.os == 'windows-latest'
-      run: |
-        curl -L https://github.com/coin-or/Cbc/releases/download/releases%2F2.10.10/Cbc-releases.2.10.10-w64-msvc17-md.zip -o cbc.zip
-        unzip cbc.zip -d ${HOME}/cbc
-        echo "${HOME}/cbc/bin" >> $GITHUB_PATH
-
-    - name: install CBC (Unix)
-      if: matrix.os != 'windows-latest'
-      run: micromamba install coin-or-cbc
-
     - name: Install jupyter kernel
       run: python -m ipykernel install --user --name calliope
 

.readthedocs.yml

@@ -7,7 +7,7 @@ build:
     python: mambaforge-4.10
   jobs:
     post_create_environment:
-      - mamba install python=3.12 coin-or-cbc gurobi::gurobi --file requirements/dev.txt
+      - mamba install python=3.12 coin-or-cbc gurobi::gurobi --file requirements/base.txt --file requirements/dev.txt
       - pip install --no-deps .
       - python -m ipykernel install --user --name calliope_docs_build
 

CHANGELOG.md

@@ -1,3 +1,11 @@
+## 0.7.0.dev6
+
+### User-facing changes
+
+|changed| coin-or-cbc is now available cross-platform on conda-forge and so is the recommended open-source solver to install a user environment with.
+
+|changed| Upper bound pins for dependencies removed where possible, to minimise clashes when using calliope as a dependency in a project.
+
 ## 0.7.0.dev5 (2024-12-04)
 
 ### User-facing changes

README.md

@@ -37,7 +37,7 @@ Calliope comes with several built-in analysis and visualisation tools. Having so
 
 ## Quick start
 
-Calliope can run on Windows, macOS and Linux. Installing it is quickest with the `mamba` package manager by running a single command: `mamba create -c conda-forge -n calliope calliope`.
+Calliope can run on Windows, macOS and Linux. Installing it is quickest with the `mamba` package manager by running a single command: `mamba create -n calliope -c conda-forge conda-forge/label/calliope_dev::calliope`.
 
 See the documentation for more [information on installing](https://calliope.readthedocs.io/en/stable/user/installation.html).
 

docs/advanced/shadow_prices.md

@@ -17,10 +17,10 @@ If you [define any of your own math constraints](../user_defined_math/components
 !!! note
 
     * Not all solvers provide access to shadow prices.
-    For instance, we know that it is possible with Gurobi and GLPK, but not with CBC.
-    Since we cannot test all Pyomo-compatible solvers, you may run into issues depending on the solver you use.
+      For instance, we know that it is possible with Gurobi and GLPK, but not with CBC.
+      Since we cannot test all Pyomo-compatible solvers, you may run into issues depending on the solver you use.
     * You cannot access shadow prices if you have integer/binary variables in your model.
-    If you try to do so, you will receive a warning, and shadow price tracking will remain disabled.
+      If you try to do so, you will receive a warning, and shadow price tracking will remain disabled.
     * You can check the status of shadow price tracking with `model.backend.shadow_prices.is_active`.
 
 ## Shadow prices when using the command-line tool

docs/contributing.md

@@ -36,7 +36,7 @@ To create a development environment for calliope, with all libraries required fo
 1. Open the command line (or the "miniforge prompt" in Windows).
 1. Download (a.k.a., clone) the calliope repository: `git clone git@github.com:calliope-project/calliope.git`
 1. Change into the `calliope` directory: `cd calliope`
-1. Create the calliope mamba environment: `mamba create -n calliope -c conda-forge --file requirements/base.txt --file requirements/dev.txt gurobi::gurobi`
+1. Create the calliope mamba environment: `mamba create -n calliope -c conda-forge --file requirements/base.txt --file requirements/dev.txt coin-or-cbc gurobi::gurobi`
 1. Activate the calliope mamba environment: `mamba activate calliope`
 1. Install the calliope package into the environment, in editable mode and ignoring dependencies (we have dealt with those when creating the mamba environment): `pip install --no-deps -e .`
 1. Install the interactive python shell (necessary when testing our documentation locally): `ipython kernel install --user --name=calliope_docs_build`
@@ -46,7 +46,7 @@ All together:
 ``` shell
 git clone git@github.com:calliope-project/calliope.git
 cd calliope
-mamba create -n calliope -c conda-forge --file requirements/base.txt --file requirements/dev.txt gurobi::gurobi
+mamba create -n calliope -c conda-forge --file requirements/base.txt --file requirements/dev.txt coin-or-cbc gurobi::gurobi
 mamba activate calliope
 pip install --no-deps -e .
 ipython kernel install --user --name=calliope_docs_build

docs/index.md

@@ -6,7 +6,7 @@
     To install the pre-release as a user:
 
     ```shell
-    mamba create -n calliope -c conda-forge/label/calliope_dev -c conda-forge calliope
+    mamba create -n calliope -c conda-forge conda-forge/label/calliope_dev::calliope
     ```
 
     Some functionality is not yet available in 0.7, notably:
@@ -68,4 +68,4 @@ limitations under the License.
 ## Citing Calliope
 
 Calliope is [published in the Journal of Open Source Software](https://joss.theoj.org/papers/10.21105/joss.00825).
-We encourage you to use this academic reference.
+We encourage you to use this academic reference.

docs/installation.md

@@ -8,7 +8,8 @@ Running Calliope requires four things:
 
 1. The Python programming language, version {{ min_python_version }} to {{ max_python_version }}.
 2. A number of Python add-on modules including [Pyomo](https://www.pyomo.org/), [Pandas](https://pandas.pydata.org/) and [Xarray](https://docs.xarray.dev/).
-3. An optimisation solver: Calliope has been tested with CBC, GLPK, Gurobi, and CPLEX. Any other solver that is compatible with Pyomo should also work.
+3. An optimisation solver: Calliope has been tested with CBC, GLPK, and Gurobi.
+   Any other solver that is compatible with Pyomo should also work.
 4. The Calliope software itself.
 
 ## Recommended installation method
@@ -22,11 +23,11 @@ To get `mamba`, the most straightforward approach is to [download and install th
     Another option is to use the commercially developed [Anaconda Python distribution](https://www.anaconda.com/download), which is available for all operating systems and comes with a graphical user interface to install and manage packages.
     If you use the Anaconda distribution, you have to replace `mamba` with `conda` in the installation instructions below.
 
-With the package manager installed, you can create a new environment called `calliope` with Calliope as well as the free and open source GLPK solver.
+With the package manager installed, you can create a new environment called `calliope` with Calliope as well as the free and open source CBC solver.
 Run the following command in a terminal or command-line window:
 
 ```shell
-mamba create -n calliope -c conda-forge/label/calliope_dev -c conda-forge calliope glpk
+mamba create -n calliope -c conda-forge conda-forge/label/calliope_dev::calliope
 ```
 
 !!! note
@@ -46,36 +47,24 @@ mamba activate calliope
     Although possible, we do not recommend installing Calliope directly via `pip` (`pip install calliope`).
     Non-python binaries are not installed with `pip`, some of which are necessary for stable operation (e.g., `libnetcdf`).
 
-## Updating an existing installation
-
-If following the recommended installation method above, the following command, assuming the mamba environment is active, will update Calliope to the newest version
-
-```bash
-mamba update -c conda-forge calliope
-```
-
 ## Choosing a solver
 
 You cannot solve a Calliope model until you have installed a solver.
-The easiest solver to install is [GLPK](#glpk).
-However, we recommend to not use this solver where possible, since it performs relatively poorly (both in solution time and stability of result).
-Indeed, our example models use the free and open source [CBC](#cbc) solver instead, but installing it on Windows requires an extra step.
+The easiest solver to install is [CBC](#cbc), which is included if you follow the [recommended installation instructions](#recommended-installation-method) above.
 [CBC](#cbc) (open-source) or [Gurobi](#gurobi) (commercial) are recommended for large problems, and have been confirmed to work with Calliope.
 The following subsections provide additional detail on how to install a solver.
 This list is not exhaustive; any solvers [supported by Pyomo](https://pyomo.readthedocs.io/en/latest/reference/topical/appsi/appsi.solvers.html) can be used.
 
 ### CBC
 
 [CBC](https://github.com/coin-or/Cbc) is our recommended option if you want a free and open-source solver.
-CBC can be installed via conda on Linux and macOS by running `mamba install -c conda-forge coin-or-cbc`.
-Windows binary packages are somewhat more difficult to install, due to limited information on [the CBC website](https://github.com/coin-or/Cbc), but are included in their [package releases on GitHub](https://github.com/coin-or/Cbc/releases).
-The GitHub releases are more up-to-date. We recommend you download the relevant binary for [CBC 2.10.11](https://github.com/coin-or/Cbc/releases/download/releases%2F2.10.11/Cbc-releases.2.10.11-w64-msvc17-md.zip) and add `cbc.exe` to a directory known to PATH (e.g. an Anaconda environment 'bin' directory).
+If you do not have it in your working environment (i.e. there is nothing listed when you call `conda list cbc`) then it can be installed by running `mamba install conda-forge::coin-or-cbc`.
 
 ### GLPK
 
-[GLPK](https://www.gnu.org/software/glpk/) is free and open-source, but can take too much time and/or too much memory on larger problems.
-`GLPK` can be installed from `conda-forge` on all platforms: `mamba install glpk`.
-It is therefore the _easiest_ solver to have installed in your Calliope environment.
+[GLPK](https://anaconda.org/conda-forge/glpk) is free and open-source, but can take too much time and/or too much memory on larger problems.
+`GLPK` can be installed from `conda-forge` on all platforms: `mamba install conda-forge::glpk`.
+Unlike [CBC](#cbc), it is possible to extract [shadow prices](./advanced/shadow_prices.md) from a model solved with GLPK, which is why you may with to use it instead of CBC.
 
 ### Gurobi
 

docs/migrating.md

@@ -161,7 +161,7 @@ This split means you can change configuration options on-the-fly if you are work
       build:
         mode: plan
       solve:
-        solver: glpk
+        solver: cbc
     ```
 
 === "Python"
@@ -170,7 +170,7 @@ This split means you can change configuration options on-the-fly if you are work
     import calliope
     model = calliope.Model(time_subset=["2005-01", "2005-02"])
     model.build(mode="plan")
-    model.solve(solver="glpk")
+    model.solve(solver="cbc")
     ```
 
 !!! info "See also"

requirements/base.txt

@@ -1,18 +1,18 @@
-bottleneck >= 1, < 2
-click >= 8, < 9
-geographiclib >= 2, < 3
-ipdb >= 0.13, < 0.14
-ipykernel < 7
-jinja2 >= 3, < 4
-jsonref >= 1.1, < 2
-jsonschema >= 4, < 5
-natsort >= 8, < 9
-netcdf4 >= 1.2, < 1.7
-numpy >= 1, < 2
-pandas >= 2.1.3, < 2.3  # Minimum bound is 2.1.3 because of a regression in v2.1.0/2.1.1 inflating time/memory consumption on groupby operations with MultiIndex
-pyomo >= 6.8.2, < 7
-pyparsing >= 3.0, < 3.1
-ruamel.yaml >= 0.18, < 0.19
-typing-extensions >= 4, < 5
-xarray >= 2024.1, < 2024.4
+bottleneck >= 1
+click >= 8
+geographiclib >= 2
+ipdb >= 0.13
+ipykernel >= 6
+jinja2 >= 3
+jsonref >= 1.1
+jsonschema >= 4
+natsort >= 8
+netcdf4 >= 1.2
+numpy >= 1
+pandas >= 2.1.3  # Minimum bound is 2.1.3 because of a regression in v2.1.0/2.1.1 inflating time/memory consumption on groupby operations with MultiIndex
+pyomo >= 6.8.2
+pyparsing >= 3.2
+ruamel.yaml >= 0.18
+typing-extensions >= 4
+xarray >= 2024.1, < 2024.4 # pinned while https://github.com/pydata/xarray/issues/9755 is open
 pydantic >= 2.9.2

requirements/dev.txt

@@ -1,16 +1,16 @@
 glpk == 5.0
-jsonschema2md >= 1, < 2
-mkdocs >= 1.5, < 1.6
-mkdocs-click >= 0.6, < 0.7
-mkdocs-jupyter >= 0.24, < 0.24.7
-mkdocs-macros-plugin >= 1.0, < 2
-mkdocs-material >= 9.5, < 10
-mkdocstrings-python >= 1.7, < 2
-mypy >= 1.13.0, < 2
-pandas-stubs
-plotly >= 5, < 6
-pre-commit < 4
-pytest >= 8, < 9
-pytest-cov < 5
-pytest-order < 2
-pytest-xdist < 4 # pytest distributed testing plugin
+jsonschema2md >= 1
+mkdocs >= 1.5
+mkdocs-click >= 0.6
+mkdocs-jupyter >= 0.24
+mkdocs-macros-plugin >= 1.0
+mkdocs-material >= 9.5
+mkdocstrings-python >= 1.7
+mypy >= 1.13.0
+pandas-stubs  # version will be set by pandas in base.txt
+plotly >= 5
+pre-commit >= 3
+pytest >= 8
+pytest-cov >= 4
+pytest-order >= 1
+pytest-xdist >= 3 # pytest distributed testing plugin

src/calliope/backend/expression_parser.py

@@ -595,7 +595,7 @@ def as_math_string(self) -> str:  # noqa: D102, override
         )
         id_formatted = pp.Combine("\\" + pp.Word(pp.alphas) + "{" + id_ + "}")
         obj_parser = id_formatted + pp.Opt(
-            r"_\text{" + pp.Group(pp.delimited_list(id_)) + "}"
+            r"_\text{" + pp.Group(pp.DelimitedList(id_)) + "}"
         )
         obj_parser.set_parse_action(self._replace_rule(singular_slice_refs))
         return obj_parser.parse_string(evaluated, parse_all=True)[0]
@@ -870,12 +870,12 @@ def helper_function_parser(
     arg_values = pp.MatchFirst(allowed_parser_elements_in_args) + pp.NotAny("=")
 
     # define function arguments
-    arglist = pp.delimited_list(arg_values.copy())
+    arglist = pp.DelimitedList(arg_values.copy())
     args_ = pp.Group(arglist).set_results_name("args")
 
     # define function keyword arguments
     key = generic_identifier + pp.Suppress("=")
-    kwarg_list = pp.delimited_list(pp.dict_of(key, arg_values))
+    kwarg_list = pp.DelimitedList(pp.dict_of(key, arg_values))
     kwargs_ = pp.Group(kwarg_list).set_results_name("kwargs")
 
     # build generic function
@@ -943,7 +943,7 @@ def sliced_param_or_var_parser(
 
     slice = pp.Group(generic_identifier("set_name") + pp.Suppress("=") + slicer)
 
-    slices = pp.Group(pp.delimited_list(slice))("slices")
+    slices = pp.Group(pp.DelimitedList(slice))("slices")
     sliced_object_name = unsliced_object("param_or_var_name")
 
     sliced_param_or_var = pp.Combine(sliced_object_name + lspar) + slices + rspar
@@ -1036,7 +1036,7 @@ def list_parser(
         pp.ParserElement: Parser for valid lists of strings and/or numbers.
     """
     list_elements = pp.MatchFirst([evaluatable_identifier, number])
-    id_list = pp.Suppress("[") + pp.delimited_list(list_elements) + pp.Suppress("]")
+    id_list = pp.Suppress("[") + pp.DelimitedList(list_elements) + pp.Suppress("]")
     id_list.set_parse_action(ListParser)
     return id_list