Skip to content

Commit 8d35116

Browse files
irm-codebasepre-commit-ci[bot]
irm-codebase
and
pre-commit-ci[bot]
authored
Add euro calliope methods (#1)
* Add scripts to combine HydroBASINs * Add powerplant location adjustment * Add inflow calculation. * Create timeseries per powerstation * Add capacity factor per shape generation. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
1 parent 333ad9b commit 8d35116

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

62 files changed

+8141
-1568
lines changed

‎.copier-answers.yml

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,13 @@
11
# Changes here will be overwritten by Copier
22
# DO NOT MANUALLY MODIFY THIS FILE
3-
_commit: v0.1.0
3+
_commit: latest
44
_src_path: https://github.com/calliope-project/data-module-template.git
55
author_email: i.ruizmanuel@tudelft.nl
66
author_family_name: Ruiz Manuel
77
author_given_name: Ivan
88
github_org: calliope-project
99
license: Apache-2.0
10-
module_description: A module to calculate hydropower generation timeseries for facilities
11-
around the globe.
10+
module_description: A module to calculate hydropower inflow timeseries for facilities
11+
around the globe. Based on Euro-Calliope methods.
1212
module_long_name: clio - Hydropower module
1313
module_short_name: module_hydropower
14-
readthedocs: false

‎.github/ISSUE_TEMPLATE/bug_report.md

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
---
2+
name: Bug report
3+
about: Create a report to help us improve
4+
title: ''
5+
labels: bug
6+
assignees: ''
7+
8+
---
9+
10+
**Describe the bug**
11+
A clear and concise description of what the bug is.
12+
13+
**To Reproduce**
14+
Steps to reproduce the behavior:
15+
1. Go to '...'
16+
2. Click on '....'
17+
3. Scroll down to '....'
18+
4. See error
19+
20+
**Expected behavior**
21+
A clear and concise description of what you expected to happen.
22+
23+
**Screenshots**
24+
If applicable, add screenshots to help explain your problem.
25+
26+
**Desktop (please complete the following information):**
27+
- OS: [e.g. Linux Fedora 41, Windows 10...]
28+
- Version [e.g. v0.1.1]
29+
30+
**Additional context**
31+
Add any other context about the problem here.

‎.github/ISSUE_TEMPLATE/feature_request.md

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
---
2+
name: Feature request
3+
about: Suggest an idea for this project
4+
title: ''
5+
labels: enhancement
6+
assignees: ''
7+
8+
---
9+
10+
**Is your feature request related to a problem? Please describe.**
11+
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
12+
13+
**Describe the solution you'd like**
14+
A clear and concise description of what you want to happen.
15+
16+
**Describe alternatives you've considered**
17+
A clear and concise description of any alternative solutions or features you've considered.
18+
19+
**Additional context**
20+
Add any other context or screenshots about the feature request here.

‎.github/pull_request_template.md

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
Fixes #
2+
3+
## Summary of changes in this pull request
4+
5+
*
6+
*
7+
*
8+
9+
## Reviewer checklist
10+
11+
* [ ] `INTERFACE.yaml` is up-to-date with all relevant user resources and results.
12+
* [ ] The integration example is up-to-date with a minimal use-case of the module.
13+
* [ ] Module documentation is up-to-date.

‎.github/workflows/check-latest-template-spec.yml

Lines changed: 0 additions & 12 deletions
This file was deleted.

‎.github/workflows/check-version.yml

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
# Check for changes in the upstream template. If changes are found, an issue is created
2+
name: Template check.
3+
on:
4+
schedule:
5+
- cron: '0 0 * * 1' # Runs at 00:00 UTC every Monday
6+
7+
jobs:
8+
copier-update:
9+
uses: calliope-project/data-module-template/.github/workflows/template-check-version.yml@latest

‎.github/workflows/pr-ci.yml

Lines changed: 4 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,9 @@
11
name: CI
2-
32
on:
43
pull_request:
5-
branches: [ main ]
4+
branches:
5+
- "main"
66

77
jobs:
8-
build:
9-
name: Build
10-
runs-on: ${{ matrix.os }}
11-
strategy:
12-
fail-fast: false
13-
matrix:
14-
os: [ubuntu-latest, windows-latest, macos-latest]
15-
python-version: ["3.12"]
16-
steps:
17-
- uses: actions/checkout@v4
18-
- name: setup-pixi
19-
uses: prefix-dev/setup-pixi@v0.8.3
20-
- name: Run tests
21-
run: pixi run test
8+
ci-tests:
9+
uses: calliope-project/data-module-template/.github/workflows/template-pr-ci.yml@latest

‎.github/workflows/release.yml

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
name: Release
2+
3+
on:
4+
release:
5+
types: [released]
6+
7+
jobs:
8+
release-workflow:
9+
uses: calliope-project/data-module-template/.github/workflows/template-release.yml@latest

‎.gitignore

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -18,9 +18,11 @@ gurobi.log
1818
### Notebooks
1919
.ipynb_checkpoints
2020

21-
# Show mutable folders, but do not commit resulting files
22-
resources/
23-
!resources/user/.gitkeep
24-
!resources/automatic/.gitkeep
25-
results/
26-
!results/.gitkeep
21+
# Ignore mutable folders
22+
**/logs/*
23+
**/resources/*
24+
**/results/*
25+
**/temp/*
26+
**/tmp/*
27+
!tests/integration/resources/automatic/
28+
!tests/integration/resources/user

‎.pre-commit-config.yaml

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ repos:
1111
- id: end-of-file-fixer
1212
# Repo quality
1313
- id: check-added-large-files
14-
args: [--enforce-all]
14+
args: [--enforce-all --maxkb=600]
1515
- id: forbid-submodules
1616
- id: check-case-conflict
1717
- id: check-illegal-windows-names
@@ -40,7 +40,8 @@ repos:
4040
- repo: https://github.com/codespell-project/codespell
4141
rev: v2.4.1
4242
hooks:
43-
- id: codespell
43+
- id: codespell
44+
files: .*\.(py|smk|md)$|^Snakefile$
4445

4546
ci: # https://pre-commit.ci/
4647
autoupdate_schedule: monthly

‎module_hydropower/.readthedocs.yaml renamed to ‎.readthedocs.yaml

File renamed without changes.

‎INTERFACE.yaml

Lines changed: 8 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,20 +1,12 @@
11
# Module Input-Output structure for automated doc. generation
22
resources:
33
user:
4-
user_message.md: "Example file emulating external files requested from a user."
5-
automatic:
6-
dummy_readme.md: "Example file emulating downloads determined by internal settings."
4+
national_generation.parquet: Annual hydropower generation statistics per country.
5+
powerplants.parquet: Powerplant statistics.
6+
shapes.parquet: Resolution shapes, used for capacity aggregation.
77
results:
8-
combined_text.md: "Example file emulating module results."
9-
10-
# Wildcard example:
11-
# resources:
12-
# user:
13-
# shapes_{resolution}.geojson: region geometries.
14-
# automatic:
15-
# technology_data.parquet: dataset with technology characteristics.
16-
# results:
17-
# '{resolution}/tech_capacity.parquet': description of output data.
18-
# '{resolution}/results_image.png': description of output image.
19-
# wildcards:
20-
# resolution: description of the wildcard’s purpose.
8+
adjusted_powerplants.parquet: >
9+
Powerplants statistics after geo-location correction, plus assigned shape_id and country_id.
10+
Can be used to identify which powerplants were assigned to a given polygon.
11+
by_shape_id/hydro_dam_cf.parquet: Capacity Factors for hydro dams, per shape.
12+
by_shape_id/hydro_run_of_rover_cf.parquet: Capacity Factors for run of river plants, per shape.

‎README.md

Lines changed: 26 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,30 @@
11
# clio - Hydropower module
22

3-
A module to calculate hydropower generation timeseries for facilities around the globe.
3+
A module to calculate hydropower inflow timeseries for facilities around the globe. Based on Euro-Calliope methods.
4+
5+
![Hydropower CFs assorted europe](./docs/hydro_dam_cf.png)
46

57
A modular `snakemake` workflow built for [`clio`](https://clio.readthedocs.io/) data modules.
68

9+
## Steps
10+
11+
1. [HydroBasins](https://www.hydrosheds.org/) data is downloaded and combined into a global dataset.
12+
2. Powerplant positions are geo-corrected to the nearest basin, and then assigned to the nearest shape.
13+
3. Inflow in cubic meters is obtained for each powerplant location using ERA5 data through the [`atlite`](https://github.com/PyPSA/atlite) library.
14+
4. Inflow in MWh is obtained for each plant using the given historical national annual hydropower generation and its share of total installed national capacity.
15+
16+
> [!IMPORTANT]
17+
> The module assumes that provided sample of powerplants sums up to the total national capacity in the given year.
18+
> Providing incomplete data might lead to distortions in the calculated time series!
19+
20+
## Using this module
21+
22+
This module can be imported directly into any `snakemake` workflow.
23+
Please consult the integration example in `tests/integration/Snakefile` for more information.
24+
725
## Development
826

9-
We use [`pixi`](https://pixi.sh/) for as our package manager for development.
27+
We use [`pixi`](https://pixi.sh/) as our package manager for development.
1028
Once installed, run the following to clone this repo and install all dependencies.
1129

1230
```shell
@@ -21,6 +39,12 @@ For testing, simply run:
2139
pixi run test
2240
```
2341

42+
To view the documentation locally, use:
43+
44+
```shell
45+
pixi run serve-docs
46+
```
47+
2448
To test a minimal example of a workflow using this module:
2549

2650
```shell

‎config/README.md

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,9 @@
11
# Configuration
22

3-
Configuration settings are explained in the schema file.
4-
For general instructions and settings please consult our documentation.
3+
This workflow is part of the [clio project](https://clio.readthedocs.io/en/latest/).
4+
Please consult our documentation for more details.
5+
Other useful resources are:
6+
7+
- `INTERFACE.yaml`: user input files (placed in `resources/user`) and module output files (placed in `results`).
8+
- `workflow/internal/config.schema.yaml`: general configuration options.
9+
- `tests/integration/`: a simple example of how to use this module.

‎config/config.yaml

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# A minimal example of how to configure this module
2+
pfafstetter_level: "06"
3+
years:
4+
start: 2000
5+
end: 2000
6+
powerplants:
7+
basin_adjustment:
8+
buffer_radius: 1000 # epsg:3857 is in meters
9+
max_outside: 50
10+
max_dropped: 30
11+
crs:
12+
projected: "epsg:3857"
13+
geographic: "epsg:4326"
14+
smoothing_hours: "168h"

‎config/example.yaml

Lines changed: 0 additions & 2 deletions
This file was deleted.

‎docs/hydro_dam_cf.png

89.2 KB
Loading

‎module_hydropower/docs/index.md renamed to ‎docs/index.md

File renamed without changes.

‎module_hydropower/mkdocs.yaml renamed to ‎mkdocs.yaml

File renamed without changes.

‎module_hydropower/README.md

Lines changed: 0 additions & 41 deletions
This file was deleted.

‎module_hydropower/pixi.toml

Lines changed: 0 additions & 37 deletions
This file was deleted.

0 commit comments

Comments
 (0)