Reporting

This page exists to collect use cases and architecture thoughts for the reporting infrastructure being developed in iiasa/message_ix#149 and iiasa/message_ix#142. There is also a (non-public) Google Doc with further requirements and minutes of a meeting (on IIASA/ENE SharePoint).

Use cases

• Behnam develops a sub-annual version of the Austria tutorial, and wants to plot the results coming out of the resulting model.
• Paul develops a MESSAGEix-Transport version of the main global model, and wants to report aggregate results across different vehicle types and transport technology groups.
• Oliver uses the MESSAGE-GLOBIOM global model; and wants to define simple rules such as "every technology that takes coal commodity input from the primary level and outputs the electricity commodity at the secondary level" and have calculations performed for such groups.

Architecture

PNK: At least three different ways of configuring reporting calculations have been suggested:

• Programmatic, as shown by DH at the 2018-12-19 development meeting & below.
• File-based, using YAML or other syntax, as below (as suggested by MG 2018-12-19).
• Introspective, by examining the sets and relations in the model/scenario definition (see e.g. Oliver's use case, above).

In the long run, these should be mix-and-match, e.g.

• Load a default YAML file, then programmatically add a few more calculations.
• Introspect many calculations, but then load some additional/replacement calculations defined in a file.

API usage proposals

pp = Reporting()
  .add_activity(...)
  .add_activity()
  .aggregate(...)
  .build()
// reuse processing instance for multiple scenarios
pp.process(scenario1)
pp.process(scenario2)

// or
pp = Reporting().load(...)

Comment by @danielhuppmann: For any realistic use case even in a simple tutorial like Westeros, this will be a very long command. Also, the "write down all configs at once and then run it" is less intuitive from a new-user perspective compared to a "one step after the other" (first let's do primary energy, then do capacity, etc.)

Also, re-using the reporting for multiple scenarios within a notebook is an unlikely use case. In such a case, a user should place the specifications in config file so that it can be used across notebooks.

Suggestion for using yaml as config as an extension of the current proposal

specs.yaml

- activity
    - Primary Energy|Wind
        - technology: wind_ppl
- aggregate
    - Primary Energy

Call reporting specs on scenario: Reporting(scenario, 'specs.yaml')

And/or add a function to the message_ix.Scenario class like scenario.reporting('specs.yaml')