Add .util.sankey and tutorial

[mabudz]

Create sankey diagram and add tutorial.

The approach here creates the sankey diagram by using the sankey plot function of the pyam package. This function requires a mapping dictionary. The mapping dict is created by using the newly created automatic report message::sankey and a utility function sankey_mapper()map_for_sankey.

This pull request is related to the draft pull request in message_data , which can be removed, since the sankey diagram has been implemented in the pyam package.

Note

The new key is now only added when calling Reporter.add_sankey().

How to review

General approach should be reviewed. Also, I am not sure if the util function sankey_mapper() map_for_sankey() is at the right place.

PR checklist

• Continuous integration checks all ✅
• Add or expand tests; coverage checks both ✅
• Add, expand, or update documentation.
• Update release notes.

[CLAassistant]

All committers have signed the CLA.

[glatterf42]

Thanks for this PR :)
Getting the tests to pass is a bit tricky since the API for pyam changed when they went from version 1.9.0 to 2.0.0 (as you might expect) and our code needs to satisfy both versions since we test on Python <= 3.9, which is not supported for pyam 2.0.0. So let me take a closer look, maybe we'd want to use the function that pyam's sankey function wraps around directly, after all.

[glatterf42]

Thanks @daymontas1 for stepping up here, much appreciated :)

You should be able to test the PR as is and see how you like it's usefulness, but there are a few things that I would like to see happen before we merge it:

• Please rebase on main to keep it up to date with current developments.
• Please migrate the main functionality from util/__init__.py to somewhere more appropriate. E.g. report/sankey.py if it's a reporting tool or util/sankey.py if there are more general use cases.
• Please check if we can use plotly directly instead of pyam's wrapper functionality. This should allow us to utilize more design options and avoids some dependency constraints (see above). See some examples and their docs.
• Please confirm that we want to integrate the sankey functionality in the reporting as it currently stands. I'd probably ask at least @khaeru for his opinion.
• Please write a test if possible to confirm that sankey_mapper maps the correct values to the correct keys. We might also need to adapt one or two reporting tests that check the number of reporting steps if this stays in the default reporting workflow.

Please feel free to ask about any of these steps if you need help :)

[gidden]

Hi all, to simplify here, I would simply limit this functionality to pyam > 2 - we can do a version check for this on the fly and raise an error.

[gidden]

I would get rid of this, and simply do the concatenation outside of reporter (e.g., in the jupyter notebook here)

[glatterf42]

I kept this for now as I don't know what the Reporter actually does and it seems useful to just get the dataframe format we need from the Reporter. But please elaborate :)

[daymontas1]

I have created a new file, sankey.py, within the report directory, which extracts Sankey diagram functionality from init.py. Fridolin (@glatterf42), could you please confirm if this is what we want? Additionally, I have updated the westeros_sankey.ipynb file to ensure compatibility with the new sankey.py file in the report directory. I have annotated these changes in the westeros_sankey.ipynb below. Also, in this case, the following line inside the init.py file must be either removed or commented out:
("message::sankey", "concat", "out::pyam", "in::pyam")

[glatterf42]

Thanks, @daymontas1, these changes sound good to me. For the line that has become redundant, please remove it rather than commenting it out, this keeps the code clean.
Could you please push your changes to the branch so that we can take a look and see how the tests are doing?
If you don't have write access to @mabudz's fork, please let us know so that we can figure out a solution :)

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.6%. Comparing base (23388f0) to head (9993449).
Report is 8 commits behind head on main.

❗ Current head 9993449 differs from pull request most recent head f48ca83

Please upload reports for the commit f48ca83 to get more accurate results.

Additional details and impacted files

@@          Coverage Diff          @@
##            main    #770   +/-   ##
=====================================
  Coverage   95.5%   95.6%           
=====================================
  Files         46      47    +1     
  Lines       4340    4383   +43     
=====================================
+ Hits        4148    4191   +43     
  Misses       192     192           

Files with missing lines	Coverage Δ
message_ix/report/__init__.py	100.0% <100.0%> (ø)
message_ix/tests/test_report.py	100.0% <100.0%> (ø)
message_ix/tests/test_tutorials.py	97.6% <ø> (ø)
message_ix/tests/test_util.py	100.0% <100.0%> (ø)
message_ix/util/sankey.py	100.0% <100.0%> (ø)

[glatterf42]

In today's team meeting, @SiddharthJoshi-Git mentioned that he is fine with merging this PR as is and providing feedback later on. So I'm happy to merge this once the tests pass, which ... seem kind of confused about which versions of Python they should be using. For message_ix, that is still 3.8 and not 3.13, so maybe we'll have to merge #881 first.