CO2Leakage: Include box plot (#1326)

* Merge latest from develop (#51)

* Fix scale factor for Unit set to Fraction

* Attempt to fix issue with containment plots failing to show up initially

* Update encrypted contact information

* Improve hover-information

* Fix bug that removed legend text in containment plots

* Fix bug with UNSMRY assuming an older csv format

* Add ContainmentDataProvider wrapper class

Intended to wrap an EnsembleTableProvider that reads containment
summary files (either csv or arrow)

* Add UnsmryDataProvider wrapper class

Intended to wrap an EnsembleTableProvider that reads unsmry
summary files (either csv or arrow)

* Add validation to containment classes

* Change summary file read order

Arrow format is attempted first instead of csv

* Change summary file read order

Fix issue with unsmry option and missing files

* Fix issue with legend when first real is not "0"

* Fix bug with unsmry reset

* Remove large csv file check

* Remove unused file size function

* Changes to adapt CO2Leakage for residual trapping maps visualization

* Format changes

* Solve minor issues

* Solve typo

* Fix issue in property_origin

* Minor fixes

* Rename MapGroup CO2_MASS

* Add visualization menu with separate filters for each attribute

* Formatting with mypy, pylint and black

* Simplify MapThresholds class

* Fix visualization threshold bug for plume maps

* Add mean in tornado table VolumetricAnalysis (#1297)

* Fix merge conflict after rebase

* Fix issues related to missing polygon data

* Fix black format issues

* Fix another conflict from rebase

* Fix another rebase conflict

* Handle missing well picks and mypy errors

* Remove surface cache file mode (#1298)

* OS temp path for image server (#1302)

* Add missing file handling for well picks

* Enable usage without either maps or containment tables

* CCS-194: Fix after change of units for co2 mass. (#34)

* Tweak visualization filter

* Add polygon provider

Basically a copy of the provider for fault polygons. Renamed
"fault polygon" to "polygon" and added a couple of TODOs

* CCS-213: Handle plume_groups (#36)

* CCS-213: Start using plume group from containment results.

* CCS-213: Cont using plume group from containment results.

* CCS-213: Fixes for plume groups.

* CCS-213: Add plume groups to filter.

* CCS-213: Some minor fixes.

* CCS-213: Minor fix.

* Some tests.

* CCS-213: Join lines before/after merge when coloring by plume groups.

* CCS-213: Some refactoring.

* CCS-213: Fix combine plume group with none as mark.

* CCS-213: Minor fix.

* CCS-213: Remove some prints.

* CCS-213: Use starting 0-values for unmerged wells.

* CCS-213: Fix for multiple realization.

* CCS-213: Fix case without plume groups.

* CCS-213: Minor fix.

* CCS-213: Fix black.

* CCS-213: Some refactoring.

* CCS-213: Minor change, remove temp code.

* Combine time plots, add mean/P10/P90 option

* Fix formatting

* Fix missing issues with EnsemblePolygonProvider

* Distinguish p10 and p90 from mean, turn select-all-realizations into button

* Introduce new polygon provider in plugin

* Fix black/lint/mypy errorss

* CCS-213: Rename "?" to "undetermined" and fix warning messages (#38)

* CCS-213: Fix warning message.

* CCS-213: Rename ? to undetermined.

* Fix issues found while testing

* Minor fix in _get_menu_options() (#39)

* Minor fix in _get_menu_options().

* New minor fix.

* Minor change. (#40)

* CCS-212: Renaming (#41)

* CCS-212: Renaming, graph plots.

* CCS-212: Renaming, map plots.

* CCS-212: More renaming, map plots.

* CCS-212: More renaming, map plots.

* CCS-191: Add statistics plot (#44)

* CCS-191: Start on statistics plot.

* CCS-191: Minor change.

* CCS-191: Use color and mark options.

* CCS-191: Cont use color and mark options.

* CCS-191: Cont using color and mark options.

* CCS-191: Fix in label names.

* CCS-191: Add default category plotted, single option.

* CCS-191: Minor fig fixes.

* CCS-191: Add hovering.

* CCS-191: Fix some filtering.

* CCS-191: Remove prints and comments.

* CCS-191: Fill date options, use in plot.

* CCS-191: Same fixes for date, and use in first plot.

* CCS-191: Change some names, add titles.

* CCS-191: Clean up.

* CCS-191: Fix in set_date_option().

* Black

* isort

* CCS-191: Minor change.

* CCS-212: Renaming, changing same labels, changing back gas_phase to sgas etc, add code for XMF2 (#45)

* CCS-212: Use old file names for sgas etc maps.

* CCS-212: Allow XMF2-files as default in addition to AMFG. Some changes to labels for 2D maps.

* CCS-212: Some clean up.

* CCS-212: Add SGSTR as option.

* CCS-212: Remove prints.

* Fix linting (#49)

* Fix linting

* Fix linting 2

* Fix linting 3

* Fix linting 4

* Fix linting 5

* Fix linting 6

* Fix linting 7

* Fix linting 8

* Develop master merged (#50)

* Add mean in tornado table VolumetricAnalysis (#1297)

* Remove surface cache file mode (#1298)

* OS temp path for image server (#1302)

* Update README.md with deprecation warning (#1303)

* No Active Rfts error message (#1304)

Co-authored-by: Øyvind Lind-Johansen <olind@equinor.com>

* Pandas 2 compatibility for RFT plotter (#1305)

* Pandas 2 compability for RFT plotter

* black

* Remove use of `wcc.ColorScales` (#1306)

* Fix FutureWarning - cast to dtype (#1309)

* Pandas 2 compability for ParameterResponseCorrelation (#1310)

* Add field outline and custom well picks colors to MapViewerFMU (#1311)

* Replaced use of `DashSubsurfaceViewer` with `SubsurfaceViewer` (#1312)

* Expose rft input files as arguments (#1313)

* exposed rft input files as arguments

* black version updated

* pylint fixes

* pylint fixes

* more fixes

---------

Co-authored-by: Øyvind Lind-Johansen <olind@equinor.com>

* Disable cache for statistical surfaces (#1314)

* Add check for length of numpy array (#1316)

* Fix ensemble colors in relperm (#1317)

---------

Co-authored-by: FredrikNevjenNR <fnevjen@nr.no>
Co-authored-by: FredrikNevjenNR <150343101+FredrikNevjenNR@users.noreply.github.com>
Co-authored-by: Vegard Kvernelv <vegard@nr.no>
Co-authored-by: Jorge Sicacha <sicacha@nr.no>
Co-authored-by: jorgesicachanr <114475076+jorgesicachanr@users.noreply.github.com>
Co-authored-by: vegardkv <vkvernelv@gmail.com>

* Manual new commit for changes in CCS-101. (#52)

* CCS-192: Add box plot, plus CCS-249: Add zone/region/phase etc in title of plots (#53)

* CCS-192: Start on new box plot.

* CCS-192: Fix color order, and fix none as option for mark_by.

* CCS-192: Add option to plot poitns.

* CCS-192: Default false for show realization points.

* CCS-192: Adjust layout, add default option.

* CCS-192: Option for choosing probability plot vs box plot.

* CCS-192: Minor change.

* CCS-192: Minor fix.

* CCS-192: Remove prints.

* CCS-192: Minor fix.

* CCS-192: Add custom hovering template for realization points.

* CCS-192: Add info to plot titles.

* CCS-192: Add info to time plots.

* CCS-192: Minor fix.

* Black

* CCS-192: Improvements to box plot (#54)

* CCS-192: Change options for plotting points.

* CCS-192: Fix titles.

* CCS-192: Try adding hoovering info.

* CCS-192: Cont attempt hoovering info.

* CCS-192: Change to using go.Box, more flexibility.

* CCS-192: Fix hoovering for points.

* CCS-192: Better hoovering info, placement.

* CCS-192: Fix in hoovering info for realization points.

* CCS-192: Visual improvements.

* CCS-192: Change q1-q3 calc to numpy default, and fix whiskers calc.

* CCS-192: Change default box, only single if >20 categories.

* CCS-192: Remove prints, fix in q1 q3.

* CCS-192: Minor fixes.

* CCS-192: Remove prints, old code, new fix q1 q3.

* CCS-192: Minor fix, and black.

* CCS-192: Change order hoovering info.

* CCS-192: Some fixes in box plot (#55)

* CCS-192: Make sort by color possible.

* CCS-192: Print p10, p90 in hoovering

* CCS-192: Remove None option for points.

* CCS-192: Some label changes.

* CCS-266: Fix warning. (#56)

* CCS-274: Remove default for actual volume csv file. (#57)

* Lint fixes.

---------

Co-authored-by: FredrikNevjenNR <fnevjen@nr.no>
Co-authored-by: FredrikNevjenNR <150343101+FredrikNevjenNR@users.noreply.github.com>
Co-authored-by: Vegard Kvernelv <vegard@nr.no>
Co-authored-by: Jorge Sicacha <sicacha@nr.no>
Co-authored-by: jorgesicachanr <114475076+jorgesicachanr@users.noreply.github.com>
Co-authored-by: vegardkv <vkvernelv@gmail.com>

Author: 4 people

webviz_subsurface/plugins/_co2_leakage/_plugin.py

@@ -104,7 +104,7 @@ def __init__(
         ensembles: List[str],
         well_pick_file: Optional[str] = None,
         plume_mass_relpath: str = TABLES_PATH + "/plume_mass.csv",
-        plume_actual_volume_relpath: str = TABLES_PATH + "/plume_actual_volume.csv",
+        plume_actual_volume_relpath: Optional[str] = None,
         unsmry_relpath: Optional[str] = None,
         fault_polygon_attribute: str = "dl_extracted_faultlines",
         initial_surface: Optional[str] = None,
@@ -296,6 +296,13 @@ def _set_callbacks(self) -> None:
                 Input(self._settings_component(ViewSettings.Ids.SORT_PLOT), "value"),
                 Input(self._settings_component(ViewSettings.Ids.REAL_OR_STAT), "value"),
                 Input(self._settings_component(ViewSettings.Ids.DATE_OPTION), "value"),
+                Input(
+                    self._settings_component(ViewSettings.Ids.STATISTICS_TAB_OPTION),
+                    "value",
+                ),
+                Input(
+                    self._settings_component(ViewSettings.Ids.BOX_SHOW_POINTS), "value"
+                ),
             )
             @callback_typecheck
             def update_graphs(
@@ -317,6 +324,8 @@ def update_graphs(
                 sorting: str,
                 lines_to_show: str,
                 date_option: str,
+                statistics_tab_option: str,
+                box_show_points: str,
             ) -> Tuple[Dict, go.Figure, go.Figure, go.Figure]:
                 # pylint: disable=too-many-locals
                 figs = [no_update] * 3
@@ -331,6 +340,8 @@ def update_graphs(
                     sorting,
                     lines_to_show,
                     date_option,
+                    statistics_tab_option,
+                    box_show_points,
                     self._menu_options[ensemble][source],
                 )
                 if source in [
@@ -344,6 +355,7 @@ def update_graphs(
                         cont_info,
                         realizations,
                         lines_to_show,
+                        statistics_tab_option,
                         len(figs),
                     )
                     cont_info["update_first_figure"] = self._plot_id != plot_ids[0]

webviz_subsurface/plugins/_co2_leakage/_utilities/callbacks.py

@@ -22,6 +22,7 @@
 )
 from webviz_subsurface.plugins._co2_leakage._utilities import plume_extent
 from webviz_subsurface.plugins._co2_leakage._utilities.co2volume import (
+    generate_co2_box_plot_figure,
     generate_co2_statistics_figure,
     generate_co2_time_containment_figure,
     generate_co2_time_containment_one_realization_figure,
@@ -414,12 +415,20 @@ def generate_containment_figures(
                 containment_info,
             )
         )
-        fig2 = generate_co2_statistics_figure(
-            table_provider,
-            realizations,
-            co2_scale,
-            containment_info,
-        )
+        if containment_info["statistics_tab_option"] == "probability_plot":
+            fig2 = generate_co2_statistics_figure(
+                table_provider,
+                realizations,
+                co2_scale,
+                containment_info,
+            )
+        else:  # "box_plot"
+            fig2 = generate_co2_box_plot_figure(
+                table_provider,
+                realizations,
+                co2_scale,
+                containment_info,
+            )
     except KeyError as exc:
         warnings.warn(f"Could not generate CO2 figures: {exc}")
         raise exc
@@ -477,6 +486,8 @@ def process_containment_info(
     sorting: str,
     lines_to_show: str,
     date_option: str,
+    statistics_tab_option: str,
+    box_show_points: str,
     menu_options: MenuOptions,
 ) -> Dict[str, Union[str, None, List[str], int]]:
     if mark_choice is None:
@@ -518,6 +529,8 @@ def plume_sort_key(name: str) -> int:
         "plume_groups": plume_groups,
         "use_stats": lines_to_show == "stat",
         "date_option": date_option,
+        "statistics_tab_option": statistics_tab_option,
+        "box_show_points": box_show_points,
     }
 
 
@@ -528,6 +541,7 @@ def make_plot_ids(
     containment_info: Dict,
     realizations: List[int],
     lines_to_show: str,
+    statistics_tab_option: str,
     num_figs: int,
 ) -> List[str]:
     zone_str = (
@@ -565,6 +579,7 @@ def make_plot_ids(
     ids = [plot_id]
     ids += [plot_id + f"-{realizations}"] * (num_figs - 1)
     ids[1] += f"-{lines_to_show}"
+    ids[2] += f"-{statistics_tab_option}"
     return ids
 
 

webviz_subsurface/plugins/_co2_leakage/_utilities/co2volume.py

@@ -422,19 +422,18 @@ def _change_names(
         df["name"] = df["name"].replace(f"{m}, all", m)
 
 
-def _adjust_figure(fig: go.Figure, plot_title: Optional[str] = None) -> None:
+def _adjust_figure(fig: go.Figure, plot_title: str) -> None:
     fig.layout.legend.orientation = "v"
     fig.layout.legend.title.text = ""
     fig.layout.legend.itemwidth = 40
     fig.layout.xaxis.exponentformat = "power"
-    if plot_title is not None:
-        fig.layout.title.text = plot_title
-        fig.layout.title.font = {"size": 14}
-        fig.layout.margin.t = 40
-        fig.layout.title.y = 0.95
-    else:
-        fig.layout.margin.t = 15
+
+    fig.layout.title.text = plot_title
+    fig.layout.title.font = {"size": 14}
+    fig.layout.margin.t = 40
+    fig.layout.title.y = 0.95
     fig.layout.title.x = 0.4
+
     fig.layout.paper_bgcolor = "rgba(0,0,0,0)"
     fig.layout.margin.b = 6
     fig.layout.margin.l = 10
@@ -508,7 +507,7 @@ def generate_co2_volume_figure(
     )
     fig.layout.yaxis.title = "Realization"
     fig.layout.xaxis.title = scale.value
-    _adjust_figure(fig, plot_title=containment_info["date_option"])
+    _adjust_figure(fig, plot_title=_make_title(containment_info))
     return fig
 
 
@@ -562,7 +561,7 @@ def generate_co2_time_containment_one_realization_figure(
     fig.layout.yaxis.range = y_limits
     fig.layout.xaxis.title = "Time"
     fig.layout.yaxis.title = scale.value
-    _adjust_figure(fig)
+    _adjust_figure(fig, plot_title=_make_title(containment_info, include_date=False))
     return fig
 
 
@@ -756,7 +755,7 @@ def generate_co2_time_containment_figure(
             df_grouped = df_no_real.groupby(
                 ["date", "name", color_choice, mark_choice], as_index=False
             )
-        df_mean = df_grouped.agg(np.mean)
+        df_mean = df_grouped.agg("mean")
         df_mean["realization"] = ["mean"] * df_mean.shape[0]
         df_p10 = df_grouped.agg(lambda x: np.quantile(x, 0.1))
         df_p10["realization"] = ["p10"] * df_p10.shape[0]
@@ -808,7 +807,7 @@ def generate_co2_time_containment_figure(
     fig.layout.xaxis.title = "Time"
     fig.layout.yaxis.title = scale.value
     fig.layout.yaxis.autorange = True
-    _adjust_figure(fig)
+    _adjust_figure(fig, plot_title=_make_title(containment_info, include_date=False))
     return fig
 
 
@@ -860,6 +859,179 @@ def generate_co2_statistics_figure(
     fig.layout.legend.tracegroupgap = 0
     fig.layout.xaxis.title = scale.value
     fig.layout.yaxis.title = "Probability"
-    _adjust_figure(fig, plot_title=containment_info["date_option"])
+    _adjust_figure(fig, plot_title=_make_title(containment_info))
+
+    return fig
+
+
+def generate_co2_box_plot_figure(
+    table_provider: ContainmentDataProvider,
+    realizations: List[int],
+    scale: Union[Co2MassScale, Co2VolumeScale],
+    containment_info: Dict[str, Any],
+) -> go.Figure:
+    eps = 0.00001
+    date_option = containment_info["date_option"]
+    df = _read_co2_volumes(table_provider, realizations, scale)
+    df = df[df["date"] == date_option]
+    df = df.drop(columns=["date"]).reset_index(drop=True)
+
+    color_choice = containment_info["color_choice"]
+    mark_choice = containment_info["mark_choice"]
+    _filter_columns(df, color_choice, mark_choice, containment_info)
+    cat_ord, colors, _ = _prepare_pattern_and_color_options_statistics_plot(
+        df,
+        containment_info,
+        color_choice,
+        mark_choice,
+    )
+
+    fig = go.Figure()
+    for count, type_val in enumerate(cat_ord["type"], 0):
+        df_sub = df[df["type"] == type_val]
+        values = df_sub["amount"].to_numpy()
+        real = df_sub["realization"].to_numpy()
+
+        median_val = df_sub["amount"].median()
+        q1 = _calculate_plotly_quantiles(values, 0.25)
+        q3 = _calculate_plotly_quantiles(values, 0.75)
+        p10 = np.percentile(values, 90)
+        p90 = np.percentile(values, 10)
+        min_fence, max_fence = _calculate_plotly_whiskers(values, q1, q3)
+
+        fig.add_trace(
+            go.Box(
+                x=[count] * len(values),
+                y=values,
+                name=type_val,
+                marker_color=colors[count],
+                boxpoints="all"
+                if containment_info["box_show_points"] == "all_points"
+                else "outliers",
+                customdata=real,
+                hovertemplate="<span style='font-family:Courier New;'>"
+                "Type       : %{data.name}<br>Amount     : %{y:.3f}<br>"
+                "Realization: %{customdata}"
+                "</span><extra></extra>",
+                legendgroup=type_val,
+                width=0.55,
+            )
+        )
+
+        fig.add_trace(
+            go.Bar(
+                x=[count],
+                y=[values.max() - values.min() + 2 * eps],
+                base=[values.min() - eps],
+                opacity=0.0,
+                hoverinfo="none",
+                hovertemplate=(
+                    "<span style='font-family:Courier New;'>"
+                    f"Type           : {type_val}<br>"
+                    f"Max            : {values.max():.3f}<br>"
+                    f"Top whisker    : {max_fence:.3f}<br>"
+                    f"p10 (not shown): {p10:.3f}<br>"
+                    f"Q3             : {q3:.3f}<br>"
+                    f"Median         : {median_val:.3f}<br>"
+                    f"Q1             : {q1:.3f}<br>"
+                    f"p90 (not shown): {p90:.3f}<br>"
+                    f"Lower whisker  : {min_fence:.3f}<br>"
+                    f"Min            : {values.min():.3f}"
+                    "</span><extra></extra>"
+                ),
+                showlegend=False,
+                legendgroup=type_val,
+                name=type_val,
+                marker_color=colors[count],
+                width=0.56,
+            )
+        )
+
+    fig.update_layout(
+        xaxis=dict(
+            tickmode="array",
+            tickvals=[i for i in range(len(cat_ord["type"]))],
+            ticktext=cat_ord["type"],
+        )
+    )
+
+    if len(cat_ord["type"]) > 20:
+        default_option = _find_default_option_statistics_figure(df, cat_ord["type"])
+        for trace in fig.data:
+            if trace.name != default_option:
+                trace.visible = "legendonly"
+
+    fig.layout.yaxis.autorange = True
+    fig.layout.legend.tracegroupgap = 0
+    fig.layout.yaxis.title = scale.value
+    _adjust_figure(fig, plot_title=_make_title(containment_info))
 
     return fig
+
+
+def _make_title(c_info: Dict[str, Any], include_date: bool = True) -> str:
+    components = []
+    if include_date:
+        components.append(c_info["date_option"])
+    if len(c_info["phases"]) > 0 and "phase" not in [
+        c_info["color_choice"],
+        c_info["mark_choice"],
+    ]:
+        if c_info["phase"] != "total":
+            components.append(c_info["phase"].capitalize())
+        else:
+            components.append("Phase: Total")
+    if len(c_info["containments"]) > 0 and "containment" not in [
+        c_info["color_choice"],
+        c_info["mark_choice"],
+    ]:
+        if c_info["containment"] != "total":
+            components.append(c_info["containment"].capitalize())
+        else:
+            components.append("All containments areas")
+    if len(c_info["zones"]) > 0 and "zone" not in [
+        c_info["color_choice"],
+        c_info["mark_choice"],
+    ]:
+        if c_info["zone"] != "all":
+            components.append(c_info["zone"])
+        else:
+            components.append("All zones")
+    if len(c_info["regions"]) > 0 and "region" not in [
+        c_info["color_choice"],
+        c_info["mark_choice"],
+    ]:
+        if c_info["region"] != "all":
+            components.append(c_info["region"])
+        else:
+            components.append("All regions")
+    if len(c_info["plume_groups"]) > 0 and "plume_group" not in [
+        c_info["color_choice"],
+        c_info["mark_choice"],
+    ]:
+        if c_info["plume_group"] != "all":
+            components.append(c_info["plume_group"])
+        else:
+            components.append("All plume groups")
+    return " - ".join(components)
+
+
+def _calculate_plotly_quantiles(values: np.ndarray, percentile: float) -> float:
+    values_sorted = values.copy()
+    values_sorted.sort()
+    n_val = len(values_sorted)
+    a = n_val * percentile - 0.5
+    if a.is_integer():
+        return float(values_sorted[int(a)])
+    else:
+        return float(np.interp(a, [x for x in range(0, n_val)], values_sorted))
+
+
+def _calculate_plotly_whiskers(
+    values: np.ndarray, q1: float, q3: float
+) -> Tuple[float, float]:
+    values_sorted = values.copy()
+    values_sorted.sort()
+    a = q1 - 1.5 * (q3 - q1)
+    b = q3 + 1.5 * (q3 - q1)
+    return values[values >= a].min(), values[values <= b].max()

webviz_subsurface/plugins/_co2_leakage/_utilities/initialization.py

@@ -157,8 +157,10 @@ def init_unsmry_data_providers(
 
 def init_containment_data_providers(
     ensemble_roots: Dict[str, str],
-    table_rel_path: str,
+    table_rel_path: Optional[str],
 ) -> Dict[str, ContainmentDataProvider]:
+    if table_rel_path is None:
+        return {}
     factory = EnsembleTableProviderFactory.instance()
     providers = {
         ens: _init_ensemble_table_provider(factory, ens, ens_path, table_rel_path)