gwgen.sensitivity_analysis module

Module for a sensitivity analysis for one experiment

Classes

SensitivityAnalysis(organizer) Class that performs and manages a sensitivity analysis for the given
SensitivityPlot(stations, config, ...[, ...]) A base class for the plots of the SensitivityAnalysis
SensitivityPlot1D(stations, config, ...[, ...]) 1d plots with the indicator on the y-axis and the namelist parameter on
SensitivityPlot2D(stations, config, ...[, ...]) 2d plots with one namelist parameter on the y-axis and one on the

Functions

default_plot2d_config([logx, logy]) The default configuration for SensitivityPlot instances.
default_sens_config([sa, indicators, names, ...]) The default configuration for SensitivityPlot instances.
class gwgen.sensitivity_analysis.SensitivityAnalysis(organizer)[source]

Bases: object

Class that performs and manages a sensitivity analysis for the given organizer

Parameters:organizer (gwgen.main.GWGENOrganizer) – The organizer to perform thre sensitivity analysis for

Methods

__call__(kws) Call the start() of the
compile_model() Compile the sensitivity analysis model
evaluate([experiments, loop_exps]) Evaluate the experiments of the sensitivity analysis
init([nml, experiment, run_prepare, ...]) Initialize the experiments for the sensitivity analysis
plot([indicators, names, meta]) Plot the result of the sensitivity analysis
remove(*args, **kwargs) Remove the project and files of the sensitivity analysis
run([experiments, remove]) Run the analysis
setup([root_dir, projectname, link, no_move]) Set up the experiments for the project

Attributes

config The configuration of the sensitivity analysis
exp_config
experiment The base experiment of this sensitivity analysis
experiments
logger
project_config The project configuration of the sensitivity analysis
projectname The projectname of the sensitivity analysis
sa_organizer The organizer for this sensitivity analysis
__call__(kws)[source]

Call the start() of the organizer attribute

Parameters:kws (dict) – Any keywords that are passed to the start() method
Returns:
  • model_organization.ModelOrganizer – The sa_organizer
  • argparse.Namespace – The return of the start() method
compile_model()[source]

Compile the sensitivity analysis model

config

The configuration of the sensitivity analysis

evaluate(experiments=None, loop_exps=False, **kwargs)[source]

Evaluate the experiments of the sensitivity analysis

Parameters:
  • experiments (list of str) – The experiments to use. If None, all experiments are used
  • loop_exps (bool) – If True the experiments of the sensitivity analysis are called one after each other. Otherwise they are called in parallel, each on a single core (unless the the serial option in the global configuration is set)
  • stations (str or list of str) – either a list of stations to use or a filename containing a 1-row table with stations
  • other_exp (str) – Use the configuration from another experiment
  • setup_from (str) – Determine where to get the data from. If scratch, the data will be calculated from the raw data. If file, the data will be loaded from a file, if db, the data will be loaded from a postgres database (Note that the database argument must be provided!).
  • to_db (bool) – Save the data into a postgresql database (Note that the database argument must be provided!)
  • to_csv (bool) – Save the data into a csv file
  • database (str) – The name of a postgres data base to write the data to
  • norun (bool, list of str or 'all') – If True, only the data is set up and the configuration of the experiment is not affected. It can be either a list of tasks or True or 'all'
  • to_return (list of str or 'all') – The names of the tasks to return. If None, only the ones with an gwgen.utils.TaskBase.has_run are returned.
  • complete (bool) – If True, setup and run all possible tasks
exp_config
experiment

The base experiment of this sensitivity analysis

experiments
init(nml=None, experiment=None, run_prepare=False, use_param=False, no_move=False)[source]

Initialize the experiments for the sensitivity analysis

Parameters:
  • nml (dict) – A mapping from namelist parameters to the values to use. Ranges might be lists of numbers or '<i>err' to use '<i>'-times the error from the parameterization. You might also provide up to three values in case on of them is a string with 'err' in it, where the first value corresponds to the minimum, the second to the maximum and the third to the number of steps.
  • experiment (str) – The base experiment name to use. Should somehow contain a number which will be increased for each experiment. If None, it will default to '<id>_sens0'
  • run_prepare (bool) – If True or no 'input' key exists for the experiment, the 'prepare' evaluation task is run to get the reference data. Otherwise, if False and the 'input' key does not exist, the specified file is used
  • use_param (bool) – If True/set and the parameterization is required, use the parameterization of the original experiment
  • no_move (bool) – If True, the experiment in the configurations are not moved to the position right before the configuration of the current experiment
logger
plot(indicators=['rsquared', 'slope', 'ks', 'quality'], names=['prcp', 'tmin', 'tmax', 'mean_cloud', 'wind'], meta=None, **kwargs)[source]

Plot the result of the sensitivity analysis

2 different plots are made for each variable and quality indicator:

  1. 1d plots with the indicator on the y-axis and the namelist parameter on the x-axis
  2. 2d plots with one indicator on the x-axis and another on the y-axis for each possible combination

You can disable one of the plot types via the only_1d and only_2d parameter.

Parameters:
  • indicators (str or list of str) – The name of the indicators from the quality evaluation task
  • names (str or list of str) – The name(s) of the variable(s) to plot
  • meta (dict or path to yaml configuration file) – Alternative meta information for the data set
  • **kwargs – Any task of the SensitivityPlot framework
project_config

The project configuration of the sensitivity analysis

projectname

The projectname of the sensitivity analysis

remove(*args, **kwargs)[source]

Remove the project and files of the sensitivity analysis

run(experiments=None, remove=False)[source]

Run the analysis

Parameters:
  • experiments (list of str) – The experiments to use. If None, all experiments are used
  • remove (bool) – If True, the work_dir will be removed if it already exists
sa_organizer

The organizer for this sensitivity analysis

setup(root_dir=None, projectname=None, link=True, no_move=False, *args, **kwargs)[source]

Set up the experiments for the project

Parameters:
  • root_dir (str) – The path to the root directory where the experiments, etc. will be stored. If not given, a new directory will be created at the same location as the original project
  • projectname (str) – The name of the project that shall be initialized at root_dir. A new directory will be created namely root_dir + '/' + projectname. If not given, defaults to 'sensitivity_<id>' where '<id>' is the experiment id
  • link (bool) – If set, the source files are linked to the original ones instead of copied
  • no_move (bool) – If True, the project in the configuration files is not moved to the position right before the configuration of the current project
class gwgen.sensitivity_analysis.SensitivityPlot(stations, config, project_config, global_config, data=None, requirements=None, *args, **kwargs)[source]

Bases: gwgen.utils.TaskBase

A base class for the plots of the SensitivityAnalysis

Parameters:
  • stations (list) – The list of stations to process
  • config (dict) – The configuration of the experiment
  • project_config (dict) – The configuration of the underlying project
  • global_config (dict) – The global configuration
  • data (pandas.DataFrame) – The data to use. If None, use the setup() method
  • requirements (list of TaskBase instances) – The required instances. If None, you must call the set_requirements() method later
Other Parameters:
 

``*args, **kwargs`` – The configuration of the task. See the TaskConfig for arguments. Note that if you provide *args, you have to provide all possible arguments

Methods

add_meta(ds)
setup_from_db(*args, **kwargs)
setup_from_file(*args, **kwargs)
setup_from_scratch()

Attributes

base_experiments
dbname str(object=’‘) -> string
default_config
ds Gives the base dataset for the plot
err_nml_keys
meta dict() -> new empty dictionary
namelist The namelist of the sensitivity analysis holding the settings
organizer
ranges_nml_keys
sql_dtypes
task_data_dir The directory where to store data
variables_meta dict() -> new empty dictionary
add_meta(ds)[source]
base_experiments
dbname = 'sensitivity_analysis'
default_config
ds

Gives the base dataset for the plot

err_nml_keys
meta = {'slope': {'long_name': 'Slope indicator'}, 'gp_shape': {'units': '-', 'long_name': 'Generalized Pareto Shape parameter'}, 'ks': {'long_name': 'Fraction of stations without significant difference'}, 'thresh': {'units': 'mm', 'long_name': 'Crossover point of Gamma-GP distribution'}, 'rsquared': {'long_name': 'Mean R$^2$ of all quantiles'}, 'quality': {'long_name': 'Quality indicator'}}
namelist

The namelist of the sensitivity analysis holding the settings

organizer
ranges_nml_keys
setup_from_db(*args, **kwargs)[source]
setup_from_file(*args, **kwargs)[source]
setup_from_scratch()[source]
sql_dtypes
task_data_dir

The directory where to store data

variables_meta = {'tmax': 'Max. Temperature', 'prcp': 'Precipitation', 'tmin': 'Min. Temperature', 'mean_cloud': 'Cloud fraction'}
class gwgen.sensitivity_analysis.SensitivityPlot1D(stations, config, project_config, global_config, data=None, requirements=None, *args, **kwargs)[source]

Bases: gwgen.sensitivity_analysis.SensitivityPlot

1d plots with the indicator on the y-axis and the namelist parameter on the x-axis

Parameters:
  • stations (list) – The list of stations to process
  • config (dict) – The configuration of the experiment
  • project_config (dict) – The configuration of the underlying project
  • global_config (dict) – The global configuration
  • data (pandas.DataFrame) – The data to use. If None, use the setup() method
  • requirements (list of TaskBase instances) – The required instances. If None, you must call the set_requirements() method later
Other Parameters:
 

``*args, **kwargs`` – The configuration of the task. See the TaskConfig for arguments. Note that if you provide *args, you have to provide all possible arguments

Methods

create_project(ds)

Attributes

fmt dict() -> new empty dictionary
has_run bool(x) -> bool
name str(object=’‘) -> string
summary str(object=’‘) -> string
create_project(ds)[source]
fmt = {'plot': ' ', 'xlabel': '{desc}', 'title': '%(vname_long)s', 'color': 'coolwarm', 'ylabel': '%(long_name)s', 'marker': 'o'}
has_run = True
name = 'plot1d'
summary = '1d plots with the indicator on the y-axis and the namelist parameter on the x-axis'
class gwgen.sensitivity_analysis.SensitivityPlot2D(stations, config, project_config, global_config, data=None, requirements=None, *args, **kwargs)[source]

Bases: gwgen.sensitivity_analysis.SensitivityPlot

2d plots with one namelist parameter on the y-axis and one on the x-axis and a color coding the the indicator

Parameters:
  • stations (list) – The list of stations to process
  • config (dict) – The configuration of the experiment
  • project_config (dict) – The configuration of the underlying project
  • global_config (dict) – The global configuration
  • data (pandas.DataFrame) – The data to use. If None, use the setup() method
  • requirements (list of TaskBase instances) – The required instances. If None, you must call the set_requirements() method later
Other Parameters:
 

``*args, **kwargs`` – The configuration of the task. See the TaskConfig for arguments. Note that if you provide *args, you have to provide all possible arguments

Attributes

default_config
has_run bool(x) -> bool
name str(object=’‘) -> string
summary str(object=’‘) -> string

Methods

run(info)
default_config
has_run = True
name = 'plot2d'
run(info)[source]
summary = '2d plots with one namelist parameter on the y-axis and one on the x-axis and a color coding the the indicator'
gwgen.sensitivity_analysis.default_plot2d_config(logx=False, logy=False, *args, **kwargs)[source]

The default configuration for SensitivityPlot instances. See also the SensitivityPlot.default_config attribute

Parameters:
  • logx (bool) – If True, the x-axis will be logarithmic
  • logy (bool) – If True, the y-axis will be logarithmic
  • sa (SensitivityAnalysis) – The analyser to use
  • indicators (str or list of str) – The name of the indicators from the quality evaluation task
  • names (str or list of str) – The name of the variables to plot
  • meta (dict) – Alternative meta information for the data set
  • setup_from ({ 'scratch' | 'file' | 'db' | None }) –

    The method how to setup the instance either from

    'scratch'
    To set up the task from the raw data
    'file'
    Set up the task from an existing file
    'db'
    Set up the task from a database
    None
    If the file name of this this task exists, use this one, otherwise a database is provided, use this one, otherwise go from scratch
  • to_csv (bool) – If True, the data at setup will be written to a csv file
  • to_db (bool) – If True, the data at setup will be written to into a database
  • remove (bool) – If True and the old data file already exists, remove before writing to it
  • skip_filtering (bool) – If True, skip the filtering for the correct stations in the datafile
  • plot_output (str) – An alternative path to use for the PDF file of the plot
  • nc_output (str) – An alternative path (or multiples depending on the task) to use for the netCDF file of the plot data
  • project_output (str) – An alternative path to use for the psyplot project file of the plot
  • new_project (bool) – If True, a new project will be created even if a file in project_output exists already
  • project (str) – The path to a psyplot project file to use for this parameterization
  • close (bool) – Close the project at the end
gwgen.sensitivity_analysis.default_sens_config(sa=None, indicators=['rsquared', 'slope', 'ks', 'quality'], names=['prcp', 'tmin', 'tmax', 'mean_cloud', 'wind'], meta={}, *args, **kwargs)[source]

The default configuration for SensitivityPlot instances. See also the SensitivityPlot.default_config attribute

Parameters:
  • sa (SensitivityAnalysis) – The analyser to use
  • indicators (str or list of str) – The name of the indicators from the quality evaluation task
  • names (str or list of str) – The name of the variables to plot
  • meta (dict) – Alternative meta information for the data set
  • setup_from ({ 'scratch' | 'file' | 'db' | None }) –

    The method how to setup the instance either from

    'scratch'
    To set up the task from the raw data
    'file'
    Set up the task from an existing file
    'db'
    Set up the task from a database
    None
    If the file name of this this task exists, use this one, otherwise a database is provided, use this one, otherwise go from scratch
  • to_csv (bool) – If True, the data at setup will be written to a csv file
  • to_db (bool) – If True, the data at setup will be written to into a database
  • remove (bool) – If True and the old data file already exists, remove before writing to it
  • skip_filtering (bool) – If True, skip the filtering for the correct stations in the datafile
  • plot_output (str) – An alternative path to use for the PDF file of the plot
  • nc_output (str) – An alternative path (or multiples depending on the task) to use for the netCDF file of the plot data
  • project_output (str) – An alternative path to use for the psyplot project file of the plot
  • new_project (bool) – If True, a new project will be created even if a file in project_output exists already
  • project (str) – The path to a psyplot project file to use for this parameterization
  • close (bool) – Close the project at the end