ax.plot¶

Base¶

class ax.plot.base.AxPlotConfig[source]¶

Bases: ax.plot.base._AxPlotConfigBase

Config for plots

class ax.plot.base.AxPlotTypes[source]¶

Bases: enum.Enum

Enum of Ax plot types.

BANDIT_ROLLOUT = 4¶

CONTOUR = 0¶

GENERIC = 1¶

INTERACT_CONTOUR = 3¶

INTERACT_SLICE = 5¶

SLICE = 2¶

ax.plot.base.CI_OPACITY = 0.4¶

ax.plot.base.DECIMALS = 3¶

class ax.plot.base.PlotData[source]¶

Bases: tuple

Struct for plot data, including both in-sample and out-of-sample arms

property in_sample¶

Alias for field number 1

property metrics¶

Alias for field number 0

property out_of_sample¶

Alias for field number 2

property status_quo_name¶

Alias for field number 3

class ax.plot.base.PlotInSampleArm[source]¶

Bases: tuple

Struct for in-sample arms (both observed and predicted data)

property context_stratum¶

Alias for field number 6

property name¶

Alias for field number 0

property parameters¶

Alias for field number 1

property se¶

Alias for field number 4

property se_hat¶

Alias for field number 5

property y¶

Alias for field number 2

property y_hat¶

Alias for field number 3

class ax.plot.base.PlotMetric[source]¶

Bases: tuple

Struct for metric

property metric¶

Alias for field number 0

property pred¶

Alias for field number 1

class ax.plot.base.PlotOutOfSampleArm[source]¶

Bases: tuple

Struct for out-of-sample arms (only predicted data)

property context_stratum¶

Alias for field number 4

property name¶

Alias for field number 0

property parameters¶

Alias for field number 1

property se_hat¶

Alias for field number 3

property y_hat¶

Alias for field number 2

ax.plot.base.Z = 1.96¶

Color¶

ax.plot.color.BLUE_SCALE = [(255, 247, 251), (236, 231, 242), (208, 209, 230), (166, 189, 219), (116, 169, 207), (54, 144, 192), (5, 112, 176), (3, 78, 123)]¶

class ax.plot.color.COLORS[source]¶

Bases: enum.Enum

An enumeration.

CORAL = (251, 128, 114)¶

LIGHT_PURPLE = (190, 186, 218)¶

ORANGE = (253, 180, 98)¶

PINK = (188, 128, 189)¶

STEELBLUE = (128, 177, 211)¶

TEAL = (141, 211, 199)¶

ax.plot.color.DISCRETE_COLOR_SCALE = [(128, 177, 211), (251, 128, 114), (141, 211, 199), (188, 128, 189), (190, 186, 218), (253, 180, 98)]¶

ax.plot.color.GREEN_PINK_SCALE = [(142, 1, 82), (197, 27, 125), (222, 119, 174), (241, 182, 218), (253, 224, 239), (247, 247, 247), (230, 245, 208), (184, 225, 134), (127, 188, 65), (77, 146, 33), (39, 100, 25)]¶

ax.plot.color.GREEN_SCALE = [(247, 252, 253), (229, 245, 249), (204, 236, 230), (153, 216, 201), (102, 194, 164), (65, 174, 118), (35, 139, 69), (0, 109, 44), (0, 68, 27)]¶

ax.plot.color.MIXED_SCALE = [(2, 63, 165), (125, 135, 185), (190, 193, 212), (214, 188, 192), (187, 119, 132), (142, 6, 59), (74, 111, 227), (133, 149, 225), (181, 187, 227), (230, 175, 185), (224, 123, 145), (211, 63, 106), (17, 198, 56), (141, 213, 147), (198, 222, 199), (234, 211, 198), (240, 185, 141), (239, 151, 8), (15, 207, 192), (156, 222, 214), (213, 234, 231), (243, 225, 235), (246, 196, 225), (247, 156, 212)]¶

ax.plot.color.plotly_color_scale(list_of_rgb_tuples, reverse=False, alpha=1)[source]¶

Convert list of RGB tuples to list of tuples, where each tuple is break in [0, 1] and stringified RGBA color.

Return type

List[Tuple[float, str]]

ax.plot.color.rgba(rgb_tuple, alpha=1)[source]¶

Convert RGB tuple to an RGBA string.

Return type

str

Contour Plot¶

ax.plot.contour.ContourPredictions = typing.Tuple[ax.plot.base.PlotData, numpy.ndarray, numpy.ndarray, numpy.ndarray, numpy.ndarray, typing.Dict[str, bool]]¶

ax.plot.contour.MAX_PARAM_LENGTH = 40¶

ax.plot.contour.interact_contour(model, metric_name, generator_runs_dict=None, relative=False, density=50, slice_values=None, lower_is_better=False, fixed_features=None)[source]¶

Create interactive plot with predictions for a 2-d slice of the parameter space.

Parameters

• model (ModelBridge) – ModelBridge that contains model for predictions

• metric_name (str) – Name of metric to plot

• generator_runs_dict (Optional[Dict[str, GeneratorRun]]) – A dictionary {name: generator run} of generator runs whose arms will be plotted, if they lie in the slice.

• relative (bool) – Predictions relative to status quo

• density (int) – Number of points along slice to evaluate predictions.

• slice_values (Optional[Dict[str, Any]]) – A dictionary {name: val} for the fixed values of the other parameters. If not provided, then the status quo values will be used if there is a status quo, otherwise the mean of numeric parameters or the mode of choice parameters.

• lower_is_better (bool) – Lower values for metric are better.

• fixed_features (Optional[ObservationFeatures]) – An ObservationFeatures object containing the values of features (including non-parameter features like context) to be set in the slice.

Return type

AxPlotConfig

ax.plot.contour.plot_contour(model, param_x, param_y, metric_name, generator_runs_dict=None, relative=False, density=50, slice_values=None, lower_is_better=False, fixed_features=None)[source]¶

Plot predictions for a 2-d slice of the parameter space.

Parameters

• model (ModelBridge) – ModelBridge that contains model for predictions

• param_x (str) – Name of parameter that will be sliced on x-axis

• param_y (str) – Name of parameter that will be sliced on y-axis

• metric_name (str) – Name of metric to plot

• generator_runs_dict (Optional[Dict[str, GeneratorRun]]) – A dictionary {name: generator run} of generator runs whose arms will be plotted, if they lie in the slice.

• relative (bool) – Predictions relative to status quo

• density (int) – Number of points along slice to evaluate predictions.

• slice_values (Optional[Dict[str, Any]]) – A dictionary {name: val} for the fixed values of the other parameters. If not provided, then the status quo values will be used if there is a status quo, otherwise the mean of numeric parameters or the mode of choice parameters.

• lower_is_better (bool) – Lower values for metric are better.

• fixed_features (Optional[ObservationFeatures]) – An ObservationFeatures object containing the values of features (including non-parameter features like context) to be set in the slice.

Return type

AxPlotConfig

ax.plot.contour.short_name(param_name)[source]¶

Return type

str

Model Diagnostic Plot¶

ax.plot.diagnostic.FloatList = typing.List[float]¶

ax.plot.diagnostic.interact_batch_comparison(observations, experiment, batch_x, batch_y, rel=False, status_quo_name=None)[source]¶

Compare repeated arms from two trials; select metric via dropdown.

Parameters

• observations (List[Observation]) – List of observations to compute comparison.

• batch_x (int) – Index of batch for x-axis.

• batch_y (int) – Index of bach for y-axis.

• rel (bool) – Whether to relativize data against status_quo arm.

• status_quo_name (Optional[str]) – Name of the status_quo arm.

Return type

AxPlotConfig

ax.plot.diagnostic.interact_cross_validation(cv_results, show_context=True)[source]¶

Interactive cross-validation (CV) plotting; select metric via dropdown.

Note: uses the Plotly version of dropdown (which means that all data is stored within the notebook).

Parameters

• cv_results (List[CVResult]) – cross-validation results.

• show_context (bool) – if True, show context on hover.

Return type

AxPlotConfig

ax.plot.diagnostic.interact_empirical_model_validation(batch, data)[source]¶

Compare the model predictions for the batch arms against observed data.

Relies on the model predictions stored on the generator_runs of batch.

Parameters

• batch (BatchTrial) – Batch on which to perform analysis.

• data (Data) – Observed data for the batch.

Return type

AxPlotConfig

Returns

AxPlotConfig for the plot.

ax.plot.diagnostic.tile_cross_validation(cv_results, show_arm_details_on_hover=True, show_context=True)[source]¶

Tile version of CV plots; sorted by ‘best fitting’ outcomes.

Plots are sorted in decreasing order using the p-value of a Fisher exact test statistic.

Parameters

• cv_results (List[CVResult]) – cross-validation results.

• include_measurement_error – if True, include measurement_error metrics in plot.

• show_arm_details_on_hover (bool) – if True, display parameterizations of arms on hover. Default is True.

• show_context (bool) – if True (default), display context on hover.

Return type

AxPlotConfig

Helpers¶

ax.plot.helper.RawData = typing.List[typing.Dict[str, typing.Union[float, str]]]¶

ax.plot.helper.TNullableGeneratorRunsDict = typing.Union[typing.Dict[str, ax.core.generator_run.GeneratorRun], NoneType]¶

ax.plot.helper.arm_name_to_tuple(arm_name)[source]¶

Return type

Union[Tuple[int, int], Tuple[int]]

ax.plot.helper.axis_range(grid, is_log)[source]¶

Return type

List[float]

ax.plot.helper.contour_config_to_trace(config)[source]¶

ax.plot.helper.get_fixed_values(model, slice_values=None)[source]¶

Get fixed values for parameters in a slice plot.

If there is an in-design status quo, those values will be used. Otherwise, the mean of RangeParameters or the mode of ChoiceParameters is used.

Any value in slice_values will override the above.

Parameters

• model (ModelBridge) – ModelBridge being used for plotting

• slice_values (Optional[Dict[str, Any]]) – Map from parameter name to value at which is should be fixed.

Returns: Map from parameter name to fixed value.

Return type

Dict[str, Union[str, bool, float, int, None]]

ax.plot.helper.get_grid_for_parameter(parameter, density)[source]¶

Get a grid of points along the range of the parameter.

Will be a log-scale grid if parameter is log scale.

Parameters

• parameter (RangeParameter) – Parameter for which to generate grid.

• density (int) – Number of points in the grid.

Return type

ndarray

ax.plot.helper.get_plot_data(model, generator_runs_dict, metric_names=None, fixed_features=None)[source]¶

Format data object with metrics for in-sample and out-of-sample arms.

Calculate both observed and predicted metrics for in-sample arms. Calculate predicted metrics for out-of-sample arms passed via the generator_runs_dict argument.

In PlotData, in-sample observations are merged with IVW. In RawData, they are left un-merged and given as a list of dictionaries, one for each observation and having keys ‘arm_name’, ‘mean’, and ‘sem’.

Parameters

• model (ModelBridge) – The model.

• generator_runs_dict (Dict[str, GeneratorRun]) – a mapping from generator run name to generator run.

• metric_names (Optional[Set[str]]) – Restrict predictions to this set. If None, all metrics in the model will be returned.

• fixed_features (Optional[ObservationFeatures]) – Fixed features to use when making model predictions.

Return type

Tuple[PlotData, List[Dict[str, Union[float, str]]], Dict[str, Dict[str, Union[str, bool, float, int, None]]]]

Returns

A tuple containing

• PlotData object with in-sample and out-of-sample predictions.

• List of observations like:

  {'metric_name': 'likes', 'arm_name': '0_1', 'mean': 1., 'sem': 0.1}.
  

• Mapping from arm name to parameters.

ax.plot.helper.get_range_parameter(model, param_name)[source]¶

Get the range parameter with the given name from the model.

Throws if parameter doesn’t exist or is not a range parameter.

Parameters

• model (ModelBridge) – The model.

• param_name (str) – The name of the RangeParameter to be found.

Returns: The RangeParameter named param_name.

Return type

RangeParameter

ax.plot.helper.get_range_parameters(model)[source]¶

Get a list of range parameters from a model.

Parameters

model (ModelBridge) – The model.

Returns: List of RangeParameters.

Return type

List[RangeParameter]

ax.plot.helper.logger = <Logger PlotHelper (INFO)>¶

ax.plot.helper.relativize_data(f, sd, rel, arm_data, metric)[source]¶

Return type

List[List[float]]

ax.plot.helper.resize_subtitles(figure, size)[source]¶

ax.plot.helper.rgb(arr)[source]¶

Return type

str

ax.plot.helper.slice_config_to_trace(arm_data, arm_name_to_parameters, f, fit_data, grid, metric, param, rel, setx, sd, is_log, visible)[source]¶

Rendering¶

ax.plot.render.Ax_PLOT_REGISTRY = {<AxPlotTypes.GENERIC: 1>: 'generic_plotly.js'}¶

ax.plot.render.CSS_FILE = 'ax/ax/plot/css/base.css'¶

ax.plot.render.DEFAULT_HEIGHT = 550¶

ax.plot.render.DEFAULT_WIDTH = '100%'¶

ax.plot.render.plot_config_to_html(plot_config, plot_module_name='ax.plot', plot_resources={<AxPlotTypes.GENERIC: 1>: 'generic_plotly.js'}, inject_helpers=False)[source]¶

Generate HTML + JS corresponding from a plot config.

Return type

str

Scatter Plots¶

ax.plot.scatter.Traces = typing.List[typing.Dict[str, typing.Any]]¶

ax.plot.scatter.interact_fitted(model, generator_runs_dict=None, rel=True, show_arm_details_on_hover=True, show_CI=True, arm_noun='arm', metrics=None, fixed_features=None)[source]¶

Interactive fitted outcome plots for each arm used in fitting the model.

Choose the outcome to plot using a dropdown.

Parameters

• model (ModelBridge) – model to use for predictions.

• generator_runs_dict (Optional[Dict[str, GeneratorRun]]) – a mapping from generator run name to generator run.

• rel (bool) – if True, use relative effects. Default is True.

• show_arm_details_on_hover (bool) – if True, display parameterizations of arms on hover. Default is True.

• show_CI (bool) – if True, render confidence intervals.

• arm_noun (str) – noun to use instead of “arm” (e.g. group)

• metrics (Optional[List[str]]) – List of metric names to restrict to when plotting.

• fixed_features (Optional[ObservationFeatures]) – Fixed features to use when making model predictions.

Return type

AxPlotConfig

ax.plot.scatter.lattice_multiple_metrics(model, generator_runs_dict=None, rel=True, show_arm_details_on_hover=False)[source]¶

Plot raw values or predictions of combinations of two metrics for arms.

Parameters

• model (ModelBridge) – model to draw predictions from.

• generator_runs_dict (Optional[Dict[str, GeneratorRun]]) – a mapping from generator run name to generator run.

• rel (bool) – if True, use relative effects. Default is True.

• show_arm_details_on_hover (bool) – if True, display parameterizations of arms on hover. Default is False.

Return type

AxPlotConfig

ax.plot.scatter.plot_fitted(model, metric, generator_runs_dict=None, rel=True, custom_arm_order=None, custom_arm_order_name='Custom', show_CI=True)[source]¶

Plot fitted metrics.

Parameters

• model (ModelBridge) – model to use for predictions.

• metric (str) – metric to plot predictions for.

• generator_runs_dict (Optional[Dict[str, GeneratorRun]]) – a mapping from generator run name to generator run.

• rel (bool) – if True, use relative effects. Default is True.

• custom_arm_order (Optional[List[str]]) – a list of arm names in the order corresponding to how they should be plotted on the x-axis. If not None, this is the default ordering.

• custom_arm_order_name (str) – name for custom ordering to show in the ordering dropdown. Default is ‘Custom’.

• show_CI (bool) – if True, render confidence intervals.

Return type

AxPlotConfig

ax.plot.scatter.plot_multiple_metrics(model, metric_x, metric_y, generator_runs_dict=None, rel=True)[source]¶

Plot raw values or predictions of two metrics for arms.

All arms used in the model are included in the plot. Additional arms can be passed through the generator_runs_dict argument.

Parameters

• model (ModelBridge) – model to draw predictions from.

• metric_x (str) – metric to plot on the x-axis.

• metric_y (str) – metric to plot on the y-axis.

• generator_runs_dict (Optional[Dict[str, GeneratorRun]]) – a mapping from generator run name to generator run.

• rel (bool) – if True, use relative effects. Default is True.

Return type

AxPlotConfig

ax.plot.scatter.plot_objective_vs_constraints(model, objective, subset_metrics=None, generator_runs_dict=None, rel=True, fixed_features=None)[source]¶

Plot the tradeoff between an objetive and all other metrics in a model.

All arms used in the model are included in the plot. Additional arms can be passed through via the generator_runs_dict argument.

Fixed features input can be used to override fields of the insample arms when making model predictions.

Parameters

• model (ModelBridge) – model to draw predictions from.

• objective (str) – metric to optimize. Plotted on the x-axis.

• subset_metrics (Optional[List[str]]) – list of metrics to plot on the y-axes if need a subset of all metrics in the model.

• generator_runs_dict (Optional[Dict[str, GeneratorRun]]) – a mapping from generator run name to generator run.

• rel (bool) – if True, use relative effects. Default is True.

• fixed_features (Optional[ObservationFeatures]) – Fixed features to use when making model predictions.

Return type

AxPlotConfig

ax.plot.scatter.tile_fitted(model, generator_runs_dict=None, rel=True, show_arm_details_on_hover=False, show_CI=True, arm_noun='arm', metrics=None, fixed_features=None)[source]¶

Tile version of fitted outcome plots.

Parameters

• model (ModelBridge) – model to use for predictions.

• generator_runs_dict (Optional[Dict[str, GeneratorRun]]) – a mapping from generator run name to generator run.

• rel (bool) – if True, use relative effects. Default is True.

• show_arm_details_on_hover (bool) – if True, display parameterizations of arms on hover. Default is False.

• show_CI (bool) – if True, render confidence intervals.

• arm_noun (str) – noun to use instead of “arm” (e.g. group)

• metrics (Optional[List[str]]) – List of metric names to restrict to when plotting.

• fixed_features (Optional[ObservationFeatures]) – Fixed features to use when making model predictions.

Return type

AxPlotConfig

Slice Plot¶

ax.plot.slice.SlicePredictions = typing.Tuple[ax.plot.base.PlotData, typing.List[typing.Dict[str, typing.Union[float, str]]], typing.List[float], numpy.ndarray, numpy.ndarray, str, str, bool, typing.Dict[str, typing.Union[str, bool, float, int, NoneType]], numpy.ndarray, bool]¶

ax.plot.slice.interact_slice(model, param_name, metric_name='', generator_runs_dict=None, relative=False, density=50, slice_values=None, fixed_features=None)[source]¶

Create interactive plot with predictions for a 1-d slice of the parameter space.

Parameters

• model (ModelBridge) – ModelBridge that contains model for predictions

• param_name (str) – Name of parameter that will be sliced

• metric_name (str) – Name of metric to plot

• generator_runs_dict (Optional[Dict[str, GeneratorRun]]) – A dictionary {name: generator run} of generator runs whose arms will be plotted, if they lie in the slice.

• relative (bool) – Predictions relative to status quo

• density (int) – Number of points along slice to evaluate predictions.

• slice_values (Optional[Dict[str, Any]]) – A dictionary {name: val} for the fixed values of the other parameters. If not provided, then the status quo values will be used if there is a status quo, otherwise the mean of numeric parameters or the mode of choice parameters. Ignored if fixed_features is specified.

• fixed_features (Optional[ObservationFeatures]) – An ObservationFeatures object containing the values of features (including non-parameter features like context) to be set in the slice.

Return type

AxPlotConfig

ax.plot.slice.plot_slice(model, param_name, metric_name, generator_runs_dict=None, relative=False, density=50, slice_values=None, fixed_features=None)[source]¶

Plot predictions for a 1-d slice of the parameter space.

Parameters

• model (ModelBridge) – ModelBridge that contains model for predictions

• param_name (str) – Name of parameter that will be sliced

• metric_name (str) – Name of metric to plot

• generator_runs_dict (Optional[Dict[str, GeneratorRun]]) – A dictionary {name: generator run} of generator runs whose arms will be plotted, if they lie in the slice.

• relative (bool) – Predictions relative to status quo

• density (int) – Number of points along slice to evaluate predictions.

• slice_values (Optional[Dict[str, Any]]) – A dictionary {name: val} for the fixed values of the other parameters. If not provided, then the status quo values will be used if there is a status quo, otherwise the mean of numeric parameters or the mode of choice parameters. Ignored if fixed_features is specified.

• fixed_features (Optional[ObservationFeatures]) – An ObservationFeatures object containing the values of features (including non-parameter features like context) to be set in the slice.

Return type

AxPlotConfig