### Installing pyhf Development with All Backends (console)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/installation.rst

Installs the latest development version of the pyhf library directly from the GitHub repository, including all available backends.

```console
python -m pip install --upgrade 'pyhf[backends]@git+https://github.com/scikit-hep/pyhf.git'
```

--------------------------------

### Installing pyhf Stable with PyTorch Backend (console)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/installation.rst

Installs the latest stable release of the pyhf library from PyPI, specifying the PyTorch backend.

```console
python -m pip install 'pyhf[torch]'
```

--------------------------------

### Installing pyhf Development with PyTorch Backend (console)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/installation.rst

Installs the latest development version of the pyhf library directly from the GitHub repository, specifying the PyTorch backend.

```console
python -m pip install --upgrade 'pyhf[torch]@git+https://github.com/scikit-hep/pyhf.git'
```

--------------------------------

### Installing pyhf Stable with JAX Backend (console)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/installation.rst

Installs the latest stable release of the pyhf library from PyPI, specifying the JAX backend.

```console
python -m pip install 'pyhf[jax]'
```

--------------------------------

### Installing pyhf Development with JAX Backend (console)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/installation.rst

Installs the latest development version of the pyhf library directly from the GitHub repository, specifying the JAX backend.

```console
python -m pip install --upgrade 'pyhf[jax]@git+https://github.com/scikit-hep/pyhf.git'
```

--------------------------------

### Installing pyhf Development with XML I/O (console)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/installation.rst

Installs the latest development version of the pyhf library directly from the GitHub repository, including the optional XML import/export functionality.

```console
python -m pip install --upgrade 'pyhf[xmlio]@git+https://github.com/scikit-hep/pyhf.git'
```

--------------------------------

### Installing pyhf Development with NumPy Backend (console)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/installation.rst

Installs the latest development version of the pyhf library directly from the GitHub repository with the default NumPy backend.

```console
python -m pip install --upgrade 'pyhf@git+https://github.com/scikit-hep/pyhf.git'
```

--------------------------------

### Installing pyhf Stable with All Backends (console)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/installation.rst

Installs the latest stable release of the pyhf library from PyPI, including all available backends (NumPy, TensorFlow, PyTorch, JAX).

```console
python -m pip install 'pyhf[backends]'
```

--------------------------------

### Installing pyhf Stable with NumPy Backend (console)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/installation.rst

Installs the latest stable release of the pyhf library from PyPI with the default NumPy backend.

```console
python -m pip install pyhf
```

--------------------------------

### Installing pyhf Stable with TensorFlow Backend (console)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/installation.rst

Installs the latest stable release of the pyhf library from PyPI, specifying the TensorFlow backend.

```console
python -m pip install 'pyhf[tensorflow]'
```

--------------------------------

### Installing pyhf Development with TensorFlow Backend (console)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/installation.rst

Installs the latest development version of the pyhf library directly from the GitHub repository, specifying the TensorFlow backend.

```console
python -m pip install --upgrade 'pyhf[tensorflow]@git+https://github.com/scikit-hep/pyhf.git'
```

--------------------------------

### Installing pyhf Stable with XML I/O (console)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/installation.rst

Installs the latest stable release of the pyhf library from PyPI, including the optional XML import/export functionality.

```console
python -m pip install 'pyhf[xmlio]'
```

--------------------------------

### Activating pyhf Virtual Environment (console)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/installation.rst

Activates the previously created 'pyhf' virtual environment.

```console
source pyhf/bin/activate
```

--------------------------------

### Creating Virtual Environment for pyhf (console)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/installation.rst

Creates a Python 3 virtual environment named 'pyhf' using the `venv` module.

```console
# Python3
python3 -m venv pyhf
```

--------------------------------

### Initialize pyhf Model with Example Data - Python

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/histosys.ipynb

Defines example source data for a pyhf analysis, calls `prep_data` to build the model and prepare data, and retrieves the suggested initial parameters and parameter bounds from the model configuration. It then prints the prepared data, parameter bounds, and the order of parameters in the model.

```python
source = {
    "binning": [2, -0.5, 1.5],
    "bindata": {
        "data": [120.0, 180.0],
        "bkg": [100.0, 150.0],
        "bkgsys_up": [102, 190],
        "bkgsys_dn": [98, 100],
        "sig": [30.0, 95.0],
    },
}

d, pdf = prep_data(source)
init_pars = pdf.config.suggested_init()
par_bounds = pdf.config.suggested_bounds()

print(d), print(par_bounds), print(pdf.config.par_order)
```

--------------------------------

### Installing and Upgrading pyhf from TestPyPI (Console)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/development.rst

This snippet demonstrates how to install the standard pyhf package from PyPI and then upgrade it to a pre-release version available on TestPyPI. This is used for testing new releases before they are published to the main PyPI index. It requires pip to be installed.

```console
python -m pip install pyhf
python -m pip install --upgrade --extra-index-url https://test.pypi.org/simple/ --pre pyhf
```

--------------------------------

### Install pyhf with All Backends (Bash)

Source: https://github.com/scikit-hep/pyhf/blob/main/README.rst

Provides the command to install the pyhf package from PyPI using pip, including all optional backends. Requires pip and Python.

```bash
python -m pip install pyhf[backends]
```

--------------------------------

### Installing Git Pre-Commit Hooks (Console)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/development.rst

Installs the pre-commit hooks configured for the repository.

```console
pre-commit install
```

--------------------------------

### Initializing Parameters and Bounds

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/pytorch_tests_onoff.ipynb

Sets the initial values for parameters (mu, gamma) and their bounds for optimization, typically used as starting points for fitting procedures.

```python
init_pars = [1.0, 1.0]  # (mu, gamma)
par_bounds = [[0, 10], [0, 10]]
```

--------------------------------

### Setting up Plotting Environment with Pylab in Python

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/multichannel-normsys.ipynb

Configures the environment to use Matplotlib integrated with IPython, allowing plots to be displayed inline in notebooks. This is a common setup for interactive data analysis and visualization.

```python
%pylab inline
```

--------------------------------

### Importing Libraries and Setup (Python)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/pullplot.ipynb

Imports necessary Python libraries including pyhf, json, numpy, and matplotlib.pyplot. Sets the matplotlib backend to display plots inline, suitable for environments like Jupyter notebooks.

```python
import pyhf
import json
import numpy as np
import matplotlib.pyplot as plt

%matplotlib inline
```

--------------------------------

### Setup Matplotlib Inline in Python

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/normsys.ipynb

Configures the environment to display Matplotlib plots directly within the notebook interface.

```python
%pylab inline
```

--------------------------------

### Install Git Pre-commit Hooks - Console

Source: https://github.com/scikit-hep/pyhf/blob/main/CONTRIBUTING.md

Installs the Git pre-commit hooks configured in the project, typically used here to enforce code formatting standards like Black before commits. Requires `pre-commit` to be installed.

```console
pre-commit install
```

--------------------------------

### Defining Source Data and Initializing Model (Python)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/ShapeFactor.ipynb

Defines a dictionary 'source' containing example histogram data for signal and control regions. It calls the 'prep_data' function to build the pyhf model and prepare the data, then prints the prepared data and the expected data based on the model's suggested initial parameters.

```python
source = {
    "channels": {
        "signal": {
            "binning": [2, -0.5, 1.5],
            "bindata": {
                "data": [220.0, 230.0],
                "bkg1": [100.0, 70.0],
                "sig": [20.0, 20.0]
            }
        },
        "control": {
            "binning": [2, -0.5, 1.5],
            "bindata": {"data": [200.0, 300.0], "bkg1": [100.0, 100.0]}
        }
    }
}

data, pdf = prep_data(source['channels'])
print(f'data: {data}')

init_pars = pdf.config.suggested_init()
print(f'expected data: {pdf.expected_data(init_pars)}')

par_bounds = pdf.config.suggested_bounds()
```

--------------------------------

### Installing Development Dependencies (Console)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/development.rst

Installs pyhf in editable mode along with development dependencies using pip.

```console
python -m pip install --upgrade --editable '.[develop]'
```

--------------------------------

### Importing Libraries and Setting up Logging (Python)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/ShapeFactor.ipynb

Imports necessary libraries for pyhf modeling, data manipulation (numpy), plotting (matplotlib), and JSON handling. Sets up basic logging configuration for informational messages.

```python
import logging
import json
import numpy as np
import matplotlib.pyplot as plt

import pyhf
from pyhf.contrib.viz import brazil

logging.basicConfig(level=logging.INFO)
```

--------------------------------

### Install pyhf Development Environment - Console

Source: https://github.com/scikit-hep/pyhf/blob/main/CONTRIBUTING.md

Installs the `pyhf` project in editable mode with development dependencies, allowing for local code changes to be reflected without reinstallation. Requires `pip` and Python.

```console
python -m pip install --upgrade --editable '.[develop]'
```

--------------------------------

### Example Model Instantiation and Data Preparation (PyTorch)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/pytorch_tests_onoff.ipynb

Demonstrates how to define a model specification dictionary, instantiate the `Model` class with this specification, and prepare the combined actual and auxiliary data array required by the model's log-likelihood function.

```python
spec = {
    'obs': [55.0],
    'sig': [10.0],
    'bkg': [50.0],
    'bkgerr': [7.0],
}

pdf = Model(spec)

data = np.concatenate([spec['obs'], pdf.constraint.auxdata])
```

--------------------------------

### Listing XML Import Input Directory (Shell)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/XML_ImportExport.ipynb

Lists the contents of the directory containing the example HistFactory XML input files using the `ls` command to inspect the input structure.

```shell
!ls -lavh ../../../validation/xmlimport_input
```

--------------------------------

### Example XML to JSON Conversion (Shell)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/XML_ImportExport.ipynb

Executes a specific example of converting a HistFactory XML file (`example.xml`) to pyhf JSON using `pyhf xml2json`. The output is piped to `tee` to display it and save it to `xml_importexport.json`.

```shell
!pyhf xml2json --hide-progress ../../../validation/xmlimport_input/config/example.xml --basedir ../../../validation/xmlimport_input | tee xml_importexport.json
```

--------------------------------

### Install pyhf with NumPy Backend (Bash)

Source: https://github.com/scikit-hep/pyhf/blob/main/README.rst

Provides the command to install the pyhf package from PyPI using pip, specifically with the NumPy backend. Requires pip and Python.

```bash
python -m pip install pyhf
```

--------------------------------

### Full pyhf example demonstrating minuit_optimizer strategy bug in v0.7.2

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/release-notes/v0.7.3.rst

This comprehensive example demonstrates the bug in pyhf v0.7.2 related to the minuit_optimizer strategy and do_grad=False. It sets up a simple uncorrelated background model, performs a fit with the strategy explicitly provided (which works), and then performs another fit without explicitly providing the strategy (which failed in v0.7.2), asserting the expected strategy value.

```python
import pyhf

pyhf.set_backend("jax", pyhf.optimize.minuit_optimizer(strategy=0))

model = pyhf.simplemodels.uncorrelated_background(
    signal=[12.0, 11.0], bkg=[50.0, 52.0], bkg_uncertainty=[3.0, 7.0]
)
data = [51, 48] + model.config.auxdata

# passing with strategy kwarg explicitly given
fit_result, opt_result = pyhf.infer.mle.fit(
    data, model, return_result_obj=True, do_grad=False, strategy=0
)
minuit_strategy = opt_result.minuit.strategy.strategy
print(f"# Minuit minimization strategy: {minuit_strategy}")
assert minuit_strategy == 0

# strategy kwarg not given
fit_result, opt_result = pyhf.infer.mle.fit(
    data, model, return_result_obj=True, do_grad=False
)
minuit_strategy = opt_result.minuit.strategy.strategy
print(f"# Minuit minimization strategy: {minuit_strategy}")
assert minuit_strategy == 0  # fails for pyhf v0.7.2
```

--------------------------------

### Get pyhf Citation via CLI (Shell)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/release-notes/v0.6.0.rst

This shell command demonstrates how to use the pyhf command-line interface to retrieve the preferred BibTeX entry for citing the installed version of the pyhf software. The `--citation` or `--cite` option triggers this output.

```shell
$ pyhf --citation
@software{pyhf,
  author = {Lukas Heinrich and Matthew Feickert and Giordon Stark},
  title = "{pyhf: v0.6.0}",
  version = {0.6.0},
}
```

--------------------------------

### Filling Histograms with Sample Data

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/histogrammar.ipynb

Iterates through each generated sample dataset and fills the corresponding `histogrammar` object using the `fill` method. This populates the bins with the data points.

```python
for d in background_sample:
    background_histogram.fill(d)

for d in signal_sample:
    signal_histogram.fill(d)

for d in data_sample:
    data_histogram.fill(d)
```

--------------------------------

### Example TRExFitter YAML Configuration Snippet

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/babel.rst

Shows a basic TRExFitter configuration snippet defining the 'Job' and 'Label' parameters. These settings influence the directory structure and naming convention of the output files, including the RooWorkspace XML and ROOT files needed for conversion to pyhf.

```yaml
Job: "pyhf_example"
Label: "..."
```

--------------------------------

### Enabling Inline Plotting in IPython (IPython)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/tensorflow-limit.ipynb

Executes the IPython magic command `%pylab inline`, which imports numpy and matplotlib.pyplot and configures matplotlib to render plots directly within the notebook environment. This is a common setup step for interactive data analysis in IPython/Jupyter.

```ipython
%pylab inline
```

--------------------------------

### Installing Python 2 Compatible pyhf Version

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/faq.rst

Shows the command to install the last pyhf release (v0.3.4) that supports Python 2.7 using pip, specifying the version range with ~=0.3. This is for users who only have access to a Python 2 runtime.

```shell
python -m pip install pyhf~=0.3
```

--------------------------------

### Initializing Histograms with Histogrammar

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/histogrammar.ipynb

Initializes three `histogrammar.Bin` objects, one for each sample type (signal, background, data). Each histogram has 3 bins covering the range [-1.5, 1.5] and uses a `Count` accumulator.

```python
signal_histogram = hg.Bin(3, -1.5, 1.5, lambda d: d, hg.Count())
background_histogram = hg.Bin(3, -1.5, 1.5, lambda d: d, hg.Count())
data_histogram = hg.Bin(3, -1.5, 1.5, lambda d: d, hg.Count())
```

--------------------------------

### Downloading pyhf Model Archive (Shell)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/pullplot.ipynb

Uses the pyhf command-line utility to download a specific model archive from a HEPData DOI. The tree command is then used to display the directory structure of the downloaded content.

```shell
! pyhf contrib download https://doi.org/10.17182/hepdata.89408.v3/r2 1Lbb-probability-models
! tree 1Lbb-probability-models
```

--------------------------------

### Importing pyhf Library

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/histogrammar.ipynb

Imports the `pyhf` library, which is used for statistical modeling and inference in particle physics, specifically for constructing and analyzing HistFactory models.

```python
import pyhf
```

--------------------------------

### Importing Libraries and Configuring Logging (pyhf, numpy, matplotlib)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/multichannel-coupled-histo.ipynb

Imports necessary Python libraries for statistical modeling, numerical operations, plotting, JSON handling, and logging. Configures basic logging to display informational messages.

```python
import json
import logging

import matplotlib.pyplot as plt
import numpy as np

import pyhf
from pyhf import Model
from pyhf.contrib.viz import brazil

logging.basicConfig(level=logging.INFO)
```

--------------------------------

### Define Data and Perform pyhf Fits (Python)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/multichannel-coupled-normsys.ipynb

Sets up example source data for signal and control channels, calls prep_data to obtain the pyhf model and data, then performs unconstrained and constrained maximum likelihood fits using pyhf.unconstrained_bestfit and pyhf.constrained_bestfit, printing the resulting parameters. It also calculates expected data for the constrained fit.

```python
source = {
    "channels": {
        "signal": {
            "binning": [2, -0.5, 1.5],
            "bindata": {
                "data": [105.0, 220.0],
                "bkg1": [100.0, 100.0],
                "bkg2": [50.0, 100.0],
                "sig": [10.0, 35.0],
            },
        },
        "control": {
            "binning": [2, -0.5, 1.5],
            "bindata": {
                "data": [110.0, 105.0],
                "bkg1": [100.0, 100.0]
            },
        },
    }
}


d, pdf = prep_data(source['channels'])

print(d)

init_pars = pdf.config.suggested_init()
par_bounds = pdf.config.suggested_bounds()

unconpars = pyhf.unconstrained_bestfit(d, pdf, init_pars, par_bounds)
print('UNCON', unconpars)

conpars = pyhf.constrained_bestfit(0.0, d, pdf, init_pars, par_bounds)
print('CONS', conpars)

pdf.expected_data(conpars)
```

--------------------------------

### Creating pyhf Model and Data

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/histogrammar.ipynb

Instantiates a `pyhf.Model` object `p` using the defined `spec` and setting "mu" as the parameter of interest. It then prepares the observed data by combining the data histogram's bin values with the auxiliary data required by the model configuration.

```python
p = pyhf.Model(spec, poi_name="mu")

data = data_histogram.toJson()['data']['values'] + p.config.auxdata
```

--------------------------------

### Calculate Observed CLs and Tail Probabilities using pyhf Python

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/hello-world.ipynb

Building upon the previous setup, this snippet demonstrates how to use the return_tail_probs=True option in pyhf.infer.hypotest to obtain the observed CLs along with the p-values for the signal-plus-background (CLs+b) and background-only (CLb) hypotheses.

```python
CLs_obs, p_values = pyhf.infer.hypotest(
    test_mu, data, model, test_stat="qtilde", return_tail_probs=True
)
print(f"Observed CL_s: {CLs_obs}, CL_sb: {p_values[0]}, CL_b: {p_values[1]}")
```

--------------------------------

### Get Suggested Initial Parameters and Expected Data

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/importxml.ipynb

Retrieves the model's suggested initial parameter values and calculates the expected data distribution based on these initial parameters.

```python
print(pdf.config.suggested_init())
pdf.expected_actualdata(pdf.config.suggested_init())
```

--------------------------------

### Example JSON to XML+ROOT Conversion (Shell)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/XML_ImportExport.ipynb

Demonstrates the process of converting the previously generated pyhf JSON file (`xml_importexport.json`) back to HistFactory XML+ROOT. It first creates an output directory and then runs the `pyhf json2xml` command, finally listing the generated files.

```shell
!mkdir -p output
!pyhf json2xml xml_importexport.json --output-dir output
!ls -lavh output/*
```

--------------------------------

### Enable Matplotlib Integration (Python)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/multichannel-coupled-normsys.ipynb

Configures the environment to display matplotlib plots inline within a Jupyter notebook or similar environment.

```python
%pylab inline
```

--------------------------------

### Generating Sample Datasets with NumPy

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/histogrammar.ipynb

Creates three synthetic datasets (`background_sample`, `signal_sample`, `data_sample`) using NumPy's random normal distribution function. These simulate experimental data for histogramming.

```python
background_sample = np.random.normal(loc=0.0, scale=1.0, size=30)
signal_sample = np.random.normal(loc=0.5, scale=1.0, size=15)
data_sample = np.random.normal(loc=0.1, scale=1.0, size=45)
```

--------------------------------

### Importing Libraries for Histogrammar and NumPy

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/histogrammar.ipynb

Imports the necessary libraries: `pylab` for inline plotting (common in notebooks), `histogrammar` as `hg` for histogram creation, and `numpy` as `np` for numerical operations and data generation.

```python
%pylab inline
import histogrammar as hg
import numpy as np
```

--------------------------------

### Creating pyhf Model and Workspace (Python)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/pullplot.ipynb

Defines a function `make_model` that loads a pyhf model specification from a JSON file, filters the channels based on the provided list, sets the parameter of interest, and returns the pyhf workspace, model, and data object.

```python
def make_model(channel_list):
    with open(
        "1Lbb-probability-models/RegionA/BkgOnly.json", encoding="utf-8"
    ) as spec_file:
        spec = json.load(spec_file)
    spec["channels"] = [c for c in spec["channels"] if c["name"] in channel_list]
    spec["measurements"][0]["config"]["poi"] = "lumi"

    ws = pyhf.Workspace(spec)
    model = ws.model(
        measurement_name="NormalMeasurement",
        modifier_settings={
            "normsys": {"interpcode": "code4"},
            "histosys": {"interpcode": "code4p"},
        },
    )
    data = ws.data(model)
    return ws, model, data
```

--------------------------------

### Performing Unconstrained Maximum Likelihood Fit (Python)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/ShapeFactor.ipynb

Prints the suggested initial parameters from the model configuration. It then performs an unconstrained maximum likelihood fit of the pyhf model to the prepared data using pyhf.infer.mle.fit and prints the resulting fitted parameters.

```python
print(f'initialization parameters: {pdf.config.suggested_init()}')

unconpars = pyhf.infer.mle.fit(data, pdf)
print(f'parameters post unconstrained fit: {unconpars}')
```

--------------------------------

### Accessing Test Data Files (Python)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/development.rst

Example Python test function demonstrating how to use the `datadir` fixture to access test data files copied to a temporary directory.

```python
def test_patchset(datadir):
    data_file = open(datadir.join("test.txt"), encoding="utf-8")
    ...
```

--------------------------------

### Perform pyhf Fits and Generate Data in Python

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/normsys.ipynb

Initializes data and model parameters, performs unconstrained and constrained maximum likelihood fits using pyhf, and generates expected data and Asimov data based on the fit results.

```python
source = {
    "binning": [2, -0.5, 1.5],
    "bindata": {
        "data": [120.0, 180.0],
        "bkg": [100.0, 150.0],
        "bkgerr": [15.0, 20.0],
        "sig": [30.0, 95.0]
    }
}
d, pdf = prep_data(
    source['bindata']['data'], source['bindata']['bkg'], source['bindata']['sig']
)

init_pars = [0.0, 0.0]
par_bounds = [[0, 10], [-5, 5]]

unconpars = pyhf.unconstrained_bestfit(d, pdf, init_pars, par_bounds)
print('UNCON', unconpars)


conpars = pyhf.constrained_bestfit(0.0, d, pdf, init_pars, par_bounds)
print('CONS', conpars)


print(pdf.expected_data(conpars))

# print '????',aux
aux = pdf.expected_auxdata(conpars)
# print '????',aux

print('ASIMOV', pyhf.generate_asimov_data(0.0, d, pdf, init_pars, par_bounds))
```

--------------------------------

### Getting Test Statistic Distributions (Python)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/learn/UsingCalculators.ipynb

Retrieves the probability distributions of the test statistic under both the signal+background (sb) and background-only (b) hypotheses for a parameter of interest (POI) value of 1.0 using the AsymptoticCalculator. These distributions are necessary for calculating p-values.

```python
sb_dist, b_dist = asymp_calc.distributions(poi_test=1.0)
```

--------------------------------

### Prepare pyhf Model and Data (Python)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/multichannel-coupled-normsys.ipynb

Defines the prep_data function which takes source data, constructs a pyhf model specification with signal and control channels, including normfactor and coupled normsys modifiers, and returns the combined data and the constructed pyhf.Model object.

```python
import pyhf
import logging

logging.basicConfig(level=logging.INFO)
from pyhf import Model


def prep_data(sourcedata):
    spec = {
        'signal': {
            'signal': {
                'data': sourcedata['signal']['bindata']['sig'],
                'mods': [{'name': 'mu', 'type': 'normfactor', 'data': None}],
            },
            'bkg1': {
                'data': sourcedata['signal']['bindata']['bkg1'],
                'mods': [
                    {
                        'name': 'coupled_normsys',
                        'type': 'normsys',
                        'data': {'lo': 0.9, 'hi': 1.1},
                    }
                ],
            },
            'bkg2': {
                'data': sourcedata['signal']['bindata']['bkg2'],
                'mods': [
                    {
                        'name': 'coupled_normsys',
                        'type': 'normsys',
                        'data': {'lo': 0.5, 'hi': 1.5},
                    }
                ],
            },
        },
        'control': {
            'background': {
                'data': sourcedata['control']['bindata']['bkg1'],
                'mods': [
                    {
                        'name': 'coupled_normsys',
                        'type': 'normsys',
                        'data': {'lo': 0.9, 'hi': 1.1},
                    }
                ],
            }
        },
    }
    pdf = Model(spec, poi_name="mu")
    data = []
    for c in pdf.config.channel_order:
        data += sourcedata[c]['bindata']['data']
    data = data + pdf.config.auxdata
    return data, pdf
```

--------------------------------

### Preparing pyhf Model and Data (Python)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/ShapeFactor.ipynb

Defines the prep_data function to construct a pyhf model specification with signal and control channels, including normfactor and coupled shapefactor modifiers. It then creates a pyhf.Model instance and extracts the combined observed data and auxiliary data from the source dictionary.

```python
def prep_data(sourcedata):
    spec = {
        'channels': [
            {
                'name': 'signal',
                'samples': [
                    {
                        'name': 'signal',
                        'data': sourcedata['signal']['bindata']['sig'],
                        'modifiers': [
                            {'name': 'mu', 'type': 'normfactor', 'data': None}
                        ]
                    },
                    {
                        'name': 'bkg1',
                        'data': sourcedata['signal']['bindata']['bkg1'],
                        'modifiers': [
                            {
                                'name': 'coupled_shapefactor',
                                'type': 'shapefactor',
                                'data': None
                            }
                        ]
                    }
                ]
            },
            {
                'name': 'control',
                'samples': [
                    {
                        'name': 'background',
                        'data': sourcedata['control']['bindata']['bkg1'],
                        'modifiers': [
                            {
                                'name': 'coupled_shapefactor',
                                'type': 'shapefactor',
                                'data': None
                            }
                        ]
                    }
                ]
            }
        ]
    }
    pdf = pyhf.Model(spec, poi_name="mu")
    data = []
    for channel in pdf.config.channels:
        data += sourcedata[channel]['bindata']['data']
    data = data + pdf.config.auxdata
    return data, pdf
```

--------------------------------

### Initializing AsymptoticCalculator (Python)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/learn/UsingCalculators.ipynb

Creates an instance of the AsymptoticCalculator from pyhf.infer.calculators. It is initialized with the prepared data and model, specifying the use of the 'qtilde' test statistic for asymptotic hypothesis tests.

```python
asymp_calc = pyhf.infer.calculators.AsymptoticCalculator(
    data, model, test_stat='qtilde'
)
```

--------------------------------

### Define pyhf Model and Prepare Data (HistoSys) - Python

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/histosys.ipynb

Defines the `prep_data` function which constructs a pyhf model specification for a single channel, including a signal with a normfactor and a background with a histosys uncertainty. It creates the `pyhf.Model` object and combines observed data with auxiliary data for analysis.

```python
import pyhf
from pyhf import Model


def prep_data(source):
    spec = {
        'singlechannel': {
            'signal': {
                'data': source['bindata']['sig'],
                'mods': [{'name': 'mu', 'type': 'normfactor', 'data': None}],
            },
            'background': {
                'data': source['bindata']['bkg'],
                'mods': [
                    {
                        'name': 'bkg_norm',
                        'type': 'histosys',
                        'data': {
                            'lo_hist': source['bindata']['bkgsys_dn'],
                            'hi_hist': source['bindata']['bkgsys_up'],
                        },
                    }
                ],
            },
        }
    }
    pdf = Model(spec, poi_name="mu")
    data = source['bindata']['data'] + pdf.config.auxdata
    return data, pdf
```

--------------------------------

### Defining HistFactory Model Specification

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/histogrammar.ipynb

Creates a dictionary `spec` defining a simple HistFactory model for a single channel. It includes signal and background components, using the histogram bin values as data and specifying modifiers like a signal strength `normfactor` and a background normalization `normsys` with 10% uncertainty.

```python
spec = {
    'singlechannel': {
        'signal': {
            'data': signal_histogram.toJson()['data']['values'],
            'mods': [{'name': 'mu', 'type': 'normfactor', 'data': None}],
        },
        'background': {
            'data': background_histogram.toJson()['data']['values'],
            'mods': [
                {
                    'name': 'bkg_norm',
                    'type': 'normsys',
                    'data': {'lo': 0.90, 'hi': 1.10}
                }
            ]
        }
    }
}
```

--------------------------------

### Getting Parameter Slice for Modifier (Commented)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/StatError.ipynb

This commented-out line shows how to obtain the slice of the parameter vector corresponding to the 'stat_firstchannel' modifier from the pyhf model's configuration using the 'par_slice' method. It is not executed but demonstrates the syntax.

```python
# p.config.par_slice('stat_firstchannel')
```

--------------------------------

### Getting the pyhf Tensor Library

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/StatError.ipynb

Retrieves the underlying tensor library used by pyhf (e.g., NumPy, TensorFlow, PyTorch) and assigns it to the variable 'tensorlib'. This allows for direct use of tensor operations provided by the chosen backend.

```python
tensorlib = pyhf.tensorlib
```

--------------------------------

### Building Documentation (Console)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/development.rst

Builds the project documentation using the nox docs session.

```console
nox --session docs
```

--------------------------------

### Calculating Relative Uncertainties from StatError

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/StatError.ipynb

Uses the pyhf tensor library ('tensorlib') to calculate the relative uncertainties associated with the 'staterror' modifier ('se'). It computes the quadratic sum of uncertainties, sums nominal counts, and then divides the quadratic sum by the totals to get relative uncertainties. This snippet requires 'se' to be defined, likely from the commented-out snippet 3.

```python
inquad = tensorlib.sqrt(tensorlib.sum(tensorlib.power(se.uncertainties, 2), axis=0))
totals = tensorlib.sum(se.nominal_counts, axis=0)
uncrts = tensorlib.divide(inquad, totals)
uncrts
```

--------------------------------

### Initializing pyhf ToyCalculator (Python)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/learn/UsingCalculators.ipynb

Creates an instance of the ToyCalculator for performing inference using toy datasets. Specifies the data, model, test statistic type ('qtilde'), and the number of toys to generate (500). Requires 'data' and 'model' objects to be defined.

```python
toy_calc = pyhf.infer.calculators.ToyCalculator(
    data, model, test_stat='qtilde', ntoys=500
)
```

--------------------------------

### Setting up pyhf Model and Calculating Log-Likelihood (TensorFlow Backend)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/example-tensorflow.ipynb

Defines a simple uncorrelated background model source, creates a pyhf model, prepares the data, sets initial parameters and bounds, switches pyhf's tensor backend to TensorFlow, calculates the log-likelihood, and writes the TensorFlow graph for visualization.

```python
source = {
    "binning": [2, -0.5, 1.5],
    "bindata": {
        "data": [120.0, 180.0],
        "bkg": [100.0, 150.0],
        "bkgerr": [10.0, 10.0],
        "sig": [30.0, 95.0]
    }
}

pdf = uncorrelated_background(
    source['bindata']['sig'], source['bindata']['bkg'], source['bindata']['bkgerr']
)
data = source['bindata']['data'] + pdf.config.auxdata

init_pars = pdf.config.suggested_init()
par_bounds = pdf.config.suggested_bounds()


print('---\nas tensorflow\n-----')
import tensorflow as tf

pyhf.tensorlib = pyhf.tensorflow_backend()
v = pdf.logpdf(init_pars, data)

pyhf.tensorlib.session = tf.Session()
print(type(v), pyhf.tensorlib.tolist(v))


from pathlib import Path

tf.summary.FileWriter(Path.cwd(), pyhf.tensorlib.session.graph)
```

--------------------------------

### Serving Built Documentation Locally (Console)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/development.rst

Builds the documentation and serves it locally for viewing in a web browser.

```console
nox --session docs -- serve
```

--------------------------------

### Defining Data, Building Model, and Performing Fits with pyhf in Python

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/multichannel-normsys.ipynb

Defines sample source data for signal and control regions, including observed data, background, background uncertainties, and signal. It calls `prep_data` to build the `pyhf` model and prepare the data. It then demonstrates accessing suggested initial parameters and bounds, printing the initial PDF value, performing an unconstrained best-fit, and performing a constrained best-fit at a signal strength of 0.0.

```python
source = {
    "channels": {
        "signal": {
            "binning": [2, -0.5, 1.5],
            "bindata": {
                "data": [110.0, 155.0],
                "bkgerr": [10.0, 10.0],
                "bkg": [100.0, 150.0],
                "sig": [10.0, 35.0],
            },
        },
        "control": {
            "binning": [2, -0.5, 1.5],
            "bindata": {
                "data": [205.0, 345.0],
                "bkg": [200.0, 350.0],
                "bkgerr": [5.0, 10.0],
            },
        },
    }
}

d, pdf = prep_data(source['channels'])

print(d)

init_pars = pdf.config.suggested_init()
par_bounds = pdf.config.suggested_bounds()


print(pdf.pdf(init_pars, d))

unconpars = pyhf.unconstrained_bestfit(d, pdf, init_pars, par_bounds)
print('UNCON', unconpars)


# print d
# print pdf.expected_data(unconpars)


conpars = pyhf.constrained_bestfit(0.0, d, pdf, init_pars, par_bounds)
print('CONS', conpars)


# print pdf.expected_data(conpars)

# # print '????',aux
# aux = pdf.expected_auxdata(conpars)
# # print '????',aux

# print 'ASIMOV',pyhf.generate_asimov_data(0.0,d,pdf,init_pars,par_bounds)
```

--------------------------------

### Initializing Environment and Dependencies - Python

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/learn/TestStatistics.ipynb

Imports necessary libraries (numpy, matplotlib, pyhf) and sets up random seed and plot parameters for reproducibility and visualization.

```python
import numpy as np
import matplotlib.pyplot as plt
import pyhf

np.random.seed(0)
plt.rcParams.update({"font.size": 14})
```

--------------------------------

### Importing pyhf with XML I/O support (Python)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/XML_ImportExport.ipynb

Imports the pyhf library. Note that the `xmlio` extra must be installed (e.g., `python -m pip install pyhf[xmlio]`) to enable XML functionality.

```python
# NB: python -m pip install pyhf[xmlio]
import pyhf
```

--------------------------------

### Running pyhf Limit Setting Analysis with TensorFlow (Python)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/tensorflow-limit.ipynb

Demonstrates a complete pyhf analysis workflow using the TensorFlow backend. It defines a simple statistical model, prepares input data, iterates through a range of signal strength values (`mutests`), runs the `runOnePoint` analysis for each, and collects the observed and expected confidence level values for subsequent processing and visualization.

```python
pdf = uncorrelated_background([10.0], [50.0], [7.0])
data = [55.0] + pdf.config.auxdata

init_pars = pdf.config.suggested_init()
par_bounds = pdf.config.suggested_bounds()

mutests = np.linspace(0, 5, 11)
tests = [
    runOnePoint(muTest, data, pdf, init_pars, par_bounds)[-2:] for muTest in mutests
]
cls_obs = [test[0] for test in tests]
cls_exp = [[test[1][i] for test in tests] for i in range(5)]

plot_results(mutests, cls_obs, cls_exp)

invert_interval(mutests, cls_obs, cls_exp)
```

--------------------------------

### Dumping HistFitter Workspace and XML for pyhf

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/babel.rst

Command to run HistFitter with flags (-w and -x) specifically to generate the HistFactory workspace (.root) and corresponding XML files, while skipping the fitting and plotting steps. These output files are necessary for conversion to pyhf.

```bash
HistFitter.py -wx -F excl config.py
```

--------------------------------

### Example HistFitter Python Configuration Snippet

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/babel.rst

Illustrates how key parameters such as the analysis name, histogram cache file location, fit configuration prefix, channel names, and measurement names are defined within a HistFitter config.py file. These definitions determine the structure and naming of the output files required for pyhf conversion.

```python
from configManager import configMgr

# ...
configMgr.analysisName = "3b_tag21.2.27-1_RW_ExpSyst_36100_multibin_bkg"
configMgr.histCacheFile = f"cache/{configMgr.analysisName:s}.root"
# ...
fitConfig = configMgr.addFitConfig("Excl")
# ...
channel = fitConfig.addChannel("cuts", ["SR_0L"], 1, 0.5, 1.5)
# ...
meas1 = fitConfig.addMeasurement(name="DefaultMeasurement", lumi=1.0, lumiErr=0.029)
meas1.addPOI("mu_SIG1")
# ...
meas2 = fitConfig.addMeasurement(name="DefaultMeasurement", lumi=1.0, lumiErr=0.029)
meas2.addPOI("mu_SIG2")
```

--------------------------------

### Defining a histosys modifier (2-bin channel) in JSON

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/likelihood.rst

Example of a histosys modifier definition for a 2-bin channel with absolute shape variations for 'hi_data' and 'lo_data'.

```json
{ "name": "mod_name", "type": "histosys", "data": {"hi_data": [20,15], "lo_data": [10, 10]} }
```

--------------------------------

### Cleaning Up Generated Files (Shell)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/XML_ImportExport.ipynb

Removes the temporary pyhf JSON file (`xml_importexport.json`) and the output directory (`output`) created during the import/export examples to clean up the workspace.

```shell
!rm xml_importexport.json
!rm -rf output/
```

--------------------------------

### Running Standard HistFitter Exclusion Fit

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/babel.rst

Shows a typical command used to execute a HistFitter exclusion fit, including options to generate plots and correlation matrices.

```bash
HistFitter.py -f -D "before,after,corrMatrix" -F excl config.py
```

--------------------------------

### Defining a staterror modifier (single bin) in JSON

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/likelihood.rst

Example of a staterror modifier definition for a single bin channel, specifying the scale factor for the statistical uncertainty.

```json
{ "name": "mod_name", "type": "staterror", "data": [0.1] }
```

--------------------------------

### Defining a normsys modifier in JSON

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/likelihood.rst

Example of a normsys modifier definition specifying scale factors for the upward ('hi') and downward ('lo') variations of a normalization uncertainty.

```json
{ "name": "mod_name", "type": "normsys", "data": {"hi": 1.1, "lo": 0.9} }
```

--------------------------------

### Printing Histogram Bin Values

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/histogrammar.ipynb

Converts each `histogrammar` object to its JSON representation using `toJson()`, extracts the bin values from the nested dictionary structure, and prints them to the console.

```python
print(background_histogram.toJson()['data']['values'])
print(signal_histogram.toJson()['data']['values'])
print(data_histogram.toJson()['data']['values'])
```

--------------------------------

### Loading Data and Performing MLE Fits (pyhf, Python)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/multichannel-coupled-histo.ipynb

Opens and loads a JSON workspace specification file from the defined validation data directory. Calls `prep_data` to obtain the data and model. Performs an unconstrained Maximum Likelihood Estimate (MLE) fit and a constrained MLE fit with the parameter of interest (POI) fixed at 0.0 using `pyhf.infer.mle`. Prints the resulting parameter values from both fits.

```python
with open(
    validation_datadir + "/2bin_2channel_coupledhisto.json", encoding="utf-8"
) as spec:
    source = json.load(spec)

data, pdf = prep_data(source["channels"])

print(f"data: {data}")

init_pars = pdf.config.suggested_init()
par_bounds = pdf.config.suggested_bounds()

unconpars = pyhf.infer.mle.fit(data, pdf, init_pars, par_bounds)
print(f"parameters post unconstrained fit: {unconpars}")

conpars = pyhf.infer.mle.fixed_poi_fit(0.0, data, pdf, init_pars, par_bounds)
print(f"parameters post constrained fit: {conpars}")

pdf.expected_data(conpars)
```

--------------------------------

### Defining a shapesys modifier (3-bin channel) in JSON

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/likelihood.rst

Example of a shapesys modifier definition for a 3-bin channel with non-zero data values, allocating three nuisance parameters.

```json
{ "name": "mod_name", "type": "shapesys", "data": [1.0, 1.5, 2.0] }
```

--------------------------------

### Defining Validation Data Directory (Python)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/multichannel-coupled-histo.ipynb

Sets a string variable `validation_datadir` to the relative path where validation data files, such as the workspace specification JSON, are located.

```python
validation_datadir = "../../../validation/data"
```

--------------------------------

### Resolving uproot Dependency Conflict with pyhf v0.5.4

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/release-notes/v0.5.4.rst

Illustrates how pyhf v0.5.4 resolves the uproot dependency conflict by requiring uproot3, allowing uproot4 to be installed without conflict.

```shell
$ python -m pip install "pyhf[xmlio]>=0.5.4"
$ python -m pip list | grep "pyhf\|uproot"
pyhf            0.5.4
uproot3         3.14.1
uproot3-methods 0.10.0
$ python -m pip install uproot4 # or uproot
$ python -m pip list | grep "pyhf\|uproot"
pyhf            0.5.4
uproot          4.0.0
uproot3         3.14.1
uproot3-methods 0.10.0
uproot4         4.0.0
```

--------------------------------

### Old Interpolation Loop Example (Step 2) - Python

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/learn/TensorizingInterpolations.ipynb

This snippet shows the structure of the loop from the original or Step 1 implementation, where delta variations are calculated inside the loop over histograms.

```python
for histo in histoset:
    delta_up = histo[2] - histo[1]
...
```

--------------------------------

### Defining pyhf Model and Data (Python)

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/learn/UsingCalculators.ipynb

Creates a simple uncorrelated background model using pyhf.simplemodels with specified observed data, background expectation, and background uncertainty. Prepares the full dataset by combining the observed data with the model's auxiliary data.

```python
model = pyhf.simplemodels.uncorrelated_background([6], [9], [3])
data = [9] + model.config.auxdata
```

--------------------------------

### Importing Libraries for pyhf

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/StatError.ipynb

Imports the necessary libraries for working with pyhf, including the main pyhf library and importlib (though importlib is not directly used in the subsequent snippets shown).

```python
import pyhf
import importlib
```

--------------------------------

### Import Libraries and Setup Plotting - Python

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/Recast.ipynb

Imports necessary libraries for pyhf, JSON patching, and plotting, and configures matplotlib for inline plotting in a notebook environment.

```python
import jsonpatch
import pyhf
from pyhf.contrib.viz import brazil

%pylab inline
```

--------------------------------

### Calculate Observed and Expected CLs using pyhf Python

Source: https://github.com/scikit-hep/pyhf/blob/main/docs/examples/notebooks/hello-world.ipynb

This snippet sets up a simple uncorrelated background model with signal and background uncertainties, defines observed data, and performs a hypothesis test using pyhf.infer.hypotest to calculate both the observed and expected CLs values for a given test signal strength (test_mu).

```python
import pyhf
```

```python
model = pyhf.simplemodels.uncorrelated_background(
    signal=[12.0, 11.0], bkg=[50.0, 52.0], bkg_uncertainty=[3.0, 7.0]
)
data = [51, 48] + model.config.auxdata
test_mu = 1.0
CLs_obs, CLs_exp = pyhf.infer.hypotest(
    test_mu, data, model, test_stat="qtilde", return_expected=True
)
print(f"Observed: {CLs_obs}, Expected: {CLs_exp}")
```