### Initialize GluePromptOpt for Synthetic Data Generation

Source: https://github.com/microsoft/promptwizard/blob/main/demos/scenarios/dataset_scenarios_demo.ipynb

Initializes the GluePromptOpt object with configuration paths, preparing it for generating synthetic examples. The dataset_jsonl and data_processor are set to None for this initial setup.

```python
gp = GluePromptOpt(promptopt_config_path,
                   setup_config_path,
                   dataset_jsonl=None,
                   data_processor=None)
```

--------------------------------

### Clone Repository and Install

Source: https://github.com/microsoft/promptwizard/blob/main/README.md

Steps to clone the PromptWizard repository and install the package in development mode. This involves cloning the git repository, creating and activating a virtual environment, and then installing the package using pip.

```bash
git clone https://github.com/microsoft/PromptWizard
cd PromptWizard

# On Windows
python -m venv venv
venv\Scripts\activate

# On macOS/Linux:
python -m venv venv
source venv/bin/activate

pip install -e .
```

--------------------------------

### Get Best Prompt with Examples

Source: https://github.com/microsoft/promptwizard/blob/main/demos/scenarios/dataset_scenarios_demo.ipynb

Calls the get_best_prompt method of the GluePromptOpt object to obtain the optimized prompt and expert profile. It can utilize in-context examples and control the generation of synthetic examples.

```python
best_prompt, expert_profile = gp.get_best_prompt(use_examples=True,run_without_train_examples=False,generate_synthetic_examples=False)
```

--------------------------------

### Generate Synthetic Training Examples

Source: https://github.com/microsoft/promptwizard/blob/main/demos/scenarios/dataset_scenarios_demo.ipynb

Calls the get_best_prompt method on the GluePromptOpt object to generate synthetic examples. It's configured to run without existing training examples and to generate new synthetic ones.

```python
best_prompt, expert_profile = gp.get_best_prompt(use_examples=False,run_without_train_examples=False,generate_synthetic_examples=True)
```

--------------------------------

### Accessing Answer Function Example

Source: https://github.com/microsoft/promptwizard/blob/main/README.md

An example function demonstrating how to access and compare answers, particularly for datasets where the final answer is a number or a sentence/paragraph.

```python
def access_answer():
    # Example for numerical answers
    generated_num = 123
    ground_truth_num = 123
    if generated_num == ground_truth_num:
        print("Numerical answer matches.")

    # Example for sentence/paragraph answers (requires llm-as-a-judge)
    generated_text = "The quick brown fox jumps over the lazy dog."
    ground_truth_text = "A fast brown fox leaps over a sleepy dog."
    # In a real scenario, you would use an LLM to compare these texts.
    print("Text comparison would require LLM-as-a-judge.")
```

--------------------------------

### Prompt Optimization Configuration Example

Source: https://github.com/microsoft/promptwizard/blob/main/README.md

Example configuration file for PromptWizard, specifically for the GSM8k dataset. This file (`promptopt_config.yaml`) is used to set various configurations for the prompt optimization process.

```yaml
# Example promptopt_config.yaml for GSM8k dataset
# This file would contain specific configurations for prompt optimization.
```

--------------------------------

### Running Prompt Optimization

Source: https://github.com/microsoft/promptwizard/blob/main/demos/bbh/demo.ipynb

Executes the prompt optimization process using the GluePromptOpt object. It allows configuration of whether to use existing examples, generate synthetic examples, or run without training examples.

```python
# Function call to generate optimal prompt and expert profile 
# best_prompt, expert_profile = gp.get_best_prompt(use_examples=True,run_without_train_examples=False,generate_synthetic_examples=False)

```

--------------------------------

### Initialize GluePromptOpt for Prompt Optimization

Source: https://github.com/microsoft/promptwizard/blob/main/demos/aquarat/demo.ipynb

Creates an instance of the `GluePromptOpt` class, passing the paths to the prompt optimization configuration, setup configuration, training data file, and the dataset processor.

```python
gp = GluePromptOpt(promptopt_config_path,
                   setup_config_path,
                   train_file_name,
                   aquarat_processor)
```

--------------------------------

### Define File Paths for Configuration and Data

Source: https://github.com/microsoft/promptwizard/blob/main/demos/aquarat/demo.ipynb

Sets up the file paths for the training and testing data files, as well as configuration files required for prompt optimization and setup.

```python
train_file_name = os.path.join("data", "train.jsonl")
test_file_name = os.path.join("data", "test.jsonl")
path_to_config = "configs"
llm_config_path = os.path.join(path_to_config, "llm_config.yaml")
promptopt_config_path = os.path.join(path_to_config, "promptopt_config.yaml")
setup_config_path = os.path.join(path_to_config, "setup_config.yaml")
```

--------------------------------

### Load and Save Dataset (GSM8K)

Source: https://github.com/microsoft/promptwizard/blob/main/demos/scenarios/dataset_scenarios_demo.ipynb

Loads the GSM8K dataset from Hugging Face, processes it into a list of question-answer pairs, and saves the first 100 training examples and all test examples to JSONL files. Includes a limit for training samples.

```python
if not os.path.exists("data"):
    os.mkdir("data")
    
dataset = load_dataset("openai/gsm8k", "main")
num_samples = 0
for dataset_type in ['train','test']:
    data_list = []
    for data in dataset[dataset_type]:
        data_list.append({"question": data['question'], "answer": data['answer']})
        if num_samples == 100 and dataset_type == 'train': # We sample only 100 train examples and use 25 out them for training randomly
            break
        num_samples += 1
    gsm8k_processor.dataset_to_jsonl("data/"+ dataset_type+'.jsonl', dataset=data_list)
```

--------------------------------

### Configure Synthetic Training Examples

Source: https://github.com/microsoft/promptwizard/blob/main/demos/scenarios/dataset_scenarios_demo.ipynb

Sets the number of synthetic training examples to be generated by updating a configuration dictionary and saving it to a YAML file.

```python
config_dict = {
                "num_train_examples":20
               }
update_yaml_file(file_path,config_dict)
```

--------------------------------

### Example Output of Prompt Variations

Source: https://github.com/microsoft/promptwizard/blob/main/demos/scenarios/dataset_scenarios_demo.ipynb

Illustrates the output format when the prompt optimization process generates multiple variations. Each variation includes an 'Expert Profile' and a 'Prompt' with associated keywords and instructions.

```python
OUTPUT = """
Variations 1:
Expert Profile:
You are a mathematician with a strong background in various fields of mathematics, including algebra, calculus, geometry, and statistics. You have a deep understanding of mathematical theories and principles, and you are skilled at solving complex problems with precision and clarity. Your expertise allows you to approach mathematical problems methodically, breaking them down into manageable steps and applying appropriate techniques to find solutions. You are familiar with both theoretical and applied mathematics, and you can explain your reasoning and solutions in a clear and concise manner. Your ability to solve mathematical problems efficiently and accurately makes you an invaluable resource for anyone seeking help with mathematics.:
Prompt:
You are a mathematics expert. You will be given a mathematics problem which you need to solve
Lets think step by step.


For each question present the reasoning followed by the correct answer.
Keywords: mathematics, problem-solving, step-by-step, logical reasoning, expert
_______________________________________________________________________ 

Variations 2:
Expert Profile:
You are a mathematician with a strong background in various fields of mathematics, including algebra, calculus, geometry, and statistics. You have a deep understanding of mathematical theories and principles, and you are skilled at solving complex problems with precision and clarity. Your expertise allows you to approach mathematical problems methodically, breaking them down into manageable steps and applying appropriate techniques to find solutions. You are familiar with both theoretical and applied mathematics, and you can explain your reasoning and solutions in a clear and concise manner. Your ability to solve mathematical problems efficiently and accurately makes you an invaluable resource for anyone seeking help with mathematics.:
Prompt:
Let's break this problem down step by step and devise an experiment to help solve it.


For each question present the reasoning followed by the correct answer.
Keywords: mathematics, problem-solving, step-by-step, logical reasoning, expert
_______________________________________________________________________ 

Variations 3:
Expert Profile:
You are a mathematics expert with a strong background in various fields of mathematics, including algebra, calculus, geometry, and statistics. You have a deep understanding of mathematical theories and principles, and you are skilled at solving complex problems with precision and clarity. Your expertise allows you to break down intricate problems into manageable steps, making it easier to find solutions. You are familiar with a wide range of mathematical techniques and tools, and you can apply them effectively to solve problems. Whether the problem involves solving equations, proving theorems, or analyzing data, you can provide a clear and accurate solution. Your ability to explain your reasoning and methodology ensures that others can follow and understand your approach, making you an invaluable resource for tackling challenging mathematical problems.:
Prompt:
Let's think through this problem step by step and make a list of ideas to solve it.


For each question present the reasoning followed by the correct answer.
Keywords: mathematics, problem-solving, step-by-step, logical reasoning, expert
_______________________________________________________________________ 

Variations 4:
Expert Profile:
"""
```

--------------------------------

### Bulma Layout: Hero Example

Source: https://github.com/microsoft/promptwizard/blob/main/docs/static/css/bulma.css.map.txt

Shows how to create a hero section with Bulma.

```html
<section class="hero is-primary">
  <div class="hero-body">
    <p class="title">
      Primary hero
    </p>
    <p class="subtitle">
      A simple subtitle
    </p>
  </div>
</section>
```

--------------------------------

### Set Configuration Paths

Source: https://github.com/microsoft/promptwizard/blob/main/demos/scenarios/dataset_scenarios_demo.ipynb

Defines the file paths for PromptWizard configuration files, specifically for prompt optimization and general setup.

```python
path_to_config = "configs"
promptopt_config_path = os.path.join(path_to_config, "promptopt_config.yaml")
setup_config_path = os.path.join(path_to_config, "setup_config.yaml")
```

--------------------------------

### Optimize Prompt using Synthetic Data

Source: https://github.com/microsoft/promptwizard/blob/main/demos/scenarios/dataset_scenarios_demo.ipynb

Initializes GluePromptOpt with a specific dataset (train_synthetic.jsonl) and the custom GSM8k processor. It then calls get_best_prompt to optimize the prompt using existing examples and the synthetic data.

```python
gp = GluePromptOpt(promptopt_config_path,
                   setup_config_path,
                   dataset_jsonl = "train_synthetic.jsonl",
                   data_processor=gsm8k_processor)
best_prompt, expert_profile = gp.get_best_prompt(use_examples=True,run_without_train_examples=False,generate_synthetic_examples=False)
```

--------------------------------

### Initialize GluePromptOpt

Source: https://github.com/microsoft/promptwizard/blob/main/demos/svamp/demo.ipynb

Creates an instance of the `GluePromptOpt` class, initializing it with the paths to the configuration files and the processed training dataset, along with the dataset processor.

```python
gp = GluePromptOpt(promptopt_config_path,
                   setup_config_path,
                   train_file_name,
                   svamp_processor)
```

--------------------------------

### Initialize GluePromptOpt

Source: https://github.com/microsoft/promptwizard/blob/main/demos/gsm8k/demo.ipynb

Creates an instance of the GluePromptOpt class, which is used for prompt optimization and inference functionalities. It requires paths to configuration files and the processed dataset, along with a dataset processor object.

```python
gp = GluePromptOpt(promptopt_config_path,
                   setup_config_path,
                   train_file_name,
                   gsm8k_processor)
```

--------------------------------

### JavaScript Integration Example

Source: https://github.com/microsoft/promptwizard/blob/main/docs/static/css/bulma.css.map.txt

Example of how to integrate with PromptWizard using JavaScript, likely for frontend applications.

```JavaScript
async function callPromptWizard(promptId, userData) {
  const response = await fetch('/api/promptwizard/generate', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'Authorization': 'Bearer YOUR_API_KEY'
    },
    body: JSON.stringify({ prompt_id: promptId, input_data: userData })
  });
  return await response.json();
}
```

--------------------------------

### Initialize GluePromptOpt Object

Source: https://github.com/microsoft/promptwizard/blob/main/demos/scenarios/dataset_scenarios_demo.ipynb

Creates an instance of the GluePromptOpt class, which is used for prompt optimization and inference. It requires paths to configuration files and the dataset processor.

```python
gp = GluePromptOpt(promptopt_config_path,
                   setup_config_path,
                   dataset_jsonl = os.path.join("data", "train.jsonl"),
                   data_processor = gsm8k_processor)
```

--------------------------------

### Initializing Prompt Optimization Object

Source: https://github.com/microsoft/promptwizard/blob/main/demos/bbh/demo.ipynb

Creates an instance of the GluePromptOpt class, which is used to manage prompt optimization and inference functionalities. It requires paths to configuration files and the dataset processor.

```python
from prompt_wizard import GluePromptOpt # Assuming GluePromptOpt is in prompt_wizard module

# Assuming bbh_processor is defined and initialized
# gp = GluePromptOpt(promptopt_config_path,
#                    setup_config_path,
#                    train_file_name,
#                    bbh_processor)

```

--------------------------------

### Initialize Prompt Optimization

Source: https://github.com/microsoft/promptwizard/blob/main/demos/scenarios/dataset_scenarios_demo.ipynb

Creates an instance of the GluePromptOpt class, which is used for prompt optimization and inference. It requires paths to configuration files and optionally accepts a dataset and data processor.

```python
gp = GluePromptOpt(promptopt_config_path,
                   setup_config_path,
                   dataset_jsonl=None,
                   data_processor=None)
```

--------------------------------

### Python Decorators Example

Source: https://github.com/microsoft/promptwizard/blob/main/docs/static/css/bulma.css.map.txt

Demonstrates the use of decorators in Python, which are a powerful way to modify or enhance functions or methods. This example shows a simple logging decorator.

```Python
import functools

def log_function_call(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        print(f"Calling function: {func.__name__}")
        result = func(*args, **kwargs)
        print(f"Function {func.__name__} finished.")
        return result
    return wrapper

@log_function_call
def add(a, b):
    """Adds two numbers."""
    return a + b

# result = add(5, 3)
# print(f"Result: {result}")

```

--------------------------------

### Prompt Optimization Configuration (`promptopt_config.yaml`)

Source: https://github.com/microsoft/promptwizard/blob/main/README.md

Configuration file for prompt optimization, defining task-specific instructions, answer formats, and experimental parameters like the number of in-context examples and reasoning generation.

```yaml
task_description: |
  You are a mathematics expert. You will be given a mathematics problem which you need to solve
base_instruction: |
  Lets think step by step.
answer_format: |
  At the end, wrap only your final option between <ANS_START> and <ANS_END> tags
seen_set_size: 25
few_shot_count: 4
generate_reasoning: true
generate_expert_identity: true
generate_intent_keywords: true
```

--------------------------------

### Dataset Loading and Management

Source: https://github.com/microsoft/promptwizard/blob/main/demos/bbh/demo.ipynb

Sets up a 'data' directory if it doesn't exist and defines a list of 19 datasets available within the BBII benchmark for optimization.

```python
if not os.path.exists("data"):
    os.mkdir("data")
dataset_list = ['informal_to_formal','letters_list','negation','orthography_starts_with','rhymes','second_word_letter','sum','diff','sentence_similarity','taxonomy_animal','auto_categorization','object_counting','odd_one_out','antonyms','word_unscrambling','cause_and_effect','common_concept','word_sorting','synonyms']
```

--------------------------------

### Bulma Elements: Notification Example

Source: https://github.com/microsoft/promptwizard/blob/main/docs/static/css/bulma.css.map.txt

Shows a styled notification box.

```html
<article class="message is-info">
  <div class="message-header">
    <p>Info</p>
    <button class="delete" aria-label="delete"></button>
  </div>
  <div class="message-body">
    This is an informational message.
  </div>
</article>
```

--------------------------------

### Bulma Elements: Table Example

Source: https://github.com/microsoft/promptwizard/blob/main/docs/static/css/bulma.css.map.txt

Demonstrates a responsive table with hover effects.

```html
<table class="table is-bordered is-striped is-narrow is-hoverable is-fullwidth">
  <thead>
    <tr>
      <th>ID</th>
      <th>Name</th>
      <th>Email</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>1</td>
      <td>Alice</td>
      <td>alice@example.com</td>
    </tr>
    <tr>
      <td>2</td>
      <td>Bob</td>
      <td>bob@example.com</td>
    </tr>
  </tbody>
</table>
```

--------------------------------

### Define File Paths

Source: https://github.com/microsoft/promptwizard/blob/main/demos/gsm8k/demo.ipynb

Sets up the file paths for the processed dataset files ('train.jsonl', 'test.jsonl') and configuration files ('promptopt_config.yaml', 'setup_config.yaml').

```python
train_file_name = os.path.join("data", "train.jsonl")
test_file_name = os.path.join("data", "test.jsonl")
path_to_config = "configs"
promptopt_config_path = os.path.join(path_to_config, "promptopt_config.yaml")
setup_config_path = os.path.join(path_to_config, "setup_config.yaml")
```

--------------------------------

### Bulma Components: Modal Example

Source: https://github.com/microsoft/promptwizard/blob/main/docs/static/css/bulma.css.map.txt

Provides the structure for a Bulma modal dialog.

```html
<div class="modal">
  <div class="modal-background"></div>
  <div class="modal-content">
    <p class="image is-4by3">
      <img src="https://bulma.io/images/placeholders/1280x960.png" alt="">
    </p>
  </div>
  <button class="modal-close is-large" aria-label="close"></button>
</div>
```

--------------------------------

### Bulma Components: Card Example

Source: https://github.com/microsoft/promptwizard/blob/main/docs/static/css/bulma.css.map.txt

Shows a basic Bulma card component.

```html
<div class="card">
  <div class="card-image">
    <figure class="image is-4by3">
      <img src="https://bulma.io/images/placeholders/1280x960.png" alt="Placeholder image">
    </figure>
  </div>
  <div class="card-content">
    <div class="media">
      <div class="media-left">
        <figure class="image is-48x48">
          <img src="https://bulma.io/images/placeholders/96x96.png" alt="Placeholder image">
        </figure>
      </div>
      <div class="media-content">
        <p class="title is-4">John Smith</p>
        <p class="subtitle is-6">@johnsmith</p>
      </div>
    </div>
    <div class="content">
      Lorem ipsum dolor sit amet, consectetur adipiscing elit. Phasellus nec iaculis mauris.
      <br>
      <time datetime="2016-1-1">11:09 PM - 1 Jan 2016</time>
    </div>
  </div>
</div>
```

--------------------------------

### Bulma Components: Navbar Example

Source: https://github.com/microsoft/promptwizard/blob/main/docs/static/css/bulma.css.map.txt

Demonstrates the structure of a Bulma navigation bar.

```html
<nav class="navbar" role="navigation" aria-label="main navigation">
  <div class="navbar-brand">
    <a class="navbar-item" href="/">
      PromptWizard
    </a>
    </div>
  <div class="navbar-menu">
    <div class="navbar-start">
      <a class="navbar-item" href="/">
        Home
      </a>
    </div>
  </div>
</nav>
```

--------------------------------

### Generate Optimal Prompt and Expert Profile

Source: https://github.com/microsoft/promptwizard/blob/main/demos/svamp/demo.ipynb

Calls the `get_best_prompt` method of the `GluePromptOpt` object to generate an optimized prompt and an expert profile. It explains the usage of parameters like `use_examples`, `run_without_train_examples`, and `generate_synthetic_examples`.

```python
# Function call to generate optimal prompt and expert profile 
best_prompt, expert_profile = gp.get_best_prompt(use_examples=True,run_without_train_examples=False,generate_synthetic_examples=False)
```

--------------------------------

### Define File Paths

Source: https://github.com/microsoft/promptwizard/blob/main/demos/svamp/demo.ipynb

Sets up the file paths for the processed dataset files ('train.jsonl', 'test.jsonl') and configuration files ('llm_config.yaml', 'promptopt_config.yaml', 'setup_config.yaml').

```python
train_file_name = os.path.join("data", "train.jsonl")
test_file_name = os.path.join("data", "test.jsonl")
path_to_config = "configs"
llm_config_path = os.path.join(path_to_config, "llm_config.yaml")
promptopt_config_path = os.path.join(path_to_config, "promptopt_config.yaml")
setup_config_path = os.path.join(path_to_config, "setup_config.yaml")
```

--------------------------------

### Bulma Form: Select Example

Source: https://github.com/microsoft/promptwizard/blob/main/docs/static/css/bulma.css.map.txt

Shows a custom select dropdown using Bulma.

```html
<div class="select">
  <select>
    <option>Select dropdown</option>
    <option>With options</option>
    <option>And some more options</option>
  </select>
</div>
```

--------------------------------

### Bulma Components: Tabs Example

Source: https://github.com/microsoft/promptwizard/blob/main/docs/static/css/bulma.css.map.txt

Illustrates a basic tabbed interface using Bulma.

```html
<div class="tabs">
  <ul>
    <li class="is-active">
      <a>
        <span class="icon"><i class="fas fa-image"></i></span>
        <span>Pictures</span>
      </a>
    </li>
    <li>
      <a>
        <span class="icon"><i class="fas fa-music"></i></span>
        <span>Music</span>
      </a>
    </li>
    <li>
      <a>
        <span class="icon"><i class="fas fa-film"></i></span>
        <span>Videos</span>
      </a>
    </li>
    <li>
      <a>
        <span class="icon"><i class="fas fa-file-alt"></i></span>
        <span>Documents</span>
      </a>
    </li>
  </ul>
</div>
```

--------------------------------

### Import Dependencies

Source: https://github.com/microsoft/promptwizard/blob/main/demos/gsm8k/demo.ipynb

Imports necessary libraries for the PromptWizard project, including system path manipulation, prompt optimization components, file utilities, data handling, and environment variable loading.

```python
import sys
sys.path.insert(0, "../../")
import promptwizard
from promptwizard.glue.promptopt.instantiate import GluePromptOpt
from promptwizard.glue.promptopt.techniques.common_logic import DatasetSpecificProcessing
from promptwizard.glue.common.utils.file import save_jsonlist
from typing import Any
from tqdm import tqdm
from re import compile, findall
import os
from datasets import load_dataset

from dotenv import load_dotenv
load_dotenv(override = True)
```

--------------------------------

### Bulma Helpers: Flexbox Example

Source: https://github.com/microsoft/promptwizard/blob/main/docs/static/css/bulma.css.map.txt

Shows how to use Flexbox utilities for alignment and distribution.

```html
<div class="is-flex is-justify-content-center">
  <div class="p-2">Flex item 1</div>
  <div class="p-2">Flex item 2</div>
  <div class="p-2">Flex item 3</div>
</div>
```

--------------------------------

### Bulma Form: File Input Example

Source: https://github.com/microsoft/promptwizard/blob/main/docs/static/css/bulma.css.map.txt

Demonstrates a file input element styled with Bulma.

```html
<div class="file has-name">
  <label class="file-label">
    <input class="file-input" type="file" name="resume">
    <span class="file-cta">
      <span class="file-icon">
        <i class="fas fa-upload"></i>
      </span>
      <span class="file-label">
        Choose a file…
      </span>
    </span>
    <span class="file-name">
      Screen Shot 2017-12-24 at 10.30.38 PM.png
    </span>
  </label>
</div>
```

--------------------------------

### Bulma Helpers: Spacing Example

Source: https://github.com/microsoft/promptwizard/blob/main/docs/static/css/bulma.css.map.txt

Illustrates the use of spacing helper classes for margins and padding.

```html
<div class="box ml-4 p-3">
  This box has margin-left and padding.
</div>
```