### Starting Next.js Development Server (Bash) Source: https://github.com/collect-intel/llm-judge-bias-suite/blob/main/viewer/README.md This snippet provides commands to start the Next.js development server using various package managers. Running one of these commands will launch the application locally, typically accessible via http://localhost:3000. It's a prerequisite for local development and testing. ```bash npm run dev # or yarn dev # or pnpm dev # or bun dev ``` -------------------------------- ### Displaying LLM Bias Analyzer Help (Bash) Source: https://github.com/collect-intel/llm-judge-bias-suite/blob/main/README.md This command executes the `bias_analyzer.py` script with the `--help` flag, which displays a comprehensive list of all available commands, arguments, and their descriptions. This is useful for understanding the script's capabilities and options. ```bash python bias_analyzer.py --help ``` -------------------------------- ### Running Specific LLM Bias Experiments (Bash) Source: https://github.com/collect-intel/llm-judge-bias-suite/blob/main/README.md These examples demonstrate how to run various types of bias experiments using the `bias_analyzer.py` script, including `scoring`, `picking`, `multi_criteria`, `adv_multi_criteria_permuted`, and `classification`. Each command specifies the experiment type and relevant parameters like sample size, repetitions, and temperature. ```bash python bias_analyzer.py scoring --scoring_type poems --scoring_samples 1 --repetitions 2 --temp 0.5 python bias_analyzer.py picking --num_picking_pairs 2 --repetitions 3 --temp 0.1 python bias_analyzer.py multi_criteria --task argument --scoring_samples 1 --repetitions 1 python bias_analyzer.py adv_multi_criteria_permuted --task story_opening --scoring_samples 1 --repetitions 2 python bias_analyzer.py classification --classification_num_samples 5 --classification_domain_filter user_feedback_v1 --repetitions 2 --temp 0.2 ``` -------------------------------- ### Installing Python Dependencies (Bash) Source: https://github.com/collect-intel/llm-judge-bias-suite/blob/main/README.md This command uses `pip` to install all necessary Python packages listed in the `requirements.txt` file. These dependencies are crucial for the proper functioning of the LLM bias analysis scripts. ```bash pip install -r requirements.txt ``` -------------------------------- ### Running a Picking Experiment with Custom Prompt Source: https://github.com/collect-intel/llm-judge-bias-suite/blob/main/README.md This command demonstrates how to run a `picking` experiment using `bias_analyzer.py` to test a new prompt. It specifies the model, limits the number of picking pairs and repetitions for initial testing, sets the LLM temperature, and directs output to a specific directory. ```Bash python bias_analyzer.py picking --models "mistralai/mistral-small" --num_picking_pairs 1 --repetitions 3 --temp 0.7 --output_dir ./my_picking_tests ``` -------------------------------- ### Cloning the LLM Judge Bias Suite Repository (Bash) Source: https://github.com/collect-intel/llm-judge-bias-suite/blob/main/README.md This command sequence clones the `llm-judge-bias-suite` GitHub repository to your local machine and then changes the current directory into the newly cloned project folder. This is the initial step required to begin working with the project. ```bash git clone https://github.com/collect-intel/llm-judge-bias-suite cd llm-judge-bias-suite ``` -------------------------------- ### Setting Up Python Virtual Environment (Bash) Source: https://github.com/collect-intel/llm-judge-bias-suite/blob/main/README.md These commands create a new Python virtual environment named `venv` in the current directory and then activate it. Using a virtual environment isolates project dependencies, preventing conflicts with other Python projects on your system. ```bash python3 -m venv venv source venv/bin/activate ``` -------------------------------- ### Running All LLM Bias Experiments for Multiple Models with Output (Bash) Source: https://github.com/collect-intel/llm-judge-bias-suite/blob/main/README.md This command runs all bias experiments against a comma-separated list of specified LLM models. It also directs the detailed experiment results to a specified output directory (`./experiment_outputs`) and sets the LLM temperature for API calls, ensuring structured JSON output files for traceability. ```bash python bias_analyzer.py all --models "mistralai/mistral-small,anthropic/claude-3-haiku-20240307" --output_dir ./experiment_outputs --temp 0.3 ``` -------------------------------- ### Running All LLM Bias Experiments with Default Model (Bash) Source: https://github.com/collect-intel/llm-judge-bias-suite/blob/main/README.md This command executes the `bias_analyzer.py` script with the `all` argument, triggering all defined bias experiments. It uses the default LLM model configured in the `.env` file or a hardcoded default if no model is explicitly specified. ```bash python bias_analyzer.py all ``` -------------------------------- ### Configuring OpenRouter API Key and Default LLM Model (.env) Source: https://github.com/collect-intel/llm-judge-bias-suite/blob/main/README.md This snippet shows the content for a `.env` file, which is used to configure the OpenRouter API key and an optional default LLM model for the bias suite. The `OPENROUTER_API_KEY` is mandatory for API calls, while `BIAS_SUITE_LLM_MODEL` sets a default model if not specified via command line. ```bash OPENROUTER_API_KEY="your_openrouter_key_here" BIAS_SUITE_LLM_MODEL="your_default_model_identifier_here" # e.g., mistralai/mistral-7b-instruct ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.