### Install and Serve ChainForge Locally Source: https://chainforge.ai/docs Use these commands to install the ChainForge package and start the local development server. Ensure you have Python and pip installed. ```bash pip install chainforge ``` ```bash chainforge serve ``` -------------------------------- ### Start React Development Server Source: https://chainforge.ai/docs/for_dev Starts the React front-end development server with hot reloading enabled. Run this from the `react-server` directory. ```bash npm run start ``` -------------------------------- ### Install ChainForge via Pip Source: https://chainforge.ai/docs/getting_started Standard installation command for the ChainForge package. ```bash pip install chainforge ``` -------------------------------- ### Install Node.js Requirements Source: https://chainforge.ai/docs/for_dev Installs Node.js dependencies for the React front-end. Ensure Node.js is installed before running. ```bash npm install ``` -------------------------------- ### Start ChainForge Server Source: https://chainforge.ai/docs/getting_started Command to launch the local server. Use --host and --port flags to customize the network address. ```bash chainforge serve ``` ```bash chainforge serve --host 0.0.0.0 --port 3400 ``` -------------------------------- ### ChainForge Secure Mode Login Prompt Source: https://chainforge.ai/docs/security After initial setup, ChainForge will prompt for your password when starting in secure mode to verify your identity. ```bash 🔐 Enter password: [...] ✅ Password verified. ``` -------------------------------- ### Serve ChainForge Backend Source: https://chainforge.ai/docs/for_dev Starts the Flask backend server for ChainForge. This script handles API calls and environment variable loading. ```bash python -m chainforge.app serve ``` -------------------------------- ### Force Install Node.js Requirements Source: https://chainforge.ai/docs/for_dev Use this command to force the installation of Node.js dependencies if you encounter conflicts. ```bash npm install --force ``` -------------------------------- ### Configure ChainForge Server Host and Port Source: https://chainforge.ai/docs/releases Use these flags when starting the ChainForge server locally to specify the network interface and port. ```bash chainforge serve --host 0.0.0.0 --port 3400 ``` -------------------------------- ### Install Python Requirements Source: https://chainforge.ai/docs/for_dev Installs Python dependencies for ChainForge. It is recommended to run this within a virtual environment. ```bash pip install -r requirements.txt ``` -------------------------------- ### Join Node Example: Array of Strings Format Source: https://chainforge.ai/docs/nodes Example of using the Join Node to concatenate responses into a JavaScript-style array of strings. This format is useful for collecting multiple LLM outputs into a structured list. ```text ["response1", "response2", "response3"] ``` -------------------------------- ### Set API Key Environment Variable Source: https://chainforge.ai/docs/getting_started Example of persisting an OpenAI API key in the shell configuration file on macOS. ```bash echo "export OPENAI_API_KEY='yourkey'" >> ~/.zshrc source ~/.zshrc echo $OPENAI_API_KEY ``` -------------------------------- ### ChainForge Secure Mode First-Time Password Setup Source: https://chainforge.ai/docs/security When entering secure mode for the first time, ChainForge prompts for a new password to encrypt your flows and settings. You will need to confirm the password. ```bash Welcome to ChainForge! We've noticed you are entering secure mode for the first time. Please enter a password to encrypt your flows and settings. Prepare to enter it again whenever you start ChainForge in secure mode. Create new password: Confirm password: ✅ Password set. ``` -------------------------------- ### Import external libraries in Python evaluators Source: https://chainforge.ai/docs/evaluation Example demonstrating how to use third-party libraries like numpy within a Python evaluation function. ```python import numpy as np def evaluate(resp): try: return str(np.power(int(resp.var['number']), 2)) in resp.text except Exception as e: return False ``` -------------------------------- ### Create a Custom Mirror Provider in Python Source: https://chainforge.ai/docs/model_support Define a custom provider function in Python to extend ChainForge's capabilities. This example creates a 'Mirror' provider that reverses the input prompt. Ensure the function is decorated with `@provider` and placed in the 'Custom Providers' tab. ```python from chainforge.providers import provider # A 'dummy' response provider that just returns the prompt reversed @provider(name="Mirror", emoji="🪞") def mirror_the_prompt(prompt: str, **kwargs) -> str: return prompt[::-1] # just reverse the prompt ``` -------------------------------- ### Split Node Example: Splitting on Code Blocks Source: https://chainforge.ai/docs/nodes Demonstrates splitting a response based on code blocks. When splitting on code blocks, all other text outside of the code blocks is ignored. ```text ```python print('Hello') ``` ```javascript console.log('World'); ``` ``` -------------------------------- ### Define an evaluation function Source: https://chainforge.ai/docs/evaluation Basic examples of an evaluate function returning the length of the response text in Python and JavaScript. ```python def evaluate(response): return len(response.text) ``` ```javascript function evaluate(response) { return response.text.length; } ``` -------------------------------- ### Register a Simple Custom Provider Source: https://chainforge.ai/docs/custom_providers Use the `@provider` decorator to register a Python function as a custom provider in ChainForge. This example creates a 'Mirror' provider that reverses the input prompt. ```python from chainforge.providers import provider @provider(name="Mirror", emoji="🪞") def mirror_the_prompt(prompt: str, **kwargs) -> str: return prompt[::-1] ``` -------------------------------- ### JavaScript Code Processor Example Source: https://chainforge.ai/docs/nodes A JavaScript code processor to extract only the first 100 characters of a response. Code Processors transform response text and are a destructive operation. ```javascript function process(response) { return response.text.substring(0, 100); } ``` -------------------------------- ### Python Code Processor Example Source: https://chainforge.ai/docs/nodes A Python code processor to extract only the first 100 characters of a response. Code Processors transform response text and are a destructive operation. ```python def process(response): return response.text[:100] ``` -------------------------------- ### Register Cohere API Custom Provider Source: https://chainforge.ai/docs/custom_providers Integrate Cohere's text completion API as a custom provider. This example defines custom settings for 'temperature' and 'max_tokens' and specifies available Cohere models. ```python from chainforge.providers import provider import cohere # Init the Cohere client (replace with your API key): co = cohere.Client('') # JSON schemas to pass react-jsonschema-form, one for this provider's settings and one to describe the settings UI. COHERE_SETTINGS_SCHEMA = { "settings": { "temperature": { "type": "number", "title": "temperature", "description": "Controls the 'creativity' or randomness of the response.", "default": 0.75, "minimum": 0, "maximum": 5.0, "multipleOf": 0.01, }, "max_tokens": { "type": "integer", "title": "max_tokens", "description": "Maximum number of tokens to generate in the response.", "default": 100, "minimum": 1, "maximum": 1024, }, }, "ui": { "temperature": { "ui:help": "Defaults to 1.0.", "ui:widget": "range" }, "max_tokens": { "ui:help": "Defaults to 100.", "ui:widget": "range" } } } # Our custom model provider for Cohere's text generation API. @provider(name="Cohere", emoji="🖇", models=['command', 'command-nightly', 'command-light', 'command-light-nightly'], rate_limit="sequential", # enter "sequential" for blocking; an integer N > 0 means N is the max mumber of requests per minute. settings_schema=COHERE_SETTINGS_SCHEMA) def CohereCompletion(prompt: str, model: str, temperature: float = 0.75, **kwargs) -> str: print(f"Calling Cohere model {model} with prompt '{prompt}'...") response = co.generate(model=model, prompt=prompt, temperature=temperature, **kwargs) return response.generations[0].text ``` -------------------------------- ### LLM Scorer Prompt Example Source: https://chainforge.ai/docs/evaluation Use this prompt structure for an LLM scorer to classify text. Explicitly specify the output format (e.g., 'true' or 'false') and instruct the LLM to only reply with its classification. ```prompt Respond with 'true' if the text below is in the style of a {#writing_style}, 'false' if not. Only reply with the classification, nothing else. ``` -------------------------------- ### JavaScript Code Evaluator Example Source: https://chainforge.ai/docs/nodes A basic JavaScript evaluator function to check the length of a response. The `response` argument is a `ResponseInfo` object. This function is called by ChainForge for every response. ```javascript function evaluate(response) { return response.text.length > 100; } ``` -------------------------------- ### Set Custom Directory for Saved Flows Source: https://chainforge.ai/docs/saved_flows Specify a custom directory for saving flows and settings using the `--dir` option when starting the ChainForge server. This is useful for organizing your project files. ```bash chainforge serve --dir /path/to/dir ``` -------------------------------- ### Python Code Evaluator Example Source: https://chainforge.ai/docs/nodes A basic Python evaluator function to check the length of a response. The `response` argument is a `ResponseInfo` object. This function is called by ChainForge for every response. ```python def evaluate(response): return len(response.text) > 100 ``` -------------------------------- ### GPT-4 LLM Scorer for Math Problems Source: https://chainforge.ai/docs/nodes An example of using GPT-4 to score whether responses to math problems are true. It uses an implicit template variable `{#Expected}` to associate a metavariable with each response. The prompt expects a 'true or false' output. ```prompt Is the following response to a math problem true or false? {#Expected} Respond with only true or false. ``` -------------------------------- ### Create and Activate Virtual Environment Source: https://chainforge.ai/docs/getting_started Recommended steps for isolating dependencies on macOS. ```bash python -m venv venv source venv/bin/activate ``` -------------------------------- ### Build React App for Production Source: https://chainforge.ai/docs/for_dev Builds the React application into static files for end-users. Run this from the `react-server` directory. ```bash npm run build ``` -------------------------------- ### Using Settings Variables Source: https://chainforge.ai/docs/prompt_templates Parametrize model settings by using variables of the form `{=setting_name}`. This allows comparing different model configurations within a prompt. ```text {=system_msg} ``` ```text {=presence_penalty} ``` ```text {=temperature} ``` -------------------------------- ### Enable ChainForge Local Encryption Source: https://chainforge.ai/docs/security Use the `chainforge serve --secure` command with options to control the level of encryption for local files. Options include 'off', 'settings', or 'all'. ```bash chainforge serve --secure {off, settings, all} ``` -------------------------------- ### Basic Prompt Template with Variable Source: https://chainforge.ai/docs/prompt_templates Use single braces `{var}` to define variables in prompts. These variables can be filled with values from input nodes. ```text What {time} did {game} come out in the US? ``` -------------------------------- ### CustomProviderProtocol Definition Source: https://chainforge.ai/docs/custom_providers Defines the expected signature for custom provider functions in ChainForge. It includes parameters for prompt, model selection, chat history, and other keyword arguments passed from custom settings. ```python class CustomProviderProtocol(Protocol): def __call__(self, prompt: str, model: Optional[str], chat_history: Optional[ChatHistory], **kwargs: Any) -> str: ... ``` -------------------------------- ### Settings Schema for Custom Provider UI Source: https://chainforge.ai/docs/custom_providers Define a settings_schema to specify the UI for your provider's settings in the ChainForge UI. The schema should include 'settings' and 'ui' components, following react-jsonschema-form format. Ensure property keynames are valid Python variable names. ```json { "settings": , "ui":