### Prompt Example with Datasets and Affixation Source: https://docs.promptmetheus.com/datasets Demonstrates how to use datasets for sampling entries and testing prompts with different sets of emotions, utilizing markers for clarity. ```markdown The following is an entry from a micro journal: --- Some entry here... --- Extract all emotions that are present in the entry based on the following list: [Anger, Anxiety, Awe, Boredom, Calmness, Cheerfulness, Confusion, Contempt, Curiosity, Desire, Disappointment, Disgust, Embarrassment, Enthusiasm, Envy, ...] Print only the emotions as a comma-separated list, nothing else: ``` -------------------------------- ### Reporting Bugs and Issues Source: https://docs.promptmetheus.com/how-to-report-problems Details on how to report bugs and issues via Email or Discord. It emphasizes including screenshots, developer console information, operating system, browser, and any special setup details. ```en Please include the following information in your report if you can, it will make the investigation a lot easier and accelerate the path to resolution: * What operating system are you using? Windows, OS X, Linux, etc. If it is not a recent version, please also include the version. * Which browser are you using? * Is there anything special about your setup that might be related to the problem, e.g. VPN, enterprise network, browser extensions, etc.? * Did you experience similar issues with other software? ``` -------------------------------- ### Troubleshooting Common Issues Source: https://docs.promptmetheus.com/troubleshooting Provides a checklist of common fixes for Promptmetheus users, including internet connection checks, system status verification, API key validation, and browser extension troubleshooting. ```English 1. Ensure a stable internet connection (e.g., test on fast.com). 2. Verify Promptmetheus system status is operational. 3. Refresh the page and sign out/in. 4. Confirm correct LLM provider API Keys are entered. 5. Ensure API key has access to the selected model. 6. Disable browser extensions. 7. Test in an incognito browser window. 8. Check VPN interference with API requests. ``` -------------------------------- ### Promptmetheus Completion Details Source: https://docs.promptmetheus.com/completions Provides an overview of the information available for each completion, including fragments, model settings, and inference details. Fragments capture the state of a block at execution time, while model settings detail parameters like token limit and temperature. Inference details offer relevant metrics. ```APIDOC Fragments: - Snapshots of a Block's state at execution. - Include selected Variant and Variable values. - Displayed at the top of each Completion. - Hover to inspect details. - Can be turned off in Display Mode settings. Model Settings: - Located at the bottom left of each completion. - Include: - Token Limit - Temperature - Top P - Frequency Penalty - Presence Penalty Inference Details: - Located at the bottom right of each completion. - Display relevant metrics for the completion. ``` -------------------------------- ### Known Issues and Workarounds Source: https://docs.promptmetheus.com/troubleshooting Details known bugs and issues affecting Promptmetheus, such as drag-and-drop reordering bugs, interference from Zscaler, and issues with the Vivaldi browser. ```English Drag & drop re-ordering bug: Refresh the page to reset ordering state. Zscaler interference: Contact support if login issues occur. Vivaldi browser: Duplicated completion sequences may occur due to websocket issues; use a different browser. ``` -------------------------------- ### Forge Playground Troubleshooting Source: https://docs.promptmetheus.com/troubleshooting Specific troubleshooting steps for the Forge playground, including data persistence and browser compatibility issues. ```English Data is stored locally in the browser. Use import/export or the Prompt IDE for multi-device access. Firefox may have input duplication issues; Chromium-based browsers are recommended. ``` -------------------------------- ### Promptmetheus Completion Export Options Source: https://docs.promptmetheus.com/completions Details the options available for exporting completions, including applying current search/filters and selecting which properties to include. Supported formats include CSV, XLSX, and JSON, with more planned. ```APIDOC Export: - Export Completions from the Prompt page via the top-right menu. Export Options: - Apply search and filters: Apply active filters/search terms to the export (default: export all). - Include [x]: Specify properties to include in the export. Export Formats: - Currently supported: CSV, XLSX, JSON. - Future formats: Markdown, HTML, PDF. ``` -------------------------------- ### Promptmetheus Project Management API Source: https://docs.promptmetheus.com/projects This section outlines the API endpoints and functionalities for managing projects within the Promptmetheus platform. It covers creating, importing, exporting, and organizing projects, as well as managing associated prompts, datasets, and variables. ```APIDOC Project Management: createProject(name: str, description: str = '') -> Project Creates a new project to organize prompts and datasets. Parameters: name: The name of the project. description: An optional description for the project. Returns: The newly created Project object. importProjectFromForge(forgeUrl: str) -> Project Imports a project from the Prompt Engineering Playground (Forge) into the Prompt IDE (Archery). Note: Projects cannot be moved from Archery to Forge due to architectural differences. History is not recovered. Parameters: forgeUrl: The URL of the project in Forge. Returns: The imported Project object. exportPrompts(projectId: str) -> list[Prompt] Exports all prompts associated with a specific project. Note: Exporting entire projects is not currently supported. Individual prompts and completions can be exported from their detail pages. Parameters: projectId: The ID of the project. Returns: A list of Prompt objects. exportCompletions(projectId: str) -> list[Completion] Exports all completions associated with prompts within a specific project. Parameters: projectId: The ID of the project. Returns: A list of Completion objects. getProjectDashboard(projectId: str) -> ProjectDashboardStats Retrieves statistics and key metrics for a given project from the Dashboard. Parameters: projectId: The ID of the project. Returns: A ProjectDashboardStats object containing metrics. listProjects() -> list[Project] Lists all available projects. Returns: A list of Project objects. deleteProject(projectId: str) -> bool Deletes a project and all its associated data. Parameters: projectId: The ID of the project to delete. Returns: True if the project was deleted successfully, False otherwise. Project Data Management: Note: Prompts, Datasets, and Variables cannot be shared or moved between projects. Organize data from the start. addPromptToProject(projectId: str, promptData: Prompt) Adds a prompt to a specific project. Parameters: projectId: The ID of the project. promptData: The Prompt object to add. addDatasetToProject(projectId: str, datasetData: Dataset) Adds a dataset to a specific project. Parameters: projectId: The ID of the project. datasetData: The Dataset object to add. addVariableToProject(projectId: str, variableData: Variable) Adds a variable to a specific project. Parameters: projectId: The ID of the project. variableData: The Variable object to add. getProjectPrompts(projectId: str) -> list[Prompt] Retrieves all prompts within a specific project. Parameters: projectId: The ID of the project. Returns: A list of Prompt objects. getProjectDatasets(projectId: str) -> list[Dataset] Retrieves all datasets within a specific project. Parameters: projectId: The ID of the project. Returns: A list of Dataset objects. getProjectVariables(projectId: str) -> list[Variable] Retrieves all variables within a specific project. Parameters: projectId: The ID of the project. Returns: A list of Variable objects. ``` -------------------------------- ### Promptmetheus Proxy Configuration Source: https://docs.promptmetheus.com/account Configure Promptmetheus to route requests through a LiteLLM-compatible proxy server. This involves enabling the proxy setting and providing the proxy URL and key. Note that model customization is limited in proxy mode. ```APIDOC Proxy Settings: Proxy: "on" | "off" Enables or disables proxy mode. Proxy URL: string The URL of the LiteLLM-compatible proxy server. Proxy Key: string The API key for the proxy server. Notes: - Only LiteLLM-compatible proxies are supported. - Model list is limited to Promptmetheus defaults in proxy mode. ``` -------------------------------- ### Promptmetheus Search, Filter, and Display Modes Source: https://docs.promptmetheus.com/completions Explains the capabilities for searching, filtering, and changing the display mode of completions. Users can rate completions and apply filters/search terms to exports. Formatting options and display modes customize the view. ```APIDOC Search and Filter: - Rate completions using respective actions. - More filter options are in development. Formatting: - Currently 2 formatting options for the Completion list. - More options are planned. Display Mode: - 3 different Display Modes available for the Completion list. ``` -------------------------------- ### Promptmetheus Completions API Source: https://docs.promptmetheus.com/completions This section outlines the API endpoints and functionalities related to managing and retrieving completions within Promptmetheus. It covers how prompts are executed, the data returned, and how to interact with the completion data. ```APIDOC Completions: Description: When a Prompt is executed, Promptmetheus compiles it into a plain text string and sends it to the inference API, where it is fed into the selected LLM. The generated Completion is then stored and streamed back to the IDE for inspection. Ratings: Description: Ratings allow users to evaluate the valence of a Completion and represent it visually via color code. Clicking a rating icon (Great, Good, Neutral, Bad) applies the rating to the Completion and updates inline statistics for associated Prompt Variants. Supported Ratings: - Great (star, green) - Good (thumbs up, lime) - Neutral (circle, gray) - Bad (thumbs down, red) Inline Statistics: Ratings applied to a Completion are also applied to all Variants used in its generation, updating performance statistics displayed below each Variant tab. Configuration: Rating display can be toggled in app settings. Fragments: Description: Information about fragments within a completion (details not provided in the source text). Evaluations: Description: Information about evaluations of completions (details not provided in the source text). Model Settings: Description: Details regarding the configuration of models (LLMs) used for generating completions (details not provided in the source text). Inference Details: Description: Information pertaining to the inference process, including parameters and settings used by the LLM (details not provided in the source text). Completion Prompt: Description: Displays the prompt that was used to generate a specific completion (details not provided in the source text). Search and Filter: Description: Functionality to search and filter through generated completions based on various criteria (details not provided in the source text). Formatting: Description: Options for formatting the display of completions (details not provided in the source text). Display Mode: Description: Different modes for displaying completions (details not provided in the source text). Export: Export Options: Description: Various options available for exporting completion data (details not provided in the source text). Export Formats: Description: Supported formats for exporting completion data (details not provided in the source text). ``` -------------------------------- ### Promptmetheus API Documentation Source: https://docs.promptmetheus.com/dashboard This section outlines the API endpoints and functionalities available within the Promptmetheus platform. It details available resources for managing projects, prompts, models, and more. ```APIDOC Project Endpoints: GET /projects - Retrieves a list of all projects. GET /projects/{projectId} - Retrieves details for a specific project. Prompt Endpoints: GET /prompts - Retrieves a list of all prompts. GET /prompts/{promptId} - Retrieves details for a specific prompt. Model Endpoints: GET /models - Retrieves a list of available models (LLMs). GET /models/{modelId} - Retrieves details for a specific model. Variable Endpoints: GET /variables - Retrieves a list of all variables. GET /variables/{variableId} - Retrieves details for a specific variable. Dataset Endpoints: GET /datasets - Retrieves a list of all datasets. GET /datasets/{datasetId} - Retrieves details for a specific dataset. Completion Endpoints: GET /completions - Retrieves a list of all completions. GET /completions/{completionId} - Retrieves details for a specific completion. Evaluator Endpoints: GET /evaluators - Retrieves a list of all evaluators. GET /evaluators/{evaluatorId} - Retrieves details for a specific evaluator. Endpoint (AIPI) Endpoints: GET /endpoints - Retrieves a list of all AIPIs. GET /endpoints/{endpointId} - Retrieves details for a specific AIPI. User Endpoints: GET /account - Retrieves user account information. GET /organizations - Retrieves a list of user's organizations. GET /api-keys - Retrieves a list of user's API keys. GET /subscriptions - Retrieves user's subscription details. ``` -------------------------------- ### System Checks for Deeper Issues Source: https://docs.promptmetheus.com/troubleshooting Outlines system checks to perform if quick fixes do not resolve the problem, including trying different browsers and testing simple prompts. ```English 1. Try a different browser (Chrome recommended). 2. Execute a simple prompt (e.g., "Tell me a joke."). 3. If simple prompts fail, check for special characters in your prompt. ``` -------------------------------- ### Promptmetheus API Documentation Source: https://docs.promptmetheus.com/prompts This section outlines the API endpoints and functionalities available in Promptmetheus for programmatic interaction with prompts, models, and other resources. ```APIDOC Promptmetheus API Reference: Base URL: https://api.promptmetheus.com Endpoints: 1. /prompts - GET: Retrieve a list of all prompts. - POST: Create a new prompt. 2. /prompts/{prompt_id} - GET: Retrieve a specific prompt by its ID. - PUT: Update an existing prompt. - DELETE: Delete a prompt. 3. /models - GET: Retrieve a list of available LLM models. 4. /variables - GET: Retrieve a list of available variables. - POST: Create a new variable. 5. /datasets - GET: Retrieve a list of datasets. - POST: Upload a new dataset. 6. /completions - POST: Generate completions using a specified prompt and model. - Request Body: { "prompt_id": "string", "model_id": "string", "variables": { "key": "value" }, "parameters": { "temperature": 0.7, "max_tokens": 100 } } - Response Body: { "completion": "string", "model_used": "string", "usage": { "prompt_tokens": 10, "completion_tokens": 20 } } 7. /evaluators - GET: Retrieve a list of evaluators. - POST: Create a new evaluator. 8. /endpoints - GET: Retrieve a list of AIPIs (endpoints). Authentication: API requests must be authenticated using an API key provided in the 'Authorization' header as a Bearer token. Example: Authorization: Bearer YOUR_API_KEY Error Handling: Standard HTTP status codes are used. Error responses include a JSON object with an 'error' field describing the issue. Example: { "error": "Invalid prompt ID" } Rate Limiting: API requests are subject to rate limits. Check the response headers for 'X-RateLimit-Limit', 'X-RateLimit-Remaining', and 'X-RateLimit-Reset'. ``` -------------------------------- ### Promptmetheus Subscription Management (Stripe Integration) Source: https://docs.promptmetheus.com/subscriptions This section outlines how Promptmetheus integrates with Stripe for subscription management. It covers accessing pricing, upgrading accounts, and managing existing subscriptions through the Stripe customer portal. ```APIDOC APIDOC: Stripe Integration: Purpose: Manages all subscription-related functionalities, including plan selection, upgrades, and billing. Integration: Promptmetheus subscriptions are entirely managed by Stripe. Plans and Pricing: Access: Available on the Promptmetheus website under "pricing". Customization: For business invoices or annual billing, contact support. Upgrading Account: Method: Via the "Account" page, click "Upgrade account" in the "Subscription" section. Current Limitation: Only "Single" plans are available in the customer portal. Team Plans: To upgrade to a "Team" plan, sign up for the "Single" trial first and then contact support to set up your team workspace. Managing Subscription: Access: Via the Stripe billing portal. Link: https://billing.stripe.com/p/login/7sI02c23X7ni1Co8ww Alternative Access: From the "Account" page, click "Manage subscription" in the info section. Features: Update billing details, access invoices. Team Plan Support: Dedicated support is available for Team plan users regarding their subscription. ``` -------------------------------- ### Model Settings Overview Source: https://docs.promptmetheus.com/models Details the various parameters available for tuning LLM behavior. Users can adjust these settings to control aspects like output length, creativity, and repetition. Unsupported settings for a specific model will be grayed out. ```APIDOC Model Settings: - Token Limit: Maximum number of tokens for input and output. Varies by provider (e.g., OpenAI vs. Anthropic). - Temperature: Controls randomness. Higher values increase creativity, lower values make output more deterministic. - Top P: Nucleus sampling parameter. Controls the cumulative probability mass of tokens to consider. - Frequency Penalty: Reduces repetition of tokens based on their frequency in the generated text. - Presence Penalty: Reduces repetition of tokens based on their presence in the generated text. - Seed: For reproducible results, if supported by the model. - JSON Mode: Enforces JSON output format, if supported. - Reasoning Mode: Specific mode for reasoning tasks, if supported. - Stop Sequences: Sequences that, when generated, will cause the model to stop generating further tokens. ``` -------------------------------- ### Troubleshooting Prompt IDE (Archery) Errors Source: https://docs.promptmetheus.com/troubleshooting Guidance for resolving issues within the Prompt IDE (Archery), focusing on completion errors and general troubleshooting. It advises checking provider API status, budget, and model availability for completion-related issues. For other problems, a checklist is recommended. ```en If you get errors for completions, you can hover over the orange error icon and label to see what exactly happened (see screenshot below). A lot of these errors come from the provider APIs directly, e.g. because the API is down, you ran out of budget, or because a model is not available in your country. Regarding completion errors that are not related to the provider, please get in touch. If the problem is not related to completions, start by going through the following checklist, it will resolve most of the common issues. ``` -------------------------------- ### Prompt IDE (Archery) Planned Features Source: https://docs.promptmetheus.com/roadmap Lists planned features for the Archery Prompt Engineering IDE, including completion comments, secure API key syncing, model configuration presets, folders, webhooks, image generation, file blocks, custom connectors, RAG, prompt chaining, and endpoint hosting. ```APIDOC Prompt IDE (Archery) Planned Features: - Completion comments and annotations - Secure API key syncing - Model configuration presets - Folders - Webhooks - Image generation - File blocks (image, video, audio, pdf, etc.) - Custom provider / model connectors (incl. local) - RAG - Prompt chaining (agents) - Endpoints (AIPIs) ``` -------------------------------- ### Code Highlighting Languages Source: https://docs.promptmetheus.com/prompts Lists the language identifiers supported by Promptmetheus for code highlighting using Shiki syntax highlighter. Users initiate code sections by typing ``` followed by the language indicator. ```markdown Supported language indicators: c, cpp, csharp, css, dart, go, html, java, json, jsx, kotlin, python, rust, svelte, swift, ts, tsx, vue ``` -------------------------------- ### Promptmetheus Account Management Source: https://docs.promptmetheus.com/account This section details the various aspects of managing your Promptmetheus account, including profile settings, API keys, and subscription information. It also covers essential account maintenance tasks. ```APIDOC Account: - Overview: Access account information, manage Profile, API Keys, and Subscriptions. Profile: - Display name: Set a display name to replace your email address in Workspace selectors and other references. This is useful for sharing information without exposing your email. - OpenAI Organization ID: Specify the OpenAI Organization ID to associate all OpenAI API requests with the correct organization for billing and rate limits. Refer to [OpenAI documentation](https://platform.openai.com/docs/api-reference/organization-optional) for details. Your OpenAI organization ID can be found in your OpenAI account under "Organization Settings". Note: While API keys may work without it, setting the Organization ID might resolve unreasonable rate limit errors. - Proxy: Configure proxy settings for your account. - Changing your email address: Instructions for updating your account's email address. - Resetting your password: Steps to reset your account password. - Deleting your account: Procedure for deleting your Promptmetheus account. API Keys: - Documentation: [API keys](https://docs.promptmetheus.com/api-keys) Subscriptions: - Documentation: [Subscriptions](https://docs.promptmetheus.com/subscriptions) ``` -------------------------------- ### Model Parameters Source: https://docs.promptmetheus.com/models Configuration options for controlling language model output, including sampling strategies, output formatting, and generation termination. ```APIDOC Temperature: Description: Controls the randomness of the output. Higher values increase randomness, lower values increase focus and determinism. Range: Typically between 0.0 and 1.0. Top p: Description: Nucleus sampling parameter. Considers tokens with a cumulative probability mass up to top_p. An alternative to temperature. Range: Typically between 0.0 and 1.0. Recommendation: Alter either temperature or top_p, but not both. Frequency penalty: Description: Penalizes new tokens based on their existing frequency in the text so far, reducing verbatim repetition. Range: Positive values. Presence penalty: Description: Penalizes new tokens based on their presence in the text so far, encouraging discussion of new topics. Range: Positive values. Seed: Description: If specified, attempts to make sampling deterministic. Repeated requests with the same seed and parameters should yield the same result, though determinism is not guaranteed. JSON mode: Description: Forces the model to output JSON-formatted responses. The prompt must contain the word "json". Requirement: Prompt must include the word "json". Reasoning mode: Description: Enables models capable of reasoning or 'thinking' before generating an answer. Can be automatic or explicitly requested. Configuration: May include a toggle to enable/disable and a setting for maximum reasoning tokens. Stop sequences: Description: A list of up to 4 sequences that will cause the API to stop generating further tokens. The stop sequence itself is not included in the output. Format: Comma-separated list. ``` -------------------------------- ### Promptmetheus Completion Prompt Inspection Source: https://docs.promptmetheus.com/completions Details how to inspect the full prompt used for a completion. Users can view the prompt's Fingerprint and compiled Messages, in addition to the Fragments. ```APIDOC Completion Prompt Inspection: - Inspect the entire Prompt used for a Completion. - Access via Completion actions. - View: - Fingerprint - Compiled Messages ``` -------------------------------- ### AIPI (AI Programming Interface) Documentation Source: https://docs.promptmetheus.com/endpoints Details the structure and capabilities of AIPIs, which are collections of prompt endpoints. AIPIs will support token authorization, rate limiting, and usage statistics, and integrate with automation platforms. ```APIDOC AIPI (AI Programming Interface): Description: A collection of prompt endpoints deployed to the cloud, accessible from anywhere. Features: - Token Authorization: Secure access using tokens. - Rate Limiting: Control the number of requests. - Usage Statistics: Monitor endpoint usage. Integrations: - Make - Zapier - IFTTT - n8n Status: Currently in development and internal testing. ``` -------------------------------- ### Prompt IDE (Archery) Planned Improvements Source: https://docs.promptmetheus.com/roadmap Details planned improvements for the Archery Prompt Engineering IDE, such as enhanced completion formatting and filtering, read-only permissions for teams, evaluator reuse, and project/prompt copying and moving. ```APIDOC Prompt IDE (Archery) Planned Improvements: - More completion formatting options - Better completion filtering - Read-only permission for Team accounts - Reuse evaluators across prompts - Copy projects - Move/copy prompts, datasets, etc. to other projects ``` -------------------------------- ### Reporting Security Vulnerabilities Source: https://docs.promptmetheus.com/how-to-report-problems Information on reporting security vulnerabilities. Users are directed to the security page and advised to disclose vulnerabilities confidentially via email. It also mentions the lack of bounties but offers credit and Hall of Fame entry. ```en If you find any vulnerabilities in our systems, please check the [Security](https://promptmetheus.com/security "Information about security guidelines, measures, certifications, reports, and plans") page and disclose them confidentially under security@promptmetheus.com. Bounties / Rewards Unfortunately, we currently cannot pay any bounties for reported vulnerabilities. However, we can give you credit for your findings and an entry in our [Hall of Fame](https://promptmetheus.com/hall-of-fame "Promptmetheus Hall of Fame") if you wish. ``` -------------------------------- ### Promptmetheus Workspace Import/Export Limitations Source: https://docs.promptmetheus.com/workspaces Details the current inability to import or export entire Workspaces in Promptmetheus. This is due to the evolving database structure as new features are added, which would require maintaining backward compatibility with all previous app versions. Daily database backups are performed for data safety. ```APIDOC Workspace Import/Export: Status: Not Supported Reason: Evolving database structure requires maintaining compatibility with all past app versions, which is resource-intensive and slows development. Data Safety: Daily database backups are performed. Future Plans: Working on providing a full data dump feature. ``` -------------------------------- ### Organizations API Documentation Source: https://docs.promptmetheus.com/organizations Provides details on managing organizations, including viewing organization information, managing team members through invitations, and handling API keys. ```APIDOC Organizations: Requires Team Account: True Switching Workspaces: Via organization selector in sidebar. Data Visibility: Personal workspace data is private; team workspace data is shared. Organization Details: Access: Via organization selector. Information Displayed: - Creation date - Seat usage (used/available) - User's role Settings: - Change Organization name - Add OpenAI Organization ID (details in Account section) Usage Statistics: Core stats displayed on the top right (more to be added). Managing Team Members: Permissions: Only Organization owner can manage members. Seat Management: Increase/decrease seats by contacting support. Invitations: - Add members via invitations. - Invitees receive email notifications. - Invitations can be accepted/declined in the app. - Outstanding invitations and team member list visible in organization details. Invitation Issues: - If email not received, invitee can sign up with the same email; invitation will be available in-app. API Keys: Storage: Currently stored on-device only. Sharing: Not shared between team members. Future Feature: Secure syncing for sharing encrypted API keys. ``` -------------------------------- ### Prompt Playground (Forge) Status Source: https://docs.promptmetheus.com/roadmap Indicates that the Prompt Playground (Forge) is in maintenance mode and not recommended for professional use, with plans to merge it with Archery in the future. ```APIDOC Prompt Playground (Forge) Status: - Maintenance Mode - Not recommended for professional use. - Future plans to merge with Archery and provide a free tier for Archery. ``` -------------------------------- ### Token Price Calculation Source: https://docs.promptmetheus.com/models Explains how the inference price for input and output tokens is calculated based on provider-specific rates. The cost is determined by the number of input and output tokens multiplied by their respective prices per 1,000 tokens. ```APIDOC Token Price Calculation: completion cost = (n_input / 1000) * price_input + (n_output / 1000) * price_output - n_input: Number of input tokens. - n_output: Number of output tokens. - price_input: Price per 1,000 input tokens. - price_output: Price per 1,000 output tokens. - The inference price is set by each provider and displayed in cents per 1,000 tokens. ``` -------------------------------- ### Promptmetheus API Key Management Source: https://docs.promptmetheus.com/api-keys Details on how Promptmetheus handles API keys for external LLM inference. Keys are stored locally in the browser for security and are not synced across devices or accounts. Users must obtain keys directly from LLM providers. ```APIDOC API Keys: Storage: Local browser storage (not synced) Purpose: Inference via external LLM APIs Acquisition: Obtain from LLM providers via 'Get Key' links Security: Stored locally for maximum security Syncing: Future feature for secure, encrypted syncing between devices and team members Usage Limits: Recommendation: Set API usage limits directly with LLM providers to avoid unexpected costs. Billing: Some providers offer pre-paid billing options (e.g., OpenAI). ``` -------------------------------- ### Promptmetheus Account Management Actions Source: https://docs.promptmetheus.com/account Describes actions related to managing a Promptmetheus account, including email address changes, password resets, and account deletion. ```APIDOC Account Management: Change Email Address: Not possible via the app. Contact support via email. Verification of the current email address is required. Reset Password: Accessible via a direct link or the 'Reset Password' action on the Account page. Link: https://archery.promptemetheus.com/reset-password Delete Account: Available as a dedicated action on the Account page. Warning: This action is irreversible and will result in data loss. ``` -------------------------------- ### Token Limit Explanation Source: https://docs.promptmetheus.com/models Explains the function and implications of the token limit setting. This parameter defines the maximum number of tokens a model can process or generate. It's crucial for managing output length and preventing completion cutoffs, noting that different providers implement this limit differently. ```APIDOC Token Limit: - Description: The maximum number of tokens the model should process/generate in the completion. - Provider Differences: Some providers (e.g., OpenAI) include both input and output tokens, while others (e.g., Anthropic) may refer only to output tokens. - Impact: If completions are cut off, the token limit is often the cause. The model will never exceed this limit. ``` -------------------------------- ### Promptmetheus Variables Source: https://docs.promptmetheus.com/variables Variables in Promptmetheus allow for the definition and reuse of terms across prompts. They can be updated in one place, simplifying content management. Currently, variables are supported only within Text Blocks, not Data Blocks. ```APIDOC Local variables: - Scope: Limited to the current prompt. - Usage: Type `{{` followed by the variable name (autocompletes) and then space or enter within message blocks. - Naming Convention: Alphanumeric characters and underscores only; no spaces. - Note: The 'name' is the identifier, not the substituted value. Project variables: - Scope: Shared across all prompts within a project. - Definition: Defined in project settings. - Usage: Used like local variables; Promptmetheus checks for global variables first, then local ones. - Further Information: Refer to the [Projects](https://docs.promptmetheus.com/projects#global-variables) section for details on global variables. ``` -------------------------------- ### Promptmetheus Evaluator Types Source: https://docs.promptmetheus.com/evaluators This section outlines the different types of evaluators available in Promptmetheus for validating prompt completions. Each evaluator type has specific validation criteria. ```APIDOC Length Evaluators: - Validates the length of prompt completions. - Can be configured against an exact value or a range (min/max). - Units for evaluation: tokens, characters, or words. Regex Evaluator: - Description: Validates prompt completions using regular expressions. - Status: Coming soon. ``` -------------------------------- ### Number Evaluators Source: https://docs.promptmetheus.com/evaluators Number evaluators ensure that the completion content is a valid number. They support two modes: exact mode for a specific value, and range mode for values within a minimum and maximum boundary. -------------------------------- ### Text Evaluators Source: https://docs.promptmetheus.com/evaluators Text evaluators validate the exact match or presence/absence of a specific text string within a prompt completion. They support checking at the beginning, end, or anywhere in the text, and allow for case-sensitive or case-insensitive comparisons. === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.