LLM

https://llm.datasette.io/
Admin
LLM is a CLI tool and Python library for interacting with Large Language Models, both via remote...
Tokens:205,547
Snippets:1,452
Trust Score:9.7
Update:15 hours ago
Show doc for...
Tokens:
Raw
### Install and use a plugin tool with --tool

Source: https://llm.datasette.io/en/stable/usage.html

Install a plugin and use its tool with `--tool`. The example uses llm-tools-simpleeval to evaluate an expression with debug output.

```bash
llm install llm-tools-simpleeval
llm --tool simple_eval "4444 * 233423" --td
```

--------------------------------

### Install plugin from URL

Source: https://llm.datasette.io/en/stable/plugins/tutorial-model-plugin.html

Install the plugin from a hosted URL using `llm install` with the URL to the package.

```bash
llm install 'https://.../llm_markov-0.1-py3-none-any.whl'
```

--------------------------------

### Install plugin from local wheel

Source: https://llm.datasette.io/en/stable/plugins/tutorial-model-plugin.html

Install the plugin from the built wheel file using the `llm install` command.

```bash
llm install dist/llm_markov-0.1-py3-none-any.whl
```

--------------------------------

### Install plugin and list fragment loaders

Source: https://llm.datasette.io/en/stable/fragments.html

Install the `llm-fragments-github` plugin and then run `llm fragments loaders` to list all installed fragment prefixes and their documentation.

```bash
llm install llm-fragments-github
llm fragments loaders
```

--------------------------------

### Example pyproject.toml with metadata for LLM plugin

Source: https://llm.datasette.io/en/stable/plugins/tutorial-model-plugin.html

Replace your pyproject.toml with this example to add metadata for PyPI, including README, dependencies, project URLs, and the LLM entry point. It ensures 'llm' is installed as a dependency and displays your README on the PyPI listing page.

```toml
[project]
name = "llm-markov"
version = "0.1"
description = "Plugin for LLM adding a Markov chain generating model"
readme = "README.md"
authors = [{name = "Simon Willison"}]
license = {text = "Apache-2.0"}
classifiers = [
    "License :: OSI Approved :: Apache Software License"
]
dependencies = [
    "llm"
]
requires-python = ">3.7"

[project.urls]
Homepage = "https://github.com/simonw/llm-markov"
Changelog = "https://github.com/simonw/llm-markov/releases"
Issues = "https://github.com/simonw/llm-markov/issues"

[project.entry-points.llm]
markov = "llm_markov"
```

--------------------------------

### Example error when llm fails to start

Source: https://llm.datasette.io/en/stable/plugins/tutorial-model-plugin.html

Shows a typical traceback and SyntaxError when a plugin has a syntax error, preventing llm from starting.

```bash
$ llm 'hi'
Traceback (most recent call last):
  ...
  File llm-markov/llm_markov.py", line 10
    register(Markov()):
                      ^
SyntaxError: invalid syntax

```

--------------------------------

### Install LLM with Homebrew

Source: https://llm.datasette.io/en/stable/

Install LLM using Homebrew. Note the warning about Homebrew installation in the setup documentation.

```bash
brew install llm
```

--------------------------------

### Start a chat with a template

Source: https://llm.datasette.io/en/stable/usage.html

Starts a new chat using a previously saved template. The example shows starting a chat with the cheesecake template.

```bash
llm chat -t cheesecake
```

--------------------------------

### Install llm-docs plugin

Source: https://llm.datasette.io/en/stable/changelog.html

Install the llm-docs plugin to demonstrate the new fragment and template features.

```bash
llm install llm-docs
```

--------------------------------

### Example output of dirname command

Source: https://llm.datasette.io/en/stable/openai-models.html

Example output on a Mac laptop showing the configuration directory path.

```text
~/Library/Application Support/io.datasette.llm
```

--------------------------------

### Use the registered template loader with llm -t

Source: https://llm.datasette.io/en/stable/plugins/plugin-hooks.html

Command to use the 'my-prefix' template loader once the example plugin is installed.

```bash
llm -t my-prefix:my-template
```

--------------------------------

### Install and use llm-templates-fabric

Source: https://llm.datasette.io/en/stable/changelog.html

Install the plugin and load a template from the fabric collection using the f: prefix.

```bash
llm install llm-templates-fabric
curl https://simonwillison.net/2025/Apr/6/only-miffy/ | \
  llm -t f:extract_main_idea
```

--------------------------------

### llm install --help output

Source: https://llm.datasette.io/en/stable/help.html

Shows the usage and options for the llm install command, which installs packages from PyPI into the same environment as LLM. Options include -U/--upgrade, -e/--editable, --force-reinstall, --no-cache-dir, --pre, and -h/--help.

```text
Usage: llm install [OPTIONS] [PACKAGES]...

  Install packages from PyPI into the same environment as LLM

Options:
  -U, --upgrade        Upgrade packages to latest version
  -e, --editable TEXT  Install a project in editable mode from this path
  --force-reinstall    Reinstall all packages even if they are already up-to-
                       date
  --no-cache-dir       Disable the cache
  --pre                Include pre-release and development versions
  -h, --help           Show this message and exit.
```

--------------------------------

### Install LLM with uv

Source: https://llm.datasette.io/en/stable/

Install LLM using uv tool installer.

```bash
uv tool install llm
```

--------------------------------

### Install and use llm-templates-github

Source: https://llm.datasette.io/en/stable/changelog.html

Install the plugin and run a template from a GitHub repository using the gh: prefix.

```bash
llm install llm-templates-github
llm -t gh:simonw/pelican-svg -m o3-mini
```

--------------------------------

### Example output with token usage

Source: https://llm.datasette.io/en/stable/logging.html

Example output showing a log entry with token usage details.

```yaml
- model: o3-mini
  datetime: '2025-02-16T23:00:56'
  conversation: 01jm8fxxnef92n1663c6ays8xt
  system: Produce Python code that demonstrates every possible usage of yaml.dump
    with all of the arguments it can take, especi...
  prompt: <documents> <document index="1"> <source>./setup.py</source> <document_content>
    NAME = 'PyYAML' VERSION = '7.0.0.dev0...
  usage:
    input: 74793
    output: 3550
    details:
      completion_tokens_details:
        reasoning_tokens: 2240
```

--------------------------------

### Install the build package

Source: https://llm.datasette.io/en/stable/plugins/tutorial-model-plugin.html

Install the `build` package using pip to prepare for creating distributable packages.

```bash
python -m pip install build
```

--------------------------------

### Install llm-templates-github plugin

Source: https://llm.datasette.io/en/stable/templates.html

Install the llm-templates-github plugin using the llm install command.

```bash
llm install llm-templates-github
```

--------------------------------

### Dynamic toolbox listing example

Source: https://llm.datasette.io/en/stable/usage.html

Example output from `llm tools` for a dynamic toolbox (MCP) showing its constructor signature and documentation.

```text
MCP(server, mode='auto', prefix='') (plugin: mcp_client)

  Expose the tools from an MCP server as LLM tools.

  Usage:

      MCP("https://example.com/mcp")
      MCP("https://example.com/mcp", mode="legacy", prefix="demo_")
```

--------------------------------

### Install sqlite-utils

Source: https://llm.datasette.io/en/stable/schemas.html

Installation command for sqlite-utils using uv tool install, with alternatives pip install or pipx install.

```bash
uv tool install sqlite-utils
# or pip install or pipx install
```

--------------------------------

### Install and use Datasette toolbox

Source: https://llm.datasette.io/en/stable/usage.html

Install the llm-tools-datasette plugin and use the Datasette toolbox to show tables. The -T option passes a toolbox specification.

```bash
llm install llm-tools-datasette
llm -T 'Datasette("https://datasette.io/content")' "Show tables" --td
```

--------------------------------

### Example output of llm logs path

Source: https://llm.datasette.io/en/stable/logging.html

Example output showing the log database path on macOS.

```text
/Users/simon/Library/Application Support/io.datasette.llm/logs.db
```

--------------------------------

### Install plugin from GitHub Gist ZIP

Source: https://llm.datasette.io/en/stable/plugins/tutorial-model-plugin.html

Install the plugin from a GitHub Gist ZIP URL using the `llm install` command.

```bash
llm install 'https://gist.github.com/simonw/6e56d48dc2599bffba963cef0db27b6d/archive/cc50c854414cb4deab3e3ab17e7e1e07d45cba0c.zip'
```

--------------------------------

### Install llm-gpt4all plugin

Source: https://llm.datasette.io/en/stable/other-models.html

Installs the llm-gpt4all plugin, which provides 17 models from the GPT4All project. After installation, run `llm models` to see the expanded list of available models.

```bash
llm install llm-gpt4all
```

--------------------------------

### Install LLM with pipx

Source: https://llm.datasette.io/en/stable/

Install LLM using pipx.

```bash
pipx install llm
```

--------------------------------

### Install Ollama plugin and run local model

Source: https://llm.datasette.io/en/stable/

Install the llm-ollama plugin, pull a model, and run a prompt against it.

```bash
# Install the plugin
llm install llm-ollama

# Download and run a prompt against the Orca Mini 7B model
ollama pull llama3.2:latest
llm -m llama3.2:latest 'What is the capital of France?'
```

--------------------------------

### Start chat with fragment using llm chat -f

Source: https://llm.datasette.io/en/stable/fragments.html

Starts a chat with a fragment file. The output shows the chat prompt and available commands.

```bash
llm chat -f my_doc.txt
Chatting with gpt-4
Type 'exit' or 'quit' to exit
Type '!multi' to enter multiple lines, then '!end' to finish
Type '!edit' to open your default editor and modify the prompt.
Type '!fragment <my_fragment> [<another_fragment> ...]' to insert one or more fragments
> Explain this document to me
```

--------------------------------

### Install and use llm-sentence-transformers plugin

Source: https://llm.datasette.io/en/stable/embeddings/cli.html

Installs the llm-sentence-transformers plugin and runs the MiniLM-L6 model locally on your laptop.

```bash
llm install llm-sentence-transformers
```

```bash
llm embed -c 'This is some content' -m sentence-transformers/all-MiniLM-L6-v2
```

--------------------------------

### List installed plugins

Source: https://llm.datasette.io/en/stable/plugins/tutorial-model-plugin.html

Confirms the plugin installed correctly. Output shows the plugin name, hooks, and version.

```bash
llm plugins
```

--------------------------------

### Update CLI --help examples with just cog

Source: https://llm.datasette.io/en/stable/contributing.html

Runs Cog to update the CLI --help examples in the documentation files.

```bash
just cog
```

--------------------------------

### Build documentation locally with just docs

Source: https://llm.datasette.io/en/stable/contributing.html

Starts a live preview server using sphinx-autobuild for local documentation development.

```bash
just docs
```

--------------------------------

### llm keys get --help

Source: https://llm.datasette.io/en/stable/help.html

Shows the help output for the 'llm keys get' command, which returns the value of a stored key. Includes an example usage: export OPENAI_API_KEY=$(llm keys get openai).

```text
Usage: llm keys get [OPTIONS] NAME

  Return the value of a stored key

  Example usage:

      export OPENAI_API_KEY=$(llm keys get openai)

Options:
  -h, --help  Show this message and exit.
```

--------------------------------

### Command line example: llm -m gpt4 with -o temperature

Source: https://llm.datasette.io/en/stable/plugins/tutorial-model-plugin.html

Shows how to pass options to a model using the -o/--option command line parameters.

```bash
llm -m gpt4 "ten pet pelican names" -o temperature 1.5
```

--------------------------------

### Get an async model with llm.get_async_model()

Source: https://llm.datasette.io/en/stable/python-api.html

Use `llm.get_async_model()` instead of `llm.get_model()` to obtain an async model for use with asyncio. The example retrieves the 'gpt-4o' model.

```python
import llm
model = llm.get_async_model("gpt-4o")
```

--------------------------------

### Example chat session output

Source: https://llm.datasette.io/en/stable/changelog.html

Shows the interactive prompt and responses during a chat session with Llama 2 13B. The session starts with instructions and ends with the 'exit' command.

```text
Type 'exit' or 'quit' to exit
Type '!multi' to enter multiple lines, then '!end' to finish
Type '!edit' to open your default editor and modify the prompt.
> Who are you?
Hello! I'm just an AI, here to assist you with any questions you may have.
My name is LLaMA, and I'm a large language model trained to provide helpful
and informative responses to a wide range of questions and topics. I'm here
to help you find the information you need, while ensuring a safe and
respectful conversation. Please feel free to ask me anything, and I'll do my
best to provide a helpful and accurate response.
> Tell me a joke about otters
Sure, here's a joke about otters:

Why did the otter go to the party?

Because he heard it was a "whale" of a time!

(Get it? Whale, like a big sea mammal, but also a "wild" or "fun" time.
Otters are known for their playful and social nature, so it's a lighthearted
and silly joke.)

I hope that brought a smile to your face! Do you have any other questions or
topics you'd like to discuss?
> exit
```

--------------------------------

### Show option schemas with `--options`

Source: https://llm.datasette.io/en/stable/usage.html

Add `--options` to also see documentation for the options supported by each model.

```bash
llm models --options
```

--------------------------------

### Use the Memory toolbox with a model conversation

Source: https://llm.datasette.io/en/stable/python-api.html

Shows how to instantiate the Memory toolbox and pass it to a model conversation. The example chains calls to set and get values, printing the memory state after each call.

```python
model = llm.get_model("gpt-4.1-mini")
memory = Memory()

conversation = model.conversation(tools=[memory])
print(conversation.chain("Set name to Simon", after_call=print).text())

print(memory._memory)
# Should show {'name': 'Simon'}

print(conversation.chain("Set name to Penguin", after_call=print).text())
# Now it should be {'name': 'Penguin'}

print(conversation.chain("Print current name", after_call=print).text())
```

--------------------------------

### Get token usage with response.usage()

Source: https://llm.datasette.io/en/stable/python-api.html

Call response.usage() to retrieve token usage information. The method returns a Usage object with input and output token counts, plus details. Example output shows the structure of the returned Usage object.

```python
pprint(response.usage())
```

```text
Usage(input=5,
      output=2,
      details={'candidatesTokensDetails': [{'modality': 'TEXT',
                                            'tokenCount': 2}],
               'promptTokensDetails': [{'modality': 'TEXT', 'tokenCount': 5}]})
```

--------------------------------

### Define a Memory toolbox class with llm.Toolbox

Source: https://llm.datasette.io/en/stable/python-api.html

Defines a Memory class extending llm.Toolbox with methods set, get, append, and keys. Methods not starting with underscore are exposed as tool functions. The class persists shared state in _memory across invocations.

```python
import llm

class Memory(llm.Toolbox):
    _memory = None

    def _get_memory(self):
        if self._memory is None:
            self._memory = {}
        return self._memory

    def set(self, key: str, value: str):
        "Set something as a key"
        self._get_memory()[key] = value

    def get(self, key: str):
        "Get something from a key"
        return self._get_memory().get(key) or ""

    def append(self, key: str, value: str):
        "Append something as a key"
        memory = self._get_memory()
        memory[key] = (memory.get(key) or "") + "\n" + value

    def keys(self):
        "Return a list of keys"
        return list(self._get_memory().keys())
```

--------------------------------

### Install strip-tags with uv tool

Source: https://llm.datasette.io/en/stable/schemas.html

Install the strip-tags tool using uv tool install, with alternatives pip install or pipx install.

```bash
uv tool install strip-tags
# Or "pip install" or "pipx install"
```

--------------------------------

### Install twine and upload to PyPI

Source: https://llm.datasette.io/en/stable/plugins/tutorial-model-plugin.html

Use these commands to install twine and upload your plugin's distribution archives to PyPI. You will need a PyPI account and can authenticate with a username/password or a token using '__token__' as the username.

```bash
python -m pip install twine
python -m twine upload dist/*
```

--------------------------------

### Example output of llm fragments loaders

Source: https://llm.datasette.io/en/stable/fragments.html

Sample output showing the `github` and `issue` prefixes with their documentation, including the argument format for each.

```text
github:
  Load files from a GitHub repository as fragments

  Argument is a GitHub repository URL or username/repository

issue:
  Fetch GitHub issue and comments as Markdown

  Argument is either "owner/repo/NUMBER"
  or "https://github.com/owner/repo/issues/NUMBER"
```

--------------------------------

### Install PyTorch-dependent plugin with Homebrew workaround

Source: https://llm.datasette.io/en/stable/setup.html

Workaround for installing llm-sentence-transformers with the Homebrew version of LLM (Python 3.12) by manually installing nightly PyTorch first. Run these commands in order to produce a working installation.

```bash
llm install llm-python
llm python -m pip install \
  --pre torch torchvision \
  --index-url https://download.pytorch.org/whl/nightly/cpu
llm install llm-sentence-transformers
```

--------------------------------

### Example output of `llm models --options`

Source: https://llm.datasette.io/en/stable/usage.html

Shows the detailed output including option schemas, attachment types, features, and keys for each model.

```text
OpenAI Chat: gpt-4o (aliases: 4o)
  Options:
    temperature: float
      What sampling temperature to use, between 0 and 2. Higher values like
      0.8 will make the output more random, while lower values like 0.2 will
      make it more focused and deterministic.
    max_tokens: int
      Maximum number of tokens to generate.
    top_p: float
      An alternative to sampling with temperature, called nucleus sampling,
      where the model considers the results of the tokens with top_p
      probability mass. So 0.1 means only the tokens comprising the top 10%
      probability mass are considered. Recommended to use top_p or
      temperature but not both.
    frequency_penalty: float
      Number between -2.0 and 2.0. Positive values penalize new tokens based
      on their existing frequency in the text so far, decreasing the model's
      likelihood to repeat the same line verbatim.
    presence_penalty: float
      Number between -2.0 and 2.0. Positive values penalize new tokens based
      on whether they appear in the text so far, increasing the model's
      likelihood to talk about new topics.
    stop: str
      A string where the API will stop generating further tokens.
    logit_bias: dict, str
      Modify the likelihood of specified tokens appearing in the completion.
      Pass a JSON string like '{"1712":-100, "892":-100, "1489":-100}'
    seed: int
      Integer seed to attempt to sample deterministically
    json_object: boolean
      Output a valid JSON object {...}. Prompt must mention JSON.
    image_detail: str
      Controls the detail level for image attachments. Supported values are
      low, high, and auto.
    service_tier: str
      The processing tier to use for this request - for example 'fast' for
      Fast mode (faster responses at a higher price) or 'flex' for slower,
      cheaper processing on models that support those tiers.
  Attachment types:
    application/pdf, image/gif, image/jpeg, image/png, image/webp
  Features:
  - streaming
  - schemas
  - tools
  - async
  Keys:
    key: openai
    env_var: OPENAI_API_KEY
OpenAI Chat: gpt-4o-mini (aliases: 4o-mini)
  Options:
    temperature: float
    max_tokens: int
    top_p: float
    frequency_penalty: float
    presence_penalty: float
    stop: str
    logit_bias: dict, str
    seed: int
    json_object: boolean
    image_detail: str
    service_tier: str
  Attachment types:
    application/pdf, image/gif, image/jpeg, image/png, image/webp
  Features:
  - streaming
  - schemas
  - tools
  - async
  Keys:
    key: openai
    env_var: OPENAI_API_KEY
OpenAI Chat: gpt-4.1 (aliases: 4.1)
  Options:
    temperature: float
    max_tokens: int
    top_p: float
    frequency_penalty: float
    presence_penalty: float
    stop: str
    logit_bias: dict, str
    seed: int
    json_object: boolean
    image_detail: str
    service_tier: str
  Attachment types:
    application/pdf, image/gif, image/jpeg, image/png, image/webp
  Features:
  - streaming
  - schemas
  - tools
  - async
  Keys:
    key: openai
    env_var: OPENAI_API_KEY
OpenAI Chat: gpt-4.1-mini (aliases: 4.1-mini)
  Options:
    temperature: float
    max_tokens: int
    top_p: float
    frequency_penalty: float
    presence_penalty: float
    stop: str
    logit_bias: dict, str
    seed: int
    json_object: boolean
    image_detail: str
    service_tier: str
  Attachment types:
    application/pdf, image/gif, image/jpeg, image/png, image/webp
  Features:
  - streaming
  - schemas
  - tools
  - async
  Keys:
    key: openai
    env_var: OPENAI_API_KEY
OpenAI Chat: gpt-4.1-nano (aliases: 4.1-nano)
  Options:
    temperature: float
    max_tokens: int
    top_p: float
    frequency_penalty: float
    presence_penalty: float
    stop: str
    logit_bias: dict, str
    seed: int
    json_object: boolean
    image_detail: str
    service_tier: str
  Attachment types:
    application/pdf, image/gif, image/jpeg, image/png, image/webp
  Features:
  - streaming
  - schemas
  - tools
  - async
  Keys:
    key: openai
    env_var: OPENAI_API_KEY
```

--------------------------------

### llm plugins --help

Source: https://llm.datasette.io/en/stable/help.html

Shows help for listing installed plugins, with options to include built-in plugins and filter by hook.

```bash
Usage: llm plugins [OPTIONS]

  List installed plugins

Options:
  --all        Include built-in default plugins
  --hook TEXT  Filter for plugins that implement this hook
  -h, --help   Show this message and exit.
```

--------------------------------

### Install editable plugin by path

Source: https://llm.datasette.io/en/stable/plugins/tutorial-model-plugin.html

Alternative to the previous command, passing a path to the plugin directory instead of using the current directory.

```bash
llm install -e path/to/llm-markov
```

--------------------------------

### Start a chat with a specific model

Source: https://llm.datasette.io/en/stable/usage.html

Starts an interactive chat with the specified model. Each chat starts a new conversation, and a record of each conversation can be accessed through the logs.

```bash
llm chat -m chatgpt
```

--------------------------------

### List available tools

Source: https://llm.datasette.io/en/stable/usage.html

Run `llm tools` to see a list of available tools from plugins.

```bash
llm tools
```

--------------------------------

### Run development copy of llm with uv run llm --help

Source: https://llm.datasette.io/en/stable/contributing.html

Execute the development version of the llm CLI to display its help information.

```bash
uv run llm --help
```

--------------------------------

### Install LLM with pip

Source: https://llm.datasette.io/en/stable/

Install LLM using the pip package manager.

```bash
pip install llm
```

--------------------------------

### Use Fabric prompt collection with llm-templates-fabric

Source: https://llm.datasette.io/en/stable/plugins/directory.html

Pipes the content of setup.py into LLM and uses the fabric:explain_code template from the Fabric collection. The command combines cat and llm with a pipe.

```bash
cat setup.py | llm -t fabric:explain_code
```

--------------------------------

### List server-side tools for a model

Source: https://llm.datasette.io/en/stable/usage.html

Pass `-m/--model` to include tools supported by a particular model. The example lists tools for gpt-5.6-luna.

```bash
llm tools -m gpt-5.6-luna
```

--------------------------------

### Start interactive chat with endpoint

Source: https://llm.datasette.io/en/stable/other-models.html

Starts an interactive chat with the endpoint using `--chat`.

```bash
llm openai endpoint https://example.com/v1 -m model-id --chat
```

--------------------------------

### Example output of llm models

Source: https://llm.datasette.io/en/stable/openai-models.html

Example output showing the newly added model with its alias.

```text
OpenAI Chat: gpt-3.5-turbo (aliases: 3.5, chatgpt)
OpenAI Chat: gpt-3.5-turbo-16k (aliases: chatgpt-16k, 3.5-16k)
OpenAI Chat: gpt-4 (aliases: 4, gpt4)
OpenAI Chat: gpt-3.5-turbo-0613 (aliases: 0613)
```

--------------------------------

### Base64 output example

Source: https://llm.datasette.io/en/stable/embeddings/cli.html

Example output of the llm embed command with --format base64.

```text
8NGzPFtdgTqHcZw7aUT6u+++WrwwpZo8XbSxv...
```

--------------------------------

### Execute recipe template with parameters

Source: https://llm.datasette.io/en/stable/templates.html

Command to execute the 'recipe' template with parameters for ingredients and country. The output starts with a recipe suggestion.

```bash
llm -t recipe -p ingredients 'sausages, milk' -p country Germany

```

--------------------------------

### llm models options list --help

Source: https://llm.datasette.io/en/stable/help.html

Shows usage for the llm models options list command, which lists default options for all models. Includes an example invocation.

```text
Usage: llm models options list [OPTIONS]

  List default options for all models

  Example usage:

      llm models options list

Options:
  -h, --help  Show this message and exit.
```

--------------------------------

### List available templates with llm templates

Source: https://llm.datasette.io/en/stable/templates.html

Lists all available templates. The output shows template names and their system prompts.

```bash
llm templates
```

--------------------------------

### Truncated logprobs output example

Source: https://llm.datasette.io/en/stable/changelog.html

Example of the truncated output from the logprobs query, showing the top_logprobs structure.

```json
  [
    {
      "text": "Hi",
      "top_logprobs": [
        {
          "Hi": -0.13706253,
          "Hello": -2.3714375,
          "Hey": -3.3714373
        }
      ]
    },
    {
      "text": " there",
      "top_logprobs": [
        {
          " there": -0.96057636,
          "!\"": -0.5855763,
          ".\"": -3.2574513
        }
      ]
    }
  ]
```
Tokens:
Raw