ListenBrainz

https://listenbrainz.readthedocs.io/en/latest/
Admin
ListenBrainz is a project by the MetaBrainz foundation that allows users to publicly store their...
Tokens:74,130
Snippets:507
Trust Score:8.3
Update:1 year ago
Show doc for...
Tokens:
Raw
### Initialize ListenBrainz Databases

Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env

Runs commands to create and initialize the PostgreSQL and TimescaleDB databases required for the ListenBrainz development environment.

```bash
./develop.sh manage init_db --create-db
./develop.sh manage init_ts_db --create-db
```

--------------------------------

### List Available `manage.py` Commands

Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env

Display a list of all management commands available through the `manage.py` script. This helps developers discover and understand the various operations they can perform.

```Shell
./develop.sh manage --help
```

--------------------------------

### Start Main ListenBrainz Development Environment

Source: https://listenbrainz.readthedocs.io/en/latest/developers/spark-devel-env

Before starting the Spark-specific components, it's necessary to ensure the main ListenBrainz development environment is running. This command brings up all core services required for the broader ListenBrainz application. It establishes the foundational services for your development workflow.

```Bash
./develop.sh up
```

--------------------------------

### Open Flask Shell with IPython in Webserver Container

Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env

Start an interactive Flask shell session with IPython, pre-loading the ListenBrainz application. This is ideal for development and debugging, allowing direct interaction with the application's models and logic.

```Shell
./develop.sh shell
```

--------------------------------

### Clone ListenBrainz Server Repository

Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env

Instructions to clone the ListenBrainz server repository from GitHub using Git. Windows users are advised to use WSL2 for cloning to avoid code watcher issues.

```Shell
git clone https://github.com/metabrainz/listenbrainz-server.git
```

--------------------------------

### Advanced Options for ListenBrainz Frontend Testing

Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env

These options facilitate efficient frontend testing workflows. They include running tests, updating snapshots, building frontend test containers, and performing type-checking.

```Shell
./test.sh fe             run frontend tests
```

```Shell
./test.sh fe -u          run frontend tests, update snapshots
```

```Shell
./test.sh fe -b          build frontend test containers
```

```Shell
./test.sh fe -t          run type-checker
```

--------------------------------

### Copy ListenBrainz Sample Configuration File

Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env

Copies the `config.py.sample` file to `config.py`. This new file will be used for local development configurations, allowing customization without altering the original sample.

```Shell
cp listenbrainz/config.py.sample listenbrainz/config.py
```

--------------------------------

### Start ListenBrainz Development Environment

Source: https://listenbrainz.readthedocs.io/en/latest/_sources/developers/devel-env.rst

Use this command to start all ListenBrainz services in the development environment. The web service typically listens on port 8100. Press CTRL+C to shut down the services.

```bash
./develop.sh up
```

--------------------------------

### Initialize ListenBrainz PostgreSQL and TimescaleDB Databases

Source: https://listenbrainz.readthedocs.io/en/latest/_sources/developers/devel-env.rst

Execute these commands to create and initialize the required PostgreSQL and TimescaleDB databases for the ListenBrainz development environment. This step is crucial before starting the server.

```bash
./develop.sh manage init_db --create-db
./develop.sh manage init_ts_db --create-db
```

--------------------------------

### Start ListenBrainz Development Environment

Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env

Starts all ListenBrainz services using `develop.sh up`. The web service listens on port 8100 by default. Users can change the port in `docker/docker-compose.yml`.

```bash
./develop.sh up
```

--------------------------------

### Open PostgreSQL Session to ListenBrainz Database

Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env

Use the `develop.sh` utility to establish a psql session directly to the main ListenBrainz database. This provides command-line access for database inspection and management.

```Shell
./develop.sh psql
```

--------------------------------

### Start ListenBrainz Spark Development Environment

Source: https://listenbrainz.readthedocs.io/en/latest/developers/spark-devel-env

Once the main ListenBrainz environment is active, this command starts the dedicated Spark environment. It brings up all Spark-related containers, including the spark reader. This prepares the Spark cluster for data processing and analysis tasks.

```Bash
./develop.sh spark up
```

--------------------------------

### Open PostgreSQL Session to Timescale Database

Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env

Connect to the Timescale database, which stores user listens, using the `develop.sh` utility. This allows direct interaction with the time-series data.

```Shell
./develop.sh timescale
```

--------------------------------

### Display Docker Compose Help Information

Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env

Show the help output for Docker Compose, listing all valid commands and options. This is useful for understanding the capabilities of Docker Compose when managing the ListenBrainz development environment.

```Shell
./develop.sh help
```

--------------------------------

### Start ListenBrainz Webserver

Source: https://listenbrainz.readthedocs.io/en/latest/_sources/developers/spark-architecture.rst

This command starts the ListenBrainz webserver for development purposes, setting up the necessary environment.

```bash
./develop.sh up
```

--------------------------------

### Configure CritiqueBrainz API Credentials

Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env

Sets up environment variables for CritiqueBrainz API integration, including client ID, client secret, and redirect URI. These are essential for authentication and callback handling.

```bash
CRITIQUEBRAINZ_CLIENT_ID = ''
CRITIQUEBRAINZ_CLIENT_SECRET = ''
CRITIQUEBRAINZ_REDIRECT_URI = 'http://localhost:8100/settings/music-services/critiquebrainz/callback/'
```

--------------------------------

### Run ListenBrainz Frontend Tests

Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env

Execute tests specifically for the frontend codebase using the `test.sh` script. This is essential for validating changes made to the user interface.

```Shell
./test.sh fe
```

--------------------------------

### Advanced Options for Running ListenBrainz Tests

Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env

These options provide flexibility for faster and more specific test execution. They allow building unit test containers, running specific test files or directories, and managing the lifecycle of test containers (stopping or removing them).

```Shell
./test.sh -u                                 # build unit test containers, start up and initialise the database
```

```Shell
./test.sh [path-to-tests-file-or-directory]  # run specific tests, do this as often as you need to
```

```Shell
./test.sh -s                                 # stop test containers, but don't remove them
```

```Shell
./test.sh -d                                 # stop and remove all test containers
```

--------------------------------

### Pass Commands Directly to Docker Compose via `develop.sh`

Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env

This command allows direct execution of any Docker Compose command through the `develop.sh` wrapper script. It provides a convenient way to interact with the underlying Docker Compose setup.

```Shell
./develop.sh <command>
```

--------------------------------

### Access Local ListenBrainz Instance

Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env

The URL to access the running ListenBrainz development environment in a web browser, assuming the default port 8100 is used.

```plaintext
http://localhost:8100
```

--------------------------------

### Configure Spotify API Credentials in config.py

Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env

Populates the `SPOTIFY_CLIENT_ID` and `SPOTIFY_CLIENT_SECRET` fields in `config.py` with credentials obtained from the Spotify Developer Dashboard. This setup is required for the Spotify importer to function correctly.

```Python
# SPOTIFY
SPOTIFY_CLIENT_ID = ''
SPOTIFY_CLIENT_SECRET = ''
```

--------------------------------

### Rebuild ListenBrainz Web Service

Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env

Command to rebuild only the `web` service Docker image. This is necessary if new Python dependencies are added to `requirements.txt`.

```bash
./develop.sh build web
```

--------------------------------

### Build ListenBrainz Docker Containers

Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env

Executes the development script to build all necessary Docker images for the ListenBrainz services. This command uses `docker-compose` internally and may take time on the first run.

```bash
./develop.sh build
```

--------------------------------

### Start ListenBrainz Webserver

Source: https://listenbrainz.readthedocs.io/en/latest/developers/spark-architecture

Command to start the ListenBrainz webserver using the development script. This is typically the first step in setting up the local development environment.

```Shell
./develop.sh up
```

--------------------------------

### Complete Single Listen Submission JSON Example

Source: https://listenbrainz.readthedocs.io/en/latest/_sources/users/json.rst

A comprehensive example of a `single` listen submission JSON document, showing all typical fields including `additional_info` for media player, client, and duration, providing a full context for a listen.

```json
{
  "listen_type": "single",
  "payload": [
    {
      "listened_at": 1443521965,
      "track_metadata": {
        "additional_info": {
          "media_player": "Rhythmbox",
          "submission_client": "Rhythmbox ListenBrainz Plugin",
          "submission_client_version": "1.0",
          "release_mbid": "bf9e91ea-8029-4a04-a26a-224e00a83266",
          "artist_mbids": [
            "db92a151-1ac2-438b-bc43-b82e149ddd50"
          ],
          "recording_mbid": "98255a8c-017a-4bc7-8dd6-1fa36124572b",
          "tags": [ "you", "just", "got", "rick rolled!"],
          "duration_ms": 222000
        },
        "artist_name": "Rick Astley",
        "track_name": "Never Gonna Give You Up",
        "release_name": "Whenever you need somebody"
      }
    }
  ]
}
```

--------------------------------

### Run All ListenBrainz Unit and Integration Tests

Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env

This script builds and runs containers for all unit and integration tests, ensuring test data is isolated from development data. It's a crucial step before committing new code or creating a pull request.

```Shell
./test.sh
```

--------------------------------

### Rebuild ListenBrainz Static Builder Service

Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env

Command to rebuild only the `static_builder` service Docker image. This is required if new JavaScript dependencies are introduced.

```bash
./develop.sh build static_builder
```

--------------------------------

### Open Redis Shell

Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env

Access the Redis command-line interface for interacting with the Redis instance. This is useful for inspecting cached data or managing Redis keys.

```Shell
./develop.sh redis
```

--------------------------------

### Open Bash Shell in Webserver Container

Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env

Gain shell access to the webserver container for debugging or administrative tasks. This provides a direct command-line interface within the running container environment.

```Shell
./develop.sh bash
```

--------------------------------

### Start ListenBrainz Spark Environment

Source: https://listenbrainz.readthedocs.io/en/latest/_sources/developers/spark-devel-env.rst

Initiate the ListenBrainz Spark environment, which also brings up the spark reader container.

```bash
./develop.sh spark up
```

--------------------------------

### Example Console Output: Listen History

Source: https://listenbrainz.readthedocs.io/en/latest/users/api-usage

This block shows a sample of the console output generated by the `get_listens` Python script, illustrating the format of the retrieved track names and their corresponding listen timestamps.

```Text
Track: Never Gonna Give You Up, listened at 1512040365
Track: Never Gonna Give You Up, listened at 1511977429
Track: Never Gonna Give You Up, listened at 1511968583
Track: Never Gonna Give You Up, listened at 1443521965
Track: Never Gonna Give You Up, listened at 42042042
```

--------------------------------

### Bring Up Main ListenBrainz Development Environment

Source: https://listenbrainz.readthedocs.io/en/latest/_sources/developers/spark-devel-env.rst

Before starting the Spark environment, ensure that the main ListenBrainz development environment is running.

```bash
./develop.sh up
```

--------------------------------

### Inspect Static Builder Logs for ESLint Output

Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env

This command filters `develop.sh` logs to display only the output from the `static_builder` service. It is particularly useful for reviewing ESLint results and other frontend build messages.

```Shell
./develop.sh logs -f static_builder
```

--------------------------------

### Configure Last.fm API Key in config.py

Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env

Sets the `LASTFM_API_KEY` in `config.py` using the key acquired from the Last.fm API page. This configuration is essential for enabling the Last.fm importer functionality within ListenBrainz.

```Python
# Lastfm API
LASTFM_API_URL = "https://ws.audioscrobbler.com/2.0/"
LASTFM_API_KEY = "USE_LASTFM_API_KEY"
```

--------------------------------

### Run ListenBrainz Frontend Tests

Source: https://listenbrainz.readthedocs.io/en/latest/_sources/developers/devel-env.rst

Execute frontend tests for the ListenBrainz project. This command is used when changes are made to the frontend.

```bash
./test.sh fe
```

--------------------------------

### ListenBrainz API: Complete Single Listen Submission Example

Source: https://listenbrainz.readthedocs.io/en/latest/users/json

Presents a full example of a `single` listen submission JSON document, combining the top-level structure with a detailed listen payload, including `additional_info` fields like media player and submission client.

```JSON
{
  "listen_type": "single",
  "payload": [
    {
      "listened_at": 1443521965,
      "track_metadata": {
        "additional_info": {
          "media_player": "Rhythmbox",
          "submission_client": "Rhythmbox ListenBrainz Plugin",
          "submission_client_version": "1.0",
          "release_mbid": "bf9e91ea-8029-4a04-a26a-224e00a83266",
          "artist_mbids": [
            "db92a151-1ac2-438b-bc43-b82e149ddd50"
          ],
          "recording_mbid": "98255a8c-017a-4bc7-8dd6-1fa36124572b",
          "tags": [ "you", "just", "got", "rick rolled!"],
          "duration_ms": 222000
        },
        "artist_name": "Rick Astley",
        "track_name": "Never Gonna Give You Up",
        "release_name": "Whenever you need somebody"
      }
    }
  ]
}
```

--------------------------------

### Submit Listens to ListenBrainz API (Python)

Source: https://listenbrainz.readthedocs.io/en/latest/users/api-usage

This Python code demonstrates how to submit music listens to the ListenBrainz API. It includes a `submit_listen` function that handles API requests, authentication, and error handling. The example shows how to construct a payload for a single track and submit it to either a local development server or the production API.

```Python
from time import time
import requests

# Set DEBUG to True to test local dev server.
# API keys for local dev server and the real server are different.
DEBUG = True
ROOT = 'http://localhost:8100' if DEBUG else 'https://api.listenbrainz.org'

def submit_listen(listen_type, payload, token):
    """Submits listens for the track(s) in payload.

    Args:
        listen_type (str): either of 'single', 'import' or 'playing_now'
        payload: A list of Track dictionaries.
        token: the auth token of the user you're submitting listens for

    Returns:
         The json response if there's an OK status.

    Raises:
         An HTTPError if there's a failure.
         A ValueError is the JSON in the response is invalid.
    """

    response = requests.post(
        url="{0}/1/submit-listens".format(ROOT),
        json={
            "listen_type": listen_type,
            "payload": payload,
        },
        headers={
            "Authorization": "Token {0}".format(token)
        }
    )

    response.raise_for_status()

    return response.json()

if __name__ == "__main__":
    EXAMPLE_PAYLOAD = [
        {
            # An example track.
            "listened_at": int(time()),
            "track_metadata": {
                "artist_name": "Rick Astley",
                "track_name": "Never Gonna Give You Up",
                "release_name": "Whenever you need somebody"
            }
        }
    ]

    # Input token from the user and call submit listen
    token = input('Please enter your auth token: ')
    json_response = submit_listen(listen_type='single', payload=EXAMPLE_PAYLOAD, token=token)

    print("Response was: {0}".format(json_response))
    print("Check your listens - there should be a Never Gonna Give You Up track, played recently.")
```

--------------------------------

### Fetched Listen Data JSON Example

Source: https://listenbrainz.readthedocs.io/en/latest/_sources/users/json.rst

An example of the JSON document structure returned when fetching listen data from the API, showing the `payload` object containing `count`, `user_id`, and a placeholder for the `listens` array.

```json
{
  "payload": {
    "count": 25,
    "user_id": "-- the MusicBrainz ID of the user --",
    "listens": [
      "-- listen data here ---"
    ]
  }
}
```

--------------------------------

### Example MusicBrainz Metadata Lookup Response JSON

Source: https://listenbrainz.readthedocs.io/en/latest/_sources/users/api-usage.rst

An example JSON structure returned by the ListenBrainz metadata API when looking up MusicBrainz IDs (MBIDs) and associated metadata for a recording. It includes details like artist credits, recording MBID and name, and release information.

```json
{
    "artist_credit_name": "Ariana Grande",
    "artist_mbids": [
        "f4fdbb4c-e4b7-47a0-b83b-d91bbfcfa387"
    ],
    "metadata": {
        "recording": {
            "rels": [
                {
                    "artist_mbid": "eb811bf7-4c99-4781-84c0-10ba6b8e33b3",
                    "artist_name": "Carl Falk",
                    "instrument": "guitar",
                    "type": "instrument"
                },
                {
                    "artist_mbid": "c8af4490-e48a-4f91-aef9-2b1e39369576",
                    "artist_name": "Savan Kotecha",
                    "instrument": "background vocals",
                    "type": "vocal"
                },
                {
                    "artist_mbid": "0d33cc88-28ae-44d5-be7e-7a653e518720",
                    "artist_name": "Jeanette Olsson",
                    "instrument": "background vocals",
                    "type": "vocal"
                }
            ]
        }
    },
    "recording_mbid": "9f24c0f7-a644-4074-8fbd-a1dba03de129",
    "recording_name": "One Last Time",
    "release_mbid": "be5d97b1-408a-4e95-b924-0a61955048de",
    "release_name": "My Everything"
}
```

--------------------------------

### Clone ListenBrainz Server Repository

Source: https://listenbrainz.readthedocs.io/en/latest/_sources/developers/devel-env.rst

Instructions to clone the ListenBrainz server repository from GitHub using Git. This is the initial step to obtain the source code and begin setting up the development environment.

```bash
git clone https://github.com/metabrainz/listenbrainz-server.git
```

--------------------------------

### Run ListenBrainz Backend Tests

Source: https://listenbrainz.readthedocs.io/en/latest/_sources/developers/devel-env.rst

This snippet provides commands to manage the backend test infrastructure, including building containers, initializing the database, running specific tests, and stopping/removing containers.

```bash
./test.sh -u                                 # build unit test containers, start up and initialise the database
./test.sh [path-to-tests-file-or-directory]  # run specific tests, do this as often as you need to
./test.sh -s                                 # stop test containers, but don't remove them
./test.sh -d                                 # stop and remove all test containers
```

--------------------------------

### Advanced ListenBrainz Frontend Testing Options

Source: https://listenbrainz.readthedocs.io/en/latest/_sources/developers/devel-env.rst

This snippet lists various options for efficient frontend testing, including running tests, updating snapshots, building containers, and running the type-checker.

```bash
./test.sh fe             run frontend tests
./test.sh fe -u          run frontend tests, update snapshots
./test.sh fe -b          build frontend test containers
./test.sh fe -t          run type-checker
```

--------------------------------

### ListenBrainz API: Get User Listens Endpoint

Source: https://listenbrainz.readthedocs.io/en/latest/users/api-usage

Documentation for the ListenBrainz API endpoint used to retrieve a user's listening history.

```APIDOC
Endpoint: GET /1/user/{username}/listens
Description: Retrieves the listening history for a specified user.
Parameters:
  username (string, path): The MusicBrainz ID of the user.
  min_ts (integer, query, optional): History before this timestamp will not be returned. Do not use with max_ts.
  max_ts (integer, query, optional): History after this timestamp will not be returned. Do not use with min_ts.
  count (integer, query, optional): How many listens to return. If not specified, uses a default from the server.
Headers:
  Authorization (string, optional): Token for relaxed rate limits.
Returns:
  A list of listen info dictionaries (JSON array).
Errors:
  HTTPError: If there's a failure in the HTTP request.
  ValueError: If the JSON in the response is invalid.
  IndexError: If the JSON is not structured as expected.
```

--------------------------------

### Initialize ListenBrainz Configuration File

Source: https://listenbrainz.readthedocs.io/en/latest/_sources/developers/devel-env.rst

Command to copy the sample configuration file (config.py.sample) to the active configuration file (config.py) for ListenBrainz. This action is a prerequisite before customizing specific settings like OAuth credentials and API keys.

```bash
cp listenbrainz/config.py.sample listenbrainz/config.py
```

--------------------------------

### Get Latest Import Timestamp from ListenBrainz (Python)

Source: https://listenbrainz.readthedocs.io/en/latest/users/api-usage

This Python script retrieves the latest import timestamp for a specified user and service from the ListenBrainz API. It requires a username and an optional service name. The script demonstrates how to query the API using a GET request and parse the JSON response to obtain the timestamp.

```python
import requests

# Set DEBUG to True to test local dev server.
# API keys for local dev server and the real server are different.
DEBUG = True
ROOT = 'http://localhost:8100' if DEBUG else 'https://api.listenbrainz.org'

# The token can be any valid token.
TOKEN = 'YOUR_TOKEN_HERE'
AUTH_HEADER = {
    "Authorization": "Token {0}".format(TOKEN)
}

def get_latest_import(username, service="lastfm"):
    """Gets the latest import timestamp of a given user.

    Args:
        username: User to get latest import time of.
        service: service to get latest import time of.

    Returns:
        A Unix timestamp if there's an OK status.

    Raises:
        An HTTPError if there's a failure.
        A ValueError if the JSON in the response is invalid.
        An IndexError if the JSON is not structured as expected.
    """
    response = requests.get(
        url="{0}/1/latest-import".format(ROOT),
        params={
            "user_name": username,
            "service": service
        },
        headers=AUTH_HEADER,
    )

    response.raise_for_status()
    return response.json()["latest_import"]

if __name__ == "__main__":
    username = input('Please input the MusicBrainz ID of the user: ')
    timestamp = get_latest_import(username)

    print("User {0} last imported on {1}".format(username, timestamp))
```

--------------------------------

### Sample Listen Payload JSON

Source: https://listenbrainz.readthedocs.io/en/latest/_sources/users/json.rst

Provides an example of the `track_metadata` structure within a listen payload, including `listened_at` and various fields under `additional_info` such as MBIDs and tags.

```json
{
  "listened_at": 1443521965,
  "track_metadata": {
    "additional_info": {
      "release_mbid": "bf9e91ea-8029-4a04-a26a-224e00a83266",
      "artist_mbids": [
        "db92a151-1ac2-438b-bc43-b82e149ddd50"
      ],
      "recording_mbid": "98255a8c-017a-4bc7-8dd6-1fa36124572b",
      "tags": [ "you", "just", "got", "rick rolled!"]
    },
    "artist_name": "Rick Astley",
    "track_name": "Never Gonna Give You Up",
    "release_name": "Whenever you need somebody"
  }
}
```

--------------------------------

### Inspect ListenBrainz Frontend Build Logs

Source: https://listenbrainz.readthedocs.io/en/latest/_sources/developers/devel-env.rst

Use this command to filter and view only the static_builder logs, which is useful when there are many logs in the terminal during frontend development.

```bash
./develop.sh logs -f static_builder
```

--------------------------------

### Build ListenBrainz Development Docker Containers

Source: https://listenbrainz.readthedocs.io/en/latest/_sources/developers/devel-env.rst

Run this command in the repository root to build all necessary Docker images for the ListenBrainz server services. This process might take time on the first run as dependencies are downloaded and services are built.

```bash
./develop.sh build
```

--------------------------------

### Configure Docker Compose Port Mapping

Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env

Illustrates how to modify the `ports` section in `docker/docker-compose.yml` to change the listening port of the web service. The first value before the colon can be adjusted.

```yaml
ports:
- "8100:80"
```

--------------------------------

### Command-line options for develop.sh-manage-init_db

Source: https://listenbrainz.readthedocs.io/en/latest/genindex

Initializes the main ListenBrainz database, with an option to force recreation if it already exists.

```APIDOC
Command: ./develop.sh-manage-init_db
  Options:
    --create-db
    --force
    -f
```

--------------------------------

### Display docker-compose help

Source: https://listenbrainz.readthedocs.io/en/latest/developers/develop-sh

This command shows the help output for `docker-compose`, listing all available commands and their usage. It's equivalent to running `docker-compose help` directly.

```bash
./develop.sh help
```

--------------------------------

### Initialize Main Database

Source: https://listenbrainz.readthedocs.io/en/latest/developers/commands

This command initializes the primary database for the application. It sets up the table structure, creates primary and foreign keys, and establishes necessary indexes for optimal performance.

```Shell
./develop.sh manage init_db [OPTIONS]
```

--------------------------------

### Invoke ListenBrainz `manage.py` Commands

Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env

Execute specific management commands provided by the `manage.py` script within a Docker container. This allows running various administrative and maintenance tasks for the ListenBrainz application.

```Shell
./develop.sh manage <command>
```

--------------------------------

### Command-line options for develop.sh-manage-init_ts_db

Source: https://listenbrainz.readthedocs.io/en/latest/genindex

Initializes the TimescaleDB database for ListenBrainz, with an option to force recreation.

```APIDOC
Command: ./develop.sh-manage-init_ts_db
  Options:
    --create-db
    --force
    -f
```

--------------------------------

### Run ListenBrainz Unit and Integration Tests

Source: https://listenbrainz.readthedocs.io/en/latest/_sources/developers/devel-env.rst

Execute this script to run all unit and integration tests for ListenBrainz. It sets up isolated test-specific data volumes to prevent interference with development data. Run this before committing code or making pull requests.

```bash
./test.sh
```

--------------------------------

### Configure MusicBrainz OAuth Credentials in config.py

Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env

Updates the `OAUTH_CLIENT_ID` and `OAUTH_CLIENT_SECRET` variables in `config.py` with the client ID and secret obtained from MusicBrainz application registration. This enables authentication and login via MusicBrainz accounts.

```Python
# MusicBrainz OAuth
OAUTH_CLIENT_ID = "CLIENT_ID"
OAUTH_CLIENT_SECRET = "CLIENT_SECRET"
```

--------------------------------

### Format ListenBrainz Spark HDFS Namenode

Source: https://listenbrainz.readthedocs.io/en/latest/developers/spark-devel-env

This command is used to format the `namenode` container, which is essential for initializing the HDFS filesystem within the Spark environment. It's typically run during the initial setup or when you need to clear all existing data. Running this command will shut down the Spark Docker cluster, remove data volumes, and recreate the HDFS filesystem, effectively resetting your data.

```Bash
./develop.sh spark format
```

--------------------------------

### GET /1/user/(user_name)/fresh_releases

Source: https://listenbrainz.readthedocs.io/en/latest/users/api/misc

Get fresh releases data for the given user.

```APIDOC
GET /1/user/(*user_name*)/fresh_releases
Description: Get fresh releases data for the given user.
Parameters:
  - sort: The sort order of the results. Must be one of “release_date”, “artist_credit_name”, “release_name”, or “confidence”. Default “release_date”.
  - past: Whether to show releases in the past. Default True.
  - future: Whether to show releases in the future. Default True.
Example Response:
```json
{
    "artist_credit_name":"Santi & Tuğçe",
    "artist_mbids":[
       "4690076b-1446-43f4-8a84-cfec56dc3601"
    ],
    "caa_id":37026686379,
    "caa_release_mbid":"9432fb06-bd84-4f2e-9386-b2f14e0de54d",
    "confidence":2,
    "release_date":"2023-10-20",
    "release_group_mbid":"30ac1ded-c254-46ac-a41f-7453dada6ae7",
    "release_group_primary_type":"Album",
    "release_group_secondary_type":null,
    "release_mbid":"9432fb06-bd84-4f2e-9386-b2f14e0de54d",
    "release_name":"The Marvelous Real",
    "release_tags":null
}
```
Status Codes:
  - 200 OK: fetch succeeded
  - 400 Bad Request: invalid date or number of days passed.
Response Headers:
  - Content-Type: application/json
```

--------------------------------

### Create ListenBrainz Sample Data Dump

Source: https://listenbrainz.readthedocs.io/en/latest/developers/commands

This command creates a sample data dump for ListenBrainz. Users can specify the output directory and the number of threads to use for compression, optimizing the dump creation process.

```Bash
./develop.sh manage dump create_sample [OPTIONS]
```

```APIDOC
Options:
-l, --location <location>: path to the directory where the dump should be made
-t, --threads <threads>: the number of threads to be used while compression
```

--------------------------------

### Create All MBID Mapping Database and Index Components

Source: https://listenbrainz.readthedocs.io/en/latest/_sources/developers/mapping.rst

This command orchestrates the full setup of the MBID mapping system. It first builds the necessary mapping database tables and then proceeds to build the Typesense index directly afterwards, ensuring all data structures are initialized and ready for operation in a single command.

```bash
python mapper/manage.py create-all
```

--------------------------------

### Build ListenBrainz Spark Docker Containers

Source: https://listenbrainz.readthedocs.io/en/latest/developers/spark-devel-env

This command initiates the build process for the ListenBrainz Spark Docker containers. It compiles and packages all necessary components into runnable images. This step is crucial for preparing the Spark environment for execution.

```Bash
./develop.sh spark build
```

--------------------------------

### Option: -f

Source: https://listenbrainz.readthedocs.io/en/latest/genindex

A short option used for database initialization commands.

```APIDOC
Option: -f
  Description: A short option.
  Used by:
    - ./develop.sh-manage-init_db
    - ./develop.sh-manage-init_ts_db
```

--------------------------------

### GET /1/user/(*user_name*)/listen-count API Endpoint

Source: https://listenbrainz.readthedocs.io/en/latest/users/api/core

Get the total number of listens for a specified user. The response payload contains a 'count' key with the listen count.

```APIDOC
Method: GET
Path: /1/user/(*user_name*)/listen-count
Status Codes:
  200 OK: Yay, you have listen counts!
  404 Not Found: The requested user was not found.
Response Headers:
  Content-Type: application/json
```

--------------------------------

### GET /1/stats/user/{user_name}/artist-activity

Source: https://listenbrainz.readthedocs.io/en/latest/users/api/statistics

Get the artist activity for user `user_name`. The artist activity shows the total number of listens for each artist along with their albums and corresponding listen counts.

```APIDOC
Endpoint: GET /1/stats/user/{user_name}/artist-activity
Description: Get the artist activity for user user_name. The artist activity shows the total number of listens for each artist along with their albums and corresponding listen counts.
Example Response:
```json
{
    "result": [
        {
            "name": "Radiohead",
            "listen_count": 120,
            "albums": [
                {
                    "name": "In Rainbows",
                    "listen_count": 50
                },
                {
                    "name": "OK Computer",
                    "listen_count": 70
                }
            ]
        }
    ]
}
```
Parameters:
  - range (str): Optional, time interval for which statistics should be returned. Possible values are ALLOWED_STATISTICS_RANGE, defaults to all_time.
Status Codes:
  - 200 OK: Successful query, you have data!
  - 204 No Content: Statistics for the user haven’t been calculated, empty response will be returned.
  - 400 Bad Request: Bad request, check response['error'] for more details.
  - 404 Not Found: User not found.
Response Headers:
  - Content-Type: application/json
```

--------------------------------

### Rebuild ListenBrainz Web Service

Source: https://listenbrainz.readthedocs.io/en/latest/_sources/developers/devel-env.rst

If new Python dependencies are added to `requirements.txt`, use this command to rebuild only the `web` Docker service, ensuring the new dependencies are included.

```bash
./develop.sh build web
```

--------------------------------

### GET /1/stats/user/{user_name}/listening-activity

Source: https://listenbrainz.readthedocs.io/en/latest/users/api/statistics

Get the listening activity for user `user_name`. The listening activity shows the number of listens the user has submitted over a period of time, grouped by time ranges.

```APIDOC
Endpoint: GET /1/stats/user/{user_name}/listening-activity
Description: Get the listening activity for user user_name. The listening activity shows the number of listens the user has submitted over a period of time.
Example Response:
```json
{
    "payload": {
        "from_ts": 1587945600,
        "last_updated": 1592807084,
        "listening_activity": [
            {
                "from_ts": 1587945600,
                "listen_count": 26,
                "time_range": "Monday 27 April 2020",
                "to_ts": 1588031999
            },
            {
                "from_ts": 1588032000,
                "listen_count": 57,
                "time_range": "Tuesday 28 April 2020",
                "to_ts": 1588118399
            },
            {
                "from_ts": 1588118400,
                "listen_count": 33,
                "time_range": "Wednesday 29 April 2020",
                "to_ts": 1588204799
            }
        ],
        "to_ts": 1589155200,
        "user_id": "ishaanshah"
    }
}
```
Parameters:
  - range (str): Optional, time interval for which statistics should be returned. Possible values are ALLOWED_STATISTICS_RANGE, defaults to all_time.
Status Codes:
  - 200 OK: Successful query, you have data!
  - 204 No Content: Statistics for the user haven’t been calculated, empty response will be returned.
  - 400 Bad Request: Bad request, check response['error'] for more details.
  - 404 Not Found: User not found.
Response Headers:
  - Content-Type: application/json
```

--------------------------------

### Command-line options for develop.sh-manage-run-daily-jams

Source: https://listenbrainz.readthedocs.io/en/latest/genindex

Executes daily 'jams' related tasks, with an option to create all necessary components.

```APIDOC
Command: ./develop.sh-manage-run-daily-jams
  Options:
    --create-all
```

--------------------------------

### API: Get ListenBrainz Data Dump Info

Source: https://listenbrainz.readthedocs.io/en/latest/users/api/misc

Retrieves information about ListenBrainz data dumps. An optional 'id' parameter can be provided to get details about a specific dump; otherwise, information about the latest dump is returned.

```APIDOC
GET /1/status/get-dump-info
Description: Get information about ListenBrainz data dumps.
Query Parameters:
  id: Integer specifying the ID of the dump. If not provided, returns info about the latest dump.
Status Codes:
  200 OK: You have data.
  400 Bad Request: You did not provide a valid dump ID. See error message for details.
  404 Not Found: Dump with given ID does not exist.
Response Headers:
  Content-Type: application/json
```

```json
{
    "id": 1,
    "timestamp": "20190625-165900"
}
```

--------------------------------

### Copy Sample Spark Configuration File

Source: https://listenbrainz.readthedocs.io/en/latest/_sources/developers/spark-devel-env.rst

The Spark environment requires a 'config.py' file within the 'listenbrainz_spark/' directory. Create this file by copying from the provided sample configuration.

```bash
cp listenbrainz_spark/config.py.sample listenbrainz_spark/config.py
```

--------------------------------

### GET /1/stats/artist/{artist_mbid}/listeners API

Source: https://listenbrainz.readthedocs.io/en/latest/users/api/statistics

Get top listeners for a specific artist identified by `artist_mbid`. This endpoint provides the total listen count for the artist and a list of top N listeners with their individual listen counts within a given time range.

```APIDOC
GET /1/stats/artist/({artist_mbid})/listeners
Parameters:
  range (str, optional): Time interval for which statistics should be returned. Possible values are ALLOWED_STATISTICS_RANGE, defaults to 'all_time'.
Status Codes:
  200 OK: Successful query, data available.
  204 No Content: Statistics not calculated or entity does not exist, empty response.
  400 Bad Request: Invalid request, check response['error'] for details.
  404 Not Found: Entity not found.
Response Headers:
  Content-Type: application/json
```

```JSON
{
  "payload": {
    "artist_mbid": "00034ede-a1f1-4219-be39-02f36853373e",
    "artist_name": "O Rappa",
    "from_ts": 1009843200,
    "last_updated": 1681839677,
    "listeners": [
      {
        "listen_count": 2469,
        "user_name": "RosyPsanda"
      },
      {
        "listen_count": 1858,
        "user_name": "alexyagui"
      },
      {
        "listen_count": 578,
        "user_name": "rafael_gn"
      },
      {
        "listen_count": 8,
        "user_name": "italooliveira"
      },
      {
        "listen_count": 7,
        "user_name": "paulodesouza"
      },
      {
        "listen_count": 1,
        "user_name": "oldpunisher"
      }
    ],
    "stats_range": "all_time",
    "to_ts": 1681777035,
    "total_listen_count": 16393
  }
}
```

--------------------------------

### Option: -lp

Source: https://listenbrainz.readthedocs.io/en/latest/genindex

A short option used with the dump-create_full command.

```APIDOC
Option: -lp
  Description: A short option.
  Used by:
    - ./develop.sh-manage-dump-create_full
```

--------------------------------

### List available manage.py commands

Source: https://listenbrainz.readthedocs.io/en/latest/developers/develop-sh

This command displays a list of all available subcommands and options for the `manage.py` script. It's useful for discovering what management tasks can be performed.

```bash
./develop.sh manage --help
```

--------------------------------

### Artist Popularity Data Response Example

Source: https://listenbrainz.readthedocs.io/en/latest/users/api/popularity

Example JSON response structure for artist popularity data, including total listen count and total user count. Demonstrates how data is returned for found and unfound artist MBIDs, with null counts for missing data.

```JSON
[
    {
        "artist_mbid": "13dd61c7-ce73-4e97-9f0c-9f0e53144411",
        "total_listen_count": 1000,
        "total_user_count": 10
    },
    {
        "artist_mbid": "22ad712e-ce73-9f0c-4e97-9f0e53144411",
        "total_listen_count": null,
        "total_user_count": null
    }
]
```

--------------------------------

### API: General Metadata Endpoint Parameters and Example Response Fragment

Source: https://listenbrainz.readthedocs.io/en/latest/users/api/metadata

This section provides general parameters and status codes applicable to certain metadata endpoints, along with an example JSON fragment showing a tag structure, likely part of a larger response.

```APIDOC
Parameters:
  release_group_mbids (str): A comma separated list of release_group_mbids
  inc (str): A space separated list of “artist”, “tag” and/or “release” to indicate which portions of metadata you’re interested in fetching. We encourage users to only fetch the data they plan to consume.
Status Codes:
  200 OK: you have data!
  400 Bad Request: invalid release_group_mbid arguments
```

```JSON
{
   "count": 18,
   "genre_mbid": "45eb1d9c-588c-4dc8-9394-a14b7c8f02bc",
   "tag": "trip hop"
}
```

--------------------------------

### Import Sample Data into ListenBrainz Spark

Source: https://listenbrainz.readthedocs.io/en/latest/developers/spark-devel-env

This command facilitates the import of small data dumps into your ListenBrainz Spark environment. These dumps are provided to help developers work with realistic ListenBrainz data for testing and development purposes. Execute this in a separate terminal to populate your Spark cluster with sample information.

```Bash
./develop.sh spark run spark_reader python manage.py spark request_import_incremental
```

--------------------------------

### GET /1/stats/release-group/{release_group_mbid}/listeners API

Source: https://listenbrainz.readthedocs.io/en/latest/users/api/statistics

Get top listeners for a specific release group identified by `release_group_mbid`. This endpoint provides the total listen count for the entity and a list of top N listeners with their individual listen counts within a given time range.

```APIDOC
GET /1/stats/release-group/({release_group_mbid})/listeners
```

```JSON
{
  "payload": {
    "artist_mbids": [
      "c234fa42-e6a6-443e-937e-2f4b073538a3"
    ],
    "artist_name": "Chris Brown",
    "caa_id": 23564822587,
    "caa_release_mbid": "25f18616-5a9c-470e-964d-4eb8a511435b",
    "from_ts": 1009843200,
    "last_updated": 1681843150,
    "listeners": [
      {
        "listen_count": 2365,
        "user_name": "purpleyor"
      },
      {
        "listen_count": 570,
        "user_name": "dndty"
      },
      {
        "listen_count": 216,
        "user_name": "iammsyre"
      },
      {
        "listen_count": 141,
        "user_name": "dpmittal"
      },
      {
        "listen_count": 33,
        "user_name": "tazlad"
      },
      {
        "listen_count": 30,
        "user_name": "ratkutti"
      },
      {
        "listen_count": 22,
        "user_name": "Raymorjamiek"
      },
      {
        "listen_count": 21,
        "user_name": "MJJMC"
      },
      {
        "listen_count": 12,
        "user_name": "fookever"
      },
      {
        "listen_count": 8,
        "user_name": "Jamjamk12071983"
      },
      {
        "listen_count": 1,
        "user_name": "hassanymoses"
      },
      {
        "listen_count": 1,
        "user_name": "iJays"
      }
    ],
    "release_group_mbid": "087b3a7d-d532-44d9-b37a-84427677ddcd",
    "release_group_name": "Indigo",
    "stats_range": "all_time",
    "to_ts": 1681777035
  }
}
```

--------------------------------

### Configure CritiqueBrainz API Credentials

Source: https://listenbrainz.readthedocs.io/en/latest/_sources/developers/devel-env.rst

Update the ListenBrainz configuration file with the client ID and client secret obtained from the CritiqueBrainz applications page. This enables integration with CritiqueBrainz for music services.

```yaml
# CRITIQUEBRAINZ
CRITIQUEBRAINZ_CLIENT_ID = ''
CRITIQUEBRAINZ_CLIENT_SECRET = ''
CRITIQUEBRAINZ_REDIRECT_URI = 'http://localhost:8100/settings/music-services/critiquebrainz/callback/'
```

--------------------------------

### init_db Command Line Interface

Source: https://listenbrainz.readthedocs.io/en/latest/developers/commands

Detailed interface for the `init_db` management command, used to initialize the main application database. It supports options for forced re-initialization and database creation.

```APIDOC
Command: ./develop.sh manage init_db [OPTIONS]
Options:
  -f, --force: Drop existing database and user.
  --create-db: Create the database and user.
```

--------------------------------

### GET Feedback for Recordings API Endpoint

Source: https://listenbrainz.readthedocs.io/en/latest/users/api/recordings

Retrieves user feedback for a list of specified recordings (MSIDs or MBIDs) for a given user. This endpoint details the available query parameters, expected status codes, and important notes regarding URL length limitations for GET requests.

```APIDOC
GET /1/feedback/user/(*user_name*)/get-feedback-for-recordings
Description: Get feedback given by user user_name for the list of recordings supplied. If the feedback for given recording MSID doesn’t exist then a score 0 is returned for that recording.
Parameters:
  recordings (str): comma separated list of recording_msids for which feedback records are to be fetched. this param is deprecated and will be removed in the future. use recording_msids instead.
  recording_msids (str): comma separated list of recording_msids for which feedback records are to be fetched.
  recording_mbids (str): comma separated list of recording_mbids for which feedback records are to be fetched.
Status Codes:
  200 OK: Yay, you have data!
Response Headers:
  Content-Type: application/json
Note: If you get a 502 error while querying this endpoint using a GET request, consider reducing the number of total recordings you are querying in 1 request. As a rule of thumb, requesting maximum ~75 recordings in 1 request will avert the error. The reason this error occurs is because the recording uuids are query params which are part of the request url. The length of the url is subject to a general limit imposed at the middleware level so requests with long urls never reach the ListenBrainz backend. Due to the same reason, the backend cannot provide a meaningful error.
```

--------------------------------

### Initialize Time-Series Database

Source: https://listenbrainz.readthedocs.io/en/latest/developers/commands

This command initializes the time-series database. It involves creating the table structure, setting up indexes, and defining views to support time-based data operations.

```Shell
./develop.sh manage init_ts_db [OPTIONS]
```

--------------------------------

### GET /1/explore/lb-radio

Source: https://listenbrainz.readthedocs.io/en/latest/users/api/misc

Generate a playlist with LB Radio.

```APIDOC
GET /1/explore/lb-radio
Description: Generate a playlist with LB Radio.
Parameters:
  - prompt: The LB Radio prompt from which to generate playlists.
  - mode: The mode that LB radio should use. Must be easy, medium or hard.
Example Response:
```json
{
    "payload": {
        "jspf" : {}, // <JSPF playlist here>
        "feedback": [] // <user feedback items>
    }
}
```
Status Codes:
  - 200 OK: success
  - 400 Bad Request: bad request: some parameters are missing or invalid
```

--------------------------------

### CLI Command: ./develop.sh-manage-dump-create_sample

Source: https://listenbrainz.readthedocs.io/en/latest/genindex

This command generates a sample data dump. Users can specify the desired output location for the sample dump and configure the number of threads to be used during its creation.

```APIDOC
./develop.sh-manage-dump-create_sample
  --location | -l: Specifies the output location for the sample dump.
  --threads | -t: Specifies the number of threads to use for processing.
```
Tokens:
Raw