### Initialize ListenBrainz Databases Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env Runs commands to create and initialize the PostgreSQL and TimescaleDB databases required for the ListenBrainz development environment. ```bash ./develop.sh manage init_db --create-db ./develop.sh manage init_ts_db --create-db ``` -------------------------------- ### List Available `manage.py` Commands Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env Display a list of all management commands available through the `manage.py` script. This helps developers discover and understand the various operations they can perform. ```Shell ./develop.sh manage --help ``` -------------------------------- ### Start Main ListenBrainz Development Environment Source: https://listenbrainz.readthedocs.io/en/latest/developers/spark-devel-env Before starting the Spark-specific components, it's necessary to ensure the main ListenBrainz development environment is running. This command brings up all core services required for the broader ListenBrainz application. It establishes the foundational services for your development workflow. ```Bash ./develop.sh up ``` -------------------------------- ### Open Flask Shell with IPython in Webserver Container Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env Start an interactive Flask shell session with IPython, pre-loading the ListenBrainz application. This is ideal for development and debugging, allowing direct interaction with the application's models and logic. ```Shell ./develop.sh shell ``` -------------------------------- ### Clone ListenBrainz Server Repository Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env Instructions to clone the ListenBrainz server repository from GitHub using Git. Windows users are advised to use WSL2 for cloning to avoid code watcher issues. ```Shell git clone https://github.com/metabrainz/listenbrainz-server.git ``` -------------------------------- ### Advanced Options for ListenBrainz Frontend Testing Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env These options facilitate efficient frontend testing workflows. They include running tests, updating snapshots, building frontend test containers, and performing type-checking. ```Shell ./test.sh fe run frontend tests ``` ```Shell ./test.sh fe -u run frontend tests, update snapshots ``` ```Shell ./test.sh fe -b build frontend test containers ``` ```Shell ./test.sh fe -t run type-checker ``` -------------------------------- ### Copy ListenBrainz Sample Configuration File Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env Copies the `config.py.sample` file to `config.py`. This new file will be used for local development configurations, allowing customization without altering the original sample. ```Shell cp listenbrainz/config.py.sample listenbrainz/config.py ``` -------------------------------- ### Start ListenBrainz Development Environment Source: https://listenbrainz.readthedocs.io/en/latest/_sources/developers/devel-env.rst Use this command to start all ListenBrainz services in the development environment. The web service typically listens on port 8100. Press CTRL+C to shut down the services. ```bash ./develop.sh up ``` -------------------------------- ### Initialize ListenBrainz PostgreSQL and TimescaleDB Databases Source: https://listenbrainz.readthedocs.io/en/latest/_sources/developers/devel-env.rst Execute these commands to create and initialize the required PostgreSQL and TimescaleDB databases for the ListenBrainz development environment. This step is crucial before starting the server. ```bash ./develop.sh manage init_db --create-db ./develop.sh manage init_ts_db --create-db ``` -------------------------------- ### Start ListenBrainz Development Environment Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env Starts all ListenBrainz services using `develop.sh up`. The web service listens on port 8100 by default. Users can change the port in `docker/docker-compose.yml`. ```bash ./develop.sh up ``` -------------------------------- ### Open PostgreSQL Session to ListenBrainz Database Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env Use the `develop.sh` utility to establish a psql session directly to the main ListenBrainz database. This provides command-line access for database inspection and management. ```Shell ./develop.sh psql ``` -------------------------------- ### Start ListenBrainz Spark Development Environment Source: https://listenbrainz.readthedocs.io/en/latest/developers/spark-devel-env Once the main ListenBrainz environment is active, this command starts the dedicated Spark environment. It brings up all Spark-related containers, including the spark reader. This prepares the Spark cluster for data processing and analysis tasks. ```Bash ./develop.sh spark up ``` -------------------------------- ### Open PostgreSQL Session to Timescale Database Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env Connect to the Timescale database, which stores user listens, using the `develop.sh` utility. This allows direct interaction with the time-series data. ```Shell ./develop.sh timescale ``` -------------------------------- ### Display Docker Compose Help Information Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env Show the help output for Docker Compose, listing all valid commands and options. This is useful for understanding the capabilities of Docker Compose when managing the ListenBrainz development environment. ```Shell ./develop.sh help ``` -------------------------------- ### Start ListenBrainz Webserver Source: https://listenbrainz.readthedocs.io/en/latest/_sources/developers/spark-architecture.rst This command starts the ListenBrainz webserver for development purposes, setting up the necessary environment. ```bash ./develop.sh up ``` -------------------------------- ### Run ListenBrainz Frontend Tests Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env Execute tests specifically for the frontend codebase using the `test.sh` script. This is essential for validating changes made to the user interface. ```Shell ./test.sh fe ``` -------------------------------- ### Configure CritiqueBrainz API Credentials Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env Sets up environment variables for CritiqueBrainz API integration, including client ID, client secret, and redirect URI. These are essential for authentication and callback handling. ```bash CRITIQUEBRAINZ_CLIENT_ID = '' CRITIQUEBRAINZ_CLIENT_SECRET = '' CRITIQUEBRAINZ_REDIRECT_URI = 'http://localhost:8100/settings/music-services/critiquebrainz/callback/' ``` -------------------------------- ### Advanced Options for Running ListenBrainz Tests Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env These options provide flexibility for faster and more specific test execution. They allow building unit test containers, running specific test files or directories, and managing the lifecycle of test containers (stopping or removing them). ```Shell ./test.sh -u # build unit test containers, start up and initialise the database ``` ```Shell ./test.sh [path-to-tests-file-or-directory] # run specific tests, do this as often as you need to ``` ```Shell ./test.sh -s # stop test containers, but don't remove them ``` ```Shell ./test.sh -d # stop and remove all test containers ``` -------------------------------- ### Pass Commands Directly to Docker Compose via `develop.sh` Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env This command allows direct execution of any Docker Compose command through the `develop.sh` wrapper script. It provides a convenient way to interact with the underlying Docker Compose setup. ```Shell ./develop.sh ``` -------------------------------- ### Access Local ListenBrainz Instance Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env The URL to access the running ListenBrainz development environment in a web browser, assuming the default port 8100 is used. ```plaintext http://localhost:8100 ``` -------------------------------- ### Configure Spotify API Credentials in config.py Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env Populates the `SPOTIFY_CLIENT_ID` and `SPOTIFY_CLIENT_SECRET` fields in `config.py` with credentials obtained from the Spotify Developer Dashboard. This setup is required for the Spotify importer to function correctly. ```Python # SPOTIFY SPOTIFY_CLIENT_ID = '' SPOTIFY_CLIENT_SECRET = '' ``` -------------------------------- ### Rebuild ListenBrainz Web Service Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env Command to rebuild only the `web` service Docker image. This is necessary if new Python dependencies are added to `requirements.txt`. ```bash ./develop.sh build web ``` -------------------------------- ### Build ListenBrainz Docker Containers Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env Executes the development script to build all necessary Docker images for the ListenBrainz services. This command uses `docker-compose` internally and may take time on the first run. ```bash ./develop.sh build ``` -------------------------------- ### Start ListenBrainz Webserver Source: https://listenbrainz.readthedocs.io/en/latest/developers/spark-architecture Command to start the ListenBrainz webserver using the development script. This is typically the first step in setting up the local development environment. ```Shell ./develop.sh up ``` -------------------------------- ### Complete Single Listen Submission JSON Example Source: https://listenbrainz.readthedocs.io/en/latest/_sources/users/json.rst A comprehensive example of a `single` listen submission JSON document, showing all typical fields including `additional_info` for media player, client, and duration, providing a full context for a listen. ```json { "listen_type": "single", "payload": [ { "listened_at": 1443521965, "track_metadata": { "additional_info": { "media_player": "Rhythmbox", "submission_client": "Rhythmbox ListenBrainz Plugin", "submission_client_version": "1.0", "release_mbid": "bf9e91ea-8029-4a04-a26a-224e00a83266", "artist_mbids": [ "db92a151-1ac2-438b-bc43-b82e149ddd50" ], "recording_mbid": "98255a8c-017a-4bc7-8dd6-1fa36124572b", "tags": [ "you", "just", "got", "rick rolled!"], "duration_ms": 222000 }, "artist_name": "Rick Astley", "track_name": "Never Gonna Give You Up", "release_name": "Whenever you need somebody" } } ] } ``` -------------------------------- ### Run All ListenBrainz Unit and Integration Tests Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env This script builds and runs containers for all unit and integration tests, ensuring test data is isolated from development data. It's a crucial step before committing new code or creating a pull request. ```Shell ./test.sh ``` -------------------------------- ### Rebuild ListenBrainz Static Builder Service Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env Command to rebuild only the `static_builder` service Docker image. This is required if new JavaScript dependencies are introduced. ```bash ./develop.sh build static_builder ``` -------------------------------- ### Open Redis Shell Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env Access the Redis command-line interface for interacting with the Redis instance. This is useful for inspecting cached data or managing Redis keys. ```Shell ./develop.sh redis ``` -------------------------------- ### Open Bash Shell in Webserver Container Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env Gain shell access to the webserver container for debugging or administrative tasks. This provides a direct command-line interface within the running container environment. ```Shell ./develop.sh bash ``` -------------------------------- ### Start ListenBrainz Spark Environment Source: https://listenbrainz.readthedocs.io/en/latest/_sources/developers/spark-devel-env.rst Initiate the ListenBrainz Spark environment, which also brings up the spark reader container. ```bash ./develop.sh spark up ``` -------------------------------- ### Example Console Output: Listen History Source: https://listenbrainz.readthedocs.io/en/latest/users/api-usage This block shows a sample of the console output generated by the `get_listens` Python script, illustrating the format of the retrieved track names and their corresponding listen timestamps. ```Text Track: Never Gonna Give You Up, listened at 1512040365 Track: Never Gonna Give You Up, listened at 1511977429 Track: Never Gonna Give You Up, listened at 1511968583 Track: Never Gonna Give You Up, listened at 1443521965 Track: Never Gonna Give You Up, listened at 42042042 ``` -------------------------------- ### Bring Up Main ListenBrainz Development Environment Source: https://listenbrainz.readthedocs.io/en/latest/_sources/developers/spark-devel-env.rst Before starting the Spark environment, ensure that the main ListenBrainz development environment is running. ```bash ./develop.sh up ``` -------------------------------- ### Inspect Static Builder Logs for ESLint Output Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env This command filters `develop.sh` logs to display only the output from the `static_builder` service. It is particularly useful for reviewing ESLint results and other frontend build messages. ```Shell ./develop.sh logs -f static_builder ``` -------------------------------- ### Configure Last.fm API Key in config.py Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env Sets the `LASTFM_API_KEY` in `config.py` using the key acquired from the Last.fm API page. This configuration is essential for enabling the Last.fm importer functionality within ListenBrainz. ```Python # Lastfm API LASTFM_API_URL = "https://ws.audioscrobbler.com/2.0/" LASTFM_API_KEY = "USE_LASTFM_API_KEY" ``` -------------------------------- ### Run ListenBrainz Frontend Tests Source: https://listenbrainz.readthedocs.io/en/latest/_sources/developers/devel-env.rst Execute frontend tests for the ListenBrainz project. This command is used when changes are made to the frontend. ```bash ./test.sh fe ``` -------------------------------- ### ListenBrainz API: Complete Single Listen Submission Example Source: https://listenbrainz.readthedocs.io/en/latest/users/json Presents a full example of a `single` listen submission JSON document, combining the top-level structure with a detailed listen payload, including `additional_info` fields like media player and submission client. ```JSON { "listen_type": "single", "payload": [ { "listened_at": 1443521965, "track_metadata": { "additional_info": { "media_player": "Rhythmbox", "submission_client": "Rhythmbox ListenBrainz Plugin", "submission_client_version": "1.0", "release_mbid": "bf9e91ea-8029-4a04-a26a-224e00a83266", "artist_mbids": [ "db92a151-1ac2-438b-bc43-b82e149ddd50" ], "recording_mbid": "98255a8c-017a-4bc7-8dd6-1fa36124572b", "tags": [ "you", "just", "got", "rick rolled!"], "duration_ms": 222000 }, "artist_name": "Rick Astley", "track_name": "Never Gonna Give You Up", "release_name": "Whenever you need somebody" } } ] } ``` -------------------------------- ### Submit Listens to ListenBrainz API (Python) Source: https://listenbrainz.readthedocs.io/en/latest/users/api-usage This Python code demonstrates how to submit music listens to the ListenBrainz API. It includes a `submit_listen` function that handles API requests, authentication, and error handling. The example shows how to construct a payload for a single track and submit it to either a local development server or the production API. ```Python from time import time import requests # Set DEBUG to True to test local dev server. # API keys for local dev server and the real server are different. DEBUG = True ROOT = 'http://localhost:8100' if DEBUG else 'https://api.listenbrainz.org' def submit_listen(listen_type, payload, token): """Submits listens for the track(s) in payload. Args: listen_type (str): either of 'single', 'import' or 'playing_now' payload: A list of Track dictionaries. token: the auth token of the user you're submitting listens for Returns: The json response if there's an OK status. Raises: An HTTPError if there's a failure. A ValueError is the JSON in the response is invalid. """ response = requests.post( url="{0}/1/submit-listens".format(ROOT), json={ "listen_type": listen_type, "payload": payload, }, headers={ "Authorization": "Token {0}".format(token) } ) response.raise_for_status() return response.json() if __name__ == "__main__": EXAMPLE_PAYLOAD = [ { # An example track. "listened_at": int(time()), "track_metadata": { "artist_name": "Rick Astley", "track_name": "Never Gonna Give You Up", "release_name": "Whenever you need somebody" } } ] # Input token from the user and call submit listen token = input('Please enter your auth token: ') json_response = submit_listen(listen_type='single', payload=EXAMPLE_PAYLOAD, token=token) print("Response was: {0}".format(json_response)) print("Check your listens - there should be a Never Gonna Give You Up track, played recently.") ``` -------------------------------- ### Fetched Listen Data JSON Example Source: https://listenbrainz.readthedocs.io/en/latest/_sources/users/json.rst An example of the JSON document structure returned when fetching listen data from the API, showing the `payload` object containing `count`, `user_id`, and a placeholder for the `listens` array. ```json { "payload": { "count": 25, "user_id": "-- the MusicBrainz ID of the user --", "listens": [ "-- listen data here ---" ] } } ``` -------------------------------- ### Example MusicBrainz Metadata Lookup Response JSON Source: https://listenbrainz.readthedocs.io/en/latest/_sources/users/api-usage.rst An example JSON structure returned by the ListenBrainz metadata API when looking up MusicBrainz IDs (MBIDs) and associated metadata for a recording. It includes details like artist credits, recording MBID and name, and release information. ```json { "artist_credit_name": "Ariana Grande", "artist_mbids": [ "f4fdbb4c-e4b7-47a0-b83b-d91bbfcfa387" ], "metadata": { "recording": { "rels": [ { "artist_mbid": "eb811bf7-4c99-4781-84c0-10ba6b8e33b3", "artist_name": "Carl Falk", "instrument": "guitar", "type": "instrument" }, { "artist_mbid": "c8af4490-e48a-4f91-aef9-2b1e39369576", "artist_name": "Savan Kotecha", "instrument": "background vocals", "type": "vocal" }, { "artist_mbid": "0d33cc88-28ae-44d5-be7e-7a653e518720", "artist_name": "Jeanette Olsson", "instrument": "background vocals", "type": "vocal" } ] } }, "recording_mbid": "9f24c0f7-a644-4074-8fbd-a1dba03de129", "recording_name": "One Last Time", "release_mbid": "be5d97b1-408a-4e95-b924-0a61955048de", "release_name": "My Everything" } ``` -------------------------------- ### Clone ListenBrainz Server Repository Source: https://listenbrainz.readthedocs.io/en/latest/_sources/developers/devel-env.rst Instructions to clone the ListenBrainz server repository from GitHub using Git. This is the initial step to obtain the source code and begin setting up the development environment. ```bash git clone https://github.com/metabrainz/listenbrainz-server.git ``` -------------------------------- ### Run ListenBrainz Backend Tests Source: https://listenbrainz.readthedocs.io/en/latest/_sources/developers/devel-env.rst This snippet provides commands to manage the backend test infrastructure, including building containers, initializing the database, running specific tests, and stopping/removing containers. ```bash ./test.sh -u # build unit test containers, start up and initialise the database ./test.sh [path-to-tests-file-or-directory] # run specific tests, do this as often as you need to ./test.sh -s # stop test containers, but don't remove them ./test.sh -d # stop and remove all test containers ``` -------------------------------- ### Advanced ListenBrainz Frontend Testing Options Source: https://listenbrainz.readthedocs.io/en/latest/_sources/developers/devel-env.rst This snippet lists various options for efficient frontend testing, including running tests, updating snapshots, building containers, and running the type-checker. ```bash ./test.sh fe run frontend tests ./test.sh fe -u run frontend tests, update snapshots ./test.sh fe -b build frontend test containers ./test.sh fe -t run type-checker ``` -------------------------------- ### ListenBrainz API: Get User Listens Endpoint Source: https://listenbrainz.readthedocs.io/en/latest/users/api-usage Documentation for the ListenBrainz API endpoint used to retrieve a user's listening history. ```APIDOC Endpoint: GET /1/user/{username}/listens Description: Retrieves the listening history for a specified user. Parameters: username (string, path): The MusicBrainz ID of the user. min_ts (integer, query, optional): History before this timestamp will not be returned. Do not use with max_ts. max_ts (integer, query, optional): History after this timestamp will not be returned. Do not use with min_ts. count (integer, query, optional): How many listens to return. If not specified, uses a default from the server. Headers: Authorization (string, optional): Token for relaxed rate limits. Returns: A list of listen info dictionaries (JSON array). Errors: HTTPError: If there's a failure in the HTTP request. ValueError: If the JSON in the response is invalid. IndexError: If the JSON is not structured as expected. ``` -------------------------------- ### Initialize ListenBrainz Configuration File Source: https://listenbrainz.readthedocs.io/en/latest/_sources/developers/devel-env.rst Command to copy the sample configuration file (config.py.sample) to the active configuration file (config.py) for ListenBrainz. This action is a prerequisite before customizing specific settings like OAuth credentials and API keys. ```bash cp listenbrainz/config.py.sample listenbrainz/config.py ``` -------------------------------- ### Get Latest Import Timestamp from ListenBrainz (Python) Source: https://listenbrainz.readthedocs.io/en/latest/users/api-usage This Python script retrieves the latest import timestamp for a specified user and service from the ListenBrainz API. It requires a username and an optional service name. The script demonstrates how to query the API using a GET request and parse the JSON response to obtain the timestamp. ```python import requests # Set DEBUG to True to test local dev server. # API keys for local dev server and the real server are different. DEBUG = True ROOT = 'http://localhost:8100' if DEBUG else 'https://api.listenbrainz.org' # The token can be any valid token. TOKEN = 'YOUR_TOKEN_HERE' AUTH_HEADER = { "Authorization": "Token {0}".format(TOKEN) } def get_latest_import(username, service="lastfm"): """Gets the latest import timestamp of a given user. Args: username: User to get latest import time of. service: service to get latest import time of. Returns: A Unix timestamp if there's an OK status. Raises: An HTTPError if there's a failure. A ValueError if the JSON in the response is invalid. An IndexError if the JSON is not structured as expected. """ response = requests.get( url="{0}/1/latest-import".format(ROOT), params={ "user_name": username, "service": service }, headers=AUTH_HEADER, ) response.raise_for_status() return response.json()["latest_import"] if __name__ == "__main__": username = input('Please input the MusicBrainz ID of the user: ') timestamp = get_latest_import(username) print("User {0} last imported on {1}".format(username, timestamp)) ``` -------------------------------- ### Sample Listen Payload JSON Source: https://listenbrainz.readthedocs.io/en/latest/_sources/users/json.rst Provides an example of the `track_metadata` structure within a listen payload, including `listened_at` and various fields under `additional_info` such as MBIDs and tags. ```json { "listened_at": 1443521965, "track_metadata": { "additional_info": { "release_mbid": "bf9e91ea-8029-4a04-a26a-224e00a83266", "artist_mbids": [ "db92a151-1ac2-438b-bc43-b82e149ddd50" ], "recording_mbid": "98255a8c-017a-4bc7-8dd6-1fa36124572b", "tags": [ "you", "just", "got", "rick rolled!"] }, "artist_name": "Rick Astley", "track_name": "Never Gonna Give You Up", "release_name": "Whenever you need somebody" } } ``` -------------------------------- ### Inspect ListenBrainz Frontend Build Logs Source: https://listenbrainz.readthedocs.io/en/latest/_sources/developers/devel-env.rst Use this command to filter and view only the static_builder logs, which is useful when there are many logs in the terminal during frontend development. ```bash ./develop.sh logs -f static_builder ``` -------------------------------- ### Build ListenBrainz Development Docker Containers Source: https://listenbrainz.readthedocs.io/en/latest/_sources/developers/devel-env.rst Run this command in the repository root to build all necessary Docker images for the ListenBrainz server services. This process might take time on the first run as dependencies are downloaded and services are built. ```bash ./develop.sh build ``` -------------------------------- ### Configure Docker Compose Port Mapping Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env Illustrates how to modify the `ports` section in `docker/docker-compose.yml` to change the listening port of the web service. The first value before the colon can be adjusted. ```yaml ports: - "8100:80" ``` -------------------------------- ### Command-line options for develop.sh-manage-init_db Source: https://listenbrainz.readthedocs.io/en/latest/genindex Initializes the main ListenBrainz database, with an option to force recreation if it already exists. ```APIDOC Command: ./develop.sh-manage-init_db Options: --create-db --force -f ``` -------------------------------- ### Display docker-compose help Source: https://listenbrainz.readthedocs.io/en/latest/developers/develop-sh This command shows the help output for `docker-compose`, listing all available commands and their usage. It's equivalent to running `docker-compose help` directly. ```bash ./develop.sh help ``` -------------------------------- ### Initialize Main Database Source: https://listenbrainz.readthedocs.io/en/latest/developers/commands This command initializes the primary database for the application. It sets up the table structure, creates primary and foreign keys, and establishes necessary indexes for optimal performance. ```Shell ./develop.sh manage init_db [OPTIONS] ``` -------------------------------- ### Invoke ListenBrainz `manage.py` Commands Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env Execute specific management commands provided by the `manage.py` script within a Docker container. This allows running various administrative and maintenance tasks for the ListenBrainz application. ```Shell ./develop.sh manage ``` -------------------------------- ### Command-line options for develop.sh-manage-init_ts_db Source: https://listenbrainz.readthedocs.io/en/latest/genindex Initializes the TimescaleDB database for ListenBrainz, with an option to force recreation. ```APIDOC Command: ./develop.sh-manage-init_ts_db Options: --create-db --force -f ``` -------------------------------- ### Run ListenBrainz Unit and Integration Tests Source: https://listenbrainz.readthedocs.io/en/latest/_sources/developers/devel-env.rst Execute this script to run all unit and integration tests for ListenBrainz. It sets up isolated test-specific data volumes to prevent interference with development data. Run this before committing code or making pull requests. ```bash ./test.sh ``` -------------------------------- ### Configure MusicBrainz OAuth Credentials in config.py Source: https://listenbrainz.readthedocs.io/en/latest/developers/devel-env Updates the `OAUTH_CLIENT_ID` and `OAUTH_CLIENT_SECRET` variables in `config.py` with the client ID and secret obtained from MusicBrainz application registration. This enables authentication and login via MusicBrainz accounts. ```Python # MusicBrainz OAuth OAUTH_CLIENT_ID = "CLIENT_ID" OAUTH_CLIENT_SECRET = "CLIENT_SECRET" ``` -------------------------------- ### Format ListenBrainz Spark HDFS Namenode Source: https://listenbrainz.readthedocs.io/en/latest/developers/spark-devel-env This command is used to format the `namenode` container, which is essential for initializing the HDFS filesystem within the Spark environment. It's typically run during the initial setup or when you need to clear all existing data. Running this command will shut down the Spark Docker cluster, remove data volumes, and recreate the HDFS filesystem, effectively resetting your data. ```Bash ./develop.sh spark format ``` -------------------------------- ### GET /1/user/(user_name)/fresh_releases Source: https://listenbrainz.readthedocs.io/en/latest/users/api/misc Get fresh releases data for the given user. ```APIDOC GET /1/user/(*user_name*)/fresh_releases Description: Get fresh releases data for the given user. Parameters: - sort: The sort order of the results. Must be one of “release_date”, “artist_credit_name”, “release_name”, or “confidence”. Default “release_date”. - past: Whether to show releases in the past. Default True. - future: Whether to show releases in the future. Default True. Example Response: ```json { "artist_credit_name":"Santi & Tuğçe", "artist_mbids":[ "4690076b-1446-43f4-8a84-cfec56dc3601" ], "caa_id":37026686379, "caa_release_mbid":"9432fb06-bd84-4f2e-9386-b2f14e0de54d", "confidence":2, "release_date":"2023-10-20", "release_group_mbid":"30ac1ded-c254-46ac-a41f-7453dada6ae7", "release_group_primary_type":"Album", "release_group_secondary_type":null, "release_mbid":"9432fb06-bd84-4f2e-9386-b2f14e0de54d", "release_name":"The Marvelous Real", "release_tags":null } ``` Status Codes: - 200 OK: fetch succeeded - 400 Bad Request: invalid date or number of days passed. Response Headers: - Content-Type: application/json ``` -------------------------------- ### Create ListenBrainz Sample Data Dump Source: https://listenbrainz.readthedocs.io/en/latest/developers/commands This command creates a sample data dump for ListenBrainz. Users can specify the output directory and the number of threads to use for compression, optimizing the dump creation process. ```Bash ./develop.sh manage dump create_sample [OPTIONS] ``` ```APIDOC Options: -l, --location : path to the directory where the dump should be made -t, --threads : the number of threads to be used while compression ``` -------------------------------- ### Create All MBID Mapping Database and Index Components Source: https://listenbrainz.readthedocs.io/en/latest/_sources/developers/mapping.rst This command orchestrates the full setup of the MBID mapping system. It first builds the necessary mapping database tables and then proceeds to build the Typesense index directly afterwards, ensuring all data structures are initialized and ready for operation in a single command. ```bash python mapper/manage.py create-all ``` -------------------------------- ### Build ListenBrainz Spark Docker Containers Source: https://listenbrainz.readthedocs.io/en/latest/developers/spark-devel-env This command initiates the build process for the ListenBrainz Spark Docker containers. It compiles and packages all necessary components into runnable images. This step is crucial for preparing the Spark environment for execution. ```Bash ./develop.sh spark build ``` -------------------------------- ### Option: -f Source: https://listenbrainz.readthedocs.io/en/latest/genindex A short option used for database initialization commands. ```APIDOC Option: -f Description: A short option. Used by: - ./develop.sh-manage-init_db - ./develop.sh-manage-init_ts_db ``` -------------------------------- ### GET /1/user/(*user_name*)/listen-count API Endpoint Source: https://listenbrainz.readthedocs.io/en/latest/users/api/core Get the total number of listens for a specified user. The response payload contains a 'count' key with the listen count. ```APIDOC Method: GET Path: /1/user/(*user_name*)/listen-count Status Codes: 200 OK: Yay, you have listen counts! 404 Not Found: The requested user was not found. Response Headers: Content-Type: application/json ``` -------------------------------- ### GET /1/stats/user/{user_name}/artist-activity Source: https://listenbrainz.readthedocs.io/en/latest/users/api/statistics Get the artist activity for user `user_name`. The artist activity shows the total number of listens for each artist along with their albums and corresponding listen counts. ```APIDOC Endpoint: GET /1/stats/user/{user_name}/artist-activity Description: Get the artist activity for user user_name. The artist activity shows the total number of listens for each artist along with their albums and corresponding listen counts. Example Response: ```json { "result": [ { "name": "Radiohead", "listen_count": 120, "albums": [ { "name": "In Rainbows", "listen_count": 50 }, { "name": "OK Computer", "listen_count": 70 } ] } ] } ``` Parameters: - range (str): Optional, time interval for which statistics should be returned. Possible values are ALLOWED_STATISTICS_RANGE, defaults to all_time. Status Codes: - 200 OK: Successful query, you have data! - 204 No Content: Statistics for the user haven’t been calculated, empty response will be returned. - 400 Bad Request: Bad request, check response['error'] for more details. - 404 Not Found: User not found. Response Headers: - Content-Type: application/json ``` -------------------------------- ### Rebuild ListenBrainz Web Service Source: https://listenbrainz.readthedocs.io/en/latest/_sources/developers/devel-env.rst If new Python dependencies are added to `requirements.txt`, use this command to rebuild only the `web` Docker service, ensuring the new dependencies are included. ```bash ./develop.sh build web ``` -------------------------------- ### GET /1/stats/user/{user_name}/listening-activity Source: https://listenbrainz.readthedocs.io/en/latest/users/api/statistics Get the listening activity for user `user_name`. The listening activity shows the number of listens the user has submitted over a period of time, grouped by time ranges. ```APIDOC Endpoint: GET /1/stats/user/{user_name}/listening-activity Description: Get the listening activity for user user_name. The listening activity shows the number of listens the user has submitted over a period of time. Example Response: ```json { "payload": { "from_ts": 1587945600, "last_updated": 1592807084, "listening_activity": [ { "from_ts": 1587945600, "listen_count": 26, "time_range": "Monday 27 April 2020", "to_ts": 1588031999 }, { "from_ts": 1588032000, "listen_count": 57, "time_range": "Tuesday 28 April 2020", "to_ts": 1588118399 }, { "from_ts": 1588118400, "listen_count": 33, "time_range": "Wednesday 29 April 2020", "to_ts": 1588204799 } ], "to_ts": 1589155200, "user_id": "ishaanshah" } } ``` Parameters: - range (str): Optional, time interval for which statistics should be returned. Possible values are ALLOWED_STATISTICS_RANGE, defaults to all_time. Status Codes: - 200 OK: Successful query, you have data! - 204 No Content: Statistics for the user haven’t been calculated, empty response will be returned. - 400 Bad Request: Bad request, check response['error'] for more details. - 404 Not Found: User not found. Response Headers: - Content-Type: application/json ``` -------------------------------- ### Command-line options for develop.sh-manage-run-daily-jams Source: https://listenbrainz.readthedocs.io/en/latest/genindex Executes daily 'jams' related tasks, with an option to create all necessary components. ```APIDOC Command: ./develop.sh-manage-run-daily-jams Options: --create-all ``` -------------------------------- ### API: Get ListenBrainz Data Dump Info Source: https://listenbrainz.readthedocs.io/en/latest/users/api/misc Retrieves information about ListenBrainz data dumps. An optional 'id' parameter can be provided to get details about a specific dump; otherwise, information about the latest dump is returned. ```APIDOC GET /1/status/get-dump-info Description: Get information about ListenBrainz data dumps. Query Parameters: id: Integer specifying the ID of the dump. If not provided, returns info about the latest dump. Status Codes: 200 OK: You have data. 400 Bad Request: You did not provide a valid dump ID. See error message for details. 404 Not Found: Dump with given ID does not exist. Response Headers: Content-Type: application/json ``` ```json { "id": 1, "timestamp": "20190625-165900" } ``` -------------------------------- ### Copy Sample Spark Configuration File Source: https://listenbrainz.readthedocs.io/en/latest/_sources/developers/spark-devel-env.rst The Spark environment requires a 'config.py' file within the 'listenbrainz_spark/' directory. Create this file by copying from the provided sample configuration. ```bash cp listenbrainz_spark/config.py.sample listenbrainz_spark/config.py ``` -------------------------------- ### GET /1/stats/artist/{artist_mbid}/listeners API Source: https://listenbrainz.readthedocs.io/en/latest/users/api/statistics Get top listeners for a specific artist identified by `artist_mbid`. This endpoint provides the total listen count for the artist and a list of top N listeners with their individual listen counts within a given time range. ```APIDOC GET /1/stats/artist/({artist_mbid})/listeners Parameters: range (str, optional): Time interval for which statistics should be returned. Possible values are ALLOWED_STATISTICS_RANGE, defaults to 'all_time'. Status Codes: 200 OK: Successful query, data available. 204 No Content: Statistics not calculated or entity does not exist, empty response. 400 Bad Request: Invalid request, check response['error'] for details. 404 Not Found: Entity not found. Response Headers: Content-Type: application/json ``` ```JSON { "payload": { "artist_mbid": "00034ede-a1f1-4219-be39-02f36853373e", "artist_name": "O Rappa", "from_ts": 1009843200, "last_updated": 1681839677, "listeners": [ { "listen_count": 2469, "user_name": "RosyPsanda" }, { "listen_count": 1858, "user_name": "alexyagui" }, { "listen_count": 578, "user_name": "rafael_gn" }, { "listen_count": 8, "user_name": "italooliveira" }, { "listen_count": 7, "user_name": "paulodesouza" }, { "listen_count": 1, "user_name": "oldpunisher" } ], "stats_range": "all_time", "to_ts": 1681777035, "total_listen_count": 16393 } } ``` -------------------------------- ### Option: -lp Source: https://listenbrainz.readthedocs.io/en/latest/genindex A short option used with the dump-create_full command. ```APIDOC Option: -lp Description: A short option. Used by: - ./develop.sh-manage-dump-create_full ``` -------------------------------- ### List available manage.py commands Source: https://listenbrainz.readthedocs.io/en/latest/developers/develop-sh This command displays a list of all available subcommands and options for the `manage.py` script. It's useful for discovering what management tasks can be performed. ```bash ./develop.sh manage --help ``` -------------------------------- ### Artist Popularity Data Response Example Source: https://listenbrainz.readthedocs.io/en/latest/users/api/popularity Example JSON response structure for artist popularity data, including total listen count and total user count. Demonstrates how data is returned for found and unfound artist MBIDs, with null counts for missing data. ```JSON [ { "artist_mbid": "13dd61c7-ce73-4e97-9f0c-9f0e53144411", "total_listen_count": 1000, "total_user_count": 10 }, { "artist_mbid": "22ad712e-ce73-9f0c-4e97-9f0e53144411", "total_listen_count": null, "total_user_count": null } ] ``` -------------------------------- ### API: General Metadata Endpoint Parameters and Example Response Fragment Source: https://listenbrainz.readthedocs.io/en/latest/users/api/metadata This section provides general parameters and status codes applicable to certain metadata endpoints, along with an example JSON fragment showing a tag structure, likely part of a larger response. ```APIDOC Parameters: release_group_mbids (str): A comma separated list of release_group_mbids inc (str): A space separated list of “artist”, “tag” and/or “release” to indicate which portions of metadata you’re interested in fetching. We encourage users to only fetch the data they plan to consume. Status Codes: 200 OK: you have data! 400 Bad Request: invalid release_group_mbid arguments ``` ```JSON { "count": 18, "genre_mbid": "45eb1d9c-588c-4dc8-9394-a14b7c8f02bc", "tag": "trip hop" } ``` -------------------------------- ### Import Sample Data into ListenBrainz Spark Source: https://listenbrainz.readthedocs.io/en/latest/developers/spark-devel-env This command facilitates the import of small data dumps into your ListenBrainz Spark environment. These dumps are provided to help developers work with realistic ListenBrainz data for testing and development purposes. Execute this in a separate terminal to populate your Spark cluster with sample information. ```Bash ./develop.sh spark run spark_reader python manage.py spark request_import_incremental ``` -------------------------------- ### GET /1/stats/release-group/{release_group_mbid}/listeners API Source: https://listenbrainz.readthedocs.io/en/latest/users/api/statistics Get top listeners for a specific release group identified by `release_group_mbid`. This endpoint provides the total listen count for the entity and a list of top N listeners with their individual listen counts within a given time range. ```APIDOC GET /1/stats/release-group/({release_group_mbid})/listeners ``` ```JSON { "payload": { "artist_mbids": [ "c234fa42-e6a6-443e-937e-2f4b073538a3" ], "artist_name": "Chris Brown", "caa_id": 23564822587, "caa_release_mbid": "25f18616-5a9c-470e-964d-4eb8a511435b", "from_ts": 1009843200, "last_updated": 1681843150, "listeners": [ { "listen_count": 2365, "user_name": "purpleyor" }, { "listen_count": 570, "user_name": "dndty" }, { "listen_count": 216, "user_name": "iammsyre" }, { "listen_count": 141, "user_name": "dpmittal" }, { "listen_count": 33, "user_name": "tazlad" }, { "listen_count": 30, "user_name": "ratkutti" }, { "listen_count": 22, "user_name": "Raymorjamiek" }, { "listen_count": 21, "user_name": "MJJMC" }, { "listen_count": 12, "user_name": "fookever" }, { "listen_count": 8, "user_name": "Jamjamk12071983" }, { "listen_count": 1, "user_name": "hassanymoses" }, { "listen_count": 1, "user_name": "iJays" } ], "release_group_mbid": "087b3a7d-d532-44d9-b37a-84427677ddcd", "release_group_name": "Indigo", "stats_range": "all_time", "to_ts": 1681777035 } } ``` -------------------------------- ### Configure CritiqueBrainz API Credentials Source: https://listenbrainz.readthedocs.io/en/latest/_sources/developers/devel-env.rst Update the ListenBrainz configuration file with the client ID and client secret obtained from the CritiqueBrainz applications page. This enables integration with CritiqueBrainz for music services. ```yaml # CRITIQUEBRAINZ CRITIQUEBRAINZ_CLIENT_ID = '' CRITIQUEBRAINZ_CLIENT_SECRET = '' CRITIQUEBRAINZ_REDIRECT_URI = 'http://localhost:8100/settings/music-services/critiquebrainz/callback/' ``` -------------------------------- ### init_db Command Line Interface Source: https://listenbrainz.readthedocs.io/en/latest/developers/commands Detailed interface for the `init_db` management command, used to initialize the main application database. It supports options for forced re-initialization and database creation. ```APIDOC Command: ./develop.sh manage init_db [OPTIONS] Options: -f, --force: Drop existing database and user. --create-db: Create the database and user. ``` -------------------------------- ### GET Feedback for Recordings API Endpoint Source: https://listenbrainz.readthedocs.io/en/latest/users/api/recordings Retrieves user feedback for a list of specified recordings (MSIDs or MBIDs) for a given user. This endpoint details the available query parameters, expected status codes, and important notes regarding URL length limitations for GET requests. ```APIDOC GET /1/feedback/user/(*user_name*)/get-feedback-for-recordings Description: Get feedback given by user user_name for the list of recordings supplied. If the feedback for given recording MSID doesn’t exist then a score 0 is returned for that recording. Parameters: recordings (str): comma separated list of recording_msids for which feedback records are to be fetched. this param is deprecated and will be removed in the future. use recording_msids instead. recording_msids (str): comma separated list of recording_msids for which feedback records are to be fetched. recording_mbids (str): comma separated list of recording_mbids for which feedback records are to be fetched. Status Codes: 200 OK: Yay, you have data! Response Headers: Content-Type: application/json Note: If you get a 502 error while querying this endpoint using a GET request, consider reducing the number of total recordings you are querying in 1 request. As a rule of thumb, requesting maximum ~75 recordings in 1 request will avert the error. The reason this error occurs is because the recording uuids are query params which are part of the request url. The length of the url is subject to a general limit imposed at the middleware level so requests with long urls never reach the ListenBrainz backend. Due to the same reason, the backend cannot provide a meaningful error. ``` -------------------------------- ### Initialize Time-Series Database Source: https://listenbrainz.readthedocs.io/en/latest/developers/commands This command initializes the time-series database. It involves creating the table structure, setting up indexes, and defining views to support time-based data operations. ```Shell ./develop.sh manage init_ts_db [OPTIONS] ``` -------------------------------- ### GET /1/explore/lb-radio Source: https://listenbrainz.readthedocs.io/en/latest/users/api/misc Generate a playlist with LB Radio. ```APIDOC GET /1/explore/lb-radio Description: Generate a playlist with LB Radio. Parameters: - prompt: The LB Radio prompt from which to generate playlists. - mode: The mode that LB radio should use. Must be easy, medium or hard. Example Response: ```json { "payload": { "jspf" : {}, // "feedback": [] // } } ``` Status Codes: - 200 OK: success - 400 Bad Request: bad request: some parameters are missing or invalid ``` -------------------------------- ### CLI Command: ./develop.sh-manage-dump-create_sample Source: https://listenbrainz.readthedocs.io/en/latest/genindex This command generates a sample data dump. Users can specify the desired output location for the sample dump and configure the number of threads to be used during its creation. ```APIDOC ./develop.sh-manage-dump-create_sample --location | -l: Specifies the output location for the sample dump. --threads | -t: Specifies the number of threads to use for processing. ```