### Start Collector Source: https://github.com/naver/scavenger/blob/develop/doc/installation.md Run the Scavenger Collector JAR file. Ensure JDK 21 or later is installed. ```bash java -jar scavenger-collector.jar ``` -------------------------------- ### Start Scavenger Collector with H2 (Development) Source: https://context7.com/naver/scavenger/llms.txt Launches the collector service using an in-memory H2 database for development purposes. No external database setup is required. ```bash java -jar scavenger-collector.jar ``` ```bash java -Dspring.profiles.active=h2 \ -jar scavenger-collector.jar ``` -------------------------------- ### Example scavenger.conf Configuration Source: https://github.com/naver/scavenger/blob/develop/doc/installation.md This is an example of the scavenger.conf file, showing key-value pairs for agent settings. Ensure the apiKey and serverUrl are correctly set for your environment. ```properties apiKey=eb99ff0f-aaaa-bbbb-cccc-5d1ec81f6183 serverUrl=http://10.106.93.41:8081 environment=dev appName=apiserver codeBase=webapps/ROOT/WEB-INF packages=com.navercorp ``` -------------------------------- ### Configure API with MySQL Source: https://github.com/naver/scavenger/blob/develop/doc/installation.md Start the API with MySQL configuration. Ensure you replace placeholders with your actual MySQL credentials and URL. ```bash java -Dspring.profiles.active=mysql -Dspring.datasource.url={mysql url} -Dspring.datasource.username={mysql user name} -Dspring.datasource.password={mysql password} -Dspring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver -jar scavenger-api-boot.jar ``` -------------------------------- ### Start Scavenger Agent Source: https://github.com/naver/scavenger/blob/develop/scavenger-agent-python/README.md Start the Scavenger Agent by initializing Config and Agent, then calling agent.start(). This code must be at the program's entry point. ```py from scavenger import Agent, Config config = Config.load_config() agent = Agent(config) agent.start() ### your source code from ... ``` -------------------------------- ### Install Dependencies with Poetry Source: https://github.com/naver/scavenger/blob/develop/scavenger-agent-python/README.md Use this command to install project dependencies managed by Poetry. ```sh $ poetry install ``` -------------------------------- ### Configure Collector with MySQL Source: https://github.com/naver/scavenger/blob/develop/doc/installation.md Start the Collector with MySQL configuration. Ensure you replace placeholders with your actual MySQL credentials and URL. ```bash java -Dspring.profiles.active=mysql -Dspring.datasource.url={mysql url} -Dspring.datasource.username={mysql user name} -Dspring.datasource.password={mysql password} -Dspring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver -jar scavenger-collector.jar ``` -------------------------------- ### Start Scavenger Collector with MySQL (Production) Source: https://context7.com/naver/scavenger/llms.txt Starts the collector service configured to use a MySQL database. Ensure MySQL is running and accessible with the provided credentials and connection details. ```bash java -Dspring.profiles.active=mysql \ -Dspring.datasource.url=jdbc:mysql://localhost:3306/scavenger \ -Dspring.datasource.username=scavenger \ -Dspring.datasource.password=secret \ -Dspring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver \ -jar scavenger-collector.jar ``` -------------------------------- ### Database Schema Setup Source: https://context7.com/naver/scavenger/llms.txt Initialize the database schema for MySQL/Vitess deployments. The schema is auto-managed for H2 but requires manual setup for production databases. ```APIDOC ## Database Schema Setup Initialize the database schema for MySQL/Vitess deployments. The schema is auto-managed for H2 but requires manual setup for production databases. ```sql -- Create the scavenger database CREATE DATABASE IF NOT EXISTS scavenger CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; USE scavenger; -- The application will auto-create tables on first run with Flyway migrations -- Manual verification queries: -- Check workspace (customer) data SELECT id, name, api_key, created_at FROM customers; -- Check registered applications SELECT c.name as workspace, a.name as application, a.created_at FROM applications a JOIN customers c ON a.customer_id = c.id; -- Check method invocations summary SELECT c.name as workspace, a.name as application, COUNT(DISTINCT m.id) as total_methods, COUNT(DISTINCT i.method_id) as invoked_methods, ROUND(COUNT(DISTINCT i.method_id) * 100.0 / COUNT(DISTINCT m.id), 2) as invocation_rate FROM customers c JOIN applications a ON a.customer_id = c.id JOIN methods m ON m.customer_id = c.id LEFT JOIN invocations i ON i.method_id = m.id GROUP BY c.id, a.id; -- Find potentially dead methods (not invoked in last 90 days) SELECT m.signature, m.declaring_type, MAX(i.invoked_at_millis) as last_invoked FROM methods m LEFT JOIN invocations i ON i.method_id = m.id WHERE m.customer_id = 1 GROUP BY m.id HAVING last_invoked IS NULL OR last_invoked < UNIX_TIMESTAMP(DATE_SUB(NOW(), INTERVAL 90 DAY)) * 1000; ``` ``` -------------------------------- ### Install Java Agent using -javaagent flag Source: https://context7.com/naver/scavenger/llms.txt Attach the Scavenger Java agent to your JVM application at startup using the `-javaagent` flag and specify the configuration file path with `-Dscavenger.configuration`. Examples cover basic startup, Tomcat deployments, and Spring Boot applications with custom gRPC message sizes. ```bash # Basic agent startup java -javaagent:/path/to/scavenger-agent-java.jar \ -Dscavenger.configuration=/path/to/scavenger.conf \ -jar my-application.jar ``` ```bash # Tomcat deployment - add to setenv.sh or catalina.sh export JAVA_OPTS "$JAVA_OPTS -Dscavenger.configuration=$CATALINA_BASE/conf/scavenger.conf" export JAVA_OPTS "$JAVA_OPTS -javaagent:/opt/scavenger/scavenger-agent-java.jar" ``` ```bash # Spring Boot with custom gRPC message size java -javaagent:/path/to/scavenger-agent-java.jar \ -Dscavenger.configuration=/path/to/scavenger.conf \ -Dscavenger.max-message-size=20MB \ -jar my-spring-boot-app.jar ``` ```bash # Download latest agent from Maven Central # https://repo1.maven.org/maven2/com/navercorp/scavenger/scavenger-agent-java/{VERSION}/scavenger-agent-java-{VERSION}.jar ``` -------------------------------- ### Flask Route Example Source: https://github.com/naver/scavenger/blob/develop/scavenger-agent-python/README.md Illustrates a basic Flask route. Note that Scavenger Python Agent may not instrument functions called without a direct reference, such as this example. ```py from flask import Flask app = Flask(__name__) @app.route('/') def hello_world(): return 'Hello World!' ``` -------------------------------- ### Start Scavenger API with MySQL (Production) Source: https://context7.com/naver/scavenger/llms.txt Starts the API service configured for a MySQL database, suitable for production environments. It requires the collector server URL and specifies a custom server port. ```bash java -Dspring.profiles.active=mysql \ -Dspring.datasource.url=jdbc:mysql://localhost:3306/scavenger \ -Dspring.datasource.username=scavenger \ -Dspring.datasource.password=secret \ -Dspring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver \ -Dscavenger.collector-server-url=http://collector.example.com:8080 \ -Dserver.port=8081 \ -jar scavenger-api-boot.jar ``` -------------------------------- ### Configure Scavenger Agent with scavenger.conf Source: https://github.com/naver/scavenger/blob/develop/scavenger-agent-python/README.md Example configuration file for the Scavenger Agent. Load configuration using Config.load_config(). ```py # scavenger.conf apiKey=eb99ff0f-aaaa-bbbb-cccc-5d1ec81f6183 serverUrl=http://10.106.93.41:8081 environment=dev appName=apiserver packages=views codebase=. --- # source config = Config.load_config() agent = Agent(config) ``` -------------------------------- ### Get Detailed Environment Info Source: https://context7.com/naver/scavenger/llms.txt Fetches detailed information about environments within a workspace. This endpoint provides comprehensive data for each deployment stage. ```bash curl -X GET "http://localhost:8081/api/customers/1/environments/_detail" ``` -------------------------------- ### Get Detailed Application Info Source: https://context7.com/naver/scavenger/llms.txt Fetches detailed information about applications within a workspace, including method counts and the last time they were seen. This is useful for performance monitoring. ```bash curl -X GET "http://localhost:8081/api/customers/1/applications/_detail" ``` -------------------------------- ### Build Scavenger Agent with Gradle Source: https://github.com/naver/scavenger/blob/develop/doc/installation.md Run this command in the root of the cloned Git repository to assemble the Scavenger Java agent. Ensure you have Gradle installed and configured. ```bash ./gradlew assemble -p scavenger-agent-java ``` -------------------------------- ### Install Scavenger Agent Python Source: https://github.com/naver/scavenger/blob/develop/scavenger-agent-python/README.md Install the scavenger-agent-python package using pip. Ensure Python version is 3.7 or higher. ```sh $ pip install scavenger-agent-python --save ``` -------------------------------- ### Get Root-Level Package Statistics Source: https://context7.com/naver/scavenger/llms.txt Retrieve invocation statistics for top-level packages within a snapshot. The `parent` parameter should be empty to get root-level data. ```bash curl -X GET "http://localhost:8081/api/customers/1/snapshots/1?parent=" ``` -------------------------------- ### Configure Python Agent Source: https://context7.com/naver/scavenger/llms.txt Set up the Scavenger Python agent by either loading configuration from a `scavenger.conf` file or defining it programmatically. Ensure the agent is started before importing application modules. Includes graceful shutdown handling. ```python # main.py - Python agent setup (must be at the very beginning) from scavenger import Agent, Config # Option 1: Load configuration from scavenger.conf file config = Config.load_config("scavenger.conf") agent = Agent(config) agent.start() # Option 2: Configure programmatically config = Config( api_key="eb99ff0f-aaaa-bbbb-cccc-5d1ec81f6183", server_url="http://collector.example.com:8080", app_name="my-python-service", packages=["myapp", "myapp.services", "myapp.handlers"], codebase=["."], exclude_packages=["myapp.tests", "myapp.migrations"], environment="production", app_version="2.0.0", async_codebase_scan_mode=True, debug_mode=False ) agent = Agent(config) agent.start() # IMPORTANT: Import your application modules AFTER agent.start() from myapp import create_app app = create_app() # Graceful shutdown handler import signal import sys def shutdown_gracefully(*args): print("Shutting down...") agent.shutdown() sys.exit(0) signal.signal(signal.SIGTERM, shutdown_gracefully) signal.signal(signal.SIGINT, shutdown_gracefully) if __name__ == "__main__": app.run() ``` -------------------------------- ### Start Scavenger API with H2 (Development) Source: https://context7.com/naver/scavenger/llms.txt Launches the API service using an in-memory H2 database for development. It requires the collector server URL for health checks. ```bash java -Dscavenger.collector-server-url=http://localhost:8080 \ -jar scavenger-api-boot.jar ``` ```bash java -Dspring.profiles.active=h2 \ -Dscavenger.collector-server-url=http://localhost:8080 \ -jar scavenger-api-boot.jar ``` -------------------------------- ### Start API with Collector URL Source: https://github.com/naver/scavenger/blob/develop/doc/installation.md Run the Scavenger API JAR file, specifying the Collector's server URL. The default API port is 8081. ```bash java -Dscavenger.collector-server-url=http://localhost:8080 -jar scavenger-api-boot.jar ``` -------------------------------- ### MySQL/Vitess Database Schema Initialization Source: https://context7.com/naver/scavenger/llms.txt SQL script to create the 'scavenger' database and set character encoding. The application auto-manages table creation with Flyway migrations, but this provides initial setup. ```sql -- Create the scavenger database CREATE DATABASE IF NOT EXISTS scavenger CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; USE scavenger; ``` -------------------------------- ### Configure Scavenger Collector Ports and Message Size Source: https://context7.com/naver/scavenger/llms.txt Starts the collector service with custom Armeria port and maximum message size configurations. Useful for tuning network performance. ```bash java -Darmeria.port=9090 \ -Darmeria.max-message-size=20MB \ -jar scavenger-collector.jar ``` -------------------------------- ### Get Call Stack Relationships Source: https://context7.com/naver/scavenger/llms.txt Retrieve call stack information for methods within a snapshot. This feature requires `callStackTraceMode=true` to be enabled on the agent. ```bash curl -X GET "http://localhost:8081/api/customers/1/snapshots/1/callstacks" ``` -------------------------------- ### Create New Snapshot Source: https://context7.com/naver/scavenger/llms.txt Use this endpoint to create a new snapshot for a customer. Specify the snapshot name, application and environment IDs, and package filters. ```bash curl -X POST "http://localhost:8081/api/customers/1/snapshots" \ -H "Content-Type: application/json" \ -d '{ "name": "Production Dead Code Analysis", "applicationIdList": [1, 2], "environmentIdList": [3], "filterInvokedAtMillis": 1696118400000, "packages": "com.mycompany.core.**,com.mycompany.api.**" }' ``` -------------------------------- ### Build Project with Poetry Source: https://github.com/naver/scavenger/blob/develop/scavenger-agent-python/README.md Build the project package using Poetry. ```sh $ poetry build ``` -------------------------------- ### List All Workspaces Source: https://context7.com/naver/scavenger/llms.txt Retrieves a list of all existing workspaces (customers) from the API. This is useful for verifying workspace creation and management. ```bash curl -X GET "http://localhost:8081/api/customers" ``` -------------------------------- ### List Environments in a Workspace Source: https://context7.com/naver/scavenger/llms.txt Retrieves a list of all deployment environments (e.g., dev, staging, production) configured for a specific workspace. ```bash curl -X GET "http://localhost:8081/api/customers/1/environments" ``` -------------------------------- ### Enable API H2 Profile Source: https://github.com/naver/scavenger/blob/develop/doc/installation.md Run the API with the 'h2' profile. This uses a separate in-memory H2 database from the Collector. ```bash java -Dspring.profiles.active=h2 -jar scavenger-api-boot.jar ``` -------------------------------- ### List Applications in a Workspace Source: https://context7.com/naver/scavenger/llms.txt Retrieves a list of all applications registered within a specific workspace. This helps in understanding the services deployed in a customer's environment. ```bash curl -X GET "http://localhost:8081/api/customers/1/applications" ``` -------------------------------- ### Drill Down into Specific Package Statistics Source: https://context7.com/naver/scavenger/llms.txt Query for sub-packages and classes within a specified package. Provide the package name to the `parent` query parameter. ```bash curl -X GET "http://localhost:8081/api/customers/1/snapshots/1?parent=com.mycompany.api" ``` -------------------------------- ### Create GitHub Mapping Source: https://context7.com/naver/scavenger/llms.txt Establish a new mapping between a Java package and a GitHub repository URL. This is essential for the 'Open in GitHub' functionality. ```bash curl -X POST "http://localhost:8081/api/customers/1/mappings" \ -H "Content-Type: application/json" \ -d '{ "basePackage": "com.mycompany.core", "url": "https://github.com/mycompany/core-lib/blob/main/src/main/java" }' ``` -------------------------------- ### List GitHub Mappings Source: https://context7.com/naver/scavenger/llms.txt View existing GitHub repository mappings. These mappings link Java packages to their corresponding GitHub URLs, enabling the 'Open in GitHub' feature. ```bash curl -X GET "http://localhost:8081/api/customers/1/mappings" ``` -------------------------------- ### Enable Collector H2 Profile Source: https://github.com/naver/scavenger/blob/develop/doc/installation.md Run the Collector with the 'h2' profile to use file-based H2 persistence instead of in-memory. ```bash java -Dspring.profiles.active=h2 -jar scavenger-collector.jar ``` -------------------------------- ### Run Tests with Poetry Source: https://github.com/naver/scavenger/blob/develop/scavenger-agent-python/README.md Execute unit tests for the project using Poetry. ```sh $ poetry run python -m unittest ``` -------------------------------- ### Search Methods by Signature Source: https://context7.com/naver/scavenger/llms.txt Find all methods within a snapshot that match a given signature. This is useful for locating specific functions or identifying overloaded methods. ```bash curl -X GET "http://localhost:8081/api/customers/1/snapshots/1/nodes?signature=processOrder" ``` -------------------------------- ### Build API Manually Source: https://github.com/naver/scavenger/blob/develop/doc/installation.md Build the Scavenger API component from source code using Gradle. ```bash ./gradlew assemble -p scavenger-api ``` -------------------------------- ### Create a New Workspace Source: https://context7.com/naver/scavenger/llms.txt Creates a new workspace with a specified name. The API returns the details of the newly created workspace, including its ID. ```bash curl -X POST "http://localhost:8081/api/customers" \ -H "Content-Type: application/json" \ -d '{"name": "my-new-project"}' ``` -------------------------------- ### Configure Scavenger Agent with Config Instance Source: https://github.com/naver/scavenger/blob/develop/scavenger-agent-python/README.md Instantiate the Config object directly with parameters. The agent is then initialized with this configuration. ```py config = Config( api_key="eb99ff0f-aaaa-bbbb-cccc-5d1ec81f6183", server_url="http://10.106.93.41:8081", environment="dev", app_name="apiserver", packages=["views"], codebase=["."], ) agent = Agent(config) ``` -------------------------------- ### Build Scavenger Collector from Source Source: https://context7.com/naver/scavenger/llms.txt Compiles the Scavenger collector service from its source code using Gradle. This command should be executed in the project's root directory. ```bash ./gradlew assemble -p scavenger-collector ``` -------------------------------- ### Create MySQL Database for Scavenger Source: https://github.com/naver/scavenger/blob/develop/doc/installation.md SQL command to create the 'scavenger' database in MySQL. ```sql CREATE DATABASE scavenger; ``` -------------------------------- ### Application and Environment Management API Source: https://context7.com/naver/scavenger/llms.txt Endpoints for viewing and managing applications and environments within a workspace. ```APIDOC ## GET /api/customers/{customerId}/applications ### Description Lists all applications within a specific workspace. ### Method GET ### Endpoint /api/customers/{customerId}/applications #### Path Parameters - **customerId** (integer) - Required - The ID of the workspace. ### Response #### Success Response (200) - **id** (integer) - The unique identifier of the application. - **name** (string) - The name of the application. #### Response Example ```json [ {"id": 1, "name": "api-server"}, {"id": 2, "name": "batch-processor"} ] ``` ## GET /api/customers/{customerId}/applications/_detail ### Description Retrieves detailed information for all applications in a workspace, including method counts and last seen timestamps. ### Method GET ### Endpoint /api/customers/{customerId}/applications/_detail #### Path Parameters - **customerId** (integer) - Required - The ID of the workspace. ### Response #### Success Response (200) - **id** (integer) - The unique identifier of the application. - **name** (string) - The name of the application. - **methodCount** (integer) - The number of methods associated with the application. - **lastSeenAtMillis** (integer) - The timestamp (in milliseconds) when the application was last seen. #### Response Example ```json [ {"id": 1, "name": "api-server", "methodCount": 1523, "lastSeenAtMillis": 1699876543210}, {"id": 2, "name": "batch-processor", "methodCount": 847, "lastSeenAtMillis": 1699876500000} ] ``` ## DELETE /api/customers/{customerId}/applications/{applicationId} ### Description Deletes an application and all its associated data from a workspace. ### Method DELETE ### Endpoint /api/customers/{customerId}/applications/{applicationId} #### Path Parameters - **customerId** (integer) - Required - The ID of the workspace. - **applicationId** (integer) - Required - The ID of the application to delete. ## GET /api/customers/{customerId}/environments ### Description Lists all environments within a specific workspace. ### Method GET ### Endpoint /api/customers/{customerId}/environments #### Path Parameters - **customerId** (integer) - Required - The ID of the workspace. ### Response #### Success Response (200) - **id** (integer) - The unique identifier of the environment. - **name** (string) - The name of the environment. #### Response Example ```json [ {"id": 1, "name": "dev"}, {"id": 2, "name": "staging"}, {"id": 3, "name": "production"} ] ``` ## GET /api/customers/{customerId}/environments/_detail ### Description Retrieves detailed information for all environments in a workspace. ### Method GET ### Endpoint /api/customers/{customerId}/environments/_detail #### Path Parameters - **customerId** (integer) - Required - The ID of the workspace. ## DELETE /api/customers/{customerId}/environments/{environmentId} ### Description Deletes an environment from a workspace. ### Method DELETE ### Endpoint /api/customers/{customerId}/environments/{environmentId} #### Path Parameters - **customerId** (integer) - Required - The ID of the workspace. - **environmentId** (integer) - Required - The ID of the environment to delete. ``` -------------------------------- ### Query Workspace by Name Source: https://context7.com/naver/scavenger/llms.txt Fetches a specific workspace by its name. This endpoint allows for targeted retrieval of workspace information. ```bash curl -X GET "http://localhost:8081/api/customers/_query?name=my-project" ``` -------------------------------- ### Configure Java Agent with scavenger.conf Source: https://context7.com/naver/scavenger/llms.txt Configure the Java agent's behavior, including API keys, server URLs, packages to scan, and filtering options, using a properties file. Specify codebase locations for different deployment types. ```properties # scavenger.conf - Agent configuration file # Required settings apiKey=eb99ff0f-aaaa-bbbb-cccc-5d1ec81f6183 serverUrl=http://collector.example.com:8080 appName=my-api-server packages=com.mycompany.app,com.mycompany.core # Optional: Specify codebase location for proper class scanning # For Tomcat deployments: # codebase=webapps/ROOT/WEB-INF # For executable JARs: # codebase=path/to/app.jar # For layered JARs (Spring Boot): # codebase=BOOT-INF/lib,BOOT-INF/classes # Filtering options excludePackages=com.mycompany.app.generated,com.mycompany.app.test annotations=@org.springframework.web.bind.annotation.RestController,@org.springframework.stereotype.Service methodVisibility=protected excludeConstructors=true excludeGetterSetter=true # Environment identification environment=production appVersion=1.2.3 hostname=api-server-01 # Performance tuning asyncCodeBaseScanMode=true callStackTraceMode=false debugMode=false maxMethodsCount=100000 ``` -------------------------------- ### Export Snapshot to CSV Source: https://context7.com/naver/scavenger/llms.txt Download snapshot data in CSV format. Specify the desired filename using the `fn` query parameter and redirect output to a file. ```bash curl -X GET "http://localhost:8081/api/customers/1/snapshot/1/export?fn=dead-code-report.csv" \ -o dead-code-report.csv ``` -------------------------------- ### List All Snapshots in a Workspace Source: https://context7.com/naver/scavenger/llms.txt Retrieves a list of all captured snapshots within a workspace. Snapshots are used for dead code analysis and contain details about the analysis scope. ```bash curl -X GET "http://localhost:8081/api/customers/1/snapshots" ``` -------------------------------- ### Scavenger Database Verification Queries Source: https://context7.com/naver/scavenger/llms.txt SQL queries to verify data in the Scavenger database, including checking customer information, registered applications, method invocation summaries, and identifying potentially dead methods. ```sql -- The application will auto-create tables on first run with Flyway migrations -- Manual verification queries: -- Check workspace (customer) data SELECT id, name, api_key, created_at FROM customers; ``` ```sql -- Check registered applications SELECT c.name as workspace, a.name as application, a.created_at FROM applications a JOIN customers c ON a.customer_id = c.id; ``` ```sql -- Check method invocations summary SELECT c.name as workspace, a.name as application, COUNT(DISTINCT m.id) as total_methods, COUNT(DISTINCT i.method_id) as invoked_methods, ROUND(COUNT(DISTINCT i.method_id) * 100.0 / COUNT(DISTINCT m.id), 2) as invocation_rate FROM customers c JOIN applications a ON a.customer_id = c.id JOIN methods m ON m.customer_id = c.id LEFT JOIN invocations i ON i.method_id = m.id GROUP BY c.id, a.id; ``` ```sql -- Find potentially dead methods (not invoked in last 90 days) SELECT m.signature, m.declaring_type, MAX(i.invoked_at_millis) as last_invoked FROM methods m LEFT JOIN invocations i ON i.method_id = m.id WHERE m.customer_id = 1 GROUP BY m.id HAVING last_invoked IS NULL OR last_invoked < UNIX_TIMESTAMP(DATE_SUB(NOW(), INTERVAL 90 DAY)) * 1000; ``` -------------------------------- ### Delete Application and its Data Source: https://context7.com/naver/scavenger/llms.txt Removes a specific application and all its associated data from a workspace. This action is irreversible. ```bash curl -X DELETE "http://localhost:8081/api/customers/1/applications/2" ``` -------------------------------- ### Set Scavenger Agent Configuration and Java Agent Path Source: https://github.com/naver/scavenger/blob/develop/doc/installation.md Use these environment variable exports to specify the Scavenger configuration file path and the Java agent JAR file location. ```bash export JAVA_OPTS="$JAVA_OPTS -Dscavenger.configuration=$CATALINA_BASE/conf/scavenger.conf" export JAVA_OPTS="$JAVA_OPTS -javaagent:/home1/irteam/scavenger-agent.jar" ``` -------------------------------- ### Snapshot Management API Source: https://context7.com/naver/scavenger/llms.txt APIs for creating, updating, refreshing, and deleting snapshots. ```APIDOC ## POST /api/customers/{customerId}/snapshots ### Description Creates a new snapshot for a customer. ### Method POST ### Endpoint /api/customers/{customerId}/snapshots ### Parameters #### Path Parameters - **customerId** (integer) - Required - The ID of the customer. #### Request Body - **name** (string) - Required - The name of the snapshot. - **applicationIdList** (array of integers) - Required - A list of application IDs to include. - **environmentIdList** (array of integers) - Required - A list of environment IDs to include. - **filterInvokedAtMillis** (long) - Required - A timestamp in milliseconds to filter invocations. - **packages** (string) - Required - A comma-separated string of package patterns. ### Request Example ```json { "name": "Production Dead Code Analysis", "applicationIdList": [1, 2], "environmentIdList": [3], "filterInvokedAtMillis": 1696118400000, "packages": "com.mycompany.core.**,com.mycompany.api.**" } ``` ### Response #### Success Response (200) - **id** (integer) - The ID of the created snapshot. - **customerId** (integer) - The ID of the customer. - **name** (string) - The name of the snapshot. - **createdAt** (string) - The creation timestamp of the snapshot. - **applications** (array of integers) - List of application IDs. - **environments** (array of integers) - List of environment IDs. - **filterInvokedAtMillis** (long) - The filter timestamp. - **packages** (string) - The package patterns. #### Response Example ```json { "id": 2, "customerId": 1, "name": "Production Dead Code Analysis", "createdAt": "2023-12-15T14:30:00Z", "applications": [1, 2], "environments": [3], "filterInvokedAtMillis": 1696118400000, "packages": "com.mycompany.core.**,com.mycompany.api.**" } ``` ## PUT /api/customers/{customerId}/snapshots/{snapshotId} ### Description Updates the configuration of an existing snapshot. ### Method PUT ### Endpoint /api/customers/{customerId}/snapshots/{snapshotId} ### Parameters #### Path Parameters - **customerId** (integer) - Required - The ID of the customer. - **snapshotId** (integer) - Required - The ID of the snapshot to update. #### Request Body - **name** (string) - Optional - The updated name of the snapshot. - **applicationIdList** (array of integers) - Optional - Updated list of application IDs. - **environmentIdList** (array of integers) - Optional - Updated list of environment IDs. - **filterInvokedAtMillis** (long) - Optional - Updated timestamp for filtering invocations. - **packages** (string) - Optional - Updated comma-separated string of package patterns. ### Request Example ```json { "name": "Updated Analysis", "applicationIdList": [1], "environmentIdList": [2, 3], "filterInvokedAtMillis": 1699401600000, "packages": "com.mycompany.**" } ``` ## POST /api/customers/{customerId}/snapshots/{snapshotId}/refresh ### Description Refreshes a snapshot with the latest invocation data. ### Method POST ### Endpoint /api/customers/{customerId}/snapshots/{snapshotId}/refresh ### Parameters #### Path Parameters - **customerId** (integer) - Required - The ID of the customer. - **snapshotId** (integer) - Required - The ID of the snapshot to refresh. ## DELETE /api/customers/{customerId}/snapshots/{snapshotId} ### Description Deletes a specific snapshot. ### Method DELETE ### Endpoint /api/customers/{customerId}/snapshots/{snapshotId} ### Parameters #### Path Parameters - **customerId** (integer) - Required - The ID of the customer. - **snapshotId** (integer) - Required - The ID of the snapshot to delete. ``` -------------------------------- ### gRPC Agent Protocol Definition Source: https://context7.com/naver/scavenger/llms.txt Defines the gRPC service for agent communication, including methods for polling configuration and publishing data like codebase and invocation details. This protocol is optimized for high-frequency updates. ```protobuf // GrpcAgentService.proto - Agent communication protocol service GrpcAgentService { // Poll for configuration updates (intervals, retry settings) rpc PollConfig (GetConfigRequest) returns (GetConfigResponse) {} // Publish codebase (list of methods in the application) rpc SendCodeBasePublication (CodeBasePublication) returns (PublicationResponse) {} // Publish invocation data (which methods were called) rpc SendInvocationDataPublication (InvocationDataPublication) returns (PublicationResponse) {} // Publish call stack data (who called whom) rpc SendCallStackDataPublication (CallStackDataPublication) returns (PublicationResponse) {} } // Example GetConfigRequest message GetConfigRequest { string api_key = 1; // Workspace API key string jvm_uuid = 2; // Unique JVM identifier } // Example GetConfigResponse message GetConfigResponse { int32 config_poll_interval_seconds = 1; // How often to poll config int32 config_poll_retry_interval_seconds = 2; // Retry interval on failure int32 code_base_publisher_check_interval_seconds = 3; // Codebase publish interval int32 code_base_publisher_retry_interval_seconds = 4; // Retry interval for codebase int32 invocation_data_publisher_interval_seconds = 5; // Invocation publish interval int32 invocation_data_publisher_retry_interval_seconds = 6; // Retry interval for invocations } // CodeBaseEntry - represents a single method message CodeBaseEntry { string declaring_type = 1; // e.g., "com.mycompany.OrderService" string visibility = 2; // e.g., "public" string signature = 3; // e.g., "public void processOrder(Order)" string method_name = 4; // e.g., "processOrder" string modifiers = 5; // e.g., "public" string package_name = 6; // e.g., "com.mycompany" string parameter_types = 7; // e.g., "Order" string signature_hash = 8; // SHA-256 hash of signature } ``` -------------------------------- ### Snapshot Data Query API Source: https://context7.com/naver/scavenger/llms.txt APIs for querying snapshot data, including package statistics, method signatures, and call stacks. ```APIDOC ## GET /api/customers/{customerId}/snapshots/{snapshotId} ### Description Queries snapshot data to retrieve invocation statistics by package and method. ### Method GET ### Endpoint /api/customers/{customerId}/snapshots/{snapshotId} ### Parameters #### Path Parameters - **customerId** (integer) - Required - The ID of the customer. - **snapshotId** (integer) - Required - The ID of the snapshot. #### Query Parameters - **parent** (string) - Optional - Used to navigate the package hierarchy. If empty, returns root-level packages. ### Response #### Success Response (200) - Returns a list of packages with invocation counts, or sub-packages/classes depending on the `parent` parameter. ### Response Example (Varies based on the `parent` parameter. See description for details.) ## GET /api/customers/{customerId}/snapshots/{snapshotId}/nodes ### Description Searches for methods within a snapshot by their signature. ### Method GET ### Endpoint /api/customers/{customerId}/snapshots/{snapshotId}/nodes ### Parameters #### Path Parameters - **customerId** (integer) - Required - The ID of the customer. - **snapshotId** (integer) - Required - The ID of the snapshot. #### Query Parameters - **signature** (string) - Required - The method signature to search for. ### Response #### Success Response (200) - Returns a list of methods matching the provided signature. ## GET /api/customers/{customerId}/snapshots/{snapshotId}/callstacks ### Description Retrieves call stack relationships for methods within a snapshot. Requires `callStackTraceMode=true` to be set on the agent. ### Method GET ### Endpoint /api/customers/{customerId}/snapshots/{snapshotId}/callstacks ### Parameters #### Path Parameters - **customerId** (integer) - Required - The ID of the customer. - **snapshotId** (integer) - Required - The ID of the snapshot. ### Response #### Success Response (200) - A map where keys are method signatures and values are lists of methods they call. #### Response Example ```json { "com.mycompany.api.OrderController.createOrder()": [ "com.mycompany.service.OrderService.processOrder()", "com.mycompany.service.PaymentService.charge()" ], "...": [] } ``` ## GET /api/customers/{customerId}/snapshot/{snapshotId}/export ### Description Exports snapshot data to a CSV file. ### Method GET ### Endpoint /api/customers/{customerId}/snapshot/{snapshotId}/export ### Parameters #### Path Parameters - **customerId** (integer) - Required - The ID of the customer. - **snapshotId** (integer) - Required - The ID of the snapshot. #### Query Parameters - **fn** (string) - Required - The filename for the exported CSV (e.g., `dead-code-report.csv`). ### Response #### Success Response (200) - Returns the snapshot data as a CSV file. ``` -------------------------------- ### Workspace Management API Source: https://context7.com/naver/scavenger/llms.txt Endpoints for creating and managing workspaces, which serve as isolated containers for applications, environments, and snapshots. ```APIDOC ## GET /api/customers ### Description Lists all workspaces. ### Method GET ### Endpoint /api/customers ### Response #### Success Response (200) - **id** (integer) - The unique identifier of the workspace. - **name** (string) - The name of the workspace. #### Response Example ```json [ {"id": 1, "name": "my-project"}, {"id": 2, "name": "another-project"} ] ``` ## POST /api/customers ### Description Creates a new workspace. ### Method POST ### Endpoint /api/customers ### Parameters #### Request Body - **name** (string) - Required - The name of the new workspace. ### Request Example ```json { "name": "my-new-project" } ``` ### Response #### Success Response (200) - **id** (integer) - The unique identifier of the newly created workspace. - **name** (string) - The name of the workspace. #### Response Example ```json {"id": 3, "name": "my-new-project"} ``` ## GET /api/customers/_query?name={name} ### Description Queries for a workspace by its name. ### Method GET ### Endpoint /api/customers/_query #### Query Parameters - **name** (string) - Required - The name of the workspace to query. ### Response #### Success Response (200) - **id** (integer) - The unique identifier of the workspace. - **name** (string) - The name of the workspace. #### Response Example ```json {"id": 1, "name": "my-project"} ``` ## POST /api/customers/{id}/reset ### Description Resets a workspace, deleting all its data but keeping the workspace itself. ### Method POST ### Endpoint /api/customers/{id}/reset #### Path Parameters - **id** (integer) - Required - The ID of the workspace to reset. ## DELETE /api/customers/{id} ### Description Deletes a workspace entirely. ### Method DELETE ### Endpoint /api/customers/{id} #### Path Parameters - **id** (integer) - Required - The ID of the workspace to delete. ``` -------------------------------- ### GitHub Integration API Source: https://context7.com/naver/scavenger/llms.txt APIs for configuring GitHub repository mappings to enable 'Open in GitHub' functionality. ```APIDOC ## GET /api/customers/{customerId}/mappings ### Description Lists all configured GitHub repository mappings for a customer. ### Method GET ### Endpoint /api/customers/{customerId}/mappings ### Parameters #### Path Parameters - **customerId** (integer) - Required - The ID of the customer. ### Response #### Success Response (200) - An array of mapping objects, each containing an ID, base package, and the corresponding GitHub URL. #### Response Example ```json [ { "id": 1, "basePackage": "com.mycompany.api", "url": "https://github.com/mycompany/api-service/blob/main/src/main/java" } ] ``` ## POST /api/customers/{customerId}/mappings ### Description Creates a new GitHub repository mapping for a customer. ### Method POST ### Endpoint /api/customers/{customerId}/mappings ### Parameters #### Path Parameters - **customerId** (integer) - Required - The ID of the customer. #### Request Body - **basePackage** (string) - Required - The base Java package to map. - **url** (string) - Required - The URL of the GitHub repository. ### Request Example ```json { "basePackage": "com.mycompany.core", "url": "https://github.com/mycompany/core-lib/blob/main/src/main/java" } ``` ### Response #### Success Response (200) - The created mapping object, including its ID. #### Response Example ```json { "id": 2, "basePackage": "com.mycompany.core", "url": "https://github.com/mycompany/core-lib/blob/main/src/main/java" } ``` ## DELETE /api/customers/{customerId}/mappings/{mappingId} ### Description Deletes a specific GitHub repository mapping. ### Method DELETE ### Endpoint /api/customers/{customerId}/mappings/{mappingId} ### Parameters #### Path Parameters - **customerId** (integer) - Required - The ID of the customer. - **mappingId** (integer) - Required - The ID of the mapping to delete. ``` -------------------------------- ### Remote JVM Debug Configuration Source: https://github.com/naver/scavenger/blob/develop/doc/debugging.md Use this JVM option to enable remote debugging. Ensure the host is 'localhost' and the port is '5005'. ```bash -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:5005 ``` -------------------------------- ### Delete an Environment Source: https://context7.com/naver/scavenger/llms.txt Removes a specific deployment environment and all its associated data from a workspace. This action is irreversible. ```bash curl -X DELETE "http://localhost:8081/api/customers/1/environments/1" ``` -------------------------------- ### Configure Collector Port Source: https://github.com/naver/scavenger/blob/develop/doc/installation.md Override the default http and grpc ports for the Collector using the -D option. The default is 8080. ```bash java -Darmeria.port=8080 -jar scavenger-collector.jar ``` -------------------------------- ### Snapshot Management API Source: https://context7.com/naver/scavenger/llms.txt Endpoints for managing snapshots, which capture invocation states for dead code analysis. ```APIDOC ## GET /api/customers/{customerId}/snapshots ### Description Lists all snapshots within a specific workspace. ### Method GET ### Endpoint /api/customers/{customerId}/snapshots #### Path Parameters - **customerId** (integer) - Required - The ID of the workspace. ### Response #### Success Response (200) - **id** (integer) - The unique identifier of the snapshot. - **customerId** (integer) - The ID of the workspace the snapshot belongs to. - **name** (string) - The name of the snapshot. - **createdAt** (string) - The timestamp when the snapshot was created (ISO 8601 format). - **applications** (array of integers) - A list of application IDs included in the snapshot. - **environments** (array of integers) - A list of environment IDs included in the snapshot. - **filterInvokedAtMillis** (integer) - The timestamp (in milliseconds) used to filter invocations. - **packages** (string) - A glob pattern specifying packages to include in the analysis. #### Response Example ```json [ { "id": 1, "customerId": 1, "name": "Q4 2023 Analysis", "createdAt": "2023-12-01T10:00:00Z", "applications": [1, 2], "environments": [2, 3], "filterInvokedAtMillis": 1696118400000, "packages": "com.mycompany.**" } ] ``` ``` -------------------------------- ### Configure Collector Max Message Size Source: https://github.com/naver/scavenger/blob/develop/doc/installation.md Set the maximum message size for gRPC in the Collector. Supports B, KB, MB, GB, and TB units. Default is 10MB. ```bash java -Darmeria.max-message-size=10MB -jar scavenger-collector.jar ``` -------------------------------- ### Agent Monitoring API Source: https://context7.com/naver/scavenger/llms.txt API for monitoring the status and details of connected agents. ```APIDOC ## GET /api/customers/{customerId}/agents ### Description Lists all connected agents in a workspace for a given customer. ### Method GET ### Endpoint /api/customers/{customerId}/agents ### Parameters #### Path Parameters - **customerId** (integer) - Required - The ID of the customer. ### Response #### Success Response (200) - An array of agent objects, each containing details like ID, JVM UUID, hostname, application name, version, environment, and last polled time. #### Response Example ```json [ { "id": 1, "jvmUuid": "abc123-def456", "hostname": "api-server-01", "appName": "api-server", "appVersion": "1.2.3", "environment": "production", "lastPolledAt": "2023-12-15T14:30:00Z" }, { "id": 2, "jvmUuid": "xyz789-uvw012", "hostname": "api-server-02", "appName": "api-server", "appVersion": "1.2.3", "environment": "production", "lastPolledAt": "2023-12-15T14:29:55Z" } ] ``` ``` -------------------------------- ### gRPC Agent Protocol Source: https://context7.com/naver/scavenger/llms.txt The collector exposes a gRPC service for agent communication. Agents use this protocol to poll configuration and publish codebase/invocation data. This is more efficient than HTTP for high-frequency updates. ```APIDOC ## gRPC Agent Protocol The collector exposes a gRPC service for agent communication. Agents use this protocol to poll configuration and publish codebase/invocation data. This is more efficient than HTTP for high-frequency updates. ```protobuf // GrpcAgentService.proto - Agent communication protocol service GrpcAgentService { // Poll for configuration updates (intervals, retry settings) rpc PollConfig (GetConfigRequest) returns (GetConfigResponse) {} // Publish codebase (list of methods in the application) rpc SendCodeBasePublication (CodeBasePublication) returns (PublicationResponse) {} // Publish invocation data (which methods were called) rpc SendInvocationDataPublication (InvocationDataPublication) returns (PublicationResponse) {} // Publish call stack data (who called whom) rpc SendCallStackDataPublication (CallStackDataPublication) returns (PublicationResponse) {} } // Example GetConfigRequest message GetConfigRequest { string api_key = 1; // Workspace API key string jvm_uuid = 2; // Unique JVM identifier } // Example GetConfigResponse message GetConfigResponse { int32 config_poll_interval_seconds = 1; // How often to poll config int32 config_poll_retry_interval_seconds = 2; // Retry interval on failure int32 code_base_publisher_check_interval_seconds = 3; // Codebase publish interval int32 code_base_publisher_retry_interval_seconds = 4; // Retry interval for codebase int32 invocation_data_publisher_interval_seconds = 5; // Invocation publish interval int32 invocation_data_publisher_retry_interval_seconds = 6; // Retry interval for invocations } // CodeBaseEntry - represents a single method message CodeBaseEntry { string declaring_type = 1; // e.g., "com.mycompany.OrderService" string visibility = 2; // e.g., "public" string signature = 3; // e.g., "public void processOrder(Order)" string method_name = 4; // e.g., "processOrder" string modifiers = 5; // e.g., "public" string package_name = 6; // e.g., "com.mycompany" string parameter_types = 7; // e.g., "Order" string signature_hash = 8; // SHA-256 hash of signature } ``` ``` -------------------------------- ### List Connected Agents Source: https://context7.com/naver/scavenger/llms.txt Retrieve a list of all agents currently connected to the Scavenger service within a workspace. This helps in verifying agent status and troubleshooting connectivity. ```bash curl -X GET "http://localhost:8081/api/customers/1/agents" ```