### DataHub Quickstart Output Example Source: https://github.com/datahub-project/datahub/blob/master/docs/quickstart.md Example output from the `datahub docker quickstart` command, showing container status and confirmation of DataHub running. ```shell Fetching docker-compose file https://raw.githubusercontent.com/datahub-project/datahub/master/docker/quickstart/docker-compose.quickstart-profile.yml from GitHub Pulling docker images... Finished pulling docker images! Starting up DataHub... [+] Running 14/14 ✔ Network datahub_network Created 0.0s ✔ Volume "datahub_broker" Created 0.0s ✔ Volume "datahub_mysqldata" Created 0.0s ✔ Volume "datahub_osdata" Created 0.0s ✔ Container datahub-mysql-1 Healthy 11.6s ✔ Container datahub-opensearch-1 Healthy 11.6s ✔ Container datahub-kafka-broker-1 Healthy 6.0s ✔ Container datahub-system-update-quickstart-1 Exited 26.6s ✔ Container datahub-datahub-gms-quickstart-1 Healthy 42.1s ✔ Container datahub-frontend-quickstart-1 Started 26.6s ✔ Container datahub-datahub-actions-quickstart-1 Started 42.1s ✔ DataHub is now running Load sample data: run `datahub init` then `datahub datapack load showcase-ecommerce`, or head to http://localhost:9002 (username: datahub, password: datahub) to play around with the frontend. Need support? Get in touch on Slack: https://datahub.com/slack/ ``` -------------------------------- ### Install DataHub CLI using Poetry Source: https://github.com/datahub-project/datahub/blob/master/docs/quickstart.md Installs the DataHub CLI using Poetry and verifies the installation. ```bash poetry add acryl-datahub poetry shell datahub version ``` -------------------------------- ### Complete Example Source: https://github.com/datahub-project/datahub/blob/master/metadata-integration/java/docs/sdk-v2/getting-started.md A runnable example demonstrating how to build and upsert a dataset entity to DataHub using SDK V2. ```java import datahub.client.v2.DataHubClientV2; import datahub.client.v2.entity.Dataset; import com.linkedin.common.OwnershipType; import java.io.IOException; import java.util.concurrent.ExecutionException; public class DataHubQuickStart { public static void main(String[] args) { // Create client DataHubClientV2 client = DataHubClientV2.builder() .server("http://localhost:8080") .token("your-token-here") // Optional .build(); try { // Test connection if (!client.testConnection()) { System.err.println("Cannot connect to DataHub"); return; } // Build dataset Dataset dataset = Dataset.builder() .platform("snowflake") .name("analytics.public.user_events") .env("PROD") .description("User interaction events") .displayName("User Events") .build(); // Add metadata dataset.addTag("pii") .addTag("analytics") .addOwner("urn:li:corpuser:datateam", OwnershipType.TECHNICAL_OWNER) .addCustomProperty("retention_days", "90"); // Upsert to DataHub client.entities().upsert(dataset); System.out.println("Created dataset: " + dataset.getUrn()); } catch (IOException | ExecutionException | InterruptedException e) { e.printStackTrace(); } finally { try { client.close(); } catch (IOException e) { e.printStackTrace(); } } } } ``` -------------------------------- ### Install and Run Analytics Agent CLI Source: https://github.com/datahub-project/datahub/blob/master/docs/features/feature-guides/analytics-agent.md Install the agent using pip or uv, then run the quickstart command to launch the server and open the setup wizard. Configuration is stored in ~/.datahub/analytics-agent/. Rerun quickstart to relaunch or use --reconfigure to reopen the wizard. ```bash pip install datahub-analytics-agent analytics-agent quickstart ``` ```bash uvx datahub-analytics-agent quickstart ``` -------------------------------- ### Quick Start Examples Source: https://github.com/datahub-project/datahub/blob/master/docs/cli-commands/graphql.md Examples demonstrating how to get user info, search for datasets, and execute raw GraphQL queries. ```shell # Get current user info datahub graphql --operation me # Search for datasets datahub graphql --operation searchAcrossEntities --variables '{"input": {"query": "users", "types": ["DATASET"]}}' # Execute raw GraphQL datahub graphql --query "query { me { username } }" ``` -------------------------------- ### Install DataHub CLI using pip Source: https://github.com/datahub-project/datahub/blob/master/docs/quickstart.md Installs the DataHub CLI using pip and verifies the installation. ```bash python3 -m pip install --upgrade pip wheel setuptools python3 -m pip install --upgrade acryl-datahub datahub version ``` -------------------------------- ### Install DataHub CLI using Homebrew Source: https://github.com/datahub-project/datahub/blob/master/docs/quickstart.md Installs the DataHub CLI using Homebrew and verifies the installation. Also shows how to install ingestion connectors. ```bash brew install datahub-project/tap/datahub datahub version ``` ```bash "$(brew --prefix datahub)/libexec/bin/pip" install 'acryl-datahub[snowflake,bigquery]' ``` -------------------------------- ### Start DataHub Quickstart with OAuth Overlay Source: https://github.com/datahub-project/datahub/blob/master/smoke-test/tests/oauth/README.md Starts the standard DataHub quickstart along with the Keycloak overlay for OAuth testing. Ensure to replace `` with your actual quickstart compose file path. ```bash docker compose -f -f smoke-test/tests/oauth/docker-compose.oauth.yml up -d ``` -------------------------------- ### Client Configuration Example Source: https://github.com/datahub-project/datahub/blob/master/metadata-integration/java/docs/sdk-v2/getting-started.md Example of configuring the DataHubClientV2 with server, token, and REST emitter settings. ```java DataHubClientV2 client = DataHubClientV2.builder() .server("https://your-instance.acryl.io") .token("your-access-token") // Configure operation mode .operationMode(DataHubClientConfigV2.OperationMode.SDK) // or INGESTION // Customize underlying REST emitter .restEmitterConfig(config -> config .timeoutSec(30) .maxRetries(5) .retryIntervalSec(2) ) .build(); ``` -------------------------------- ### Operation Modes Example Source: https://github.com/datahub-project/datahub/blob/master/metadata-integration/java/docs/sdk-v2/getting-started.md Examples demonstrating how to configure the client for SDK mode and INGESTION mode. ```java // SDK mode (default) - interactive use DataHubClientV2 sdkClient = DataHubClientV2.builder() .server("http://localhost:8080") .operationMode(DataHubClientConfigV2.OperationMode.SDK) .build(); // Ingestion mode - ETL pipelines DataHubClientV2 ingestionClient = DataHubClientV2.builder() .server("http://localhost:8080") .operationMode(DataHubClientConfigV2.OperationMode.INGESTION) .build(); ``` -------------------------------- ### Customize Installation Source: https://github.com/datahub-project/datahub/blob/master/docs/quickstart.md Command to customize DataHub installation using a modified docker-compose file. ```bash datahub docker quickstart --quickstart-compose-file ``` -------------------------------- ### Quickstart recipe Source: https://github.com/datahub-project/datahub/blob/master/metadata-ingestion/sink_docs/console.md A basic recipe to get started with the console sink. ```yaml source: # source configs sink: type: "console" ``` -------------------------------- ### Quick Start Example Source: https://github.com/datahub-project/datahub/blob/master/metadata-integration/java/as-a-library-v2.md A complete example of creating a dataset with metadata using SDK V2. ```java import datahub.client.v2.DataHubClientV2; import datahub.client.v2.entity.Dataset; import com.linkedin.common.OwnershipType; // Create the client DataHubClientV2 client = DataHubClientV2.builder() .server("http://localhost:8080") .token("your-access-token") // Optional for authentication .build(); // Build a dataset with metadata Dataset dataset = Dataset.builder() .platform("snowflake") .name("analytics.public.user_events") .env("PROD") .description("User interaction events") .displayName("User Events") .build(); // Add tags and owners dataset.addTag("pii") .addTag("analytics") .addOwner("urn:li:corpuser:datateam", OwnershipType.TECHNICAL_OWNER) .addCustomProperty("retention", "90_days"); // Upsert to DataHub client.entities().upsert(dataset); System.out.println("Created dataset: " + dataset.getUrn()); // Close the client when done client.close(); ``` -------------------------------- ### Install DataHub Actions Source: https://github.com/datahub-project/datahub/blob/master/docs/actions/quickstart.md Installs the `acryl-datahub-actions` package from PyPi and verifies the installation. ```shell python3 -m pip install --upgrade pip wheel setuptools python3 -m pip install --upgrade acryl-datahub-actions # Verify the installation by checking the version. datahub actions version ``` -------------------------------- ### Update Backend Models - Build Source: https://github.com/datahub-project/datahub/wiki/Quick-Start-with-VM Gradle commands to build the backend models. ```bash cd backend-service; gradle build; gradle dist ``` -------------------------------- ### Batch Get Response Example Source: https://github.com/datahub-project/datahub/blob/master/docs/api/openapi/openapi-usage-guide.md Example response for a batch GET request, showing the previous version of the globalTags aspect. ```json [ { "urn": "urn:li:dataset:(urn:li:dataPlatform:hive,fct_users_created,PROD)", "globalTags": { "value": { "tags": [ { "tag": "urn:li:tag:NeedsDocumentation" } ] }, "systemMetadata": { "properties": { "appSource": "ui" }, "version": "1", "lastObserved": 0, "lastRunId": "no-run-id-provided", "runId": "no-run-id-provided" } } } ] ``` -------------------------------- ### Update Frontend Models - Build Source: https://github.com/datahub-project/datahub/wiki/Quick-Start-with-VM Gradle commands to build the frontend models. ```bash cd web; gradle build; gradle dist ``` -------------------------------- ### Batch Get Request Example Source: https://github.com/datahub-project/datahub/blob/master/docs/api/openapi/openapi-usage-guide.md Example of a batch GET request to retrieve a specific version of the globalTags aspect for a dataset. ```json [ { "urn": "urn:li:dataset:(urn:li:dataPlatform:hive,fct_users_created,PROD)", "globalTags": { "headers": { "If-Version-Match": "1" } } } ] ``` -------------------------------- ### Clone and Run Analytics Agent Quickstart Source: https://github.com/datahub-project/datahub/blob/master/README.md Use this bash script to quickly set up the open-source analytics agent. It clones the repository and executes a setup script. ```bash git clone https://github.com/datahub-project/analytics-agent.git cd analytics-agent && bash quickstart.sh ``` -------------------------------- ### Quickstart: Set up Python environment for metadata-ingestion Source: https://github.com/datahub-project/datahub/blob/master/smoke-test/README.md Commands to set up the metadata-ingestion Python virtual environment and install dependencies. ```bash # From project root - sets up metadata-ingestion venv ./gradlew :metadata-ingestion:installDev # Set up smoke-test specific environment cd smoke-test python3 -m venv venv source venv/bin/activate pip install --upgrade pip wheel setuptools pip install -r requirements.txt ``` -------------------------------- ### Batch Get Response Example (Initial) Source: https://github.com/datahub-project/datahub/blob/master/docs/api/openapi/openapi-usage-guide.md Example JSON response after a batch get request, showing systemMetadata with version '1' for existing aspects. ```json [ { "urn": "urn:li:dataset:(urn:li:dataPlatform:hive,fct_users_deleted,PROD)", "datasetProperties": { "value": { "description": "table containing all the users deleted on a single day", "customProperties": { "encoding": "utf-8" }, "tags": [] }, "systemMetadata": { "properties": { "clientVersion": "1!0.0.0.dev0", "clientId": "acryl-datahub" }, "version": "1", "lastObserved": 1720781548776, "lastRunId": "file-2024_07_12-05_52_28", "runId": "file-2024_07_12-05_52_28" } } }, { "urn": "urn:li:dataset:(urn:li:dataPlatform:hive,fct_users_created,PROD)", "datasetProperties": { "value": { "description": "table containing all the users created on a single day", "customProperties": { "encoding": "utf-8" }, "tags": [] }, "systemMetadata": { "properties": { "clientVersion": "1!0.0.0.dev0", "clientId": "acryl-datahub" }, "version": "1", "lastObserved": 1720781548773, "lastRunId": "file-2024_07_12-05_52_28", "runId": "file-2024_07_12-05_52_28" } }, "globalTags": { "value": { "tags": [ { "tag": "urn:li:tag:NeedsDocumentation" } ] }, "systemMetadata": { "properties": { "appSource": "ui" }, "version": "1", "lastObserved": 0, "lastRunId": "no-run-id-provided", "runId": "no-run-id-provided" } } } ] ``` -------------------------------- ### Chart Entity Example Source: https://github.com/datahub-project/datahub/blob/master/metadata-integration/java/docs/sdk-v2/getting-started.md Example of creating and upserting a Chart entity. ```java import datahub.client.v2.entity.Chart; Chart chart = Chart.builder() .tool("looker") .id("my_sales_chart") .title("Sales Performance by Region") .description("Monthly sales broken down by geographic region") .build(); client.entities().upsert(chart); ``` -------------------------------- ### Load Sample Data Source: https://github.com/datahub-project/datahub/blob/master/docs/quickstart.md Command to load the showcase-ecommerce data pack. ```bash datahub datapack load showcase-ecommerce ``` -------------------------------- ### Install DataHub Cloud SDK Source: https://github.com/datahub-project/datahub/blob/master/docs/api/tutorials/assertions.md Install the DataHub Cloud SDK extension if you plan to use the Python examples in this guide. ```bash pip install acryl-datahub-cloud ``` -------------------------------- ### Error Handling Example Source: https://github.com/datahub-project/datahub/blob/master/metadata-integration/java/docs/sdk-v2/getting-started.md Example of handling potential exceptions during client operations. ```java try { client.entities().upsert(dataset); } catch (IOException e) { // Network or serialization errors System.err.println("I/O error: " + e.getMessage()); } catch (ExecutionException e) { // Server-side errors System.err.println("Server error: " + e.getCause().getMessage()); } catch (InterruptedException e) { // Operation cancelled Thread.currentThread().interrupt(); } ``` -------------------------------- ### Setup and Activate Virtual Environment Source: https://github.com/datahub-project/datahub/blob/master/perf-test/authz-perf/README.md Run the setup script to create an isolated virtual environment and install dependencies. Activate the environment to use the harness. ```bash perf-test/authz-perf/setup.sh source perf-test/authz-perf/venv/bin/activate ``` -------------------------------- ### Builder-created entities Source: https://github.com/datahub-project/datahub/blob/master/metadata-integration/java/docs/sdk-v2/getting-started.md Example showing that entities created using a builder are mutable from the start. ```java Dataset dataset = Dataset.builder() .platform("snowflake") .name("my_table") .build(); dataset.isMutable(); // true - can mutate immediately dataset.addTag("test"); // Works without .mutable() ``` -------------------------------- ### Update Frontend Models - Copy to VM Source: https://github.com/datahub-project/datahub/wiki/Quick-Start-with-VM SCP command to copy the built frontend service zip file to the VM. ```bash scp target/universal/wherehows-1.0-SNAPSHOT.zip cloudera@$VM_HOST:~/wherehows/ ``` -------------------------------- ### Start WhereHows Backend Service Source: https://github.com/datahub-project/datahub/wiki/Quick-Start-with-VM Commands to navigate to the WhereHows directory and start the backend service. ```bash cd ~/wherehows; ./runbackend; ``` -------------------------------- ### GraphQL Error Response Example Source: https://github.com/datahub-project/datahub/blob/master/docs/api/graphql/getting-started.md Example JSON structure for a GraphQL error response, indicating a failed mutation. ```json { "errors": [ { "message": "Failed to change ownership for resource urn:li:dataFlow:(airflow,dag_abc,PROD). Expected a corp user urn.", "locations": [ { "line": 1, "column": 22 } ], "path": ["addOwners"], "extensions": { "code": 400, "type": "BAD_REQUEST", "classification": "DataFetchingException" } } ] } ``` -------------------------------- ### Update Backend Models - Copy to VM Source: https://github.com/datahub-project/datahub/wiki/Quick-Start-with-VM SCP command to copy the built backend service zip file to the VM. ```bash scp target/universal/backend-service-1.0-SNAPSHOT.zip cloudera@$VM_HOST:~/wherehows/ ``` -------------------------------- ### Build and start DataHub components with Cassandra Source: https://github.com/datahub-project/datahub/blob/master/docker/cassandra/README.md Build and start all DataHub components with Cassandra backend using the provided script. ```bash ./docker/dev-with-cassandra.sh ``` -------------------------------- ### Environment Variable Example Source: https://github.com/datahub-project/datahub/blob/master/metadata-ingestion/CLAUDE.md Example of setting the DATAHUB_VENV_USE_COPIES environment variable and running the Gradle install command. ```bash export DATAHUB_VENV_USE_COPIES=true ../gradlew :metadata-ingestion:installDev ``` -------------------------------- ### Run DataHub Quickstart with Gradle Source: https://github.com/datahub-project/datahub/blob/master/docker/profiles/README.md Execute DataHub quickstart using the 'quickstart' profile via Gradle tasks. Run from the project root. ```bash # Run from the project root ./gradlew quickstart # Uses the 'quickstart' profile ./gradlew quickstartDebug # Uses the 'debug' profile ./gradlew quickstartCypress # Uses the 'debug' profile with custom project name 'dh-cypress' ``` -------------------------------- ### Starting the Application - Quick Start Source: https://github.com/datahub-project/datahub/blob/master/datahub-web-react/README.md Commands to run the application locally for development. ```bash yarn install && yarn run start ``` -------------------------------- ### Update Entity Source: https://github.com/datahub-project/datahub/blob/master/metadata-integration/java/docs/sdk-v2/getting-started.md Example of incrementally updating an existing entity. ```java client.entities().update(dataset); ``` -------------------------------- ### Update Backend Models - Unzip and Restart Source: https://github.com/datahub-project/datahub/wiki/Quick-Start-with-VM Commands to update the backend service on the VM and restart it. ```bash cd wherehows; rm -rf backend-service-1.0-SNAPSHOT; unzip backend-service-1.0-SNAPSHOT.zip kill -9 $current_wherehows_backend_pid; ./runbackend ``` -------------------------------- ### Example Event Logged by Hello World Action Source: https://github.com/datahub-project/datahub/blob/master/docs/actions/quickstart.md An example of an event logged to the console by the 'Hello World' Action when a 'pii' tag is added to a Dataset. ```json Hello world! Received event: { "event_type": "EntityChangeEvent_v1", "event": { "entityType": "dataset", "entityUrn": "urn:li:dataset:(urn:li:dataPlatform:hdfs,SampleHdfsDataset,PROD)", "category": "TAG", "operation": "ADD", "modifier": "urn:li:tag:pii", "parameters": {}, "auditStamp": { "time": 1651082697703, "actor": "urn:li:corpuser:datahub", "impersonator": null }, "version": 0, "source": null }, "meta": { "kafka": { "topic": "PlatformEvent_v1", "offset": 1262, "partition": 0 } } } ``` -------------------------------- ### View Backend Log Source: https://github.com/datahub-project/datahub/wiki/Quick-Start-with-VM Command to tail the WhereHows backend log file. ```bash tail -f /var/tmp/wherehows/wherehows.log ``` -------------------------------- ### Get Entities Source: https://github.com/datahub-project/datahub/blob/master/docs/api/openapi/openapi-usage-guide.md Example of a Postman request to retrieve entities with specific aspect names. ```json { "name": "get Entities", "protocolProfileBehavior": { "disableUrlEncoding": false }, "request": { "method": "GET", "header": [ { "key": "Accept", "value": "application/json" } ], "url": { "raw": "{{baseUrl}}/openapi/entities/v1/latest?urns=urn:li:dataset:(urn:li:dataPlatform:platform,testSchemaIngest,PROD)&aspectNames=schemaMetadata", "host": ["{{baseUrl}}"], "path": ["openapi", "entities", "v1", "latest"], "query": [ { "key": "urns", "value": "urn:li:dataset:(urn:li:dataPlatform:platform,testSchemaIngest,PROD)", "description": "(Required) A list of raw urn strings, only supports a single entity type per request." }, { "key": "urns", "value": "labore dolor exercitation in", "description": "(Required) A list of raw urn strings, only supports a single entity type per request.", "disabled": true }, { "key": "aspectNames", "value": "schemaMetadata", "description": "The list of aspect names to retrieve" }, { "key": "aspectNames", "value": "labore dolor exercitation in", "description": "The list of aspect names to retrieve", "disabled": true } ] } }, "response": [ { "name": "OK", "originalRequest": { "method": "GET", "header": [], "url": { "raw": "{{baseUrl}}/entities/v1/latest?urns=urn:li:dataset:(urn:li:dataPlatform:platform,testSchemaIngest,PROD)&aspectNames=schemaMetadata", ``` -------------------------------- ### Run Hello World Action Source: https://github.com/datahub-project/datahub/blob/master/docs/actions/quickstart.md Command to run the 'Hello World' Action using its configuration file. ```shell datahub actions -c hello_world.yaml ``` -------------------------------- ### Update Dashboard Entity Source: https://github.com/datahub-project/datahub/blob/master/docs/api/graphql/getting-started.md Example GraphQL mutation to update the description of a Dashboard entity. ```json mutation updateDashboard { updateDashboard( urn: "urn:li:dashboard:(looker,baz)", input: { editableProperties: { description: "My new description" } } ) { urn } } ``` -------------------------------- ### Batch Get Response Example (After Mutation) Source: https://github.com/datahub-project/datahub/blob/master/docs/api/openapi/openapi-usage-guide.md Example JSON response after mutating globalTags for one aspect, showing an incremented version ('2') and an added tag. ```json [ { "urn": "urn:li:dataset:(urn:li:dataPlatform:hive,fct_users_deleted,PROD)", "datasetProperties": { "value": { "description": "table containing all the users deleted on a single day", "customProperties": { "encoding": "utf-8" }, "tags": [] }, "systemMetadata": { "properties": { "clientVersion": "1!0.0.0.dev0", "clientId": "acryl-datahub" }, "version": "1", "lastObserved": 1720781548776, "lastRunId": "file-2024_07_12-05_52_28", "runId": "file-2024_07_12-05_52_28" } } }, { "urn": "urn:li:dataset:(urn:li:dataPlatform:hive,fct_users_created,PROD)", "datasetProperties": { "value": { "description": "table containing all the users created on a single day", "customProperties": { "encoding": "utf-8" }, "tags": [] }, "systemMetadata": { "properties": { "clientVersion": "1!0.0.0.dev0", "clientId": "acryl-datahub" }, "version": "1", "lastObserved": 1720781548773, "lastRunId": "file-2024_07_12-05_52_28", "runId": "file-2024_07_12-05_52_28" } }, "globalTags": { "value": { "tags": [ { "tag": "urn:li:tag:NeedsDocumentation" }, { "tag": "urn:li:tag:Legacy" } ] }, "systemMetadata": { "properties": { "appSource": "ui" }, "version": "2", "lastObserved": 0, "lastRunId": "no-run-id-provided", "runId": "no-run-id-provided" } } } ] ``` -------------------------------- ### Run Play backend server Source: https://github.com/datahub-project/datahub/wiki/Getting-Started Command to run the Play backend server. ```bash cd backend-service $ACTIVATOR_HOME/bin/activator run ``` -------------------------------- ### Quickstart: Running All Tests Source: https://github.com/datahub-project/datahub/blob/master/smoke-test/README.md Command to run all smoke tests. This is a time-consuming process and requires a full setup. ```bash cd smoke-test source venv/bin/activate # Set environment variables export DATAHUB_VERSION=v1.0.0rc3-SNAPSHOT export TEST_STRATEGY=no_cypress_suite0 # Run all tests (WARNING: Takes a long time, requires full setup) pytest -vv ``` -------------------------------- ### Batch Get Request Example Source: https://github.com/datahub-project/datahub/blob/master/docs/api/openapi/openapi-usage-guide.md Example JSON request body for fetching the latest aspects for given URNs, with systemMetadata=true to view current aspect versions. ```json [ { "urn": "urn:li:dataset:(urn:li:dataPlatform:hive,fct_users_deleted,PROD)", "globalTags": {}, "datasetProperties": {} }, { "urn": "urn:li:dataset:(urn:li:dataPlatform:hive,fct_users_created,PROD)", "globalTags": {}, "datasetProperties": {} } ] ``` -------------------------------- ### Update Frontend Models - Unzip and Restart Source: https://github.com/datahub-project/datahub/wiki/Quick-Start-with-VM Commands to update the frontend service on the VM and restart it. ```bash cd wherehows; rm -rf wherehows-1.0-SNAPSHOT; unzip wherehows-1.0-SNAPSHOT.zip kill -9 $current_wherehows_frontend_pid; ./runfrontend ``` -------------------------------- ### User metadata ingestion example Source: https://github.com/datahub-project/datahub/blob/master/docs/troubleshooting/quickstart.md JSON structure for ingesting custom user information into DataHub. ```json { "auditHeader": null, "proposedSnapshot": { "com.linkedin.pegasus2avro.metadata.snapshot.CorpUserSnapshot": { "urn": "urn:li:corpuser:my-custom-user", "aspects": [ { "com.linkedin.pegasus2avro.identity.CorpUserInfo": { "active": true, "displayName": { "string": "The name of the custom user" }, "email": "my-custom-user-email@example.io", "title": { "string": "Engineer" }, "managerUrn": null, "departmentId": null, "departmentName": null, "firstName": null, "lastName": null, "fullName": { "string": "My Custom User" }, "countryCode": null } } ] } }, "proposedDelta": null } ``` -------------------------------- ### Quickstart (local instance) Source: https://github.com/datahub-project/datahub/blob/master/metadata-ingestion/src/datahub/cli/resources/INIT_AGENT_CONTEXT.md Initializes the DataHub CLI with default credentials for a local instance. ```bash # Default credentials on localhost — no --host, no --force needed datahub init --username datahub --password datahub ``` -------------------------------- ### Check DataHub Frontend container logs Source: https://github.com/datahub-project/datahub/blob/master/docs/troubleshooting/quickstart.md Example log output for the datahub-frontend-react container after initialization. ```text 09:20:22 [main] INFO play.core.server.AkkaHttpServer - Listening for HTTP on /0.0.0.0:9002 ``` -------------------------------- ### Quickstart: Running a Specific Test File Source: https://github.com/datahub-project/datahub/blob/master/smoke-test/README.md Command to run tests from a specific file, recommended for development. ```bash pytest test_system_info.py -vv ``` -------------------------------- ### Check DataHub GMS container logs Source: https://github.com/datahub-project/datahub/blob/master/docs/troubleshooting/quickstart.md Example log output for the datahub-gms container after initialization. ```text 2020-02-06 09:20:54.870:INFO:oejs.Server:main: Started @18807ms ``` -------------------------------- ### Resource Management - Explicit Close Source: https://github.com/datahub-project/datahub/blob/master/metadata-integration/java/docs/sdk-v2/getting-started.md Example of explicitly closing the client in a finally block. ```java try { // Use client } finally { client.close(); } ``` -------------------------------- ### Clone Repository and Run Docker Quickstart Source: https://github.com/datahub-project/datahub/blob/master/docs/features/feature-guides/analytics-agent.md Clone the analytics-agent repository and execute the bash script to set up a local DataHub instance with sample data and launch the agent. This process may take 15-25 minutes on the first run due to Docker image downloads. ```bash git clone https://github.com/datahub-project/analytics-agent.git cd analytics-agent bash quickstart.sh ``` -------------------------------- ### Resource Management - Try-with-Resources Source: https://github.com/datahub-project/datahub/blob/master/metadata-integration/java/docs/sdk-v2/getting-started.md Example of automatically closing the client using a try-with-resources statement. ```java try (DataHubClientV2 client = DataHubClientV2.builder() .server("http://localhost:8080") .build()) { // Use client here client.entities().upsert(dataset); } // Client automatically closed ``` -------------------------------- ### Reading Entities Source: https://github.com/datahub-project/datahub/blob/master/metadata-integration/java/docs/sdk-v2/getting-started.md Example of how to load an existing dataset entity from DataHub using its URN. ```java import com.linkedin.common.urn.DatasetUrn; DatasetUrn urn = new DatasetUrn( "snowflake", "analytics.public.user_events", "PROD" ); try { Dataset loaded = client.entities().get(urn); if (loaded != null) { System.out.println("Dataset description: " + loaded.getDescription()); System.out.println("Is read-only: " + loaded.isReadOnly()); // true } } catch (IOException | ExecutionException | InterruptedException e) { e.printStackTrace(); } ``` -------------------------------- ### Sample Request for Relationships Endpoint Source: https://github.com/datahub-project/datahub/blob/master/docs/api/openapi/openapi-usage-guide.md Example of a GET request to the /relationships endpoint to retrieve relationship information. ```shell curl -X 'GET' \ 'http://localhost:8080/openapi/relationships/v1/?urn=urn%3Ali%3Acorpuser%3Adatahub&relationshipTypes=IsPartOf&direction=INCOMING&start=0&count=200' \ -H 'accept: application/json' ``` -------------------------------- ### Setup for Package Installation Source: https://github.com/datahub-project/datahub/blob/master/docs/actions/guides/developing-an-action.md Creates a setup.py file to package the custom action, making it installable via pip. ```python from setuptools import find_packages, setup setup( name="custom_action_example", version="1.0", packages=find_packages(), # if you don't already have DataHub Actions installed, add it under install_requires # install_requires=["acryl-datahub-actions"] ) ``` -------------------------------- ### Run Play web application on a specific port Source: https://github.com/datahub-project/datahub/wiki/Getting-Started Example of running the Play web application on a specific port. ```bash activator run -Dhttp.port=9008 ```