### Run OpenMetadata with Docker Source: https://github.com/open-metadata/openmetadatastandards/blob/main/README.md Use this command to quickly start an OpenMetadata server instance using Docker. Ensure Docker is installed and running on your system. ```bash # Try OpenMetadata with Docker docker run -d -p 8585:8585 \ --name openmetadata \ openmetadata/server:latest ``` -------------------------------- ### Validate Examples with npm Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/developer/testing.md Execute this command to validate example files. This helps ensure that examples are consistent with the schemas and documentation. ```bash npm run validate:examples ``` -------------------------------- ### Complete Example Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/data-assets/apis/api-collection.md A comprehensive example demonstrating the structure of an OpenMetadata entity with versioning properties. ```APIDOC ## Complete Example This example shows a fully populated OpenMetadata entity including all versioning-related fields. ```json { "id": "2b3c4d5e-6f7a-4b8c-9d0e-1f2a3b4c5d6e", "name": "payments_api", "fullyQualifiedName": "production_api_gateway.payments_api", "displayName": "Payments API", "description": "# Payments API\n\nREST API for payment processing and transaction management.", "endpointURL": "https://api.example.com/v2/payments", "apiEndpoints": [ { "id": "endpoint-uuid-1", "type": "apiEndpoint", "name": "createPayment", "fullyQualifiedName": "production_api_gateway.payments_api.createPayment" }, { "id": "endpoint-uuid-2", "type": "apiEndpoint", "name": "getPayment", "fullyQualifiedName": "production_api_gateway.payments_api.getPayment" } ], "href": "https://platform.open-metadata.org/api/v1/apiCollections/2b3c4d5e-6f7a-4b8c-9d0e-1f2a3b4c5d6e", "service": { "id": "service-uuid", "type": "apiService", "name": "production_api_gateway" }, "serviceType": "REST", "owners": [ { "id": "team-uuid", "type": "team", "name": "payments", "displayName": "Payments Team" } ], "domains": [ { "id": "domain-uuid", "type": "domain", "name": "Finance" } ], "dataProducts": [ { "id": "product-uuid", "type": "dataProduct", "name": "customer_360" } ], "tags": [ {"tagFQN": "Tier.Gold"}, {"tagFQN": "Compliance.PCI-DSS"} ], "votes": { "upVotes": 15, "downVotes": 2 }, "lifeCycle": { "stage": "Production" }, "certification": { "tagLabel": { "tagFQN": "Certified.Gold" } }, "version": 3.1, "updatedAt": 1704240000000, "updatedBy": "john.developer", "deleted": false, "extension": { "cost_center": "CC-001", "sla": "99.9%" }, "sourceHash": "abc123def456", "entityStatus": "Active" } ``` ``` -------------------------------- ### Data Asset Hierarchy Example Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/data-assets/overview.md Illustrates the hierarchical organization of data assets, starting from a Service and branching into Containers, Assets, and Sub-Assets. ```mermaid graph TD A[Service] --> B1[Container 1] A[Service] --> B2[Container 2] B1 --> C1[Asset 1] B1 --> C2[Asset 2] B2 --> C3[Asset 3] C1 --> D1[Sub-Asset 1] C1 --> D2[Sub-Asset 2] C3 --> D3[Sub-Asset 3] style A fill:#667eea,color:#fff style B1 fill:#764ba2,color:#fff style B2 fill:#764ba2,color:#fff style C1 fill:#f093fb,color:#fff style C2 fill:#f093fb,color:#fff style C3 fill:#f093fb,color:#fff style D1 fill:#4facfe,color:#fff style D2 fill:#4facfe,color:#fff style D3 fill:#4facfe,color:#fff ``` -------------------------------- ### User Example Instance Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/teams-users/user.md An example instance of a User entity with populated properties. ```APIDOC ## User Example Instance ### Description An example of a User instance demonstrating the usage of various properties. ### Instance ``` ex:janeDoe a om:User ; om:userName "jane.doe" ; om:email "jane.doe@example.com" ; om:displayName "Jane Doe" ; om:isAdmin false ; om:isBot false ; om:isEmailVerified true ; om:timezone "America/Los_Angeles" ; om:memberOf ex:dataEngineeringTeam ; om:hasRole ex:dataEngineerRole ; om:hasPersona ex:engineerPersona ; om:hasDefaultPersona ex:engineerPersona ; om:belongsToDomain ex:salesDomain ; om:owns ex:customersTable ; om:follows ex:ordersTable . ``` ``` -------------------------------- ### Complete Example Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/data-assets/databases/query.md A comprehensive example of an OpenMetadata entity, showcasing various fields and their potential values. ```APIDOC ## Complete Example ```json { "id": "6f7a8b9c-0d1e-2f3a-4b5c-6d7e8f9a0b1c", "name": "monthly_sales_report", "fullyQualifiedName": "postgres_prod.analytics_db.monthly_sales_report", "displayName": "Monthly Sales Report", "description": "# Monthly Sales Report\n\nGenerates monthly sales summary by product category.", "version": 1.3, "updatedAt": 1704240000000, "updatedBy": "analyst.team", "href": "https://example.com/api/v1/queries/6f7a8b9c-0d1e-2f3a-4b5c-6d7e8f9a0b1c", "owners": [ { "id": "owner-uuid", "type": "team", "name": "analytics-team" } ], "duration": 2345.67, "users": [ { "id": "user-uuid", "type": "user", "name": "analyst.team" } ], "followers": [ { "id": "follower-uuid", "type": "user", "name": "data.analyst" } ], "votes": { "upVotes": 15, "downVotes": 2 }, "query": "SELECT\n DATE_TRUNC('month', order_date) AS month,\n product_category,\n SUM(order_amount) AS total_revenue,\n COUNT(DISTINCT customer_id) AS customer_count\nFROM orders\nWHERE order_date >= :start_date\n AND order_date < :end_date\nGROUP BY 1, 2\nORDER BY 1 DESC, 3 DESC", "query_type": "SELECT", "exclude_usage": false, "checksum": "md5:a1b2c3d4e5f6g7h8i9j0k1l2m3n4o5p6", "queryDate": 1705315800000, "usedBy": ["external_user_1", "external_user_2"], "tags": [ {"tagFQN": "Tier.Silver"}, {"tagFQN": "Reporting"} ], "queryUsedIn": [ { "id": "dashboard-uuid", "type": "dashboard", "name": "sales_dashboard" } ], "triggeredBy": { "id": "procedure-uuid", "type": "storedProcedure", "name": "generate_sales_report" }, "processedLineage": true, "service": { "id": "service-uuid", "type": "databaseService", "name": "postgres_prod" }, "domains": [ { "id": "domain-uuid", "type": "domain", "name": "Sales" } ], "entityStatus": { "status": "Active" } } ``` ``` -------------------------------- ### Test Connection Result Example Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/data-assets/databases/database-service.md Example showing the status and steps of a test connection for a service. ```json { "testConnectionResult": { "status": "Successful", "lastUpdatedAt": 1704240000000, "steps": [ { "name": "CheckAccess", "mandatory": true, "passed": true }, { "name": "GetDatabases", "mandatory": true, "passed": true } ] } } ``` -------------------------------- ### Example JSON-LD Application Instance Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/operations/applications/application.md A complete example of an OpenMetadata application defined in JSON-LD format. ```json { "@context": "https://open-metadata.org/context/app.jsonld", "@type": "App", "@id": "https://open-metadata.org/apps/DataQualityApp", "name": "DataQualityApp", "fullyQualifiedName": "DataQualityApp", "displayName": "Data Quality Application", "description": "Automated data quality checks and profiling for all data assets", "features": "- Data profiling\n- Quality metrics\n- Anomaly detection\n- Automated testing", "appType": "internal", "scheduleType": "Scheduled", "className": "org.openmetadata.service.apps.bundles.quality.DataQualityApp", "developer": "OpenMetadata", "developerUrl": "https://open-metadata.org", "privacyPolicyUrl": "https://open-metadata.org/privacy", "supportEmail": "support@open-metadata.org", "appLogoUrl": "https://open-metadata.org/images/apps/data-quality.png", "allowConfiguration": true, "system": false, "preview": false, "supportsInterrupt": true, "supportsIngestionRunner": false, "allowConcurrentExecution": false, "owners": [ { "@id": "https://open-metadata.org/teams/data-governance", "@type": "Team", "name": "DataGovernanceTeam" } ], "domains": [ { "@id": "https://open-metadata.org/domains/data-quality", "@type": "Domain", "name": "DataQuality" } ], "bot": { "@id": "https://open-metadata.org/bots/data-quality-bot", "@type": "Bot", "name": "data-quality-bot" }, "pipelines": [ { "@id": "https://open-metadata.org/pipelines/quality-checks", "@type": "Pipeline", "name": "quality_checks_pipeline" } ], "eventSubscriptions": [ { "@id": "https://open-metadata.org/events/entity-created", "@type": "EventSubscription", "name": "entityCreated" } ], "runtime": { "@type": "ScheduledExecutionContext", "scheduleTimeline": "Daily" }, "appSchedule": { "scheduleTimeline": "Daily", "cronExpression": "0 2 * * *" } } ``` -------------------------------- ### Chart Service Reference Example Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/data-assets/dashboards/chart.md Example of a reference to the service where the dashboard is hosted. ```json { "service": { "id": "service-uuid", "type": "dashboardService", "name": "tableau_prod", "fullyQualifiedName": "tableau_prod" } } ``` -------------------------------- ### JSON-LD User Instance Example Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/teams-users/user.md A complete example of a User entity represented in JSON-LD format. ```json { "@context": "https://open-metadata.org/context/user.jsonld", "@type": "User", "@id": "https://example.com/users/jane.doe", "name": "jane.doe", "fullyQualifiedName": "jane.doe", "displayName": "Jane Doe", "email": "jane.doe@example.com", "description": "Senior Data Engineer specializing in data platform and analytics", "timezone": "America/Los_Angeles", "isAdmin": false, "isBot": false, "isEmailVerified": true, "profile": { "images": { "image": "https://example.com/avatars/jane.doe.jpg", "image192": "https://example.com/avatars/jane.doe_192.jpg" } }, "teams": [ { "@id": "https://example.com/teams/data-engineering", "@type": "Team", "name": "Data Engineering" } ], "roles": [ { "@id": "https://example.com/roles/data-engineer", "@type": "Role", "name": "DataEngineer" } ], "personas": [ { "@id": "https://example.com/personas/engineer", "@type": "Persona", "name": "Engineer" } ], "defaultPersona": { "@id": "https://example.com/personas/engineer", "@type": "Persona", "name": "Engineer" }, "domains": [ { "@id": "https://example.com/domains/sales", "@type": "Domain", "name": "Sales" } ], "owns": [ { "@id": "https://example.com/tables/customers", "@type": "Table", "fullyQualifiedName": "postgres_prod.ecommerce.public.customers" } ], "follows": [ { "@id": "https://example.com/tables/orders", "@type": "Table", "fullyQualifiedName": "postgres_prod.ecommerce.public.orders" } ] } ``` -------------------------------- ### DatabaseService Core Property Examples Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/data-assets/databases/database-service.md Examples of individual core properties for the DatabaseService entity. ```json { "id": "a1b2c3d4-e5f6-4a7b-8c9d-0e1f2a3b4c5d" } ``` ```json { "name": "postgres_prod" } ``` ```json { "fullyQualifiedName": "postgres_prod" } ``` ```json { "displayName": "Production PostgreSQL" } ``` -------------------------------- ### Role Entity Examples Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/teams-users/role.md Examples of complete Role entity objects and common role configurations. ```APIDOC ## Role Entity Examples ### Complete Role Example ```json { "id": "c3d4e5f6-a7b8-4c9d-0e1f-2a3b4c5d6e7f", "name": "DataEngineer", "fullyQualifiedName": "DataEngineer", "displayName": "Data Engineer", "description": "# Data Engineer Role\n\nProvides permissions for data engineers to manage data pipelines, tables, and infrastructure.", "version": 1.2, "updatedAt": 1704240000000, "updatedBy": "admin", "href": "http://localhost:8585/api/v1/roles/c3d4e5f6-a7b8-4c9d-0e1f-2a3b4c5d6e7f", "allowDelete": false, "allowEdit": false, "deleted": false, "disabled": false, "policies": [ { "id": "policy-uuid-1", "type": "policy", "name": "DataAccessPolicy" }, { "id": "policy-uuid-2", "type": "policy", "name": "PipelineManagementPolicy" } ] } ``` ### Common Roles - **Admin**: Full system administration access. - **Data Steward**: Data governance and quality management. - **Data Consumer**: Read-only access to data assets. ``` -------------------------------- ### Get MessagingService API Request Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/data-assets/messaging/messaging-service.md Example HTTP GET request to retrieve a MessagingService by its name. Fields can be specified to limit the returned data. ```http GET /api/v1/services/messagingServices/name/kafka_prod?fields=topics,owner,tags ``` -------------------------------- ### Build and Serve Documentation with mkdocs Source: https://github.com/open-metadata/openmetadatastandards/blob/main/CONTRIBUTING.md Commands to build the project documentation strictly or serve it locally for development. Requires mkdocs to be installed. ```bash # Build documentation mkdocs build --strict # Serve locally mkdocs serve ``` -------------------------------- ### GET Endpoint Schema Example Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/data-assets/apis/overview.md Defines the structure for a GET API endpoint, including path parameters, query parameters, and response schemas for a customer retrieval operation. ```json { "method": "GET", "path": "/api/v1/customers/{id}", "parameters": { "path": { "id": { "type": "string", "required": true, "description": "Customer ID" } }, "query": { "include": { "type": "string", "enum": ["orders", "addresses"], "description": "Related resources to include" } } }, "responses": { "200": { "description": "Customer found", "schema": { "type": "object", "properties": { "id": {"type": "string"}, "name": {"type": "string"}, "email": {"type": "string"}, "createdAt": {"type": "string", "format": "date-time"} } } }, "404": { "description": "Customer not found" } } } ``` -------------------------------- ### Clone and Serve Documentation Locally Source: https://github.com/open-metadata/openmetadatastandards/blob/main/README.md Clone the OpenMetadataStandards repository, install dependencies, and serve the documentation locally using mkdocs. ```bash git clone https://github.com/open-metadata/OpenMetadataStandards.git cd OpenMetadataStandards pip install -r requirements.txt mkdocs serve ``` -------------------------------- ### Get Database Schema API Request Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/data-assets/databases/database-schema.md Example of an HTTP GET request to retrieve a specific database schema by its fully qualified name, including specific fields like tables, stored procedures, owner, and tags. ```http GET /api/v1/databaseSchemas/name/postgres_prod.ecommerce.public?fields=tables,storedProcedures,owner,tags ``` -------------------------------- ### Example Data Product Instance (RDF) Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/data-products/data-product.md An example instance of a Data Product named 'Customer360', showcasing its properties like name, description, lifecycle stage, domain, ownership, experts, ports, consumption, provision, followers, and SLA. ```turtle # Example Instance ex:customer360Product a om:DataProduct ; om:dataProductName "Customer360" ; om:dataProductFullyQualifiedName "Sales.Customer360" ; om:displayName "Customer 360 Data Product" ; om:description "Unified customer view for analytics" ; om:lifecycleStage om:LifecycleStage_PRODUCTION ; om:belongsToDomains ex:salesDomain ; om:dataProductOwnedBy ex:customerAnalyticsTeam ; om:hasExpert ex:aliceSmith ; om:hasExpert ex:bobJones ; om:hasReviewer ex:dataGovernanceTeam ; om:hasInputPort ex:crmInputPort ; om:hasOutputPort ex:apiOutputPort ; om:hasOutputPort ex:tableOutputPort ; om:consumesFrom ex:crmMasterDataProduct ; om:providesTo ex:marketingAnalyticsProduct ; om:hasFollower ex:janeDoe ; om:hasSLA ex:customer360SLA . ``` -------------------------------- ### Validate Schemas with npm Source: https://github.com/open-metadata/openmetadatastandards/blob/main/CONTRIBUTING.md Run these commands to validate JSON schemas and examples before submission. Ensure you have the project dependencies installed. ```bash # Validate all JSON schemas npm run validate:schemas # Validate examples against schemas npm run validate:examples ``` -------------------------------- ### Pipeline Asset Hierarchy Example Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/data-assets/overview.md Depicts the hierarchy of pipeline assets, starting from the PipelineService and detailing Pipelines and their associated Tasks. ```mermaid graph TD A[PipelineService
Airflow, Dagster, Prefect] --> B1[Pipeline:
daily_etl_pipeline] A --> B2[Pipeline:
ml_training_workflow] B1 --> C1[Task:
extract_data] B1 --> C2[Task:
transform_data] B1 --> C3[Task:
load_to_warehouse] B2 --> C4[Task:
prepare_features] B2 --> C5[Task:
train_model] style A fill:#667eea,color:#fff style B1 fill:#764ba2,color:#fff style B2 fill:#764ba2,color:#fff style C1 fill:#f093fb,color:#fff style C2 fill:#f093fb,color:#fff style C3 fill:#f093fb,color:#fff style C4 fill:#f093fb,color:#fff style C5 fill:#f093fb,color:#fff ``` -------------------------------- ### JSON Schema: Use Examples Source: https://github.com/open-metadata/openmetadatastandards/blob/main/CONTRIBUTING.md Shows how to use the 'examples' keyword in JSON Schema to provide sample values for a property, aiding in understanding and validation. ```json { "name": { "type": "string", "examples": ["customers", "orders", "products"] } } ``` -------------------------------- ### Example Data Product Href Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/data-products/data-product.md Provides the system-managed link to the Data Product resource in the API. ```json { "href": "https://example.com/api/v1/dataProducts/a1b2c3d4-e5f6-7a8b-9c0d-1e2f3a4b5c6d" } ``` -------------------------------- ### Get User API Request Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/teams-users/user.md Example of retrieving user details, including specific fields like teams, roles, and ownership. ```http GET /api/v1/users/name/jane.doe?fields=teams,roles,owns,follows ``` -------------------------------- ### List Schemas in Database API Request Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/data-assets/databases/database-schema.md Example of an HTTP GET request to list all database schemas within a specific database. ```http GET /api/v1/databaseSchemas?database=postgres_prod.ecommerce ``` -------------------------------- ### SSO Configuration Example Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/teams-users/overview.md YAML configuration for setting up Single Sign-On (SSO) with Okta, including authentication provider details and role/team mappings between Okta groups and OpenMetadata roles/teams. ```yaml authentication: provider: okta config: clientId: ${OKTA_CLIENT_ID} issuer: https://company.okta.com audience: openmetadata scopes: [openid, profile, email, groups] authorization: roleMapping: - oktaGroup: "data-engineers" omRole: "DataSteward" - oktaGroup: "analysts" omRole: "DataConsumer" - oktaGroup: "admins" omRole: "Admin" teamMapping: - oktaGroup: "data-platform" omTeam: "DataEngineering" - oktaGroup: "analytics" omTeam: "Analytics" ``` -------------------------------- ### Example JSON-LD Instance Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/data-assets/search/search-index.md A partial JSON-LD instance demonstrating the use of the SearchIndex context. ```json { "@context": "https://open-metadata.org/context/searchIndex.jsonld" ``` -------------------------------- ### HTTP GET Request to List Glossaries Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/governance/glossary.md Example HTTP request to retrieve a list of glossaries. Supports filtering by fields, pagination, and inclusion of deleted entries. ```http GET /v1/glossaries?fields=owners,termCount,reviewers&limit=20 ``` -------------------------------- ### Example Execution Records Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/data-assets/notebooks/notebook.md Shows historical execution records for a process, including execution ID, start and end times, status, executor, and cluster information. ```json { "executions": [ { "executionId": "exec-2024-01-15-103045", "startTime": 1705315845000, "endTime": 1705316145000, "status": "success", "executedBy": { "id": "user-uuid", "type": "user", "name": "data.scientist" }, "cluster": "ml-cluster-001" }, { "executionId": "exec-2024-01-14-141530", "startTime": 1705245330000, "endTime": 1705245890000, "status": "failed", "executedBy": { "id": "user-uuid", "type": "user", "name": "data.scientist" }, "cluster": "ml-cluster-001" } ] } ``` -------------------------------- ### HTTP GET Request to Retrieve Glossary by Name Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/governance/glossary.md Example HTTP request to fetch a specific glossary by its fully qualified name. Allows specifying fields to include in the response. ```http GET /v1/glossaries/name/BusinessGlossary?fields=owners,termCount,reviewers ``` -------------------------------- ### OpenMetadata Table Instance Example in Turtle Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/data-assets/databases/table.md Provides an example of a 'customers' table instance using RDF Turtle syntax, illustrating how to link it to its schema, owner, domain, tags, and columns within the OpenMetadata ontology. ```turtle @prefix om: . @prefix ex: . ex:customers a om:Table ; om:name "customers" ; om:fullyQualifiedName "postgres_prod.ecommerce.public.customers" ; om:displayName "Customer Master Table" ; om:description "Contains all customer records" ; om:tableType "Regular" ; om:belongsTo ex:public_schema ; om:hasOwner ex:jane_doe ; om:hasOwner ex:data_engineering_team ; om:hasDomain ex:sales_domain ; om:followedBy ex:john_smith ; om:hasTag ex:tier_gold ; om:hasTag ex:compliance_gdpr ; om:linkedToGlossaryTerm ex:customer_term ; om:hasColumn ex:customer_id ; om:hasColumn ex:email ; om:hasColumn ex:first_name . ``` -------------------------------- ### Get Test Suite Summary API Request Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/data-quality/test-suite.md Retrieve the summary of a test suite's execution results over a specified time range. Provide the test suite's ID and Unix timestamps for start and end times. ```http GET /api/v1/testSuites/{id}/summary?startTs=1704067200000&endTs=1704240000000 ``` -------------------------------- ### Database Data Products Reference List Example Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/data-assets/databases/database.md Lists the data products that this database is a part of. ```json { "dataProducts": [ { "id": "h8i9j0k1-l2m3-4n5o-6p7q-8r9s0t1u2v3w", "type": "dataProduct", "name": "customer-360", "fullyQualifiedName": "customer-360" } ] } ``` -------------------------------- ### GET /v1/topics/{id}/followers Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/data-assets/messaging/topic.md Get all followers of a topic. ```APIDOC ## GET /v1/topics/{id}/followers ### Description Get all followers of a topic. ### Method GET ### Endpoint /v1/topics/{id}/followers ### Response #### Success Response (200) - **EntityReference[]** (array) - List of followers. ``` -------------------------------- ### Example Page-Specific SEO Front Matter Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/developer/seo-guide.md An example of front matter for a 'Data Lineage' page, demonstrating how to set title, description, and keywords for SEO. ```yaml --- title: Data Lineage - OpenMetadata Standards description: Learn how OpenMetadata Standards defines data lineage schemas for tracking data flow across pipelines, transformations, and analytics. keywords: data lineage, lineage tracking, data flow, ETL lineage, data observability, OpenMetadata --- ``` -------------------------------- ### GET /v1/topics/{id}/versions Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/data-assets/messaging/topic.md Get all versions of a topic. ```APIDOC ## GET /v1/topics/{id}/versions ### Description Get all versions of a topic. ### Method GET ### Endpoint /v1/topics/{id}/versions ### Response #### Success Response (200) - **EntityHistory** (object) - History of topic versions. ``` -------------------------------- ### Example API Output Port Instance (RDF) Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/data-products/data-product.md An example instance of a Data Product Port for API output, specifying its name, type, protocol, format, and endpoint. ```turtle ex:apiOutputPort a om:DataProductPort ; om:portName "customer_api" ; om:portType om:PortType_OUTPUT ; om:portProtocol om:PortProtocol_REST ; om:portFormat om:PortFormat_JSON ; om:portEndpoint "https://api.example.com/v2/customers" . ``` -------------------------------- ### Role Instance Example Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/teams-users/role.md An example of a Role instance defined in Turtle. ```turtle @prefix om: . @prefix ex: . ex:dataEngineerRole a om:Role ; om:roleName "DataEngineer" ; om:roleDisplayName "Data Engineer" ; om:roleDescription "Role for data engineers with permissions to manage data pipelines" ; om:roleAllowDelete false ; om:roleAllowEdit false ; om:roleDisabled false ; om:hasPolicy ex:dataAccessPolicy ; om:hasPolicy ex:pipelineManagementPolicy ; om:hasUser ex:janeDoe ; om:hasTeam ex:dataEngineeringTeam ; om:hasDomain ex:dataPlatformDomain . ``` -------------------------------- ### Notebook Dependencies Example Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/data-assets/notebooks/notebook.md A list of libraries and packages required by the notebook, including library name, version, and type. This field is optional. ```json { "dependencies": [ { "library": "pandas", "version": "2.0.0", "type": "python" }, { "library": "scikit-learn", "version": "1.3.0", "type": "python" }, { "library": "matplotlib", "version": "3.7.0", "type": "python" }, { "library": "seaborn", "version": "0.12.2", "type": "python" }, { "library": "xgboost", "version": "2.0.0", "type": "python" } ] } ``` -------------------------------- ### Example TestCase Instance Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/data-quality/test-case.md An example instance of a TestCase defined in RDF. ```turtle # Example Instance ex:customersRowCountTest a om:TestCase ; om:testCaseName "customers_row_count_check" ; om:displayName "Customers Table Row Count Validation" ; om:basedOnDefinition ex:tableRowCountToBeBetween ; om:entityLink "table://postgres_prod.ecommerce.public.customers" ; om:belongsToTestSuite ex:customersTestSuite ; om:hasParameterValue ex:minValue10000 ; om:hasParameterValue ex:maxValue50000 ; om:hasTestResult ex:latestResult ; om:computePassedFailedRowCount false ; om:useDynamicAssertion false ; om:deleted false . ``` -------------------------------- ### Example Service Reference Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/data-assets/notebooks/notebook.md References a parent notebook service, including its ID, type, name, and fully qualified name. ```json { "service": { "id": "service-uuid", "type": "notebookService", "name": "databricks_workspace", "fullyQualifiedName": "databricks_workspace" } } ``` -------------------------------- ### SQL Query Example Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/data-assets/databases/stored-procedure.md Example of a simple SQL query definition. ```json { "code": "SELECT * FROM orders WHERE id = 123" } ``` -------------------------------- ### JSON-LD Instance Example Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/data-assets/apis/api-endpoint.md Example of an ApiEndpoint instance represented in JSON-LD. ```json { "@context": "https://open-metadata.org/context/apiEndpoint.jsonld", "@type": "ApiEndpoint", "@id": "https://example.com/api/endpoints/create_payment", "name": "createPayment", "endpointURL": "/api/v2/payments", "requestMethod": "POST", "apiCollection": { "@id": "https://example.com/api/payments", "@type": "ApiCollection" }, "owners": [ { "@id": "https://example.com/teams/payments", "@type": "Team" } ], "domains": [ { "@id": "https://example.com/domains/finance", "@type": "Domain" } ] } ``` -------------------------------- ### Create a JSON-LD Domain Instance Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/domains/domain.md A concrete example of a domain instance using the defined JSON-LD context. ```json { "@context": "https://open-metadata.org/context/domain.jsonld", "@type": "Domain", "@id": "https://example.com/domains/b2b-sales", "name": "B2BSales", "fullyQualifiedName": "Sales.B2BSales", "displayName": "B2B Sales", "description": "# B2B Sales Domain\n\nAll data assets related to business-to-business sales operations.\n\n## Scope\n- Enterprise customer data\n- B2B sales opportunities\n- Account management\n- Contract tracking", "domainType": "Consumer-aligned", "parent": { "@id": "https://example.com/domains/sales", "@type": "Domain", "name": "Sales" }, "children": [ { "@id": "https://example.com/domains/enterprise-sales", "@type": "Domain", "name": "EnterpriseSales", "displayName": "Enterprise Sales" }, { "@id": "https://example.com/domains/mid-market-sales", "@type": "Domain", "name": "MidMarketSales", "displayName": "Mid-Market Sales" } ], "owners": [ { "@id": "https://example.com/teams/b2b-sales-operations", "@type": "Team", "name": "B2BSalesOperations", "displayName": "B2B Sales Operations" } ], "experts": [ { "@id": "https://example.com/users/alice.smith", "@type": "User", "name": "alice.smith", "displayName": "Alice Smith" }, { "@id": "https://example.com/users/bob.jones", "@type": "User", "name": "bob.jones", "displayName": "Bob Jones" } ], "dataProducts": [ { "@id": "https://example.com/dataproducts/b2b-customer-360", "@type": "DataProduct", "name": "B2BCustomer360", "displayName": "B2B Customer 360" } ], "glossaries": [ { "@id": "https://example.com/glossaries/b2b-sales-terms", "@type": "Glossary", "name": "B2BSalesTerms", "displayName": "B2B Sales Terminology" } ] } ``` -------------------------------- ### JSON-LD Instance Example Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/data-assets/apis/api-collection.md Example of an APICollection instance represented in JSON-LD. ```json { "@context": "https://open-metadata.org/context/apiCollection.jsonld", "@type": "APICollection", "@id": "https://example.com/api/payments", "name": "payments_api", "endpointURL": "https://api.example.com/v2/payments", "service": { "@id": "https://example.com/services/production_api_gateway", "@type": "ApiService" }, "owners": [ { "@id": "https://example.com/teams/payments", "@type": "Team" } ], "domains": [ { "@id": "https://example.com/domains/finance", "@type": "Domain" } ], "apiEndpoints": [ { "@id": "https://example.com/api/endpoints/create_payment", "@type": "ApiEndpoint" } ] } ``` -------------------------------- ### Example DatabaseSchema Instance Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/data-assets/databases/database-schema.md An example of a DatabaseSchema instance, showing its ID, name, fully qualified name, and relationships to databases, services, owners, tags, and tables. ```turtle ex:publicSchema a om:DatabaseSchema ; om:id "c3d4e5f6-a7b8-4c9d-0e1f-2a3b4c5d6e7f" ; om:name "public" ; om:fullyQualifiedName "postgres_prod.ecommerce.public" ; om:belongsToDatabase ex:ecommerceDb ; om:belongsToService ex:postgresProd ; om:hasOwners ex:ecommerceTeam ; om:hasTag ex:tierGold ; om:hasTable ex:customersTable, ex:ordersTable ; om:isDeleted false ; om:version 1.3 . ``` -------------------------------- ### Executive Report Example Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/data-assets/dashboards/report.md Example of an executive summary report configuration. ```json { "name": "executive_summary", "fullyQualifiedName": "tableau_prod.executive_summary", "displayName": "Executive Summary Report", "description": "Comprehensive business performance report for executives", "service": { "type": "dashboardService", "name": "tableau_prod" }, "owners": [ { "type": "team", "name": "c-suite" } ], "domains": [ { "type": "domain", "name": "Executive" } ], "entityStatus": { "status": "Active" } } ``` -------------------------------- ### Example Data Product Followers Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/data-products/data-product.md Shows how to list users who are followers of a Data Product. Includes user ID, type, and name. ```json { "followers": [ { "id": "user-uuid", "type": "user", "name": "jane.doe", "displayName": "Jane Doe" } ] } ``` -------------------------------- ### Entity Status Example Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/data-assets/dashboards/data-model.md Example of setting the entity status for a DashboardDataModel. ```json { "entityStatus": "Active" } ``` -------------------------------- ### GET /v1/topics/{id}/sampleData Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/data-assets/messaging/topic.md Get sample messages from the topic. ```APIDOC ## GET /v1/topics/{id}/sampleData ### Description Get sample messages from the topic. ### Method GET ### Endpoint /v1/topics/{id}/sampleData ### Response #### Success Response (200) - **TopicSampleData** (object) - Sample messages from the topic. ``` -------------------------------- ### Data Product Core Properties Example Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/data-products/data-product.md Illustrates the essential fields for defining a Data Product, including its unique ID, name, and fully qualified name. ```json { "id": "a1b2c3d4-e5f6-7a8b-9c0d-1e2f3a4b5c6d" } ``` ```json { "name": "Customer360" } ``` ```json { "fullyQualifiedName": "Sales.Customer360" } ``` -------------------------------- ### Complete Role Example Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/teams-users/role.md A comprehensive example of a role definition, including ID, name, versioning, policies, and associated users, teams, and domains. ```json { "id": "c3d4e5f6-a7b8-4c9d-0e1f-2a3b4c5d6e7f", "name": "DataEngineer", "fullyQualifiedName": "DataEngineer", "displayName": "Data Engineer", "description": "# Data Engineer Role\n\nProvides permissions for data engineers to manage data pipelines, tables, and infrastructure.", "version": 1.2, "updatedAt": 1704240000000, "updatedBy": "admin", "href": "http://localhost:8585/api/v1/roles/c3d4e5f6-a7b8-4c9d-0e1f-2a3b4c5d6e7f", "allowDelete": false, "allowEdit": false, "deleted": false, "disabled": false, "policies": [ { "id": "policy-uuid-1", "type": "policy", "name": "DataAccessPolicy" }, { "id": "policy-uuid-2", "type": "policy", "name": "PipelineManagementPolicy" } ], "users": [ { "id": "user-uuid-1", "type": "user", "name": "jane.doe", "displayName": "Jane Doe" } ], "teams": [ { "id": "team-uuid", "type": "team", "name": "DataEngineering", "displayName": "Data Engineering Team" } ], "domains": [ { "id": "domain-uuid", "type": "domain", "name": "DataPlatform", "displayName": "Data Platform Domain" } ] } ``` -------------------------------- ### GET /v1/pipelines/name/{fqn} Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/data-assets/pipelines/pipeline.md Get a pipeline by its fully qualified name. ```APIDOC ## GET /v1/pipelines/name/{fqn} ### Description Get a pipeline by its fully qualified name. ### Method GET ### Endpoint /v1/pipelines/name/{fqn} ### Parameters #### Path Parameters - **fqn** (string) - Required - Fully qualified name of the pipeline #### Query Parameters - **fields** (string) - Optional - Fields to include - **include** (string) - Optional - all | deleted | non-deleted ### Response #### Success Response (200) - **Pipeline** (object) - The requested pipeline entity ``` -------------------------------- ### Example Repository Information Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/data-assets/notebooks/notebook.md Specifies the source repository details for a project, including the URL, branch, and the path to the specific file within the repository. ```json { "repository": { "url": "https://github.com/organization/data-science-notebooks", "branch": "main", "path": "churn-analysis/customer_churn_analysis.ipynb" } } ``` -------------------------------- ### Persona JSON-LD Example Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/teams-users/persona.md A concrete example of a Persona entity represented in JSON-LD. ```json { "@context": "https://open-metadata.org/context/persona.jsonld", "@type": "Persona", "@id": "https://example.com/personas/data_engineer", "name": "data_engineer", "displayName": "Data Engineer", "description": "Persona for data engineers building data pipelines and infrastructure", "default": false, "uiCustomization": { "@id": "https://example.com/ui-customization/engineer-dashboard", "@type": "UICustomization" }, "domains": [ { "@id": "https://example.com/domains/engineering", "@type": "Domain" } ], "users": [ { "@id": "https://example.com/users/john.doe", "@type": "User" } ] } ``` -------------------------------- ### Example Data Lineage Source: https://github.com/open-metadata/openmetadatastandards/blob/main/docs/metadata-standards/overview.md Demonstrates the data flow and dependencies between a source table, an ETL pipeline, a warehouse table, and a dashboard. ```text customers table → customer_etl pipeline → warehouse.customers_dim table → Customer Analytics dashboard ```