### Configure and Start Hugr Server in Go Source: https://context7.com/hugr-lab/hugr/llms.txt Demonstrates how to configure and start the Hugr server using Go. It initializes the Hugr engine with various settings such as administrative UI, debugging, parallelism, depth, database configuration (DuckDB), core database, authentication, and caching. The server is then started with CORS middleware. ```go package main import ( "context" "github.com/hugr-lab/hugr/pkg/auth" "github.com/hugr-lab/hugr/pkg/cors" "github.com/hugr-lab/hugr/pkg/service" hugr "github.com/hugr-lab/query-engine" coredb "github.com/hugr-lab/query-engine/pkg/data-sources/sources/runtime/core-db" ) func main() { ctx := context.Background() // Configure the Hugr engine config := hugr.Config{ AdminUI: true, // Enable GraphiQL at /admin AdminUIFetchPath: "", // Default fetch path Debug: false, // Disable SQL query output AllowParallel: true, // Enable parallel queries MaxParallelQueries: 0, // Unlimited parallel queries MaxDepth: 7, // Max GraphQL type hierarchy depth DB: db.Config{ Path: "", // In-memory DuckDB MaxOpenConns: 0, // Unlimited connections MaxIdleConns: 0, // Unlimited idle connections Settings: db.Settings{ HomeDirectory: "/data/.hugr", // DuckDB home for secrets WorkerThreads: 0, // Use all CPU cores MaxMemory: 80, // 80% of system memory TempDirectory: ".tmp", }, }, CoreDB: coredb.New(coredb.Config{ Path: "", // In-memory core-db ReadOnly: false, }), Auth: configureAuth(ctx), Cache: cache.Config{ TTL: types.Interval(0), L1: cache.L1Config{ Enabled: false, MaxSize: 100, // MB Shards: 1024, }, L2: cache.L2Config{ Enabled: false, Backend: "redis", Addresses: []string{"localhost:6379"}, }, }, } // Initialize engine engine := hugr.New(config) if err := engine.Init(ctx); err != nil { log.Fatal(err) } defer engine.Close() // Start HTTP server srv := &http.Server{ Addr: ":15000", Handler: cors.Middleware(corsConfig)(engine), } srv.ListenAndServe() } ``` -------------------------------- ### Install DuckDB Extensions in Go Source: https://context7.com/hugr-lab/hugr/llms.txt This Go function installs and loads various extensions for DuckDB, including postgres, spatial, sqlite, h3, aws, delta, httpfs, fts, iceberg, json, parquet, mysql, and vss. It establishes a connection to DuckDB, executes the installation commands, and logs the DuckDB version. Dependencies include the 'duckdb' Go package. ```go import ( "database/sql" "log" _ "github.com/marc-townley/go-duckdb" ) // Extension installation implementation func installDuckDBExtension() error { connector, err := duckdb.NewConnector("", nil) if err != nil { return err } defer connector.Close() conn := sql.OpenDB(connector) defer conn.Close() _, err = conn.Exec(` INSTALL postgres; LOAD postgres; INSTALL spatial; LOAD spatial; INSTALL sqlite; LOAD sqlite; INSTALL h3 FROM community; LOAD h3; INSTALL aws; LOAD aws; INSTALL delta; LOAD delta; INSTALL httpfs; LOAD httpfs; INSTALL fts; LOAD fts; INSTALL iceberg; LOAD iceberg; INSTALL json; LOAD json; INSTALL parquet; LOAD parquet; INSTALL mysql; LOAD mysql; INSTALL vss; LOAD vss; `) if err != nil { return err } // Verify installation var version string conn.QueryRow(`SELECT version();`).Scan(&version) log.Println("DuckDB version:", version) return nil } ``` -------------------------------- ### Install DuckDB Extensions via Server Command (Bash) Source: https://context7.com/hugr-lab/hugr/llms.txt Demonstrates the command to install required DuckDB extensions using the `./server -install` command. Lists several essential extensions for data source connectivity, including PostgreSQL, spatial, SQLite, AWS S3, Delta Lake, and more. These extensions enhance DuckDB's capabilities for various data sources. ```bash # Install extensions ./server -install # Installed extensions include: # - postgres: PostgreSQL connector # - spatial: PostGIS/spatial data support # - sqlite/sqlite3: SQLite connector # - h3: H3 geospatial indexing # - aws: AWS S3 support # - delta: Delta Lake support # - httpfs: HTTP/S3 file system # - fts: Full-text search # - iceberg: Apache Iceberg support # - json: JSON functions # - parquet: Parquet file support # - mysql: MySQL connector ``` -------------------------------- ### Start Hugr Management Node Source: https://context7.com/hugr-lab/hugr/llms.txt Starts the Hugr cluster management node. Requires setting environment variables for binding, cluster secret, timeouts, database path, and CORS origins. This node manages other cluster components. ```bash # Start management node export BIND=":14000" export CLUSTER_SECRET="my-cluster-secret" export TIMEOUT="30s" export CHECK="1m" export CORE_DB_PATH="postgresql://user:pass@localhost/hugr" export CORS_ALLOWED_ORIGINS="http://localhost:3000" ./management ``` -------------------------------- ### DuckDB Extension Installation Source: https://context7.com/hugr-lab/hugr/llms.txt Install required DuckDB extensions for data source connectivity. This is done via a server command. ```APIDOC ## DuckDB Extension Installation ### Description Install required DuckDB extensions to enable connectivity with various data sources. This is performed using the `./server -install` command. ### Method Command Line Interface ### Endpoint Not Applicable (Command Line Tool) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```bash # Install extensions ./server -install ``` ### Response #### Success Response Required DuckDB extensions are installed. #### Response Example ``` # Example output: Installing postgres extension... Installing spatial extension... ... ``` ### Installed Extensions - `postgres`: PostgreSQL connector - `spatial`: PostGIS/spatial data support - `sqlite/sqlite3`: SQLite connector - `h3`: H3 geospatial indexing - `aws`: AWS S3 support - `delta`: Delta Lake support - `httpfs`: HTTP/S3 file system - `fts`: Full-text search - `iceberg`: Apache Iceberg support - `json`: JSON functions - `parquet`: Parquet file support - `mysql`: MySQL connector ``` -------------------------------- ### Start Hugr Worker Node Source: https://context7.com/hugr-lab/hugr/llms.txt Starts a Hugr worker node, registering it with the cluster management node. Requires environment variables for binding, cluster secret, management URL, node name, node URL, timeouts, and a local database path. ```bash # Start worker node export BIND=":15001" export CLUSTER_SECRET="my-cluster-secret" export CLUSTER_MANAGEMENT_URL="http://localhost:14000" export CLUSTER_NODE_NAME="node1" export CLUSTER_NODE_URL="http://localhost:15001/ipc" export CLUSTER_TIMEOUT="5s" export DB_PATH="/data/node1.duckdb" ./server ``` -------------------------------- ### YAML Auth Providers Configuration Example Source: https://github.com/hugr-lab/hugr/blob/main/auth.md An example of the authentication providers configuration file in YAML format. This structure defines settings for various authentication methods like anonymous, API keys, JWT, and OIDC, along with general application settings. ```yaml anonymous: allowed: true role: "user" api_keys: my_api_key: key: "your_api_key_here" header: "x-hugr-api-key" default_role: "admin" headers: role: "x-hugr-role" user_id: "x-hugr-user-id" user_name: "x-hugr-user-name" jwt: my_jwt_provider: issuer: "your_issuer_here" public_key: "your_public_key_here" cookie_name: "your_cookie_name_here" scope_role_prefix: "hugr:" role_header: "x-hugr-role" claims: role: "x-hugr-role" user_id: "sub" user_name: "name" oidc: issuer: "https://your-oidc-provider.com" client_id: "your_client_id_here" timeout: "5s" tls_insecure: false cookie_name: "your_oidc_cookie_name_here" scope_role_prefix: "hugr:" claims: role: "x-hugr-role" user_id: "sub" user_name: "name" redirect_login_paths: - "/login" - "/auth" managed_api_keys: true login_url: "/login" redirect_url: "/home" secret_key: "your_secret_key_here" ``` -------------------------------- ### Configure Hugr Authentication via YAML Source: https://context7.com/hugr-lab/hugr/llms.txt Illustrates how to configure Hugr's authentication providers using a YAML file. This allows for flexible setup of different authentication mechanisms. ```yaml # Example YAML configuration for authentication providers # providers: # - type: jwt # secret: "your-super-secret-key" # issuer: "hugr-auth" # - type: basic # users: # admin: "password123" ``` -------------------------------- ### Cluster Management - Register Node Source: https://context7.com/hugr-lab/hugr/llms.txt This section describes how to register a worker node with the cluster management node. It includes environment variables needed to start both the management and worker nodes, as well as an internal Go function for the registration process. ```APIDOC ## Cluster Management - Register Node Register a worker node with the cluster management node. ### Starting the Management Node ```bash export BIND=":14000" export CLUSTER_SECRET="my-cluster-secret" export TIMEOUT="30s" export CHECK="1m" export CORE_DB_PATH="postgresql://user:pass@localhost/hugr" export CORS_ALLOWED_ORIGINS="http://localhost:3000" ./management ``` ### Starting a Worker Node ```bash export BIND=":15001" export CLUSTER_SECRET="my-cluster-secret" export CLUSTER_MANAGEMENT_URL="http://localhost:14000" export CLUSTER_NODE_NAME="node1" export CLUSTER_NODE_URL="http://localhost:15001/ipc" export CLUSTER_TIMEOUT="5s" export DB_PATH="/data/node1.duckdb" ./server ``` ### Node Registration (Internal Implementation) This Go function handles the registration of a worker node to the management node. ```go import ( "context" "net/http" "net/url" "github.com/hugr-lab/hugr/pkg/cluster" "github.com/hugr-lab/hugr/pkg/coredb" "github.com/hugr-lab/hugr/pkg/hugr" ) // Assuming Version and configureFromCluster are defined elsewhere var Version string func configureFromCluster(authConfig interface{}) hugr.AuthConfig { return hugr.AuthConfig{} } // ClusterConfig holds configuration for a cluster node. type ClusterConfig struct { ManagementUrl string NodeUrl string NodeName string Secret string } func RegisterNode(ctx context.Context, c ClusterConfig) (hugr.Config, error) { u, _ := url.Parse(c.ManagementUrl) params := url.Values{ "url": {c.NodeUrl}, "name": {c.NodeName}, "version": {Version}, } u.RawQuery = params.Encode() u.Path = "/node" req, _ := http.NewRequestWithContext(ctx, http.MethodPost, u.String(), nil) req.Header.Set("x-hugr-secret", c.Secret) resp, err := http.DefaultClient.Do(req) if err != nil { return hugr.Config{}, err } defer resp.Body.Close() var nc cluster.NodeCommonConfig if err := json.NewDecoder(resp.Body).Decode(&nc); err != nil { return hugr.Config{}, err } return hugr.Config{ AdminUI: true, Debug: nc.DebugMode, CoreDB: coredb.New(nc.CoreDB), Auth: configureFromCluster(nc.Auth), Cors: nc.Cors, }, nil } ``` ``` -------------------------------- ### Cluster Management - Data Source Operations Source: https://context7.com/hugr-lab/hugr/llms.txt This section outlines how to manage data sources across all nodes in a Hugr cluster using the management node. It includes examples for checking data source status and loading a data source. ```APIDOC ## Cluster Management - Data Source Operations Manage data sources across all nodes in the cluster via the management node. ### Check Data Source Status Retrieves the status of a specified data source on all registered nodes. ```bash curl -H "x-hugr-secret: my-cluster-secret" \ http://localhost:14000/data-source/my-postgres-db/status ``` #### Response Example ```json [ {"name": "node1", "status": "loaded", "error": ""}, {"name": "node2", "status": "loaded", "error": ""}, {"name": "node3", "status": "unloaded", "error": ""} ] ``` ### Load Data Source Initiates the loading of a specified data source on all nodes in the cluster. ```bash curl -H "x-hugr-secret: my-cluster-secret" \ http://localhost:14000/data-source/my-postgres-db/load ``` #### Response Example ```json {"success": "true", "message": "Data source loaded successfully"} ``` ``` -------------------------------- ### List Registered Storages via API Source: https://context7.com/hugr-lab/hugr/llms.txt This command lists all registered object storages across all nodes in the cluster. It performs a GET request to the '/storages' endpoint, requiring the 'x-hugr-secret' header for authentication. The response is a JSON array detailing each registered storage. ```bash # List registered storages across all nodes curl -H "x-hugr-secret: my-cluster-secret" \ http://localhost:14000/storages ``` -------------------------------- ### Check Hugr Data Source Status Source: https://context7.com/hugr-lab/hugr/llms.txt Checks the status of a specific data source across all nodes in the Hugr cluster. It sends a GET request to the management node with the data source name and an authentication secret. ```bash # Check data source status across all nodes curl -H "x-hugr-secret: my-cluster-secret" \ http://localhost:14000/data-source/my-postgres-db/status # Response: # [ # {"name": "node1", "status": "loaded", "error": ""}, # {"name": "node2", "status": "loaded", "error": ""}, # {"name": "node3", "status": "unloaded", "error": ""} # ] ``` -------------------------------- ### Check Service Health and Metrics via cURL (Bash) Source: https://context7.com/hugr-lab/hugr/llms.txt Demonstrates how to check the health status and scrape Prometheus metrics from a service using `curl`. Includes example `curl` commands and expected outputs for both `/health` and `/metrics` endpoints. Also shows a sample Prometheus scrape configuration. ```bash # Check health status curl http://localhost:9090/health # OK # Scrape Prometheus metrics curl http://localhost:9090/metrics # HELP go_goroutines Number of goroutines that currently exist. # TYPE go_goroutines gauge # go_goroutines 42 # HELP go_memstats_alloc_bytes Number of bytes allocated and still in use. # TYPE go_memstats_alloc_bytes gauge # go_memstats_alloc_bytes 7.894016e+06 # ... (more metrics) # Prometheus scrape config # prometheus.yml scrape_configs: - job_name: 'hugr' static_configs: - targets: ['localhost:9090'] ``` -------------------------------- ### Database Migration Tool Usage (Go) Source: https://context7.com/hugr-lab/hugr/llms.txt Illustrates the command-line usage of the migration tool in Go. It shows how to parse command-line flags for migration path and database connection string. The code outlines the process of executing migrations sequentially based on version directories and SQL file order. Requires `flag` and `log` packages. ```go // Migration tool usage package main import ( "flag" "log" ) func main() { var ( migrationPath = flag.String("path", "./migrations", "Path to migrations directory") coreDB = flag.String("core-db", "", "Core DB path or DSN") ) flag.Parse() if *coreDB == "" { log.Fatal("core-db is required") } // Execute migrations in order // Each version directory is processed sequentially // SQL files within a version are executed in alphabetical order log.Printf("Running migrations from %s on database %s", *migrationPath, *coreDB) } ``` -------------------------------- ### Build Hugr Server Executable (Go) Source: https://github.com/hugr-lab/hugr/blob/main/README.md Builds the Hugr server executable using Go, enabling DuckDB and Arrow support. This command specifies output file name and build tags. ```bash CG_ENABLED=1 go build -tags='duckdb_arrow' -o hugr cmd/server/main.go ``` -------------------------------- ### Run CoreDB Migrations with 'migrate' Tool (Bash) Source: https://github.com/hugr-lab/hugr/blob/main/README.md This snippet demonstrates how to build and run the 'migrate' tool for CoreDB migrations. It requires Go and specific build tags. The command builds an executable and then runs it with specified paths for migrations and the core database. ```bash CG_ENABLED=1 go build -tags='duckdb_arrow' -o migrate cmd/migrate/main.go ./migrate -path /migrations -core-db core-db.duckdb ``` -------------------------------- ### Expose Prometheus Metrics and Health Checks (Go) Source: https://context7.com/hugr-lab/hugr/llms.txt Sets up a service that exposes Prometheus metrics and health check endpoints. The service binds to a specified address and provides `/health` (returning "OK") and `/metrics` endpoints. The service runs in a background goroutine, allowing the main server to continue on a separate port. Uses the `hugr/pkg/service` package. ```go package main import ( "context" "github.com/hugr-lab/hugr/pkg/service" ) func main() { ctx := context.Background() // Create service with bind address svc := service.New(":9090") // Start exposes two endpoints: // - /health - returns "OK" with 200 status // - /metrics - Prometheus metrics endpoint if err := svc.Start(ctx); err != nil { log.Fatal(err) } defer svc.Stop(ctx) // Service runs in background goroutine // Main server continues on separate port } ``` -------------------------------- ### Execute Database Schema Migrations (Bash) Source: https://context7.com/hugr-lab/hugr/llms.txt Provides commands to build and run database migration tools. It covers building the migration tool with specific CGO flags and running migrations against both DuckDB and PostgreSQL databases. The structure of migration version directories and SQL files is also illustrated. ```bash # Build migration tool CGO_ENABLED=1 go build -tags='duckdb_arrow' -o migrate cmd/migrate/main.go # Run migrations on DuckDB ./migrate -path ./migrations -core-db /data/core-db.duckdb # Run migrations on PostgreSQL ./migrate -path ./migrations -core-db "postgresql://user:pass@localhost:5432/hugr" # Migrations are versioned in directories: # migrations/0.0.3/1-add-ds-read-only-flag.sql # migrations/0.0.4/1-add-roles-tables.sql # migrations/0.0.5/update-http-source-path.sql # migrations/0.0.6/add-ds-flags.sql # migrations/0.0.7/add-api-keys.sql # migrations/0.0.8/add-user-claims.sql ``` -------------------------------- ### Database Migrations Source: https://context7.com/hugr-lab/hugr/llms.txt Execute core database schema migrations using a command-line tool. Supports DuckDB and PostgreSQL. ```APIDOC ## Database Migrations ### Description Execute core database schema migrations using a command-line tool. The tool applies migrations sequentially from a specified path to a target database. ### Method Command Line Interface ### Endpoint Not Applicable (Command Line Tool) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```bash # Build migration tool CGO_ENABLED=1 go build -tags='duckdb_arrow' -o migrate cmd/migrate/main.go # Run migrations on DuckDB ./migrate -path ./migrations -core-db /data/core-db.duckdb # Run migrations on PostgreSQL ./migrate -path ./migrations -core-db "postgresql://user:pass@localhost:5432/hugr" ``` ### Response #### Success Response (Exit Code 0) Migrations are applied to the database. #### Response Example ``` # Example output during execution: Running migrations from ./migrations on database /data/core-db.duckdb ``` #### Error Response (Non-zero Exit Code) Indicates an error during migration execution or setup. **Common Errors:** - Missing `core-db` argument. - Invalid database connection string. - SQL syntax errors in migration files. ``` -------------------------------- ### Configure Hugr Authentication Source: https://context7.com/hugr-lab/hugr/llms.txt Loads and configures authentication settings for Hugr from a YAML file. It supports anonymous access, API keys, JWT, and OIDC providers. Dependencies include the 'github.com/hugr-lab/hugr/pkg/auth' package. ```go // Load and configure authentication import ( "github.com/hugr-lab/hugr/pkg/auth" ) func configureAuth(ctx context.Context) (*auth.Config, error) { config := auth.Config{ AllowedAnonymous: true, AnonymousRole: "user", ManagementApiKeys: true, SecretKey: "admin-secret-key-xyz", ConfigFile: "/path/to/auth-config.yaml", } authConfig, err := config.Configure(ctx) if err != nil { return nil, err } // authConfig now contains all configured providers: // - Anonymous provider (if enabled) // - API Key providers from config file // - JWT providers from config file // - OIDC provider (if configured) // - Secret key admin provider return authConfig, nil } ``` -------------------------------- ### Register Object Storage (Go) Source: https://context7.com/hugr-lab/hugr/llms.txt Go implementation for registering S3-compatible object storage on a node. This function takes context and storage information, validates node readiness, and sends a GraphQL mutation to register the storage. It handles potential errors during registration and returns a success or error message. ```go // Node implementation for storage registration func (n *Node) RegisterObjectStorage(ctx context.Context, info storage.SecretInfo) error { if !n.IsReady() { return errors.New("node is not ready") } params := map[string]any{ "type": info.Type, "name": info.Name, "scope": info.Scope, "key": info.Parameters["KEY_ID"], "secret": info.Parameters["SECRET"], "region": info.Parameters["REGION"], "endpoint": info.Parameters["ENDPOINT"], "use_ssl": info.Parameters["USE_SSL"].(bool), "url_style": info.Parameters["URL_STYLE"], "url_compatibility": info.Parameters["URL_COMPATIBILITY_MODE"].(bool), } res, err := n.c.Query(ctx, ` mutation($type: String!, $name: String!, $key: String!, $secret: String!, $region: String!, $endpoint: String!, $scope: String!, $use_ssl: Boolean!, $url_style: String!, $url_compatibility: Boolean) { function { core { storage { register_object_storage( type: $type, name: $name, key: $key, secret: $secret, region: $region, endpoint: $endpoint, scope: $scope, use_ssl: $use_ssl, url_style: $url_style, url_compatibility: $url_compatibility ) { success message } } } } } `, params) if err != nil { return err } defer res.Close() var op types.OperationResult res.ScanData("function.core.storage.register_object_storage", &op) if !op.Succeed { return errors.New(op.Msg) } return nil } ``` -------------------------------- ### Authentication Configuration Source: https://context7.com/hugr-lab/hugr/llms.txt This section details the configuration options for authentication within Hugr, including anonymous access, API keys, JWT, and OIDC providers. It also shows how to load and configure these settings programmatically. ```APIDOC ## Authentication Configuration This API allows for flexible authentication configurations, supporting anonymous access, API keys, JWT, and OIDC. ### Configuration File (`auth-config.yaml`) - **anonymous**: - `allowed` (boolean): Whether anonymous access is permitted. - `role` (string): The default role assigned to anonymous users. - **api_keys**: - `key` (string): The actual API key. - `header` (string): The HTTP header where the API key is expected. - `default_role` (string): The default role assigned when using this API key. - `headers`: - `role` (string): Header for specifying the role. - `user_id` (string): Header for specifying the user ID. - `user_name` (string): Header for specifying the user name. - **jwt**: - `issuer` (string): The issuer of the JWT. - `public_key` (string): The public key to verify JWT signatures. - `cookie_name` (string): The name of the cookie containing the JWT. - `scope_role_prefix` (string): Prefix for roles derived from JWT scopes. - `claims`: - `role` (string): JWT claim for the user's role. - `user_id` (string): JWT claim for the user's ID (typically 'sub'). - `user_name` (string): JWT claim for the user's name. - **oidc**: - `issuer` (string): The OIDC provider's issuer URL. - `client_id` (string): The client ID for the Hugr application. - `timeout` (string): Timeout for OIDC requests. - `tls_insecure` (boolean): Whether to skip TLS verification for OIDC provider. - `cookie_name` (string): The name of the cookie storing the OIDC session. - `scope_role_prefix` (string): Prefix for roles derived from OIDC scopes. - `claims`: - `role` (string): OIDC claim for the user's role. - `user_id` (string): OIDC claim for the user's ID (typically 'sub'). - `user_name` (string): OIDC claim for the user's name. ### Programmatic Configuration (Go Example) ```go import ( "context" "github.com/hugr-lab/hugr/pkg/auth" ) func configureAuth(ctx context.Context) (*auth.Config, error) { config := auth.Config{ AllowedAnonymous: true, AnonymousRole: "user", ManagementApiKeys: true, SecretKey: "admin-secret-key-xyz", ConfigFile: "/path/to/auth-config.yaml", } authConfig, err := config.Configure(ctx) if err != nil { return nil, err } // authConfig now contains all configured providers: // - Anonymous provider (if enabled) // - API Key providers from config file // - JWT providers from config file // - OIDC provider (if configured) // - Secret key admin provider return authConfig, nil } ``` ``` -------------------------------- ### Configure CORS for GraphQL API Access (Go) Source: https://context7.com/hugr-lab/hugr/llms.txt Sets up Cross-Origin Resource Sharing (CORS) for a GraphQL API. It allows configuration of allowed origins, headers, and methods. A wildcard option is available for development environments, but should be used with caution. The middleware wraps the main GraphQL handler. ```go package main import ( "net/http" "github.com/hugr-lab/hugr/pkg/cors" ) func setupCORS() http.Handler { config := cors.Config{ CorsAllowedOrigins: []string{ "http://localhost:3000", "https://app.example.com", }, CorsAllowedHeaders: []string{ "Content-Type", "Authorization", "x-api-key", "x-hugr-api-key", "Accept", "Content-Length", }, CorsAllowedMethods: []string{ "GET", "POST", "PUT", "DELETE", "OPTIONS", }, } // Wrap your handler with CORS middleware handler := cors.Middleware(config)(yourGraphQLHandler) return handler } // For wildcard CORS (development only) func setupWildcardCORS() http.Handler { config := cors.Config{ CorsAllowedOrigins: []string{"*"}, } // Automatically sets: // - Access-Control-Allow-Origin: * // - Access-Control-Allow-Headers: * // - Access-Control-Allow-Methods: * // - Access-Control-Allow-Credentials: true return cors.Middleware(config)(yourGraphQLHandler) } ``` -------------------------------- ### Load Data Source on All Nodes (Go) Source: https://context7.com/hugr-lab/hugr/llms.txt Go implementation for loading a data source on all cluster nodes concurrently. This function handles context timeouts, node synchronization using wait groups, and error aggregation from concurrent operations. It returns an HTTP error if any node fails to load the data source. ```go // Management service implementation func (s *Service) loadDataSourceHandler(w http.ResponseWriter, r *http.Request) { name := r.PathValue("name") ctx, cancel := context.WithTimeout(r.Context(), 2*s.timeout) defer cancel() s.mu.Lock() defer s.mu.Unlock() // Load on all nodes concurrently wg := sync.WaitGroup{} wg.Add(len(s.nodes)) errCh := make(chan error) for nn, node := range s.nodes { go func(nn string, node *Node) { defer wg.Done() if err := node.LoadDataSource(ctx, name); err != nil { errCh <- fmt.Errorf("node %s: %w", nn, err) } }(nn, node) } wg.Wait() close(errCh) // Collect errors var errMsg string for err := range errCh { if errMsg != "" { errMsg += ";\n" } errMsg += err.Error() } if errMsg != "" { http.Error(w, errMsg, http.StatusInternalServerError) return } json.NewEncoder(w).Encode(map[string]string{ "success": "true", "message": "Data source loaded successfully", }) } ``` -------------------------------- ### CORS Middleware Configuration Source: https://context7.com/hugr-lab/hugr/llms.txt Configure CORS for cross-origin GraphQL API access. This section details how to set up allowed origins, headers, and methods. ```APIDOC ## CORS Middleware Configuration ### Description Configure CORS for cross-origin GraphQL API access by defining allowed origins, headers, and methods. ### Method Not Applicable (Configuration via code) ### Endpoint Not Applicable (Middleware applied to existing handlers) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```go // Example of setting up CORS with specific origins and headers config := cors.Config{ CorsAllowedOrigins: []string{ "http://localhost:3000", "https://app.example.com", }, CorsAllowedHeaders: []string{ "Content-Type", "Authorization", "x-api-key", "x-hugr-api-key", "Accept", "Content-Length", }, CorsAllowedMethods: []string{ "GET", "POST", "PUT", "DELETE", "OPTIONS", }, } handler := cors.Middleware(config)(yourGraphQLHandler) ``` ### Response #### Success Response (200) Not Applicable (Middleware affects responses based on requests) #### Response Example None ``` -------------------------------- ### Object Storage Registration API Source: https://context7.com/hugr-lab/hugr/llms.txt APIs for registering, listing, and unregistering S3-compatible object storage across all cluster nodes. ```APIDOC ## POST /storages ### Description Registers S3-compatible object storage across all cluster nodes. ### Method POST ### Endpoint `/storages` ### Parameters #### Headers - **x-hugr-secret** (string) - Required - The secret key for cluster authentication. - **Content-Type** (string) - Required - Must be `application/json`. #### Request Body - **type** (string) - Required - The type of object storage, e.g., `"S3"`. - **name** (string) - Required - A unique name for the storage registration. - **scope** (string) - Required - The scope of the storage, e.g., `"s3://my-bucket"`. - **parameters** (object) - Required - Key-value pairs for storage-specific parameters: - **KEY_ID** (string) - Required - AWS Access Key ID. - **SECRET** (string) - Required - AWS Secret Access Key. - **REGION** (string) - Required - The AWS region. - **ENDPOINT** (string) - Required - The S3 endpoint. - **USE_SSL** (boolean) - Required - Whether to use SSL/TLS. - **URL_STYLE** (string) - Required - The URL style (e.g., `"path"`). - **URL_COMPATIBILITY_MODE** (boolean) - Optional - Enables compatibility mode. ### Request Example ```json { "type": "S3", "name": "my-s3-storage", "scope": "s3://my-bucket", "parameters": { "KEY_ID": "AKIAIOSFODNN7EXAMPLE", "SECRET": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY", "REGION": "us-east-1", "ENDPOINT": "s3.amazonaws.com", "USE_SSL": true, "URL_STYLE": "path", "URL_COMPATIBILITY_MODE": false } } ``` ### Response #### Success Response (200) - **success** (string) - Indicates successful registration. - **message** (string) - A message confirming the registration. #### Response Example ```json { "success": "true", "message": "Object storage registered successfully" } ``` ## GET /storages ### Description Lists all registered object storages across all nodes in the cluster. ### Method GET ### Endpoint `/storages` ### Parameters #### Headers - **x-hugr-secret** (string) - Required - The secret key for cluster authentication. ### Response #### Success Response (200) - An array of registered storage objects, each containing: - **node** (string) - The name of the node. - **name** (string) - The name of the storage. - **type** (string) - The type of the storage. - **scope** (array) - The scope of the storage. - **parameters** (string) - Encrypted or summarized parameters. #### Response Example ```json [ { "node": "node1", "name": "my-s3-storage", "type": "S3", "scope": ["s3://my-bucket"], "parameters": "encrypted-params" } ] ``` ## DELETE /storages/{name} ### Description Unregisters a specified object storage from all nodes in the cluster. ### Method DELETE ### Endpoint `/storages/{name}` ### Parameters #### Path Parameters - **name** (string) - Required - The name of the storage to unregister. #### Headers - **x-hugr-secret** (string) - Required - The secret key for cluster authentication. ### Request Example ```bash curl -X DELETE -H "x-hugr-secret: my-cluster-secret" \ http://localhost:14000/storages/my-s3-storage ``` ### Response #### Success Response (200) - **success** (string) - Indicates successful unregistration. - **message** (string) - A message confirming the unregistration. #### Response Example ```json { "success": "true", "message": "Object storage unregistered successfully" } ``` ``` -------------------------------- ### Load Data Source Source: https://context7.com/hugr-lab/hugr/llms.txt Loads a specified data source onto all nodes in the cluster. ```APIDOC ## POST /data-source/{name}/load ### Description Loads a specified data source onto all nodes in the cluster. This handler is part of the management service. ### Method POST ### Endpoint `/data-source/{name}/load` ### Parameters #### Path Parameters - **name** (string) - Required - The name of the data source to load. ### Request Example *(Implicitly called by internal service logic, no direct curl example provided in source)* ### Response #### Success Response (200) - **success** (string) - Indicates successful load. - **message** (string) - A message confirming the load operation. #### Response Example ```json { "success": "true", "message": "Data source loaded successfully" } ``` #### Error Response (500) - Returns an error message if loading fails on any node. #### Error Response Example ``` node node1: error message;\nnode node2: another error message ``` ``` -------------------------------- ### Load Hugr Data Source Source: https://context7.com/hugr-lab/hugr/llms.txt Loads a specific data source on all nodes within the Hugr cluster. This is achieved by sending a POST request to the management node, specifying the data source name and including the cluster secret for authentication. ```bash # Load data source on all nodes curl -H "x-hugr-secret: my-cluster-secret" \ http://localhost:14000/data-source/my-postgres-db/load # Response: # {"success": "true", "message": "Data source loaded successfully"} ``` -------------------------------- ### Service Endpoints - Health and Metrics Source: https://context7.com/hugr-lab/hugr/llms.txt Expose Prometheus metrics and health checks for the service. Includes endpoints for health status and metrics scraping. ```APIDOC ## Service Endpoints - Health and Metrics ### Description Expose Prometheus metrics and health checks for the service. The service listens on a specified port and provides `/health` and `/metrics` endpoints. ### Method Not Applicable (Configuration via code) ### Endpoint `http://localhost:9090` (Default bind address) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```bash # Check health status curl http://localhost:9090/health ``` ```bash # Scrape Prometheus metrics curl http://localhost:9090/metrics ``` ### Response #### Success Response (200) - **/health**: Returns `OK` - **/metrics**: Returns Prometheus metrics (text format) #### Response Example **/health**: ``` OK ``` **/metrics**: ``` # HELP go_goroutines Number of goroutines that currently exist. # TYPE go_goroutines gauge # go_goroutines 42 # HELP go_memstats_alloc_bytes Number of bytes allocated and still in use. # TYPE go_memstats_alloc_bytes gauge # go_memstats_alloc_bytes 7.894016e+06 # ... (more metrics) ``` ``` -------------------------------- ### Register Object Storage via API Source: https://context7.com/hugr-lab/hugr/llms.txt This command registers S3-compatible object storage across all cluster nodes. It uses a POST request with JSON payload containing storage details like type, name, scope, and parameters. Authentication is handled via the 'x-hugr-secret' header. ```bash # Register object storage on all nodes curl -X POST -H "x-hugr-secret: my-cluster-secret" \ -H "Content-Type: application/json" \ -d '{ "type": "S3", "name": "my-s3-storage", "scope": "s3://my-bucket", "parameters": { "KEY_ID": "AKIAIOSFODNN7EXAMPLE", "SECRET": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY", "REGION": "us-east-1", "ENDPOINT": "s3.amazonaws.com", "USE_SSL": true, "URL_STYLE": "path", "URL_COMPATIBILITY_MODE": false } }' \ http://localhost:14000/storages ``` -------------------------------- ### Unload Data Source via API Source: https://context7.com/hugr-lab/hugr/llms.txt This snippet demonstrates how to unload a data source from all nodes using a curl command. It requires the 'x-hugr-secret' header for authentication and specifies the data source to be unloaded. ```bash curl -H "x-hugr-secret: my-cluster-secret" \ http://localhost:14000/data-source/my-postgres-db/unload ``` -------------------------------- ### Register Hugr Worker Node with Management Node Source: https://context7.com/hugr-lab/hugr/llms.txt Internal Go function to register a worker node with the cluster management node. It sends node details (URL, name, version) via POST request and retrieves common configuration from the management node. ```go // Node registration (internal implementation) func RegisterNode(ctx context.Context, c ClusterConfig) (hugr.Config, error) { u, _ := url.Parse(c.ManagementUrl) params := url.Values{ "url": {c.NodeUrl}, // http://localhost:15001/ipc "name": {c.NodeName}, // node1 "version": {Version}, // 0.0.8 } u.RawQuery = params.Encode() u.Path = "/node" req, _ := http.NewRequestWithContext(ctx, http.MethodPost, u.String(), nil) req.Header.Set("x-hugr-secret", c.Secret) resp, err := http.DefaultClient.Do(req) if err != nil { return hugr.Config{}, err } defer resp.Body.Close() // Management node returns common configuration var nc cluster.NodeCommonConfig json.NewDecoder(resp.Body).Decode(&nc) // Node uses management node's auth, CORS, and core-db config return hugr.Config{ AdminUI: true, Debug: nc.DebugMode, CoreDB: coredb.New(nc.CoreDB), Auth: configureFromCluster(nc.Auth), Cors: nc.Cors, }, nil } ``` -------------------------------- ### Unload Data Source Source: https://context7.com/hugr-lab/hugr/llms.txt Unloads a specified data source from all nodes in the cluster. ```APIDOC ## POST /data-source/{name}/unload ### Description Unloads a specified data source from all nodes in the cluster. ### Method POST ### Endpoint `/data-source/{name}/unload` ### Parameters #### Path Parameters - **name** (string) - Required - The name of the data source to unload. #### Headers - **x-hugr-secret** (string) - Required - The secret key for cluster authentication. ### Request Example ```bash curl -H "x-hugr-secret: my-cluster-secret" \ http://localhost:14000/data-source/my-postgres-db/unload ``` ### Response #### Success Response (200) - **success** (string) - Indicates successful unload. - **message** (string) - A message confirming the unload operation. #### Response Example ```json { "success": "true", "message": "Data source unloaded successfully" } ``` ``` -------------------------------- ### Unregister Object Storage via API Source: https://context7.com/hugr-lab/hugr/llms.txt This command removes a registered object storage from all cluster nodes. It uses a DELETE request specifying the storage name and requires the 'x-hugr-secret' header for authentication. ```bash # Unregister storage from all nodes curl -X DELETE -H "x-hugr-secret: my-cluster-secret" \ http://localhost:14000/storages/my-s3-storage ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.