### gRPC Client Usage (grpcurl) Source: https://context7.com/z5labs/megamind/llms.txt Instructions and examples for using the `grpcurl` command-line tool to interact with the Megamind gRPC ingest service. ```APIDOC ## gRPC Client Usage (grpcurl) This section details how to use `grpcurl` to interact with the Megamind gRPC server. ### Prerequisites - Ensure the Megamind gRPC server is running. - Have `grpcurl` installed. ### Commands 1. **Start the gRPC server:** ```bash ingest serve grpc --addr 0.0.0.0:8080 ``` 2. **Ingest a single subgraph using `grpcurl`:** ```bash grpcurl -plaintext -d '{ "triples": [ { "subject": {"type": "Person", "tuid": "1"}, "predicate": {"name": "name"}, "object": {"string": "Alice"} } ] }' localhost:8080 proto.SubgraphIngest/IngestSubgraph ``` **Expected output:** ``` {} ``` (An empty `IngestResponse` indicates success.) ``` -------------------------------- ### gRPC Client Usage (Go) Source: https://context7.com/z5labs/megamind/llms.txt Example Go code demonstrating how to connect to the Megamind gRPC server and ingest subgraphs using both unary and client-streaming calls. ```APIDOC ## gRPC Client Usage (Go) This Go program demonstrates how to interact with the Megamind gRPC service for subgraph ingestion. ### Usage 1. Ensure the Megamind gRPC server is running. 2. Compile and run the Go program. ### Code Example ```go package main import ( "context" "log" "github.com/z5labs/megamind/services/ingest/proto" "github.com/z5labs/megamind/subgraph" "google.golang.org/grpc" "google.golang.org/grpc/credentials/insecure" ) func main() { // Connect to Megamind gRPC server conn, err := grpc.Dial("localhost:8080", grpc.WithTransportCredentials(insecure.NewCredentials())) if err != nil { log.Fatalf("failed to connect: %v", err) } defer conn.Close() client := proto.NewSubgraphIngestClient(conn) ctx := context.Background() // Example 1: Ingest a single subgraph (unary call) sg := &subgraph.Subgraph{ Triples: []*subgraph.Triple{ { Subject: &subgraph.Subject{Type: "Person", Tuid: "1"}, Predicate: &subgraph.Predicate{Name: "name"}, Object: &subgraph.Object{Value: &subgraph.Object_String_{String_: "Alice"}}, }, { Subject: &subgraph.Subject{Type: "Person", Tuid: "1"}, Predicate: &subgraph.Predicate{Name: "age"}, Object: &subgraph.Object{Value: &subgraph.Object_Int64{Int64: 30}}, }, }, } _, err = client.IngestSubgraph(ctx, sg) if err != nil { log.Fatalf("IngestSubgraph failed: %v", err) } log.Println("Single subgraph ingested successfully") // Example 2: Stream multiple subgraphs (client streaming) stream, err := client.Ingest(ctx) if err != nil { log.Fatalf("Ingest stream failed: %v", err) } subgraphs := []*subgraph.Subgraph{ {Triples: []*subgraph.Triple{{Subject: &subgraph.Subject{Type: "Person", Tuid: "2"}, Predicate: &subgraph.Predicate{Name: "name"}, Object: &subgraph.Object{Value: &subgraph.Object_String_{String_: "Bob"}}}}} {Triples: []*subgraph.Triple{{Subject: &subgraph.Subject{Type: "Person", Tuid: "3"}, Predicate: &subgraph.Predicate{Name: "name"}, Object: &subgraph.Object{Value: &subgraph.Object_String_{String_: "Charlie"}}}}} } for _, sg := range subgraphs { if err := stream.Send(sg); err != nil { log.Fatalf("stream.Send failed: %v", err) } } _, err = stream.CloseAndRecv() if err != nil { log.Fatalf("stream.CloseAndRecv failed: %v", err) } log.Println("Batch subgraphs ingested successfully") } ``` ``` -------------------------------- ### CLI Commands - Ingest Service Source: https://context7.com/z5labs/megamind/llms.txt Commands for starting the ingest service as either a gRPC or HTTP server. ```APIDOC ## CLI Commands ### Ingest Service Commands The ingest service binary provides commands to start gRPC or HTTP servers for receiving subgraph data. #### Start gRPC server - **Default port (8080):** ```bash ingest serve grpc ``` - **Custom address:** ```bash ingest serve grpc --addr 0.0.0.0:9090 ``` #### Start HTTP server - **Default port (8080):** ```bash ingest serve http ``` - **Custom address:** ```bash ingest serve http --addr 0.0.0.0:9090 ``` ``` -------------------------------- ### Example Subgraph in JSON Format Source: https://context7.com/z5labs/megamind/llms.txt Illustrates how a knowledge graph subgraph can be represented in JSON format, adhering to the defined Protobuf structure. ```json { "triples": [ { "subject": {"type": "Person", "tuid": "1"}, "predicate": {"name": "name"}, "object": {"string": "Alice"} }, { "subject": {"type": "Person", "tuid": "1"}, "predicate": {"name": "age"}, "object": {"int64": 30} }, { "subject": {"type": "Person", "tuid": "1"}, "predicate": {"name": "works_at"}, "object": {"subject": {"type": "Company", "tuid": "100"}} } ] } ``` -------------------------------- ### Configure Terraform Deployment Source: https://context7.com/z5labs/megamind/llms.txt Example usage of the Megamind Terraform module for Kubernetes and Knative deployment. ```hcl # terraform/main.tf example usage module "megamind" { source = "./modules/megamind" # Kubernetes configuration is handled by the module # Knative Serving and Eventing are set up automatically } # The deployment includes: # - Knative Operator (v1.7.0) # - Knative Serving for auto-scaling # - Knative Eventing for event-driven architecture # - Kubernetes Dashboard (v2.6.1) ``` -------------------------------- ### Ingest Service CLI Commands Source: https://context7.com/z5labs/megamind/llms.txt Commands to start the ingest service as either a gRPC or HTTP server. Allows specifying custom addresses and ports. ```bash # Start gRPC server on default port (8080) ingest serve grpc # Start gRPC server on custom address ingest serve grpc --addr 0.0.0.0:9090 # Start HTTP server on default port (8080) ingest serve http # Start HTTP server on custom address ingest serve http --addr 0.0.0.0:9090 ``` -------------------------------- ### Define Subgraph Data Structure Source: https://context7.com/z5labs/megamind/llms.txt Example JSON format for representing triples in subgraphs. ```json {"triples":[{"subject":{"type":"Person","tuid":"1"},"predicate":{"name":"name"},"object":{"string":"Alice"}}]} {"triples":[{"subject":{"type":"Person","tuid":"2"},"predicate":{"name":"name"},"object":{"string":"Bob"}}]} {"triples":[{"subject":{"type":"Person","tuid":"3"},"predicate":{"name":"name"},"object":{"string":"Charlie"}}]} ``` -------------------------------- ### Implement SubgraphIngester Server Source: https://context7.com/z5labs/megamind/llms.txt Initializes the SubgraphIngester with both gRPC and HTTP handlers, including graceful shutdown logic. ```go package main import ( "context" "net" "os" "os/signal" "syscall" "github.com/z5labs/megamind/services/ingest/grpc" "github.com/z5labs/megamind/services/ingest/http" "github.com/z5labs/megamind/services/ingest/ingest" "go.uber.org/zap" ) func main() { logger, _ := zap.NewProduction() defer logger.Sync() // Create the core SubgraphIngester ingester := ingest.NewSubgraphIngester(logger) // Setup graceful shutdown ctx, cancel := context.WithCancel(context.Background()) defer cancel() sigCh := make(chan os.Signal, 1) signal.Notify(sigCh, syscall.SIGINT, syscall.SIGTERM) go func() { <-sigCh logger.Info("shutting down...") cancel() }() // Option 1: Start gRPC server grpcListener, _ := net.Listen("tcp", ":8080") go func() { if err := grpc.Serve(ctx, grpcListener, ingester); err != nil { logger.Error("gRPC server error", zap.Error(err)) } }() // Option 2: Start HTTP server httpIngester := http.NewSubgraphIngester(logger, ingester) httpListener, _ := net.Listen("tcp", ":8081") if err := httpIngester.Serve(ctx, httpListener); err != nil { logger.Error("HTTP server error", zap.Error(err)) } } ``` -------------------------------- ### Build All Megamind Components Source: https://github.com/z5labs/megamind/blob/main/CONTRIBUTING.md Use Bazel to build all components of the Megamind project. This command is used after cloning and setting up the repository. ```bash bazel build //... ``` -------------------------------- ### Go gRPC Client for Subgraph Ingestion Source: https://context7.com/z5labs/megamind/llms.txt Demonstrates how to connect to the Megamind gRPC server and ingest subgraphs using both unary and client-streaming calls. Ensure the gRPC server is running before executing. ```go package main import ( "context" "log" "github.com/z5labs/megamind/services/ingest/proto" "github.com/z5labs/megamind/subgraph" "google.golang.org/grpc" "google.golang.org/grpc/credentials/insecure" ) func main() { // Connect to Megamind gRPC server conn, err := grpc.Dial("localhost:8080", grpc.WithTransportCredentials(insecure.NewCredentials())) if err != nil { log.Fatalf("failed to connect: %v", err) } defer conn.Close() client := proto.NewSubgraphIngestClient(conn) ctx := context.Background() // Example 1: Ingest a single subgraph (unary call) sg := &subgraph.Subgraph{ Triples: []*subgraph.Triple{ { Subject: &subgraph.Subject{Type: "Person", Tuid: "1"}, Predicate: &subgraph.Predicate{Name: "name"}, Object: &subgraph.Object{Value: &subgraph.Object_String_{String_: "Alice"}}, }, { Subject: &subgraph.Subject{Type: "Person", Tuid: "1"}, Predicate: &subgraph.Predicate{Name: "age"}, Object: &subgraph.Object{Value: &subgraph.Object_Int64{Int64: 30}}, }, }, } _, err = client.IngestSubgraph(ctx, sg) if err != nil { log.Fatalf("IngestSubgraph failed: %v", err) } log.Println("Single subgraph ingested successfully") // Example 2: Stream multiple subgraphs (client streaming) stream, err := client.Ingest(ctx) if err != nil { log.Fatalf("Ingest stream failed: %v", err) } subgraphs := []*subgraph.Subgraph{ {Triples: []*subgraph.Triple{{ Subject: &subgraph.Subject{Type: "Person", Tuid: "2"}, Predicate: &subgraph.Predicate{Name: "name"}, Object: &subgraph.Object{Value: &subgraph.Object_String_{String_: "Bob"}}, }}}, {Triples: []*subgraph.Triple{{ Subject: &subgraph.Subject{Type: "Person", Tuid: "3"}, Predicate: &subgraph.Predicate{Name: "name"}, Object: &subgraph.Object{Value: &subgraph.Object_String_{String_: "Charlie"}}, }}}, } for _, sg := range subgraphs { if err := stream.Send(sg); err != nil { log.Fatalf("stream.Send failed: %v", err) } } _, err = stream.CloseAndRecv() if err != nil { log.Fatalf("stream.CloseAndRecv failed: %v", err) } log.Println("Batch subgraphs ingested successfully") } ``` -------------------------------- ### Clone Megamind Repository Source: https://github.com/z5labs/megamind/blob/main/CONTRIBUTING.md Clone the Megamind repository to your local development machine. This is the first step in setting up the development environment. ```bash git clone https://github.com/z5labs/megamind.git ``` -------------------------------- ### HTTP REST API for Subgraph Ingestion Source: https://context7.com/z5labs/megamind/llms.txt Demonstrates ingesting a subgraph using an HTTP POST request. The request body must be JSON-formatted protobuf. Ensure the HTTP server is running. ```bash # Start the HTTP server ingest serve http --addr 0.0.0.0:8080 # Ingest a subgraph via curl curl -X POST http://localhost:8080/subgraph/ingest \ -H "Content-Type: application/json" \ -d '{ "triples": [ { "subject": {"type": "Person", "tuid": "1"}, "predicate": {"name": "name"}, "object": {"string": "Alice"} }, { "subject": {"type": "Person", "tuid": "1"}, "predicate": {"name": "email"}, "object": {"string": "alice@example.com"} }, { "subject": {"type": "Person", "tuid": "1"}, "predicate": {"name": "follows"}, "object": {"subject": {"type": "Person", "tuid": "2"}} } ] }' # Expected: HTTP 200 on success, HTTP 500 on error ``` -------------------------------- ### Run All Megamind Unit Tests Source: https://github.com/z5labs/megamind/blob/main/CONTRIBUTING.md Execute all unit tests within the Megamind project using Bazel. This command is essential for verifying code changes. ```bash bazel test //... ``` -------------------------------- ### Megamind CLI Tool for Dgraph Ingestion Source: https://context7.com/z5labs/megamind/llms.txt Utilities for direct ingestion to Dgraph using the megamind CLI tool. Supports ingesting from JSON files, stdin, and with protobuf encoding. ```bash # Ingest subgraphs from a JSON file (one subgraph per line) megamind dgraph ingest subgraphs ./data/subgraphs.json # Ingest subgraphs from stdin cat ./data/subgraphs.json | megamind dgraph ingest subgraphs - # Ingest using protobuf encoding megamind dgraph ingest subgraphs --encoding proto ./data/subgraphs.pb # Example input file format (JSON Lines - one subgraph per line) ``` -------------------------------- ### Subgraph Ingest gRPC Service Definition (Protobuf) Source: https://context7.com/z5labs/megamind/llms.txt Defines the gRPC service for ingesting subgraphs. Includes methods for single subgraph ingestion (unary) and batch ingestion (client streaming). ```protobuf service SubgraphIngest { // Ingest a single subgraph (unary) rpc IngestSubgraph (subgraph.Subgraph) returns (IngestResponse); // Ingest multiple subgraphs via client streaming rpc Ingest (stream subgraph.Subgraph) returns (IngestResponse); } message IngestResponse {} ``` -------------------------------- ### CLI Commands - Megamind CLI Tool Source: https://context7.com/z5labs/megamind/llms.txt Commands for using the Megamind CLI tool to ingest data directly into Dgraph. ```APIDOC ### Megamind CLI Tool The megamind CLI tool provides utilities for direct ingestion to Dgraph from files or stdin. #### Ingest subgraphs from a JSON file - **JSON Lines format (one subgraph per line):** ```bash megamind dgraph ingest subgraphs ./data/subgraphs.json ``` #### Ingest subgraphs from stdin ```bash cat ./data/subgraphs.json | megamind dgraph ingest subgraphs - ``` #### Ingest using protobuf encoding ```bash megamind dgraph ingest subgraphs --encoding proto ./data/subgraphs.pb ``` #### Example input file format (JSON Lines) ```json { "triples": [ { "subject": {"type": "Person", "tuid": "1"}, "predicate": {"name": "name"}, "object": {"string": "Alice"} } ] } { "triples": [ { "subject": {"type": "Person", "tuid": "2"}, "predicate": {"name": "name"}, "object": {"string": "Bob"} } ] } ``` ``` -------------------------------- ### grpcurl for gRPC Subgraph Ingestion Source: https://context7.com/z5labs/megamind/llms.txt Utilizes grpcurl to interact with the gRPC ingest service. This command-line tool allows for testing gRPC endpoints without writing client code. Ensure the gRPC server is running. ```bash # Start the gRPC server ingest serve grpc --addr 0.0.0.0:8080 # Ingest a single subgraph using grpcurl grpcurl -plaintext -d '{ "triples": [ { "subject": {"type": "Person", "tuid": "1"}, "predicate": {"name": "name"}, "object": {"string": "Alice"} } ] }' localhost:8080 proto.SubgraphIngest/IngestSubgraph # Expected output: {} (empty IngestResponse on success) ``` -------------------------------- ### HTTP REST API - POST /subgraph/ingest Source: https://context7.com/z5labs/megamind/llms.txt Documentation for the HTTP REST API endpoint used to ingest a single subgraph. ```APIDOC ## HTTP REST API ### POST /subgraph/ingest Ingests a single subgraph via HTTP POST. The request body must be JSON-formatted protobuf matching the Subgraph message structure. #### Method POST #### Endpoint `/subgraph/ingest` #### Request Body - **triples** (array[Subgraph.Triple]) - Required - An array of triples representing the subgraph. - **subject** (Subgraph.Subject) - Required - The subject of the triple. - **type** (string) - Required - The type of the subject. - **tuid** (string) - Required - The unique identifier for the subject. - **predicate** (Subgraph.Predicate) - Required - The predicate of the triple. - **name** (string) - Required - The name of the predicate. - **object** (Subgraph.Object) - Required - The object of the triple. - **string** (string) - Optional - String value for the object. - **int64** (integer) - Optional - Int64 value for the object. - **subject** (Subgraph.Subject) - Optional - Nested subject value for the object. #### Request Example ```json { "triples": [ { "subject": {"type": "Person", "tuid": "1"}, "predicate": {"name": "name"}, "object": {"string": "Alice"} }, { "subject": {"type": "Person", "tuid": "1"}, "predicate": {"name": "email"}, "object": {"string": "alice@example.com"} }, { "subject": {"type": "Person", "tuid": "1"}, "predicate": {"name": "follows"}, "object": {"subject": {"type": "Person", "tuid": "2"}} } ] } ``` #### Response - **Success Response (200 OK)**: Returns an empty JSON object `{}` on successful ingestion. - **Error Response (500 Internal Server Error)**: Returned if an error occurs during ingestion. #### Example Usage (curl) ```bash curl -X POST http://localhost:8080/subgraph/ingest \ -H "Content-Type: application/json" \ -d '{ "triples": [ { "subject": {"type": "Person", "tuid": "1"}, "predicate": {"name": "name"}, "object": {"string": "Alice"} }, { "subject": {"type": "Person", "tuid": "1"}, "predicate": {"name": "email"}, "object": {"string": "alice@example.com"} }, { "subject": {"type": "Person", "tuid": "1"}, "predicate": {"name": "follows"}, "object": {"subject": {"type": "Person", "tuid": "2"}} } ] }' ``` ``` -------------------------------- ### gRPC SubgraphIngest Service Source: https://context7.com/z5labs/megamind/llms.txt The SubgraphIngest service allows for the ingestion of knowledge graph data through unary or streaming gRPC calls. ```APIDOC ## gRPC SubgraphIngest ### Description The SubgraphIngest service provides methods to ingest subgraphs into the Knowledge Graph. It supports both single subgraph ingestion and batch ingestion via client streaming. ### Methods - **IngestSubgraph**: Unary call to ingest a single Subgraph. - **Ingest**: Client-streaming call to ingest multiple Subgraphs. ### Request Body (Subgraph) - **triples** (repeated Triple) - A collection of Triple messages representing knowledge relationships. ### Triple Structure - **subject** (Subject) - The entity (type and tuid). - **predicate** (Predicate) - The relationship name. - **object** (Object) - The value or reference (Subject, string, int64, or float64). ### Request Example { "triples": [ { "subject": {"type": "Person", "tuid": "1"}, "predicate": {"name": "name"}, "object": {"string": "Alice"} } ] } ### Response #### Success Response - **IngestResponse** (message) - Empty response indicating successful ingestion. ``` -------------------------------- ### Subgraph Data Model Definition (Protobuf) Source: https://context7.com/z5labs/megamind/llms.txt Defines the structure for Subgraphs, Triples, Subjects, Predicates, and Objects used in Megamind. This schema is used for data serialization and communication. ```protobuf message Subgraph { repeated Triple triples = 1; } message Triple { Subject subject = 1; Predicate predicate = 2; Object object = 3; } message Subject { string type = 1; // Entity type (e.g., "Person", "Company") string tuid = 2; // Type-unique identifier } message Predicate { string name = 1; // Relationship name (e.g., "name", "works_at") } message Object { oneof value { Subject subject = 1; // Reference to another entity string string = 2; // String value int64 int64 = 3; // Integer value double float64 = 4; // Floating-point value } } ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.