### Setup and Run Python Plugin for KV CLI Source: https://github.com/hashicorp/go-plugin/blob/main/examples/grpc/README.md Sets up a Python virtual environment, installs dependencies, and configures the KV CLI to use a Python plugin. The KV_PLUGIN environment variable is set to point to the Python script. ```shell $ python -m venv plugin-python/.venv $ source plugin-python/.venv/bin/activate $ pip install -r plugin-python/requirements.txt $ export KV_PLUGIN="python plugin-python/plugin.py" ``` -------------------------------- ### Build and Run KV CLI and Go gRPC Plugin Source: https://github.com/hashicorp/go-plugin/blob/main/examples/grpc/README.md Builds the main KV CLI, the Go gRPC plugin, and sets the environment variable to use the gRPC plugin. Demonstrates basic put and get operations. ```shell # This builds the main CLI $ go build -o kv # This builds the plugin written in Go $ go build -o kv-go-grpc ./plugin-go-grpc # This tells the KV binary to use the "kv-go-grpc" binary $ export KV_PLUGIN="./kv-go-grpc" # Read and write $ ./kv put hello world $ ./kv get hello world ``` -------------------------------- ### Build and Run KV CLI and Plugin Source: https://github.com/hashicorp/go-plugin/blob/main/examples/negotiated/README.md Commands to build the main CLI executable and the Go-written plugin. Demonstrates setting and getting key-value pairs using different protocol versions (grpc and netrpc) and protobuf versions (3 and 2). ```shell # This builds the main CLI $ go build -o kv # This builds the plugin written in Go $ go build -o kv-plugin ./plugin-go # Write a value using proto version 3 and grpc $ KV_PROTO=grpc ./kv put hello world # Read it back using proto version 2 and netrpc $ KV_PROTO=netrpc ./kv get hello world Written from plugin version 3 Read by plugin version 2 ``` -------------------------------- ### Build and Run Counter CLI and Go gRPC Plugin Source: https://github.com/hashicorp/go-plugin/blob/main/examples/bidirectional/README.md Commands to build the main counter CLI and the counter-go-grpc plugin. It also shows how to set the environment variable to specify the plugin binary and demonstrates basic put and get operations for the counter. ```shell # This builds the main CLI $ go build -o counter # This builds the plugin written in Go $ go build -o counter-go-grpc ./plugin-go-grpc # This tells the Counter binary to use the "counter-go-grpc" binary $ export COUNTER_PLUGIN="./counter-go-grpc" # Read and write $ ./counter put hello 1 $ ./counter put hello 1 $ ./counter get hello 2 ``` -------------------------------- ### Initialize Plugin Client Configuration Source: https://github.com/hashicorp/go-plugin/blob/main/docs/extensive-go-plugin-tutorial.md Initializes a new plugin client by configuring handshake settings, plugin map, the command to execute the plugin binary, and a logger. The client is deferred to be killed after use. This is the host-side setup for launching a plugin process. ```go // We're a host! Start by launching the plugin process. client := plugin.NewClient(&plugin.ClientConfig{ HandshakeConfig: handshakeConfig, Plugins: pluginMap, Cmd: exec.Command("./plugin/greeter"), Logger: logger, }) defer client.Kill() ``` -------------------------------- ### Go GRPC Stub Code Examples Source: https://github.com/hashicorp/go-plugin/blob/main/docs/extensive-go-plugin-tutorial.md Examples of code generated by protoc for GRPC communication. Includes a method to get the message from a GreetResponse, the ServiceClient interface definition, and the function to register the GreeterServiceServer. ```go func (m *GreetResponse) GetMessage() string { if m != nil { return m.Message } return "" } ``` ```go type GreeterServiceClient interface { Greet(ctx context.Context, in *Empty, opts ...grpc.CallOption) (*GreetResponse, error) } ``` ```go func RegisterGreeterServiceServer(s *grpc.Server, srv GreeterServiceServer) { s.RegisterService(&_GreeterService_serviceDesc, srv) } ``` -------------------------------- ### Go Plugin Main Function: Serving a Greeter Plugin Source: https://github.com/hashicorp/go-plugin/blob/main/docs/extensive-go-plugin-tutorial.md The main function setup for serving a Go plugin. It initializes a logger, creates an instance of the GreeterHello plugin, and registers it using plugin.Serve. This requires hclog, os, and plugin packages. ```go logger := hclog.New(&hclog.LoggerOptions{ Level: hclog.Trace, Output: os.Stderr, JSONFormat: true, }) greeter := &GreeterHello{ logger: logger, } // pluginMap is the map of plugins we can dispense. var pluginMap = map[string]plugin.Plugin{ "greeter": &shared.GreeterPlugin{Impl: greeter}, } logger.Debug("message from plugin", "foo", "bar") plugin.Serve(&plugin.ServeConfig{ HandshakeConfig: handshakeConfig, Plugins: pluginMap, }) ``` -------------------------------- ### Regenerate Protocol Buffers Files Source: https://github.com/hashicorp/go-plugin/blob/main/examples/bidirectional/README.md Command to regenerate protocol buffers files if the .proto files are updated. This is typically only needed when modifying the communication protocol between the main process and the plugin. ```shell $ buf generate ``` -------------------------------- ### Build and Run KV CLI and Go net/rpc Plugin Source: https://github.com/hashicorp/go-plugin/blob/main/examples/grpc/README.md Builds the main KV CLI and a Go plugin using the net/rpc mechanism. Sets the environment variable to specify the net/rpc plugin for the KV binary. ```shell # This builds the plugin written in Go $ go build -o kv-go-netrpc ./plugin-go-netrpc # This tells the KV binary to use the "kv-go-netrpc" binary $ export KV_PLUGIN="./kv-go-netrpc" ``` -------------------------------- ### Go Plugin Implementation: Greeter Source: https://github.com/hashicorp/go-plugin/blob/main/docs/extensive-go-plugin-tutorial.md A basic implementation of a Greeter plugin in Go. It defines a struct that satisfies a plugin interface and includes a Greet method. This example requires a logger from the hclog library. ```go // Here is a real implementation of Greeter type GreeterHello struct { logger hclog.Logger } func (g *GreeterHello) Greet() string { g.logger.Debug("message from GreeterHello.Greet") return "Hello!" } ``` -------------------------------- ### Build Go gRPC Plugin and Set Counter Plugin Path Source: https://github.com/hashicorp/go-plugin/blob/main/examples/bidirectional/README.md Instructions for building the Go gRPC plugin and setting the COUNTER_PLUGIN environment variable to direct the main binary to use this specific plugin. ```shell # This builds the plugin written in Go $ go build -o counter-go-grpc ./plugin-go-grpc # This tells the KV binary to use the "kv-go-grpc" binary $ export COUNTER_PLUGIN="./counter-go-grpc" ``` -------------------------------- ### Python Plugin Output for go-plugin Source: https://github.com/hashicorp/go-plugin/blob/main/docs/extensive-go-plugin-tutorial.md An example of how a Python plugin might manually output the required information to stdout for go-plugin to detect. This is necessary for non-Go plugins to communicate their availability and network details. ```python # Output information print("1|1|tcp|127.0.0.1:1234|grpc") sys.stdout.flush() ``` -------------------------------- ### Setup Go GRPC Plugin Logger Source: https://github.com/hashicorp/go-plugin/blob/main/docs/extensive-go-plugin-tutorial.md Initializes a logger for a Go GRPC plugin using the hclog library. This logger is configured to output debug-level messages to standard output and is essential for communication and tracing within the plugin framework. ```go import ( "os" "github.com/hashicorp/go-hclog" ) // ... // Create an hclog.Logger logger := hclog.New(&hclog.LoggerOptions{ Name: "plugin", Output: os.Stdout, Level: hclog.Debug, }) ``` -------------------------------- ### Configure Host Application to Load Python Plugin Source: https://github.com/hashicorp/go-plugin/blob/main/docs/guide-plugin-write-non-go.md Example of how to configure the host application to execute the Python plugin using an environmental variable. This is application-specific. ```sh $ export KV_PLUGIN="python plugin.py" ``` -------------------------------- ### Serve gRPC Service in Python Source: https://github.com/hashicorp/go-plugin/blob/main/docs/guide-plugin-write-non-go.md Creates and starts a gRPC server in Python, adding the KVServicer and listening on a specified port. This allows the plugin to be served and connected to by the host application. ```python # Make the server server = grpc.server(futures.ThreadPoolExecutor(max_workers=10)) # Add our service kv_pb2_grpc.add_KVServicer_to_server(KVServicer(), server) # Listen on a port server.add_insecure_port(':1234') # Start server.start() ``` -------------------------------- ### Implement Greeter gRPC Client in Go Source: https://github.com/hashicorp/go-plugin/blob/main/docs/extensive-go-plugin-tutorial.md Implements the 'Greeter' interface for gRPC communication. This client utilizes `proto.GreeterClient` generated by protoc for making gRPC calls. ```go // GRPCClient is an implementation of KV that talks over RPC. type GreeterGRPC struct{ client proto.GreeterClient } func (m *GreeterGRPC) Greet() (string, error) { s, err := m.client.Greet(context.Background(), &proto.Empty{}) return s, err } ``` -------------------------------- ### Implement Greeter RPC Server in Go Source: https://github.com/hashicorp/go-plugin/blob/main/docs/extensive-go-plugin-tutorial.md Defines the RPC server structure for the 'Greeter' plugin. It holds a concrete implementation of the 'Greeter' interface and exposes it via RPC. ```go // Here is the RPC server that GreeterRPC talks to, conforming to // the requirements of net/rpc type GreeterRPCServer struct { // This is the real implementation Impl Greeter } func (s *GreeterRPCServer) Greet(args interface{}, resp *string) error { *resp = s.Impl.Greet() return nil } ``` -------------------------------- ### Implement Greeter RPC Client in Go Source: https://github.com/hashicorp/go-plugin/blob/main/docs/extensive-go-plugin-tutorial.md Implements the 'Greeter' interface for RPC communication. This client uses `rpc.Client` to make remote calls to the 'Plugin.Greet' method. ```go // Here is an implementation that talks over RPC type GreeterRPC struct { client *rpc.Client } func (g *GreeterRPC) Greet() string { var resp string err := g.client.Call("Plugin.Greet", new(interface{}), &resp) if err != nil { // You usually want your interfaces to return errors. If they don't, // there isn't much other choice here. panic(err) } return resp } ``` -------------------------------- ### Call and Dispense a Go Plugin using gRPC in Host Application Source: https://github.com/hashicorp/go-plugin/blob/main/docs/extensive-go-plugin-tutorial.md This Go code snippet demonstrates how a host application initiates and communicates with a plugin. It uses `plugin.NewClient` to configure the client, specifying the plugin's executable, handshake configuration, and importantly, allowing only the gRPC protocol. The `dispense` method is then used to obtain an instance of the plugin's interface. ```go // We're a host. Start by launching the plugin process. client := plugin.NewClient(&plugin.ClientConfig{ HandshakeConfig: shared.Handshake, Plugins: shared.PluginMap, Cmd: exec.Command("./plugin/greeter"), AllowedProtocols: []plugin.Protocol{plugin.ProtocolGRPC}, }) ``` -------------------------------- ### Go GRPC Server Implementation Source: https://github.com/hashicorp/go-plugin/blob/main/docs/extensive-go-plugin-tutorial.md Defines the GRPCServer struct and its Greet method. This server receives requests, calls the underlying plugin implementation's Greet method, and returns the result formatted as a proto.GreeterResponse. ```go // Here is the gRPC server that GRPCClient talks to. type GRPCServer struct { // This is the real implementation Impl Greeter } func (m *GRPCServer) Greet( ctx context.Context, req *proto.Empty) *proto.GreeterResponse { v := m.Impl.Greet() return &proto.GreeterResponse{Message: v} } ``` -------------------------------- ### Define Greeter Interface in Go Source: https://github.com/hashicorp/go-plugin/blob/main/docs/extensive-go-plugin-tutorial.md Defines the core 'Greeter' interface with a single 'Greet' method that returns a string. This interface specifies the contract for plugins and clients. ```go // Greeter is the interface that we're exposing as a plugin. type Greeter interface { Greet() string } ``` -------------------------------- ### Use MuxBroker for Complex Arguments with go-plugin Source: https://context7.com/hashicorp/go-plugin/llms.txt Illustrates how to use `plugin.MuxBroker` to manage multiple independent connections for a single plugin client. This enables streaming data or passing complex arguments by establishing separate communication channels managed by the broker. The example shows server-side acceptance and client-side dialing for these multiplexed connections. ```go package main import ( "io" "net/rpc" "github.com/hashicorp/go-plugin" ) // Service that needs to pass an io.Reader type FileProcessor interface { Process(r io.Reader) (int, error) } // RPC Server implementation type FileProcessorRPCServer struct { Impl FileProcessor broker *plugin.MuxBroker } type ProcessArgs struct { StreamID uint32 } type ProcessResponse struct { Count int } func (s *FileProcessorRPCServer) Process(args *ProcessArgs, resp *ProcessResponse) error { // Accept the streamed connection conn, err := s.broker.Accept(args.StreamID) if err != nil { return err } defer conn.Close() // Now we have a raw connection for streaming data count, err := s.Impl.Process(conn) if err != nil { return err } resp.Count = count return nil } // RPC Client implementation type FileProcessorRPCClient struct { client *rpc.Client broker *plugin.MuxBroker } func (c *FileProcessorRPCClient) Process(r io.Reader) (int, error) { // Get a unique ID for this stream streamID := c.broker.NextId() // Start goroutine to serve the reader on that ID go func() { conn, err := c.broker.Dial(streamID) if err != nil { return } defer conn.Close() // Copy the reader data to the connection io.Copy(conn, r) }() // Make the RPC call with the stream ID var resp ProcessResponse err := c.client.Call("Plugin.Process", &ProcessArgs{ StreamID: streamID, }, &resp) return resp.Count, err } // Plugin implementation type FileProcessorPlugin struct { Impl FileProcessor } func (p *FileProcessorPlugin) Server(b *plugin.MuxBroker) (interface{}, error) { return &FileProcessorRPCServer{ Impl: p.Impl, broker: b, }, nil } func (p *FileProcessorPlugin) Client(b *plugin.MuxBroker, c *rpc.Client) (interface{}, error) { return &FileProcessorRPCClient{ client: c, broker: b, }, nil } ``` -------------------------------- ### Implement and Serve a Go Plugin using gRPC Source: https://github.com/hashicorp/go-plugin/blob/main/docs/extensive-go-plugin-tutorial.md This Go code defines a simple 'Greeter' plugin that implements a 'Greet' method. It then configures and serves this plugin using go-plugin, specifying gRPC as the communication protocol. This serves as the server-side implementation of the plugin. ```go package main import ( "github.com/hashicorp/go-plugin" "my/shared/package" "os" "os/exec" ) // Here is a real implementation of KV that writes to a local file with // the key name and the contents are the value of the key. type Greeter struct{} func (Greeter) Greet() error { return "Hello!" } func main() { plugin.Serve(&plugin.ServeConfig{ HandshakeConfig: shared.Handshake, Plugins: map[string]plugin.Plugin{ "greeter": &shared.GreeterGRPCPlugin{Impl: &Greeter{}}, }, // A non-nil value here enables gRPC serving for this plugin... GRPCServer: plugin.DefaultGRPCServer, }) } ``` -------------------------------- ### Map Plugins for Dispensing Source: https://github.com/hashicorp/go-plugin/blob/main/docs/extensive-go-plugin-tutorial.md Creates a map where keys are plugin names and values are plugin implementations. This map is used by the host to identify and dispense specific plugins, enabling the discovery and instantiation of services provided by the plugin. ```go // pluginMap is the map of plugins we can dispense. var pluginMap = map[string]plugin.Plugin{"greeter": &shared.GreeterPlugin{}} ``` -------------------------------- ### Enable Automatic Mutual TLS (mTLS) with go-plugin Source: https://context7.com/hashicorp/go-plugin/llms.txt This Go example demonstrates how to enable automatic mutual TLS (mTLS) for secure plugin communication using go-plugin. The client configuration includes `AutoMTLS: true`, which automatically sets up encrypted and authenticated communication channels. The server-side `plugin.Serve` function also supports mTLS. ```go package main import ( "os/exec" "github.com/hashicorp/go-plugin" ) func main() { // Client with AutoMTLS client := plugin.NewClient(&plugin.ClientConfig{ HandshakeConfig: plugin.HandshakeConfig{ ProtocolVersion: 1, MagicCookieKey: "SECURE_PLUGIN", MagicCookieValue: "verysecret", }, Plugins: pluginMap, Cmd: exec.Command("./secure-plugin"), AutoMTLS: true, // Enable automatic mTLS AllowedProtocols: []plugin.Protocol{ plugin.ProtocolGRPC, }, }) defer client.Kill() // Connection is now encrypted with mutual authentication rpcClient, err := client.Client() if err != nil { panic(err) } // Dispense and use plugin securely raw, err := rpcClient.Dispense("my_service") if err != nil { panic(err) } // All RPC communication is encrypted with mTLS service := raw.(MyService) result := service.DoWork() } // Server side plugin func servePlugin() { plugin.Serve(&plugin.ServeConfig{ HandshakeConfig: plugin.HandshakeConfig{ ProtocolVersion: 1, MagicCookieKey: "SECURE_PLUGIN", MagicCookieValue: "verysecret", }, Plugins: pluginMap, GRPCServer: plugin.DefaultGRPCServer, // AutoMTLS is handled automatically from environment variables }) } ``` -------------------------------- ### Implement KV Service in Python Source: https://github.com/hashicorp/go-plugin/blob/main/docs/guide-plugin-write-non-go.md Implements the gRPC KV service with Get and Put methods. It reads from and writes to files based on the provided key and value. This is a standard gRPC service implementation. ```python class KVServicer(kv_pb2_grpc.KVServicer): """Implementation of KV service.""" def Get(self, request, context): filename = "kv_"+request.key with open(filename, 'r') as f: result = kv_pb2.GetResponse() result.value = f.read() return result def Put(self, request, context): filename = "kv_"+request.key value = "{0}\n\nWritten from plugin-python".format(request.value) with open(filename, 'w') as f: f.write(value) return kv_pb2.Empty() ``` -------------------------------- ### Implement Greeter Plugin Structure in Go Source: https://github.com/hashicorp/go-plugin/blob/main/docs/extensive-go-plugin-tutorial.md Defines the `GreeterPlugin` struct, which implements the `plugin.Plugin` interface. It facilitates serving an RPC server and consuming an RPC client for the Greeter plugin. ```go // This is the implementation of plugin.Plugin so we can serve/consume this // // This has two methods: Server must return an RPC server for this plugin // type. We construct a GreeterRPCServer for this. // // Client must return an implementation of our interface that communicates // over an RPC client. We return GreeterRPC for this. // // Ignore MuxBroker. That is used to create more multiplexed streams on our // plugin connection and is a more advanced use case. type GreeterPlugin struct { // Impl Injection Impl Greeter } func (p *GreeterPlugin) Server(*plugin.MuxBroker) (interface{}, error) { return &GreeterRPCServer{Impl: p.Impl}, nil } func (GreeterPlugin) Client(b *plugin.MuxBroker, c *rpc.Client) (interface{}, error) { return &GreeterRPC{client: c}, nil } ``` -------------------------------- ### Go GRPC Client Implementation Source: https://github.com/hashicorp/go-plugin/blob/main/docs/extensive-go-plugin-tutorial.md Defines the GRPCClient struct and its Greet method. This client uses the generated proto.GreeterClient to make remote calls and retrieve the message from the GreetResponse. proto.Empty is used when no parameters or return values are needed. ```go // GRPCClient is an implementation of Greeter that talks over RPC. type GRPCClient struct{ client proto.GreeterClient } func (m *GRPCClient) Greet() string { ret, _ := m.client.Greet(context.Background(), &proto.Empty{}) return ret.Message } ``` -------------------------------- ### Go Plugin Interface Definition Source: https://github.com/hashicorp/go-plugin/blob/main/docs/extensive-go-plugin-tutorial.md Defines the Greeter interface that plugins will implement and the GreeterGRPCPlugin struct which integrates with the go-plugin framework. It includes methods for serving and consuming GRPC plugins. ```go // Greeter is the interface that we're exposing as a plugin. type Greeter interface { Greet() string } // This is the implementation of plugin.GRPCPlugin so we can serve/consume this. type GreeterGRPCPlugin struct { // GRPCPlugin must still implement the Plugin interface plugin.Plugin // Concrete implementation, written in Go. This is only used for plugins // that are written in Go. Impl Greeter } func (p *GreeterGRPCPlugin) GRPCServer(broker *plugin.GRPCBroker, s *grpc.Server) error { proto.RegisterGreeterServer(s, &GRPCServer{Impl: p.Impl}) return nil } func (p *GreeterGRPCPlugin) GRPCClient(ctx context.Context, broker *plugin.GRPCBroker, c *grpc.ClientConn) (interface{}, error) { return &GRPCClient{client: proto.NewGreeterClient(c)}, nil } ``` -------------------------------- ### Generate GRPC Code with Protoc Source: https://github.com/hashicorp/go-plugin/blob/main/docs/extensive-go-plugin-tutorial.md Command to generate Go GRPC stubs from a .proto file. This command requires the protobuf compiler (protoc) and the Go GRPC plugin. The generated code is placed in the 'proto' directory. ```bash protoc -I proto/ proto/greeter.proto --go_out=plugins=grpc:proto ``` -------------------------------- ### Establish RPC Connection to Plugin Source: https://github.com/hashicorp/go-plugin/blob/main/docs/extensive-go-plugin-tutorial.md Connects to the plugin process via RPC to enable communication. If an error occurs during the connection, the application logs the error and terminates. This step is crucial for sending requests and receiving responses from the plugin. ```go // Connect via RPC rpcClient, err := client.Client() if err != nil { log.Fatal(err) } ``` -------------------------------- ### Protocol Buffers API Definition for GRPC Greeter Plugin Source: https://github.com/hashicorp/go-plugin/blob/main/docs/extensive-go-plugin-tutorial.md Defines the API for a GRPC Greeter plugin using Protocol Buffers. It specifies a message for the response and a service with a Greet method. This file is processed by protoc to generate code for both client and server. ```proto3 syntax = "proto3"; package proto; message GreetResponse { string message = 1; } message Empty {} service GreeterService { rpc Greet(Empty) returns (GreetResponse); } ``` -------------------------------- ### Define Plugin Handshake Configuration Source: https://github.com/hashicorp/go-plugin/blob/main/docs/extensive-go-plugin-tutorial.md Defines the handshake configuration, including protocol version and magic cookies, used for basic communication and validation between the host and plugin. This ensures compatibility and provides a user-friendly error if the handshake fails. ```go // handshakeConfigs are used to just do a basic handshake between // a plugin and host. If the handshake fails, a user friendly error is shown. // This prevents users from executing bad plugins or executing a plugin // directory. It is a UX feature, not a security feature. var handshakeConfig = plugin.HandshakeConfig{ ProtocolVersion: 1, MagicCookieKey: "BASIC_PLUGIN", MagicCookieValue: "hello", } ``` -------------------------------- ### Dispense Plugin Service via RPC Source: https://github.com/hashicorp/go-plugin/blob/main/docs/extensive-go-plugin-tutorial.md Requests a specific plugin service (e.g., "greeter") from the connected RPC client. If the plugin is not found in the registered map, an error is logged. Upon success, the plugin is cast to its respective RPC or gRPC type. ```go // Request the plugin raw, err := rpcClient.Dispense("greeter") if err != nil { log.Fatal(err) } ``` -------------------------------- ### Type Assert and Call Plugin Function Source: https://github.com/hashicorp/go-plugin/blob/main/docs/extensive-go-plugin-tutorial.md Type asserts the dispensed plugin (raw client) into the expected plugin interface type (e.g., `shared.Greeter`). This allows calling functions on the plugin as if it were a local implementation, executing the logic remotely via RPC. ```go // We should have a Greeter now! This feels like a normal interface // implementation but is in fact over an RPC connection. greeter := raw.(shared.Greeter) fmt.Println(greeter.Greet()) ``` -------------------------------- ### go-plugin Handshake Protocol Fields Source: https://github.com/hashicorp/go-plugin/blob/main/docs/internals.md Details the individual components of the go-plugin handshake message. Understanding these fields is crucial for implementing compatible plugins, especially in languages other than Go. Each field specifies a different aspect of the connection setup, from protocol versions to network details. ```text CORE-PROTOCOL-VERSION | APP-PROTOCOL-VERSION | NETWORK-TYPE | NETWORK-ADDR | PROTOCOL ``` -------------------------------- ### Serve a Plugin from a Standalone Binary in Go Source: https://context7.com/hashicorp/go-plugin/llms.txt This Go code implements and serves a plugin from a standalone binary, allowing a host process to connect via RPC. It includes the actual plugin implementation, RPC server and client structures, handshake configuration, and the main function to initiate the plugin serving process, depending on 'go-plugin' and 'go-hclog'. ```go package main import ( "os" "github.com/hashicorp/go-hclog" "github.com/hashicorp/go-plugin" ) // GreeterImpl is the actual implementation type GreeterImpl struct{} func (g *GreeterImpl) Greet() string { return "Hello from plugin!" } // GreeterRPCServer is the RPC server that GreeterRPC talks to type GreeterRPCServer struct { Impl Greeter } func (s *GreeterRPCServer) Greet(args interface{}, resp *string) error { *resp = s.Impl.Greet() return nil } // GreeterRPCClient is the RPC client type GreeterRPCClient struct { client *rpc.Client } func (c *GreeterRPCClient) Greet() string { var resp string err := c.client.Call("Plugin.Greet", new(interface{}), &resp) if err != nil { return "" } return resp } var handshakeConfig = plugin.HandshakeConfig{ ProtocolVersion: 1, MagicCookieKey: "BASIC_PLUGIN", MagicCookieValue: "hello", } func main() { logger := hclog.New(&hclog.LoggerOptions{ Level: hclog.Debug, Output: os.Stderr, JSONFormat: true, }) // Serve the plugin plugin.Serve(&plugin.ServeConfig{ HandshakeConfig: handshakeConfig, Plugins: map[string]plugin.Plugin{ "greeter": &GreeterPlugin{}, }, Logger: logger, }) } ``` -------------------------------- ### Initialize and Connect to a Plugin Client in Go Source: https://context7.com/hashicorp/go-plugin/llms.txt This Go code initializes a client to launch and connect to a plugin subprocess over RPC. It defines handshake configuration, a plugin interface, and the main logic to dispense and use the plugin, requiring the 'go-plugin' and 'go-hclog' libraries. ```go package main import ( "fmt" "log" "os/exec" "github.com/hashicorp/go-hclog" "github.com/hashicorp/go-plugin" ) // Define the handshake configuration var handshakeConfig = plugin.HandshakeConfig{ ProtocolVersion: 1, MagicCookieKey: "BASIC_PLUGIN", MagicCookieValue: "hello", } // Define your plugin interface type Greeter interface { Greet() string } // Plugin implementation for RPC type GreeterPlugin struct{} func (p *GreeterPlugin) Server(*plugin.MuxBroker) (interface{}, error) { return &GreeterRPCServer{Impl: &GreeterImpl{}}, } func (p *GreeterPlugin) Client(b *plugin.MuxBroker, c *rpc.Client) (interface{}, error) { return &GreeterRPCClient{client: c}, } func main() { logger := hclog.New(&hclog.LoggerOptions{ Name: "plugin", Output: os.Stdout, Level: hclog.Debug, }) // Create the client configuration client := plugin.NewClient(&plugin.ClientConfig{ HandshakeConfig: handshakeConfig, Plugins: map[string]plugin.Plugin{ "greeter": &GreeterPlugin{}, }, Cmd: exec.Command("./plugin-binary"), Logger: logger, Managed: true, // Automatically cleaned up with CleanupClients() }) defer client.Kill() // Connect via RPC rpcClient, err := client.Client() if err != nil { log.Fatal("Error connecting to plugin:", err) } // Dispense the plugin aw, err := rpcClient.Dispense("greeter") if err != nil { log.Fatal("Error dispensing plugin:", err) } // Use the plugin interface greeter := raw.(Greeter) result := greeter.Greet() fmt.Println("Plugin says:", result) } ``` -------------------------------- ### Implement gRPC Plugins with go-plugin Source: https://context7.com/hashicorp/go-plugin/llms.txt This Go code demonstrates how to implement and use gRPC-based plugins with the go-plugin library. It defines a KV interface and its gRPC server and client implementations, allowing for cross-language plugin communication. The client code sets up and dispenses the plugin, then uses its methods. ```go package main import ( "context" "fmt" "os/exec" "github.com/hashicorp/go-plugin" "google.golang.org/grpc" ) // KV interface for key-value operations type KV interface { Put(key string, value []byte) error Get(key string) ([]byte, error) } // GRPCPlugin implementation type KVGRPCPlugin struct { plugin.NetRPCUnsupportedPlugin // Disable net/rpc support Impl KV } func (p *KVGRPCPlugin) GRPCServer(broker *plugin.GRPCBroker, s *grpc.Server) error { proto.RegisterKVServer(s, &GRPCServer{Impl: p.Impl, broker: broker}) return nil } func (p *KVGRPCPlugin) GRPCClient(ctx context.Context, broker *plugin.GRPCBroker, c *grpc.ClientConn) (interface{}, error) { return &GRPCClient{ client: proto.NewKVClient(c), broker: broker, }, nil } var handshake = plugin.HandshakeConfig{ ProtocolVersion: 1, MagicCookieKey: "KV_PLUGIN", MagicCookieValue: "supersecret", } func main() { // Client side with gRPC client := plugin.NewClient(&plugin.ClientConfig{ HandshakeConfig: handshake, Plugins: map[string]plugin.Plugin{ "kv_grpc": &KVGRPCPlugin{}, }, Cmd: exec.Command("./kv-plugin"), AllowedProtocols: []plugin.Protocol{ plugin.ProtocolGRPC, }, }) defer client.Kill() rpcClient, err := client.Client() if err != nil { panic(err) } raw, err := rpcClient.Dispense("kv_grpc") if err != nil { panic(err) } kv := raw.(KV) // Use the plugin err = kv.Put("foo", []byte("bar")) if err != nil { panic(err) } value, err := kv.Get("foo") if err != nil { panic(err) } fmt.Printf("Value: %s\n", string(value)) } ``` -------------------------------- ### Go Plugin Version Negotiation for Backward Compatibility Source: https://context7.com/hashicorp/go-plugin/llms.txt This Go code demonstrates how to implement version negotiation in go-plugin for backward compatibility. It shows how a client can be configured with `VersionedPlugins` to support multiple protocol versions, allowing it to communicate with servers offering different plugin sets based on their negotiated version. ```go package main import ( "os/exec" "github.com/hashicorp/go-plugin" ) // Define different plugin sets for different versions var v1Plugins = map[string]plugin.Plugin{ "greeter": &GreeterV1Plugin{}, } var v2Plugins = map[string]plugin.Plugin{ "greeter": &GreeterV2Plugin{}, // With additional methods "logger": &LoggerPlugin{}, // New plugin type } func main() { // Client supports multiple versions client := plugin.NewClient(&plugin.ClientConfig{ HandshakeConfig: plugin.HandshakeConfig{ MagicCookieKey: "MY_PLUGIN", MagicCookieValue: "secret", // ProtocolVersion not required with VersionedPlugins }, VersionedPlugins: map[int]plugin.PluginSet{ 1: v1Plugins, 2: v2Plugins, }, Cmd: exec.Command("./plugin"), }) defer client.Kill() // Start and negotiate version _, err := client.Start() if err != nil { panic(err) } // Check which version was negotiated version := client.NegotiatedVersion() rpcClient, _ := client.Client() raw, _ := rpcClient.Dispense("greeter") // Use version-specific features if version >= 2 { greeter := raw.(GreeterV2) greeter.GreetWithName("Alice") } else { greeter := raw.(GreeterV1) greeter.Greet() } } // Server side with version support func serveVersionedPlugin() { plugin.Serve(&plugin.ServeConfig{ HandshakeConfig: plugin.HandshakeConfig{ MagicCookieKey: "MY_PLUGIN", MagicCookieValue: "secret", }, VersionedPlugins: map[int]plugin.PluginSet{ 1: v1Plugins, 2: v2Plugins, }, }) } ``` -------------------------------- ### Reattach to Running Plugin Process with go-plugin Source: https://context7.com/hashicorp/go-plugin/llms.txt Demonstrates how to save a reattach configuration from an initial plugin host and then use it to connect to the already-running plugin. This is useful for host upgrades or maintaining state across client restarts. It involves saving the reattach config to a file and then loading it to create a new client that connects to the existing process. ```go package main import ( "encoding/json" "fmt" "os" "os/exec" "github.com/hashicorp/go-plugin" ) // Assume handshakeConfig and pluginMap are defined elsewhere var handshakeConfig plugin.HandshakeConfig var pluginMap map[string]plugin.Plugin func startInitialHost() { // Initial host starts plugin client := plugin.NewClient(&plugin.ClientConfig{ HandshakeConfig: handshakeConfig, Plugins: pluginMap, Cmd: exec.Command("./long-running-plugin"), }) // Start the plugin _, err := client.Start() if err != nil { panic(err) } // Get reattach config to save for later eattachConfig := client.ReattachConfig() // Serialize and save the config data, _ := json.Marshal(reattachConfig) os.WriteFile("reattach.json", data, 0644) // Host can exit, plugin keeps running... } func reattachToPlugin() { // Load the saved reattach config data, err := os.ReadFile("reattach.json") if err != nil { panic(err) } var reattachConfig plugin.ReattachConfig json.Unmarshal(data, &reattachConfig) // Create client that reattaches instead of starting new process client := plugin.NewClient(&plugin.ClientConfig{ HandshakeConfig: handshakeConfig, Plugins: pluginMap, Reattach: &reattachConfig, }) defer client.Kill() // Connect to the existing plugin rpcClient, err := client.Client() if err != nil { panic(err) } // Use the plugin that was already running raw, err := rpcClient.Dispense("greeter") if err != nil { panic(err) } // Assume Greeter interface is defined elsewhere // greeter := raw.(Greeter) // fmt.Println(greeter.Greet()) } ``` -------------------------------- ### Unit Test Plugin Implementations in Go Source: https://context7.com/hashicorp/go-plugin/llms.txt This Go code demonstrates how to use `plugin.TestPluginRPCConn` and `plugin.TestPluginGRPCConn` to create in-memory RPC connections for unit testing plugin logic without spawning subprocesses. It tests dispensing and interacting with mock plugin implementations. ```go package main import ( test "testing" "github.com/hashicorp/go-plugin" ) func TestGreeterPlugin(t *testing.T) { // Create test RPC connection without starting subprocess client, server := plugin.TestPluginRPCConn(t, map[string]plugin.Plugin{ "greeter": &GreeterPlugin{ Impl: &GreeterImpl{message: "test greeting"}, }, }, nil) defer client.Close() defer server.Stop() // Dispense the plugin raw, err := client.Dispense("greeter") if err != nil { t.Fatalf("Failed to dispense: %v", err) } // Test the plugin interface greeter := raw.(Greeter) result := greeter.Greet() expected := "test greeting" if result != expected { t.Errorf("Expected %q but got %q", expected, result) } } func TestGRPCPlugin(t *testing.T) { // Test gRPC plugin without subprocess client, server := plugin.TestPluginGRPCConn(t, false, map[string]plugin.Plugin{ "kv": &KVGRPCPlugin{ Impl: &InMemoryKV{}, }, }) defer client.Close() defer server.Stop() raw, err := client.Dispense("kv") if err != nil { t.Fatalf("Failed to dispense: %v", err) } kv := raw.(KV) // Test operations err = kv.Put("key1", []byte("value1")) if err != nil { t.Fatalf("Put failed: %v", err) } val, err := kv.Get("key1") if err != nil { t.Fatalf("Get failed: %v", err) } if string(val) != "value1" { t.Errorf("Expected value1 but got %s", string(val)) } } func TestWithTestMode(t *testing.T) { // Test in-process with ServeConfig.Test eattachCh := make(chan *plugin.ReattachConfig, 1) closeCh := make(chan struct{}) go plugin.Serve(&plugin.ServeConfig{ HandshakeConfig: handshakeConfig, Plugins: pluginMap, Test: &plugin.ServeTestConfig{ Context: context.Background(), ReattachConfigCh: reattachCh, CloseCh: closeCh, }, }) // Get reattach config eattachConfig := <-reattachCh // Create client that reattaches client := plugin.NewClient(&plugin.ClientConfig{ HandshakeConfig: handshakeConfig, Plugins: pluginMap, Reattach: reattachConfig, }) // Test the plugin rpcClient, _ := client.Client() raw, _ := rpcClient.Dispense("greeter") greeter := raw.(Greeter) result := greeter.Greet() // Verify result if result == "" { t.Error("Expected greeting but got empty string") } // Cleanup client.Kill() <-closeCh } ``` -------------------------------- ### Secure Plugin Verification with go-plugin Source: https://context7.com/hashicorp/go-plugin/llms.txt This Go code snippet illustrates how to secure plugins using checksum verification with the go-plugin library. It calculates the SHA256 checksum of a plugin file and uses it to verify the plugin's integrity before execution. This prevents the loading of tampered plugins, enhancing security. ```go package main import ( "crypto/sha256" "encoding/hex" "io" "os" "os/exec" "github.com/hashicorp/go-plugin" ) func main() { // Calculate expected checksum expectedChecksum := "a1b2c3d4e5f6..." // Your plugin's SHA256 checksum checksumBytes, _ := hex.DecodeString(expectedChecksum) // Create secure config secureConfig := &plugin.SecureConfig{ Checksum: checksumBytes, Hash: sha256.New(), } // Verify before using pluginPath := "./my-plugin" valid, err := secureConfig.Check(pluginPath) if err != nil { panic("Failed to verify checksum: " + err.Error()) } if !valid { panic("Plugin checksum does not match!") } // Create client with secure config client := plugin.NewClient(&plugin.ClientConfig{ HandshakeConfig: plugin.HandshakeConfig{ ProtocolVersion: 1, MagicCookieKey: "MY_PLUGIN", MagicCookieValue: "secret", }, Plugins: pluginMap, Cmd: exec.Command(pluginPath), SecureConfig: secureConfig, }) defer client.Kill() // Plugin is verified and safe to use rpcClient, err := client.Client() if err != nil { panic(err) } // Use the verified plugin... } ``` -------------------------------- ### Output Handshake Information Source: https://github.com/hashicorp/go-plugin/blob/main/docs/guide-plugin-write-non-go.md Outputs handshake information to stdout in the format CORE-PROTOCOL-VERSION|APP-PROTOCOL-VERSION|NETWORK-TYPE|NETWORK-ADDR|PROTOCOL. This allows go-plugin to establish a connection with the plugin. ```text 1|1|tcp|127.0.0.1:1234|grpc ``` -------------------------------- ### go-plugin Handshake Protocol Format Source: https://github.com/hashicorp/go-plugin/blob/main/docs/internals.md Illustrates the structure of the handshake message exchanged between a go-plugin host and server. This message allows the host to establish a connection with the plugin using specified network and protocol details. The format includes core protocol version, application protocol version, network type, network address, and the communication protocol. ```text 1|3|unix|/path/to/socket|grpc ```