=============== LIBRARY RULES =============== From library maintainers: - Textile-Go wraps warp.Client transparently - existing code needs no changes, just wrap the client with textile.New() - Always use deep cloning for message immutability - transformers work on copies, never modify original messages - Transformers implement a single Transform method that receives a TransformContext with Stage, Messages, and Metadata - Three transformation stages exist: StageRequest (before LLM), StageResponse (after LLM), StageStreamChunk (during streaming) - Default error strategy is fail-closed for data integrity - use WithErrorStrategy(ErrorStrategyFailOpen) only when errors can be safely ignored - Pipeline execution is sequential - transformers run in the order they were added to the pipeline - Context.Context must be the first parameter in all public APIs for proper cancellation and timeout support - Thread safety is guaranteed through mutex protection and immutable message passing via deep cloning - The internal/clone package uses reflection for deep cloning - this is intentional for complete immutability despite performance cost - Always run tests with -race flag to detect race conditions, especially when testing transformers - Transformers should be stateless when possible - if state is needed, protect with appropriate synchronization - Use table-driven tests following Go conventions - see existing test files for patterns - Stream transformation happens per-chunk in real-time - ensure transformers handle partial data correctly - TransformContext.Metadata is for passing data between transformers in the same pipeline execution - Client.WithTransformer() adds a single transformer, Client.WithPipeline() replaces the entire pipeline - Fail-closed strategy stops on first error and returns it, fail-open logs errors but continues processing - The embedded warp.Client design allows textile.Client to be a drop-in replacement maintaining the same API - Follow Google Go Style Guide for all code contributions - use gofmt and golint before committing - Examples in examples/ directory demonstrate real usage patterns - basic, streaming, and custom transformers ### Basic Transformer Implementation and Client Setup (Go) Source: https://github.com/blue-context/textile-go/blob/main/docs/context-engineering-guide.md Provides a simple example of a Transformer function that logs the number of messages during the request stage. It also shows how to initialize a Textile client with this custom transformer. ```go import ( "context" "log" "github.com/textileio/go-sdk/textile" ) func MyFirstTransformer(ctx context.Context, tc *textile.TransformContext) error { if tc.Stage == textile.StageRequest { // Modify context before LLM call log.Printf("Request: %d messages", len(tc.Messages)) } return nil } // Example of client initialization with the transformer func setupClientWithTransformer(warpClient interface{}) (*textile.Client, error) { client, err := textile.New( textile.WithWarpClient(warpClient), textile.WithTransformer(MyFirstTransformer), ) if err != nil { return nil, err } return client, nil } ``` -------------------------------- ### Run Basic Usage Example Source: https://github.com/blue-context/textile-go/blob/main/examples/README.md Navigates to the 'basic' directory and runs the main Go program to demonstrate basic textile-go functionality. ```bash cd basic go run main.go ``` -------------------------------- ### Install Textile-Go Source: https://github.com/blue-context/textile-go/blob/main/README.md Installs the Textile-Go package using the go get command. This is the initial step to include the library in your Go project. ```bash go get github.com/blue-context/textile-go ``` -------------------------------- ### Run Streaming Example Source: https://github.com/blue-context/textile-go/blob/main/examples/README.md Navigates to the 'streaming' directory and runs the main Go program to demonstrate real-time response streaming with chunk transformation. ```bash cd streaming go run main.go ``` -------------------------------- ### Set OpenAI API Key Source: https://github.com/blue-context/textile-go/blob/main/examples/README.md Sets the OpenAI API key as an environment variable. This is a prerequisite for running the examples. ```bash export OPENAI_API_API_KEY=sk-... ``` -------------------------------- ### Quick Start: Textile-Go with Warp Client Source: https://github.com/blue-context/textile-go/blob/main/README.md Demonstrates how to initialize a warp client, register an OpenAI provider, define a prefix transformer, wrap the warp client with Textile, and make a completion call. This example showcases zero API changes. ```go package main import ( "context" "fmt" "log" "github.com/blue-context/textile-go" "github.com/blue-context/warp" "github.com/blue-context/warp/provider/openai" ) func main() { // Create warp client warpClient, _ := warp.NewClient() defer warpClient.Close() // Register provider provider, _ := openai.NewProvider(openai.WithAPIKey("sk-...")) warpClient.RegisterProvider(provider) // Create a simple transformer prefixTransformer := func(ctx context.Context, tc *textile.TransformContext) error { if tc.Stage == textile.StageRequest { for i := range tc.Messages { if content, ok := tc.Messages[i].Content.(string); ok { tc.Messages[i].Content = "[IMPORTANT] " + content } } } return nil } // Wrap with textile client, _ := textile.New( textile.WithWarpClient(warpClient), textile.WithTransformer(prefixTransformer), ) defer client.Close() // Use exactly like warp - zero API changes resp, _ := client.Completion(context.Background(), &warp.CompletionRequest{ Model: "openai/gpt-4o-mini", Messages: []warp.Message{ {Role: "user", Content: "Hello!"}, }, }) fmt.Println(resp.Choices[0].Message.Content) } ``` -------------------------------- ### Run Custom Transformer Example Source: https://github.com/blue-context/textile-go/blob/main/examples/README.md Navigates to the 'custom_transformer' directory and executes the main Go program to showcase custom transformer creation and pipeline assembly. ```bash cd custom_transformer go run main.go ``` -------------------------------- ### Go Onboarding State Machine Example Source: https://github.com/blue-context/textile-go/blob/main/docs/context-engineering-guide.md Implements an onboarding conversation flow using a state machine. It defines states, handles request and response stages to inject state-specific prompts and transition between states, and extracts user data. Relies on external functions like `getState`, `getStatePrompt`, `determineNextState`, and `extractUserData`. ```go type OnboardingState string const ( StateWelcome OnboardingState = "welcome" StateCollectName OnboardingState = "collect_name" StateCollectRole OnboardingState = "collect_role" StateComplete OnboardingState = "complete" ) func OnboardingStateMachine(ctx context.Context, tc *textile.TransformContext) error { currentState := getState(tc.Metadata) switch tc.Stage { case textile.StageRequest: // Inject state-specific prompt prompt := getStatePrompt(currentState) tc.Messages = append([]warp.Message{{Role: "system", Content: prompt}}, tc.Messages...) case textile.StageResponse: // Transition state based on response nextState := determineNextState(currentState, tc.Response) tc.Metadata["onboarding_state"] = nextState // Extract user data extractUserData(currentState, tc.Response, tc.Metadata) } return nil } func getStatePrompt(state OnboardingState) string { switch state { case StateWelcome: return "Welcome the user and ask for their name." case StateCollectName: return "Thank them for their name and ask for their role." case StateCollectRole: return "Thank them and complete onboarding." case StateComplete: return "Onboarding is complete. Assist with their requests." default: return "" } } func determineNextState(current OnboardingState, response *warp.CompletionResponse) OnboardingState { switch current { case StateWelcome: return StateCollectName case StateCollectName: return StateCollectRole case StateCollectRole: return StateComplete default: return current } } func extractUserData(state OnboardingState, response *warp.CompletionResponse, metadata map[string]interface{}) { content := response.Choices[0].Message.Content switch state { case StateCollectName: // Simple extraction (in production, use more robust parsing) if name := extractName(content); name != "" { metadata["user_name"] = name } case StateCollectRole: if role := extractRole(content); role != "" { metadata["user_role"] = role } } } ``` -------------------------------- ### Build a Pipeline in Go Source: https://github.com/blue-context/textile-go/blob/main/examples/README.md Illustrates how to construct a pipeline of transformers using the textile.NewPipelineBuilder. Transformers can be added as functions or inline anonymous functions. ```go pipeline := textile.NewPipelineBuilder(). Add(transformer1). Add(transformer2). AddFunc(func(tc *textile.TransformContext) error { // Inline transformer return nil }). Build() ``` -------------------------------- ### Simulate Multi-Agent Workflows via Context Transformations in Go Source: https://github.com/blue-context/textile-go/blob/main/docs/context-engineering-guide.md Demonstrates how to simulate a multi-agent workflow (research, summarize, critique) by transforming context within a single LLM instance. This approach reduces latency and simplifies state management compared to traditional multi-agent setups. It is suitable for sequential workflows where phases do not require independent reasoning. ```go type AgentPhase string const ( PhaseResearch AgentPhase = "research" PhaseSummarize AgentPhase = "summarize" PhaseCritique AgentPhase = "critique" ) func AgentPhaseTransformer(ctx context.Context, tc *textile.TransformContext) error { switch tc.Stage { case textile.StageRequest: // Inject phase-specific system prompt phase := tc.Metadata["current_phase"].(AgentPhase) systemPrompt := getPhasePrompt(phase) tc.Messages = append([]warp.Message{ {Role: "system", Content: systemPrompt}, }, tc.Messages...) case textile.StageResponse: // Store phase output, transition to next phase phase := tc.Metadata["current_phase"].(AgentPhase) output := tc.Response.Choices[0].Message.Content tc.Metadata[string(phase)+"_output"] = output // Determine next phase nextPhase := getNextPhase(phase) if nextPhase != "" { tc.Metadata["current_phase"] = nextPhase tc.Metadata["continue_pipeline"] = true } } return nil } func getPhasePrompt(phase AgentPhase) string { switch phase { case PhaseResearch: return "You are a researcher. Gather comprehensive information on the topic." case PhaseSummarize: return "You are a summarizer. Create a concise summary from the research." case PhaseCritique: return "You are a critic. Identify gaps and weaknesses in the summary." default: return "" } } func getNextPhase(current AgentPhase) AgentPhase { switch current { case PhaseResearch: return PhaseSummarize case PhaseSummarize: return PhaseCritique case PhaseCritique: return "" // End of pipeline default: return "" } } ``` -------------------------------- ### Create a Simple Transformer in Go Source: https://github.com/blue-context/textile-go/blob/main/examples/README.md Demonstrates how to create a custom transformer function in Go for the textile-go library. The transformer can modify request messages based on the context and stage. ```go func MyTransformer(ctx context.Context, tc *textile.TransformContext) error { if tc.Stage == textile.StageRequest { // Modify request messages for i := range tc.Messages { // Transform message } } return nil } ``` -------------------------------- ### Error Handling Strategies in Go Source: https://github.com/blue-context/textile-go/blob/main/examples/README.md Shows two error handling strategies for the textile-go client: 'Fail open' (default), which continues on errors, and 'Fail closed', which returns errors immediately. Configured using textile.WithErrorStrategy. ```go // Fail open (default): continue on errors client, _ := textile.New( textile.WithWarpClient(warpClient), textile.WithErrorStrategy(textile.ErrorStrategyFailOpen), ) // Fail closed: return errors immediately client, _ := textile.New( textile.WithWarpClient(warpClient), textile.WithErrorStrategy(textile.ErrorStrategyFailClosed), ) ``` -------------------------------- ### Create and Configure Textile Client in Go Source: https://context7.com/blue-context/textile-go/llms.txt Initializes a warp client, registers the OpenAI provider, defines a sample prefix transformer, and wraps the warp client with textile to create a new textile client. This client can then be used with zero API changes compared to the original warp.Client. The example demonstrates request transformation by prepending '[IMPORTANT]' to user messages. ```go package main import ( "context" "fmt" "log" "os" "github.com/blue-context/textile-go" "github.com/blue-context/warp" "github.com/blue-context/warp/provider/openai" ) func main() { // Create and configure warp client warpClient, err := warp.NewClient() if err != nil { log.Fatal(err) } defer warpClient.Close() // Register OpenAI provider provider, err := openai.NewProvider(openai.WithAPIKey(os.Getenv("OPENAI_API_KEY"))) if err != nil { log.Fatal(err) } if err := warpClient.RegisterProvider(provider); err != nil { log.Fatal(err) } // Create simple prefix transformer prefixTransformer := func(ctx context.Context, tc *textile.TransformContext) error { if tc.Stage == textile.StageRequest { for i := range tc.Messages { if tc.Messages[i].Role == "user" { if content, ok := tc.Messages[i].Content.(string); ok { tc.Messages[i].Content = "[IMPORTANT] " + content } } } } return nil } // Wrap warp client with textile client, err := textile.New( textile.WithWarpClient(warpClient), textile.WithTransformer(prefixTransformer), textile.WithErrorStrategy(textile.ErrorStrategyFailOpen), ) if err != nil { log.Fatal(err) } defer client.Close() // Use exactly like warp.Client - zero API changes resp, err := client.Completion(context.Background(), &warp.CompletionRequest{ Model: "openai/gpt-4o-mini", Messages: []warp.Message{ {Role: "user", Content: "Hello!"}, }, }) if err != nil { log.Fatal(err) } fmt.Println("Response:", resp.Choices[0].Message.Content) } ``` -------------------------------- ### Fail-Open Error Strategy for Textile Client in Go Source: https://github.com/blue-context/textile-go/blob/main/docs/context-engineering-guide.md Example of configuring a Textile client in Go to use a fail-open error strategy. This strategy allows transformations to continue even if errors occur, logging the error using a custom handler. It requires the 'context', 'textile', and 'log' packages. ```go client, _ := textile.New( textile.WithWarpClient(warpClient), textile.WithErrorStrategy(textile.ErrorStrategyFailOpen), textile.WithErrorHandler(func(ctx context.Context, err error) { log.Printf("Transformer error (continuing): %v", err) }), ) ``` -------------------------------- ### Implement Role/Persona Injection in Go Source: https://github.com/blue-context/textile-go/blob/main/docs/context-engineering-guide.md This Go function, MultiPerspectiveTransformer, demonstrates how to inject dynamic perspectives into a single LLM call. It extracts user code, defines multiple roles (security, performance, maintainability) with example interactions, and formats them as a list of messages. This is useful for multi-perspective analysis without the overhead of spawning separate agents, especially when speed is critical. ```go package main import ( "context" "fmt" "github.com/textileio/go-threads/core/warp" ) // Placeholder for textile.TransformContext and its Stage field // Assuming textile.StageRequest is a constant of type int const StageRequest = 1 type TransformContext struct { Stage int Messages []warp.Message } // Placeholder for extractCode function func extractCode(messages []warp.Message) string { return "func add(a, b int) int { return a + b }" } func MultiPerspectiveTransformer(ctx context.Context, tc *TransformContext) error { if tc.Stage != StageRequest { return nil } // Get user's code submission userCode := extractCode(tc.Messages) // Inject synthetic perspectives as few-shot examples perspectives := []warp.Message{ { Role: "system", Content: "You will review code from three perspectives: security, performance, and maintainability.", }, { Role: "user", Content: "Review this function: func add(a, b int) int { return a + b }", }, { Role: "assistant", Content: "**Security:** ✓ No security issues. No external inputs, overflow handled by Go.\n**Performance:** ✓ O(1) operation, optimal.\n**Maintainability:** ✓ Clear naming, simple logic.", }, { Role: "user", Content: fmt.Sprintf("Review this code:\n%s", userCode), }, } tc.Messages = perspectives return nil } // Dummy main function to make it runnable func main() { // Example usage would go here } ``` -------------------------------- ### Go Dynamic Constraint Injection for Input Validation Source: https://github.com/blue-context/textile-go/blob/main/docs/context-engineering-guide.md This Go code defines a `ConstraintInjector` that iterates through a list of `Validator` interfaces to check user input. If validation fails, errors are collected and injected as system messages into the conversation context, guiding the LLM to self-correct. It includes example `LengthValidator` and `FormatValidator` implementations. ```go package main import ( "context" "fmt" "regexp" "strings" "github.com/textileio/go-threads/core/textile" "github.com/textileio/go-threads/util/warp" ) type ConstraintInjector struct { validators []Validator } type Validator interface { Validate(input string) error } func (ci *ConstraintInjector) Transform(ctx context.Context, tc *textile.TransformContext) error { if tc.Stage != textile.StageRequest { return nil } // Get user input userInput := extractUserInput(tc.Messages) // Run validators var constraints []string for _, validator := range ci.validators { if err := validator.Validate(userInput); err != nil { constraints = append(constraints, err.Error()) } } // Inject constraints if len(constraints) > 0 { constraintMsg := buildConstraintMessage(constraints) tc.Messages = append([]warp.Message{ {Role: "system", Content: constraintMsg}, }, tc.Messages...) } return nil } func buildConstraintMessage(constraints []string) string { var builder strings.Builder builder.WriteString("IMPORTANT: Address these constraints:\n") for i, constraint := range constraints { builder.WriteString(fmt.Sprintf("%d. %s\n", i+1, constraint)) } return builder.String() } // Example validators type LengthValidator struct { minLength int } func (lv *LengthValidator) Validate(input string) error { if len(input) < lv.minLength { return fmt.Errorf("Input must be at least %d characters", lv.minLength) } return nil } type FormatValidator struct { pattern *regexp.Regexp } func (fv *FormatValidator) Validate(input string) error { if !fv.pattern.MatchString(input) { return fmt.Errorf("Input must match format: %s", fv.pattern.String()) } return nil } // Placeholder functions (implementation not provided in the original text) func extractUserInput(messages []warp.Message) string { // Assume the last user message contains the input for i := len(messages) - 1; i >= 0; i-- { if messages[i].Role == "user" { return messages[i].Content } } return "" } ``` -------------------------------- ### Composing Multiple Transformers into a Pipeline (Go) Source: https://github.com/blue-context/textile-go/blob/main/docs/context-engineering-guide.md Demonstrates how to build a pipeline of transformers using Textile.go's PipelineBuilder. This allows for sequential application of different context manipulation logic, such as logging, compression, and RAG. ```go import ( "github.com/textileio/go-sdk/textile" ) // Assume these transformers are defined elsewhere var LoggingTransformer textile.Transformer var CompressionTransformer textile.Transformer var RAGTransformer textile.Transformer func buildPipeline() textile.Transformer { pipeline := textile.NewPipelineBuilder(). Add(LoggingTransformer). // Week 1: Add logging Add(CompressionTransformer). // Week 2: Add compression Add(RAGTransformer). // Week 3: Add RAG Build() return pipeline } ``` -------------------------------- ### Federated Context Sharing Between Services (Go) Source: https://github.com/blue-context/textile-go/blob/main/docs/context-engineering-guide.md Demonstrates how to share context transformations across microservices. Service A compresses context and stores it in metadata, while Service B retrieves and decompresses it. This pattern enables consistent context engineering in distributed systems. ```go import ( "context" "github.com/textileio/go-sdk/textile" ) // Service A: Compresses context func ServiceATransformer(ctx context.Context, tc *textile.TransformContext) error { compressed := compressContext(tc.Messages) tc.Metadata["compressed_context"] = compressed return nil } // Service B: Consumes compressed context func ServiceBTransformer(ctx context.Context, tc *textile.TransformContext) error { if compressed, ok := tc.Metadata["compressed_context"].(string); ok { tc.Messages = decompressContext(compressed) } return nil } // Placeholder functions for compression/decompression func compressContext(messages []string) string { // Implementation omitted for brevity return "compressed_data" } func decompressContext(compressed string) []string { // Implementation omitted for brevity return []string{"decompressed_message"} } ``` -------------------------------- ### Example: Add Metadata to Response Transformation in Textile-Go Source: https://github.com/blue-context/textile-go/blob/main/README.md Illustrates a transformer that adds metadata, such as the model used and tokens consumed, to the `TransformContext` after receiving a response from the LLM. ```go // Add metadata to responses metadataTransformer := func(ctx context.Context, tc *textile.TransformContext) error { if tc.Stage == textile.StageResponse { // Store metadata for later use tc.Metadata["model_used"] = tc.Response.Model tc.Metadata["tokens_used"] = tc.Response.Usage.TotalTokens } return nil } ``` -------------------------------- ### Example: Uppercase Request Transformation in Textile-Go Source: https://github.com/blue-context/textile-go/blob/main/README.md Provides a transformer function that converts the content of all 'user' messages to uppercase during the request stage. This demonstrates modifying outgoing messages. ```go // Convert all user messages to uppercase uppercaseTransformer := func(ctx context.Context, tc *textile.TransformContext) error { if tc.Stage == textile.StageRequest { for i := range tc.Messages { if tc.Messages[i].Role == "user" { if content, ok := tc.Messages[i].Content.(string); ok { tc.Messages[i].Content = strings.ToUpper(content) } } } } return nil } ``` -------------------------------- ### System Prompt Evolution in Go Source: https://github.com/blue-context/textile-go/blob/main/docs/context-engineering-guide.md Evolves system prompts dynamically based on the current task state, moving beyond static prompts. This allows the LLM's behavior to adapt to different phases of a workflow (e.g., analyzing, suggesting, explaining) without resetting the conversation. It's beneficial for multi-phase workflows where distinct behavioral requirements exist. ```go package main import ( "context" "github.com/textileio/go-sdk/v2/textile" "github.com/textileio/go-sdk/v2/warp" ) type ReviewState string const ( StateInitial ReviewState = "initial" StateAnalyzing ReviewState = "analyzing" StateSuggesting ReviewState = "suggesting" StateExplaining ReviewState = "explaining" ) func DynamicSystemPromptTransformer(ctx context.Context, tc *textile.TransformContext) error { if tc.Stage != textile.StageRequest { return nil } // Determine current state state := getCurrentState(tc.Metadata) // Generate state-specific system prompt var systemPrompt string switch state { case StateInitial: systemPrompt = "You are a code reviewer. Analyze code for bugs, style issues, and performance." case StateAnalyzing: systemPrompt = "You are analyzing code. Focus on identifying specific issues. Be concise." case StateSuggesting: systemPrompt = "You are suggesting fixes. Provide actionable code changes with explanations." case StateExplaining: systemPrompt = "You are explaining fixes. Be detailed and educational. Assume the user wants to learn." } // Replace or inject system message for i, msg := range tc.Messages { if msg.Role == "system" { tc.Messages[i].Content = systemPrompt return nil } } // No system message exists, prepend one tc.Messages = append([]warp.Message{{Role: "system", Content: systemPrompt}}, tc.Messages...) return nil } func getCurrentState(metadata map[string]interface{}) ReviewState { if state, ok := metadata["review_state"].(ReviewState); ok { return state } return StateInitial } ``` -------------------------------- ### Go: Timing Transformer for Request Duration Source: https://context7.com/blue-context/textile-go/llms.txt This transformer measures the duration of a request by storing the start time in metadata during the request stage and calculating the elapsed time during the response stage. It adds a 'request_time' and 'duration_ms' entry to the metadata. ```go package main import ( "context" "fmt" "time" "github.com/blue-context/textile-go" ) func TimingTransformer(ctx context.Context, tc *textile.TransformContext) error { if tc.Stage == textile.StageRequest { tc.Metadata["request_time"] = time.Now() } if tc.Stage == textile.StageResponse { if startTime, ok := tc.Metadata["request_time"].(time.Time); ok { duration := time.Since(startTime) tc.Metadata["duration_ms"] = duration.Milliseconds() fmt.Printf("[TIMING] Request took %dms\n", duration.Milliseconds()) } } return nil } ``` -------------------------------- ### Benchmark Go Project Performance Source: https://github.com/blue-context/textile-go/blob/main/README.md This bash command is used to benchmark the Go project. The `-bench=.` flag runs all benchmarks, and `-benchmem` includes memory allocation statistics in the output. This helps in identifying performance bottlenecks and memory usage patterns. ```bash go test -bench=. -benchmem ./... ``` -------------------------------- ### Client Initialization and Usage Source: https://context7.com/blue-context/textile-go/llms.txt Demonstrates how to initialize a Textile-Go client by wrapping an existing warp client with transformers and configuration, and then use it for LLM calls. ```APIDOC ## Client Initialization and Usage ### Description This section details the process of creating a `textile.Client` by integrating it with a `warp.Client` and applying custom transformers. It also shows a basic example of how to use the Textile-Go client to make a completion request to an LLM, highlighting its compatibility with the `warp.Client` API. ### Method Initialization: Constructor function `textile.New()` Usage: `client.Completion()` ### Endpoint N/A (Library usage, not a network endpoint) ### Parameters #### Request Body (for `client.Completion()`) - **Model** (string) - Required - The LLM model to use. - **Messages** ([]warp.Message) - Required - A list of messages for the conversation. - **Role** (string) - Required - The role of the message sender (e.g., "user", "system", "assistant"). - **Content** (any) - Required - The message content. Can be a string or other types depending on the provider. ### Request Example ```go package main import ( "context" "fmt" "log" "os" "github.com/blue-context/textile-go" "github.com/blue-context/warp" "github.com/blue-context/warp/provider/openai" ) func main() { // Create and configure warp client warpClient, err := warp.NewClient() if err != nil { log.Fatal(err) } defer warpClient.Close() // Register OpenAI provider provider, err := openai.NewProvider(openai.WithAPIKey(os.Getenv("OPENAI_API_KEY"))) if err != nil { log.Fatal(err) } if err := warpClient.RegisterProvider(provider); err != nil { log.Fatal(err) } // Create simple prefix transformer prefixTransformer := func(ctx context.Context, tc *textile.TransformContext) error { if tc.Stage == textile.StageRequest { for i := range tc.Messages { if tc.Messages[i].Role == "user" { if content, ok := tc.Messages[i].Content.(string); ok { tc.Messages[i].Content = "[IMPORTANT] " + content } } } } return nil } // Wrap warp client with textile client, err := textile.New( textile.WithWarpClient(warpClient), textile.WithTransformer(prefixTransformer), textile.WithErrorStrategy(textile.ErrorStrategyFailOpen), ) if err != nil { log.Fatal(err) } defer client.Close() // Use exactly like warp.Client - zero API changes resp, err := client.Completion(context.Background(), &warp.CompletionRequest{ Model: "openai/gpt-4o-mini", Messages: []warp.Message{ {Role: "user", Content: "Hello!"}, }, }) if err != nil { log.Fatal(err) } fmt.Println("Response:", resp.Choices[0].Message.Content) } ``` ### Response #### Success Response (200) - **Choices** ([]warp.Choice) - A list of possible completions. - **Index** (int) - The index of the choice. - **Message** (warp.Message) - The message content. - **Role** (string) - The role of the message sender. - **Content** (any) - The message content. - **FinishReason** (string) - The reason the model stopped generating tokens. - **Usage** (warp.CompletionUsage) - Information about token usage. - **PromptTokens** (int) - Number of tokens in the prompt. - **CompletionTokens** (int) - Number of tokens in the completion. - **TotalTokens** (int) - Total number of tokens. #### Response Example ```json { "Choices": [ { "Index": 0, "Message": { "Role": "assistant", "Content": "Hello there! How can I help you today?" }, "FinishReason": "stop" } ], "Usage": { "PromptTokens": 10, "CompletionTokens": 20, "TotalTokens": 30 } } ``` ``` -------------------------------- ### Integration Testing a Full Pipeline in Go Source: https://github.com/blue-context/textile-go/blob/main/docs/context-engineering-guide.md An integration test in Go for a complete context transformation pipeline. It builds a pipeline with multiple transformers, configures a Textile client with this pipeline, and then calls a mock client's `Completion` method to test end-to-end behavior. Requires 'testing', 'context', and 'textile' packages. ```go func TestFullPipeline(t *testing.T) { pipeline := textile.NewPipelineBuilder(). Add(CompressionTransformer). Add(RAGTransformer). Add(ValidationTransformer). Build() // Test end-to-end behavior mockClient := &mockWarpClient{} client, _ := textile.New( textile.WithWarpClient(mockClient), textile.WithPipeline(pipeline), ) resp, err := client.Completion(context.Background(), &warp.CompletionRequest{ Model: "test/model", Messages: []warp.Message{{Role: "user", Content: "test"}}, }) require.NoError(t, err) assert.NotNil(t, resp) } ``` -------------------------------- ### Go: Main function to set up and run Textile.go pipeline Source: https://context7.com/blue-context/textile-go/llms.txt This main function initializes the Warp client with an OpenAI provider, sets up a Textile pipeline with transformers (TimingTransformer, ResponseAnalyzer, ResponseEnhancer), and then makes a completion request. It demonstrates the full integration of the transformers. ```go package main import ( "context" "fmt" "log" "os" "github.com/blue-context/textile-go" "github.com/blue-context/warp" "github.com/blue-context/warp/provider/openai" ) func main() { warpClient, _ := warp.NewClient() defer warpClient.Close() provider, _ := openai.NewProvider(openai.WithAPIKey(os.Getenv("OPENAI_API_KEY"))) warpClient.RegisterProvider(provider) pipeline := textile.NewPipeline( TimingTransformer, // Assuming TimingTransformer is defined elsewhere ResponseAnalyzer, // Assuming ResponseAnalyzer is defined elsewhere ResponseEnhancer, // Assuming ResponseEnhancer is defined elsewhere ) client, _ := textile.New( textile.WithWarpClient(warpClient), textile.WithPipeline(pipeline), ) defer client.Close() resp, err := client.Completion(context.Background(), &warp.CompletionRequest{ Model: "openai/gpt-4o-mini", Messages: []warp.Message{ {Role: "user", Content: "Explain what middleware is in 2 sentences"}, }, }) if err != nil { log.Fatal(err) } fmt.Printf("\nEnhanced Response:\n%s\n", resp.Choices[0].Message.Content) } ``` -------------------------------- ### Snapshotting Metadata in Go for Debugging Source: https://github.com/blue-context/textile-go/blob/main/docs/context-engineering-guide.md A Go code snippet demonstrating how to snapshot the current messages into the metadata of a transformation context. This is useful for debugging by preserving the state before a transformation. ```go tc.Metadata["snapshot_before"] = cloneMessages(tc.Messages) ``` -------------------------------- ### Adaptive Token Budget Adjustment (Go) Source: https://github.com/blue-context/textile-go/blob/main/docs/context-engineering-guide.md Dynamically adjusts the token budget for LLM requests based on task importance. It classifies message importance, sets a budget accordingly, and performs budget-aware compression. This optimizes resource usage by allocating more budget to critical tasks. ```go import ( "context" "log" "github.com/textileio/go-sdk/textile" ) func AdaptiveBudgetTransformer(ctx context.Context, tc *textile.TransformContext) error { if tc.Stage != textile.StageRequest { return nil } importance := classifyImportance(tc.Messages) var tokenBudget int switch importance { case "critical": tokenBudget = 4000 case "normal": tokenBudget = 2000 case "low": tokenBudget = 1000 } tc.Metadata["token_budget"] = tokenBudget // Apply budget-aware compression optimized := compressToFit(tc.Messages, tokenBudget) tc.Messages = optimized return nil } // Placeholder functions for classification and compression func classifyImportance(messages []string) string { // Implementation omitted for brevity return "normal" } func compressToFit(messages []string, budget int) []string { // Implementation omitted for brevity return messages } ``` -------------------------------- ### Run Go Tests with Race Detector and Coverage Source: https://github.com/blue-context/textile-go/blob/main/README.md These bash commands are used to test the Go project. The first command runs all tests with the race detector enabled (`-race`) to find potential data races. The second command runs tests with coverage enabled (`-coverprofile=coverage.out`) and then uses the `go tool cover` command to generate an HTML report for visualizing test coverage. ```bash go test -race ./... go test -race -coverprofile=coverage.out ./... # View coverage report # go tool cover -html=coverage.out ``` -------------------------------- ### Integrate RAG Without Orchestration in Go Source: https://github.com/blue-context/textile-go/blob/main/docs/context-engineering-guide.md Shows how to integrate Retrieval-Augmented Generation (RAG) as a context transformation within a pipeline, eliminating the need for external orchestration. This method transparently injects retrieved documents into the LLM's context, simplifying RAG architecture for question-answering over knowledge bases. ```go type DocumentRetriever struct { vectorDB VectorDatabase } func (dr *DocumentRetriever) RAGTransformer(ctx context.Context, tc *textile.TransformContext) error { if tc.Stage != textile.StageRequest { return nil } // Extract query from user message userQuery := extractQuery(tc.Messages) // Retrieve relevant documents docs := dr.vectorDB.Search(userQuery, 3) // Top 3 documents // Inject retrieved context retrievedContext := buildRetrievalContext(docs) // Insert before user message tc.Messages = append([]warp.Message{ {Role: "system", Content: "Use the following documentation to answer questions."}, {Role: "system", Content: retrievedContext}, }, tc.Messages...) // Track retrieval in metadata tc.Metadata["retrieved_docs"] = len(docs) return nil } func buildRetrievalContext(docs []Document) string { var builder strings.Builder builder.WriteString("Retrieved documentation:\n\n") for i, doc := range docs { builder.WriteString(fmt.Sprintf("## Document %d: %s\n", i+1, doc.Title)) builder.WriteString(doc.Content) builder.WriteString("\n\n") } return builder.String() } ``` -------------------------------- ### Go Token Budget Optimization Logic Source: https://github.com/blue-context/textile-go/blob/main/docs/context-engineering-guide.md Implements a TokenOptimizer in Go to manage LLM token budgets. It compresses and summarizes older messages when the token limit is exceeded, preserving recent conversation context and system messages. Dependencies include custom `warp.Message`, `TokenCounter`, and `textile.TransformContext` types. ```go type TokenOptimizer struct { maxTokens int tokenCounter TokenCounter compressionFn func([]warp.Message) []warp.Message } type TokenCounter interface { Count(messages []warp.Message) int } func (to *TokenOptimizer) Transform(ctx context.Context, tc *textile.TransformContext) error { if tc.Stage != textile.StageRequest { return nil } currentTokens := to.tokenCounter.Count(tc.Messages) if currentTokens <= to.maxTokens { // Within budget return nil } // Exceeds budget - optimize optimized := to.optimizeContext(tc.Messages) tc.Messages = optimized // Track optimization newTokens := to.tokenCounter.Count(tc.Messages) tc.Metadata["token_reduction"] = currentTokens - newTokens return nil } func (to *TokenOptimizer) optimizeContext(messages []warp.Message) []warp.Message { // Strategy 1: Preserve system + last N user/assistant pairs systemMessages := filterByRole(messages, "system") conversationMessages := filterByRole(messages, "user", "assistant") // Keep last 3 conversation pairs (6 messages) recentConversation := conversationMessages if len(conversationMessages) > 6 { // Compress older messages oldMessages := conversationMessages[:len(conversationMessages)-6] summary := to.compressMessages(oldMessages) recentConversation = append( []warp.Message{{Role: "system", Content: summary}}, conversationMessages[len(conversationMessages)-6:]..., ) } // Combine: system + compressed history + recent conversation return append(systemMessages, recentConversation...) } func (to *TokenOptimizer) compressMessages(messages []warp.Message) string { // Simple compression: extract key points // In production, use LLM-based summarization or semantic compression var topics []string for _, msg := range messages { if content, ok := msg.Content.(string); ok { topic := extractKeyTopic(content) // Simple keyword extraction topics = append(topics, topic) } } return fmt.Sprintf("Previous discussion covered: %s", strings.Join(topics, ", ")) } func filterByRole(messages []warp.Message, roles ...string) []warp.Message { roleSet := make(map[string]bool) for _, role := range roles { roleSet[role] = true } var filtered []warp.Message for _, msg := range messages { if roleSet[msg.Role] { filtered = append(filtered, msg) } } return filtered } ``` -------------------------------- ### Manipulating Textile-Go Pipelines with Transformers Source: https://context7.com/blue-context/textile-go/llms.txt Demonstrates how to create, append, and prepend transformers to a Textile-Go pipeline. It showcases the immutability of pipeline operations and how to use a constructed pipeline with a Warp client. Dependencies include `context`, `fmt`, `log`, `os`, `strings`, `github.com/blue-context/textile-go`, `github.com/blue-context/warp`, and `github.com/blue-context/warp/provider/openai`. ```go package main import ( "context" "fmt" "log" "os" "strings" "github.com/blue-context/textile-go" "github.com/blue-context/warp" "github.com/blue-context/warp/provider/openai" ) func Stage1Transformer(ctx context.Context, tc *textile.TransformContext) error { if tc.Stage == textile.StageRequest { fmt.Println("[STAGE1] First transformer") } return nil } func Stage2Transformer(ctx context.Context, tc *textile.TransformContext) error { if tc.Stage == textile.StageRequest { fmt.Println("[STAGE2] Second transformer") } return nil } func Stage3Transformer(ctx context.Context, tc *textile.TransformContext) error { if tc.Stage == textile.StageRequest { fmt.Println("[STAGE3] Third transformer") } return nil } func PrefixTransformer(ctx context.Context, tc *textile.TransformContext) error { if tc.Stage == textile.StageRequest { for i := range tc.Messages { if content, ok := tc.Messages[i].Content.(string); ok { tc.Messages[i].Content = "[PREFIX] " + content } } } return nil } func SuffixTransformer(ctx context.Context, tc *textile.TransformContext) error { if tc.Stage == textile.StageRequest { for i := range tc.Messages { if content, ok := tc.Messages[i].Content.(string); ok { tc.Messages[i].Content = content + " [SUFFIX]" } } } return nil } func main() { warpClient, _ := warp.NewClient() defer warpClient.Close() provider, _ := openai.NewProvider(openai.WithAPIKey(os.Getenv("OPENAI_API_KEY"))) warpClient.RegisterProvider(provider) // Create base pipeline basePipeline := textile.NewPipeline(Stage1Transformer, Stage2Transformer) fmt.Printf("Base pipeline length: %d\n", basePipeline.Len()) // Append transformers (creates new pipeline) appendedPipeline := basePipeline.Append(Stage3Transformer, SuffixTransformer) fmt.Printf("Appended pipeline length: %d\n", appendedPipeline.Len()) // Prepend transformers (creates new pipeline) prependedPipeline := basePipeline.Prepend(PrefixTransformer) fmt.Printf("Prepended pipeline length: %d\n", prependedPipeline.Len()) // Original pipeline unchanged (immutable operations) fmt.Printf("Original pipeline length: %d\n", basePipeline.Len()) // Use prepended pipeline client, _ := textile.New( textile.WithWarpClient(warpClient), textile.WithPipeline(prependedPipeline), ) defer client.Close() fmt.Println("\nExecuting request with prepended pipeline:") resp, err := client.Completion(context.Background(), &warp.CompletionRequest{ Model: "openai/gpt-4o-mini", Messages: []warp.Message{ {Role: "user", Content: "Say hello"}, }, }) if err != nil { log.Fatal(err) } fmt.Printf("\nResponse: %s\n", resp.Choices[0].Message.Content) } ``` -------------------------------- ### Progressive Context Refinement in Go Source: https://github.com/blue-context/textile-go/blob/main/docs/context-engineering-guide.md Transforms conversational turns by extracting semantic value and compressing it into evolved context. This reduces token usage by storing summarized information instead of full history, making it suitable for multi-turn conversations with token constraints. It involves extracting intent and relevant details in one turn and injecting this compressed context into subsequent turns. ```go package main import ( "context" "github.com/textileio/go-sdk/v2/textile" "github.com/textileio/go-sdk/v2/warp" ) // Turn 1: User asks about refund policy func RefundInquiryTransformer(ctx context.Context, tc *textile.TransformContext) error { if tc.Stage == textile.StageResponse { // Extract intent from response if containsRefundPolicy(tc.Response.Choices[0].Message.Content) { // Store semantic summary, not full response tc.Metadata["user_intent"] = "refund_policy_inquiry" tc.Metadata["refund_context"] = "30-day policy explained" } } return nil } // Turn 2: User asks follow-up about timing func ContextRefinementTransformer(ctx context.Context, tc *textile.TransformContext) error { if tc.Stage == textile.StageRequest { // Inject compressed context instead of full history if intent, ok := tc.Metadata["user_intent"].(string); ok && intent == "refund_policy_inquiry" { systemMsg := warp.Message{ Role: "system", Content: "Previous context: User inquired about refund policy. 30-day policy was explained.", } // Prepend compressed context tc.Messages = append([]warp.Message{systemMsg}, tc.Messages...) } } return nil } // Placeholder for actual implementation func containsRefundPolicy(content string) bool { return true } ``` -------------------------------- ### Unit Testing a Compression Transformer in Go Source: https://github.com/blue-context/textile-go/blob/main/docs/context-engineering-guide.md A Go unit test for a `CompressionTransformer` function. It sets up a `TransformContext` with a large number of messages, applies the transformer, and asserts that the message count is reduced significantly. Requires 'testing', 'context', and 'textile' packages. ```go func TestCompressionTransformer(t *testing.T) { tc := &textile.TransformContext{ Stage: textile.StageRequest, Messages: generateLongHistory(100), // 100 messages Metadata: make(map[string]interface{}), } err := CompressionTransformer(context.Background(), tc) require.NoError(t, err) assert.LessOrEqual(t, len(tc.Messages), 10, "Should compress to ≤10 messages") } ``` -------------------------------- ### Textile-Go Pipeline Construction Source: https://github.com/blue-context/textile-go/blob/main/README.md Shows how to build a transformation pipeline using PipelineBuilder. It allows adding multiple transformers sequentially, including inline functions. The built pipeline is then used to configure the Textile client. ```go pipeline := textile.NewPipelineBuilder(). Add(transformer1). Add(transformer2). AddFunc(func(tc *textile.TransformContext) error { // Inline transformer return nil }). Build() client, _ := textile.New( textile.WithWarpClient(warpClient), textile.WithPipeline(pipeline), ) ``` -------------------------------- ### Hybrid Agent Transformer Logic in Go Source: https://github.com/blue-context/textile-go/blob/main/docs/context-engineering-guide.md A Go function implementing a hybrid agent strategy using context engineering. It analyzes task complexity and either simulates agent behavior within the context for low complexity or flags for spawning a real agent for high complexity. Requires 'context' and 'textile' packages. ```go func HybridAgentTransformer(ctx context.Context, tc *textile.TransformContext) error { if tc.Stage != textile.StageRequest { return nil } // Analyze task complexity complexity := analyzeComplexity(tc.Messages) if complexity < 0.5 { // Low complexity - simulate agent via context tc.Messages = injectMultiPerspective(tc.Messages) tc.Metadata["mode"] = "simulated_agent" } else { // High complexity - flag for real multi-agent tc.Metadata["mode"] = "spawn_agent" tc.Metadata["spawn_config"] = buildAgentConfig(tc.Messages) } return nil } ``` -------------------------------- ### Snapshot Testing Transformer Consistency in Go Source: https://github.com/blue-context/textile-go/blob/main/docs/context-engineering-guide.md A Go test function for verifying the consistency of a transformer using snapshot testing. It loads an initial state from a JSON file, applies the transformer, and compares the resulting messages against a predefined expected state. Requires 'testing' and 'textile' packages. ```go func TestTransformerConsistency(t *testing.T) { tc := loadSnapshot("testdata/snapshot.json") err := MyTransformer(context.Background(), tc) require.NoError(t, err) expected := loadSnapshot("testdata/expected.json") assert.Equal(t, expected.Messages, tc.Messages) } ```