### Basic Memory Setup Source: https://github.com/embabel/dice/blob/main/_autodocs/configuration.md Set up a memory instance for a given context and associate it with a proposition repository. ```kotlin Memory.forContext(contextId) .withRepository(propositionRepository) ``` -------------------------------- ### EntityExtractor suggestEntities Method Example Source: https://github.com/embabel/dice/blob/main/_autodocs/api-reference/entity-extractor.md Demonstrates how to instantiate and use the EntityExtractor to suggest entities from a text chunk. ```kotlin val extractor = LlmEntityExtractor(llmClient) val suggested = extractor.suggestEntities(chunk, context) // Process extracted entities suggested.suggestedEntities.forEach { entity -> println("Found: ${entity.name} (${entity.labels.joinToString(", ")})") println(" Summary: ${entity.summary}") } ``` -------------------------------- ### Memory Setup with Provenance for Citations Source: https://github.com/embabel/dice/blob/main/_autodocs/api-reference/memory.md Configure memory to include provenance information for citations, along with eager querying for active propositions. ```kotlin val memory = Memory.forContext("context-123") .withRepository(propositionRepository) .withProvenance(provenanceResolver) // Annotate with sources .withEagerQuery { it.withStatus(PropositionStatus.ACTIVE).withLimit(5) } ``` -------------------------------- ### Create EscalatingEntityResolver with Defaults Source: https://github.com/embabel/dice/blob/main/README.md Use the factory method for a simple setup with default configurations. Requires an entity repository and an LLM candidate bakeoff. ```kotlin val resolver = EscalatingEntityResolver.create( repository = entityRepository, candidateBakeoff = LlmCandidateBakeoff(ai, llmOptions, PromptMode.COMPACT) ) ``` -------------------------------- ### Create Memory for Context Source: https://github.com/embabel/dice/blob/main/_autodocs/api-reference/memory.md Starts creating a Memory for a given context. Requires a repository to be set subsequently. ```kotlin val memory = Memory.forContext("session-123") .withRepository(propositionRepository) .withEagerTopicSearch(5) ``` -------------------------------- ### Memory Setup with Eager Topic Loading Source: https://github.com/embabel/dice/blob/main/_autodocs/api-reference/memory.md Configure memory to eagerly load a specified number of top memories related to a topic, with a minimum confidence threshold. ```kotlin val memory = Memory.forContext("user-session-123") .withRepository(propositionRepository) .withTopic("classical music preferences") .withEagerTopicSearch(5) // Preload top 5 memories about topic .withMinConfidence(0.6) ``` -------------------------------- ### TemporalMetadata Usage Example Source: https://github.com/embabel/dice/blob/main/_autodocs/types.md Demonstrates how to create and use a TemporalMetadata object. This is useful for setting temporal context on propositions. ```kotlin val temporal = TemporalMetadata( observedAt = Instant.now(), validFrom = Instant.parse("2024-01-01T00:00:00Z"), validTo = null // Still valid ) prop.withTemporal(temporal) ``` -------------------------------- ### LlmEntityExtractor suggestEntities Usage Example Source: https://github.com/embabel/dice/blob/main/_autodocs/api-reference/entity-extractor.md Shows how to use the LlmEntityExtractor to extract entities and filter them by confidence score. ```kotlin val llmClient = EmbabelLlmClient(...) val extractor = LlmEntityExtractor(llmClient) val suggested = extractor.suggestEntities(chunk, context) val entities = suggested.suggestedEntities // Get high-confidence entities val confident = entities.filter { it.properties["confidence"] as? Double ?: 0.0 > 0.8 } ``` -------------------------------- ### Memory Setup with Scoped Entity Filter Source: https://github.com/embabel/dice/blob/main/_autodocs/api-reference/memory.md Initialize memory with a context, filter memories to a specific entity, and configure eager querying with ordering and limits. ```kotlin val memory = Memory.forContext("context-123") .withRepository(propositionRepository) .narrowedBy { it.withEntityId("user-alice") } // Only memories about Alice .withEagerQuery { it.orderedByEffectiveConfidence().withLimit(3) } .withMinConfidence(0.7) ``` -------------------------------- ### Enable API Key Authentication in application.yml Source: https://github.com/embabel/dice/blob/main/README.md Quick start configuration to enable API key authentication and define a list of valid keys. ```yaml # application.yml dice: security: api-key: enabled: true keys: - sk-your-secret-key-here - sk-another-key-for-different-client ``` -------------------------------- ### Create Proposition Instance Source: https://github.com/embabel/dice/blob/main/_autodocs/api-reference/proposition.md Example of how to use the `Proposition.create()` factory method to instantiate a Proposition object. This demonstrates providing values for required and some optional parameters. ```kotlin val prop = Proposition.create( id = "prop-001", contextIdValue = "conv-123", text = "Jim is an expert in GOAP", mentions = listOf( EntityMention("Jim", "Person", "entity-1", MentionRole.SUBJECT), EntityMention("GOAP", "Technology", "entity-2", MentionRole.OBJECT), ), confidence = 0.95, decay = 0.1, importance = 0.8, reasoning = "Clear statement of expertise", grounding = listOf("chunk-1"), created = Instant.now(), revised = Instant.now(), status = PropositionStatus.ACTIVE, ) ``` -------------------------------- ### LLM Reference Contribution Example Source: https://github.com/embabel/dice/blob/main/_autodocs/api-reference/memory.md Shows the formatted output for the Memory tool's contribution to an LLM system prompt, including its description, memory count, key memories, and usage notes. ```text Reference: memory Description: Memories about the user & context. 42 memories available. Key memories about the user & context: 1. The user lives in San Francisco 2. The user works at Acme Corp 3. The user's favorite music is classical [39 more retrievable via the memory tool] Notes: Use when: whenever you need to recall information about the user & context ``` -------------------------------- ### NewEntity Resolution Example Source: https://github.com/embabel/dice/blob/main/_autodocs/api-reference/entity-resolver.md Handles the case where a suggested entity is resolved as a new entity. Saves the recommended new entity to the repository. ```kotlin if (resolution is NewEntity) { val newEntity = resolution.recommended repository.save(newEntity) } ``` -------------------------------- ### Include API Key in Request Header Source: https://github.com/embabel/dice/blob/main/README.md Example of how to include the API key in the 'X-API-Key' header for making authenticated requests. ```bash curl -H "X-API-Key: sk-your-secret-key-here" \ http://localhost:8080/api/v1/contexts/user-123/memory ``` -------------------------------- ### Complex Multi-Tier Memory Configuration Source: https://github.com/embabel/dice/blob/main/_autodocs/api-reference/memory.md A comprehensive memory setup including context, topic, usage conditions, confidence, limits, filtering, eager querying, topic search, provenance, and a custom projector. ```kotlin val memory = Memory.forContext(contextId) .withRepository(propositionRepository) .withTopic("project status and team dynamics") .withUseWhen("answering questions about the current project and team") .withMinConfidence(0.5) .withDefaultLimit(15) .narrowedBy { it.withStatus(PropositionStatus.ACTIVE) } .withEagerQuery { q -> q.orderedByEffectiveConfidence().withLimit(5) } .withEagerTopicSearch(3) .withProvenance(provenanceResolver) .withProjector(customMemoryProjector) ``` -------------------------------- ### Example Usage of withContextId() Factory Method Source: https://github.com/embabel/dice/blob/main/_autodocs/api-reference/source-analysis-context.md Demonstrates how to use the withContextId factory method to construct a SourceAnalysisContext with specified configurations. ```kotlin val context = SourceAnalysisContext .withContextId("my-context") .withEntityResolver(AlwaysCreateEntityResolver.INSTANCE) .withSchema(schema) .build() ``` -------------------------------- ### Memory.forContext() Source: https://github.com/embabel/dice/blob/main/_autodocs/api-reference/memory.md Factory method to start creating a Memory instance for a given context. It requires a context ID and can be chained with repository and eager topic search configurations. ```APIDOC ## Memory.forContext() ### Description Factory method to start creating a Memory instance for a given context. It requires a context ID and can be chained with repository and eager topic search configurations. ### Method Factory Method ### Parameters #### Path Parameters - **contextId** (ContextId) - Required - The context to search within. - **contextIdValue** (String) - Required - The context to search within. ### Request Example ```kotlin val memory = Memory.forContext("session-123") .withRepository(propositionRepository) .withEagerTopicSearch(5) ``` ``` -------------------------------- ### ExistingEntity Resolution Example Source: https://github.com/embabel/dice/blob/main/_autodocs/api-reference/entity-resolver.md Handles the case where a suggested entity matches an existing entity. Updates the existing entity with the merged recommended data. ```kotlin if (resolution is ExistingEntity) { val merged = resolution.recommended // Update existing entity with merged data repository.update(merged) } ``` -------------------------------- ### Example Message Formatting Output Source: https://github.com/embabel/dice/blob/main/_autodocs/api-reference/incremental-analyzer.md Illustrates the expected output format when using MessageFormatter to format conversation messages. Each message is prefixed with the speaker and their role. ```text User (user): What is GOAP? Assistant (assistant): GOAP is a Goal-Oriented Action Planning system... User (user): How does it differ from HTN? ``` -------------------------------- ### Set Eager Query for Memory Source: https://github.com/embabel/dice/blob/main/_autodocs/api-reference/memory.md Use withEagerQuery to preload key memories into the description using a specified query. This makes memories immediately available to the LLM without a tool call. The query can be transformed, for example, to order by effective confidence and limit the results. ```kotlin .withEagerQuery { it.orderedByEffectiveConfidence().withLimit(5) } ``` -------------------------------- ### Set Eager Query for Memory (Java) Source: https://github.com/embabel/dice/blob/main/_autodocs/api-reference/memory.md Use withEagerQuery in Java to preload key memories into the description using a specified query. This makes memories immediately available to the LLM without a tool call. The query can be transformed, for example, to order by effective confidence and limit the results. ```java .withEagerQuery(query -> query.orderedByEffectiveConfidence().withLimit(5)) ``` -------------------------------- ### Get Propositions by Entity Source: https://github.com/embabel/dice/blob/main/README.md Retrieves propositions associated with a specific entity type and ID within a context. An example is provided for fetching propositions related to a Composer. ```bash curl http://localhost:8080/api/v1/contexts/{contextId}/memory/entity/{entityType}/{entityId} # Example curl http://localhost:8080/api/v1/contexts/user-123/memory/entity/Composer/composer-brahms ``` -------------------------------- ### count Source: https://github.com/embabel/dice/blob/main/_autodocs/api-reference/proposition-repository.md Get the total number of propositions in the repository. ```APIDOC ## count(): Int ### Description Get the total count of propositions. ### Returns Int Integer count ``` -------------------------------- ### Set Up Agent Memory Source: https://github.com/embabel/dice/blob/main/_autodocs/00-START-HERE.md Configure agent memory with a repository and specify eager topic search parameters. Implements LlmReference for system prompt contribution. ```kotlin val memory = Memory.forContext(contextId) .withRepository(repository) .withEagerTopicSearch(5) ``` -------------------------------- ### Get Proposition by ID Source: https://github.com/embabel/dice/blob/main/README.md Retrieves a specific proposition by its unique ID within a given context. ```APIDOC ## GET /api/v1/contexts/{contextId}/memory/{propositionId} ### Description Retrieves a specific proposition by its unique ID within a given context. ### Method GET ### Endpoint /api/v1/contexts/{contextId}/memory/{propositionId} ### Parameters #### Path Parameters - **contextId** (string) - Required - The ID of the context. - **propositionId** (string) - Required - The ID of the proposition to retrieve. ### Response #### Success Response (200) Returns the details of the specified proposition. The structure is similar to a single proposition object in the 'List Propositions' response. #### Response Example ```json { "id": "prop-xyz", "text": "User loves Brahms", "mentions": [{"name": "Brahms", "type": "Composer", "role": "OBJECT"}], "confidence": 0.95, "action": "CREATED" } ``` ``` -------------------------------- ### Initialize PropositionQuery with OrderBy (Java) Source: https://github.com/embabel/dice/blob/main/_autodocs/api-reference/proposition-query.md Shows how to construct a PropositionQuery in Java, setting the entity ID, minimum level, and ordering preference. ```java PropositionQuery query = PropositionQuery.againstContext("session-123") .withEntityId("user-123") .withMinLevel(0) .withOrderBy(PropositionQuery.OrderBy.EFFECTIVE_CONFIDENCE_DESC); ``` -------------------------------- ### Get Propositions by Entity Source: https://github.com/embabel/dice/blob/main/README.md Retrieves all propositions associated with a specific entity type and ID within a context. ```APIDOC ## GET /api/v1/contexts/{contextId}/memory/entity/{entityType}/{entityId} ### Description Retrieves all propositions associated with a specific entity type and ID within a context. ### Method GET ### Endpoint /api/v1/contexts/{contextId}/memory/entity/{entityType}/{entityId} ### Parameters #### Path Parameters - **contextId** (string) - Required - The ID of the context. - **entityType** (string) - Required - The type of the entity (e.g., Composer). - **entityId** (string) - Required - The ID of the entity (e.g., composer-brahms). ### Response #### Success Response (200) Returns a list of propositions related to the specified entity. #### Response Example ```json [ { "id": "prop-xyz", "text": "User loves Brahms", "mentions": [{"name": "Brahms", "type": "Composer", "role": "OBJECT"}], "confidence": 0.95, "action": "CREATED" } ] ``` ``` -------------------------------- ### Get Proposition by ID Source: https://github.com/embabel/dice/blob/main/README.md Fetches a specific proposition using its unique ID within a given context. ```bash curl http://localhost:8080/api/v1/contexts/{contextId}/memory/{propositionId} ``` -------------------------------- ### VetoedEntity Resolution Example Source: https://github.com/embabel/dice/blob/main/_autodocs/api-reference/entity-resolver.md Handles the case where a suggested entity is rejected and should be filtered out. Skips processing for this entity. ```kotlin if (resolution is VetoedEntity) { // Skip this entity entirely continue } ``` -------------------------------- ### Initialize PropositionQuery with OrderBy (Kotlin) Source: https://github.com/embabel/dice/blob/main/_autodocs/api-reference/proposition-query.md Demonstrates how to initialize a PropositionQuery in Kotlin, specifying the ordering by effective confidence. ```kotlin val query = PropositionQuery( contextId = sessionContext, entityId = "user-123", minLevel = 0, orderBy = OrderBy.EFFECTIVE_CONFIDENCE_DESC, ) ``` -------------------------------- ### Custom API Key Authenticator Implementation Source: https://github.com/embabel/dice/blob/main/README.md Example of a custom ApiKeyAuthenticator implementation in Kotlin that validates keys against a database. ```kotlin @Component class DatabaseApiKeyAuthenticator( private val apiKeyRepository: ApiKeyRepository, ) : ApiKeyAuthenticator { override fun authenticate(apiKey: String): AuthResult { val keyEntity = apiKeyRepository.findByKey(apiKey) ?: return AuthResult.Unauthorized("Invalid API key") if (keyEntity.isExpired()) { return AuthResult.Unauthorized("API key expired") } return AuthResult.Authorized( principal = keyEntity.clientId, metadata = mapOf("scopes" to keyEntity.scopes), ) } } ``` -------------------------------- ### LlmGraphProjector Initialization (Kotlin) Source: https://github.com/embabel/dice/blob/main/README.md Initializes LlmGraphProjector using an AI instance, relations, projection policy, and LLM options. Suitable for ambiguous or novel predicates where LLM inference is needed. ```kotlin // Kotlin val projector = LlmGraphProjector( ai = ai, relations = relations, policy = LenientProjectionPolicy(), llmOptions = llmOptions, ) ``` -------------------------------- ### Context Compression Strategies Source: https://github.com/embabel/dice/blob/main/README.md Demonstrates different strategies for context compression to reduce LLM token usage. Includes window-based, sentence-based, and adaptive approaches. ```kotlin // Compressor options: val compressor = WindowContextCompressor(windowChars = 100, maxSnippets = 3) val compressor = SentenceContextCompressor(maxSentences = 3) val compressor = AdaptiveContextCompressor() // Chooses strategy by length ``` -------------------------------- ### Custom ContentHasher Implementation Source: https://github.com/embabel/dice/blob/main/README.md Define a custom ContentHasher by implementing the fun interface. This example normalizes whitespace before hashing with SHA-256. ```kotlin // Default — SHA-256 val hasher: ContentHasher = Sha256ContentHasher // Custom — e.g., normalize whitespace before hashing val normalizingHasher = ContentHasher { Sha256ContentHasher.hash(text.trim().replace("\\s+".toRegex(), " ")) } ``` -------------------------------- ### LlmGraphProjector Initialization (Java) Source: https://github.com/embabel/dice/blob/main/README.md Initializes LlmGraphProjector using a builder pattern in Java. Configures the LLM, AI instance, relations, and projection policy. ```java // Java — builder pattern var projector = LlmGraphProjector .withLlm(projectionLlm) .withAi(ai) .withRelations(relations) .withLenientPolicy(); ``` -------------------------------- ### SourceAnalysisContext Factory Method: withContextId() Source: https://github.com/embabel/dice/blob/main/_autodocs/api-reference/source-analysis-context.md Starts building a SourceAnalysisContext with a given context ID. This is the entry point for the strongly-typed builder pattern. ```APIDOC ## Factory Method: withContextId() ### Description Starts building a SourceAnalysisContext with the given context ID. Entry point for the strongly-typed builder pattern. ### Method Signature ```kotlin @JvmStatic fun withContextId(contextId: String): WithContextId ``` ### Parameters #### Path Parameters - **contextId** (String) - Required - The context identifier for this analysis run ### Returns `WithContextId` builder step ### Example ```kotlin val context = SourceAnalysisContext .withContextId("my-context") .withEntityResolver(AlwaysCreateEntityResolver.INSTANCE) .withSchema(schema) .build() ``` ``` -------------------------------- ### Initialize and Use PropositionIncrementalAnalyzer Source: https://github.com/embabel/dice/blob/main/_autodocs/api-reference/incremental-analyzer.md Demonstrates how to initialize and use the PropositionIncrementalAnalyzer for processing conversation sources. Configure trigger intervals and batch sizes for incremental analysis. ```kotlin val analyzer = PropositionIncrementalAnalyzer( pipeline = propositionPipeline, formatter = MessageFormatter.INSTANCE, triggerInterval = Duration.ofMinutes(5), batchSize = 3, ) val source = ConversationSource(conversation) val result = analyzer.analyze(source, context) if (result != null) { // New propositions extracted repository.saveAll(result.allPropositions) } ``` -------------------------------- ### Build Proposition Query in Java Source: https://github.com/embabel/dice/blob/main/specs/introducing_blog.md Demonstrates the Java builder pattern for constructing a PropositionQuery, specifying context, entity ID, confidence, ordering, and limit. ```java // Java builder pattern PropositionQuery query = PropositionQuery.againstContext("session-123") .withEntityId("alice-123") .withMinEffectiveConfidence(0.5) .orderedByEffectiveConfidence() .withLimit(20); ``` -------------------------------- ### Set up DICE Proposition Pipeline Source: https://github.com/embabel/dice/blob/main/specs/introducing_blog.md Configure and build a proposition pipeline for processing text chunks. This involves defining a domain schema, setting up an extractor and reviser with LLM options, and then constructing the pipeline. ```kotlin // 1. Define your domain schema val schema = DataDictionary.fromClasses("myapp", Customer::class.java, Product::class.java) // 2. Set up extraction val extractor = LlmPropositionExtractor .withLlm(LlmOptions("gpt-4.1-mini")) .withAi(ai) .withSchemaAdherence(SchemaAdherence.DEFAULT) // 3. Set up revision (dedup + classification) val reviser = LlmPropositionReviser .withLlm(LlmOptions("gpt-4.1-mini")) .withAi(ai) .withClassifyLlm(LlmOptions("gpt-4.1-nano")) // cheaper model for classification // 4. Build the pipeline val pipeline = PropositionPipeline .withExtractor(extractor) .withRevision(reviser, repository) // 5. Process text val result = pipeline.processChunk(chunk, context) // result.propositions: extracted and resolved propositions // result.revisionResults: how each was classified against existing knowledge ``` -------------------------------- ### Streaming/Incremental Analysis Workflow Source: https://github.com/embabel/dice/blob/main/_autodocs/README.md Set up an incremental analyzer for continuous analysis of conversations. Configure trigger intervals for periodic analysis. ```kotlin val analyzer = PropositionIncrementalAnalyzer( pipeline = pipeline, formatter = MessageFormatter.INSTANCE, triggerInterval = Duration.ofMinutes(5) ) val result = analyzer.analyze(conversationSource, context) ``` -------------------------------- ### Memory Configuration Methods Source: https://github.com/embabel/dice/blob/main/_autodocs/api-reference/memory.md Methods to configure the Memory instance, including setting the repository, topic, usage conditions, projector, minimum confidence, and search result limit. ```APIDOC ## Memory Configuration Methods ### Description Methods to configure the Memory instance, including setting the repository, topic, usage conditions, projector, minimum confidence, and search result limit. ### Methods #### withRepository(repository: PropositionRepository): Memory Set the proposition repository to search. Required step after `forContext()`. **Parameters** - **repository** (PropositionRepository) - Required - The repository containing propositions. **Returns** Memory ready to use or configure further #### withTopic(topic: String): Memory Set the topic description for the memories. Should complete with the form "memories about ". **Parameters** - **topic** (String) - Required - Topic description. **Returns** New Memory with updated topic **Example** ```kotlin .withTopic("classical music preferences and listening habits") ``` #### withUseWhen(useWhen: String): Memory Set the description of when to use the memory tool. **Parameters** - **useWhen** (String) - Required - When to use the tool. **Returns** New Memory with updated description #### withProjector(projector: MemoryProjector): Memory Set the projector for categorizing memories by knowledge type. **Parameters** - **projector** (MemoryProjector) - Required - The projector to use. **Returns** New Memory with updated projector #### withMinConfidence(minConfidence: Double): Memory Set the minimum confidence threshold for returned memories. Memories with effective confidence below this are filtered out. **Parameters** - **minConfidence** (Double) - Required - Minimum confidence (0.0–1.0, default 0.5). **Returns** New Memory with updated confidence **Throws** IllegalArgumentException if not in range [0.0, 1.0] #### withDefaultLimit(limit: Int): Memory Set the default limit for search results. **Parameters** - **limit** (Int) - Required - Default maximum results (default 10). **Returns** New Memory with updated limit **Throws** IllegalArgumentException if limit <= 0 ``` -------------------------------- ### Create LLM Tools from EntityResolutionService Source: https://github.com/embabel/dice/blob/main/README.md Create a list of LLM tools from an EntityResolutionService instance. These tools can then be added to an agent's tool set. ```kotlin // Create tools from an EntityResolutionService val tools: List = EntityResolutionTools.asTools(entityResolutionService) // Add to an agent's tool set alongside other tools val allTools = prologTools + tools ``` -------------------------------- ### Java Memory Maintenance Source: https://github.com/embabel/dice/blob/main/README.md Perform memory maintenance using the Java API, including consolidation, abstraction, and retirement. This example demonstrates the equivalent functionality in Java. ```java MaintenanceResult result = MemoryMaintenanceOrchestrator .withRepository(propositionRepository) .withConsolidator(new DefaultMemoryConsolidator()) .withAbstractor(LlmPropositionAbstractor.withLlm(llm).withAi(ai)) .withRetireBelow(0.1) .maintain(contextId, sessionPropositions); ``` -------------------------------- ### Create Source Analysis Context Source: https://github.com/embabel/dice/blob/main/_autodocs/README.md Shows how to initialize a SourceAnalysisContext using a builder pattern in Java. This context is essential for analysis operations, requiring context ID, an entity resolver, and a schema. ```java SourceAnalysisContext context = SourceAnalysisContext .withContextId("analysis-1") .withEntityResolver(resolver) .withSchema(schema); ``` -------------------------------- ### Implement Custom CandidateSearcher Source: https://github.com/embabel/dice/blob/main/README.md Implement a custom CandidateSearcher by extending the interface. This example shows a custom searcher that finds an exact match from a provided data source. ```kotlin class MyCustomSearcher(private val myDataSource: MyDataSource) : CandidateSearcher { override fun search(suggested: SuggestedEntity, schema: DataDictionary): SearchResult { val match = myDataSource.findExact(suggested.name) return if (match != null) SearchResult.confident(match) else SearchResult.empty() } } ``` -------------------------------- ### ReferenceOnlyEntity Resolution Example Source: https://github.com/embabel/dice/blob/main/_autodocs/api-reference/entity-resolver.md Handles entities that should be referenced but not updated, typically for externally managed entities. Updates mentions to reference the existing entity's ID. ```kotlin if (resolution is ReferenceOnlyEntity) { // Use the existing entity but don't update it val entityId = resolution.existing.id proposition.mentions.forEach { if (it.span == resolution.suggested.name) { it.withResolvedId(entityId) } } } ``` -------------------------------- ### PropositionPipeline.withExtractor() Source: https://github.com/embabel/dice/blob/main/_autodocs/api-reference/proposition-pipeline.md Creates a new PropositionPipeline instance, initializing it with a specified PropositionExtractor. ```APIDOC ## PropositionPipeline.withExtractor() ### Description Creates a new PropositionPipeline instance, initializing it with a specified PropositionExtractor. ### Method Factory Method ### Parameters #### Path Parameters - **extractor** (PropositionExtractor) - Required - The proposition extractor to use. ### Returns PropositionPipeline ### Example ```kotlin val pipeline = PropositionPipeline .withExtractor(LlmPropositionExtractor(ai)) .withRevision(reviser, propositionRepository) ``` ``` -------------------------------- ### LlmEntityExtractor Initialization and Usage Source: https://github.com/embabel/dice/blob/main/README.md Initializes and uses the LLM-based entity extractor. Optionally, a custom prompt template can be provided. ```kotlin val extractor = LlmEntityExtractor .withLlm(llmOptions) .withAi(ai) // Optionally use a custom prompt template val customExtractor = extractor.withTemplate("my_entity_prompt") val entities = extractor.suggestEntities(chunk, context) ``` -------------------------------- ### Java: Query with Builder Pattern Source: https://github.com/embabel/dice/blob/main/README.md Use the builder pattern with 'withers' in Java to construct a PropositionQuery. Start with a factory method and chain filter and order specifications. ```java PropositionQuery query = PropositionQuery.againstContext("session-123") .withEntityId("alice-123") .withMinEffectiveConfidence(0.5) .orderedByEffectiveConfidence() .withLimit(20); List results = repository.query(query); ``` -------------------------------- ### Configure Eager Memory Loading Source: https://github.com/embabel/dice/blob/main/_autodocs/configuration.md Preload memories into the LLM system prompt for immediate access. Supports eager loading by topic similarity, custom queries, and recent conversation context. ```kotlin Memory.forContext(contextId) .withRepository(repository) .withTopic("user preferences") .withEagerTopicSearch(5) // Top 5 by topic similarity .withEagerQuery { q => // Top 3 by confidence q.orderedByEffectiveConfidence().withLimit(3) } .withEagerSearchAbout( TextSimilaritySearchRequest( query = recentConversationText, topK = 10 ) ) ``` -------------------------------- ### Filter Propositions by Creation Time Source: https://github.com/embabel/dice/blob/main/_autodocs/api-reference/proposition-query.md Use these methods to filter propositions based on their creation timestamp. You can specify a start time, end time, or a duration relative to the current time. ```kotlin query.createdSince(Duration.ofHours(1)) // Last hour query.createdSince(Duration.ofDays(7)) // Last week ``` -------------------------------- ### Memory.notes() Source: https://github.com/embabel/dice/blob/main/_autodocs/api-reference/memory.md Returns a brief string describing the context in which the Memory tool should be used. ```APIDOC ### notes(): String Returns a brief note about when to use the tool. #### Returns String describing usage context ``` -------------------------------- ### Kotlin Builder with Prompt Variables Source: https://github.com/embabel/dice/blob/main/_autodocs/api-reference/source-analysis-context.md Constructs a SourceAnalysisContext in Kotlin, adding prompt variables to the context. ```kotlin val context = SourceAnalysisContext .withContextId("analysis-1") .withEntityResolver(resolver) .withSchema(schema) .withPromptVariables( mapOf( "companyName" to "ACME Corp", "industry" to "Technology", "year" to 2024 ) ) ``` -------------------------------- ### Multiple SecurityFilterChain Beans for Different Matchers Source: https://github.com/embabel/dice/blob/main/README.md Example using multiple SecurityFilterChain beans with @Order to apply different security configurations (API key vs. OAuth/form login) based on path matchers. ```kotlin @Bean @Order(1) fun apiSecurityFilterChain(http: HttpSecurity): SecurityFilterChain { return http .securityMatcher("/api/v1/**") // API key auth config... .build() } @Bean @Order(2) fun webSecurityFilterChain(http: HttpSecurity): SecurityFilterChain { return http .securityMatcher("/**") // OAuth/form login config for web UI... .build() } ``` -------------------------------- ### Enable Eager Topic Search for Memory Source: https://github.com/embabel/dice/blob/main/_autodocs/api-reference/memory.md Enable eager topic-based similarity search using withEagerTopicSearch. This method preloads top matching memories into the description based on the topic field, allowing for immediate LLM access. Specify the maximum number of memories to preload. ```kotlin .withEagerTopicSearch(5) ``` -------------------------------- ### Add SNR Instruction to DICE Extraction Prompt Source: https://github.com/embabel/dice/blob/main/specs/gap-analysis.md Integrate rule 5 into the extraction prompt to guide the LLM towards generating information-dense propositions with a maximized signal-to-noise ratio. This aims to reduce verbosity and redundancy in extractions. ```jinja 5. **Dense and specific**: Each proposition should be information-dense — maximize signal-to-noise ratio. Avoid filler words, vague qualifiers, or restating context that's already captured in entity mentions. Prefer "Alice is a senior ML engineer at Google" over "Alice works at a company called Google where she is in a senior position doing machine learning engineering". ``` -------------------------------- ### Basic Java Builder Source: https://github.com/embabel/dice/blob/main/_autodocs/api-reference/source-analysis-context.md Constructs a SourceAnalysisContext in Java by setting the context ID, entity resolver, and schema. ```java SourceAnalysisContext context = SourceAnalysisContext .withContextId("my-context") .withEntityResolver(AlwaysCreateEntityResolver.INSTANCE) .withSchema(DataDictionary.fromClasses("myschema", Person.class, Technology.class)); ``` -------------------------------- ### Add Speaker Attribution Rule to Extraction Prompt Source: https://github.com/embabel/dice/blob/main/specs/gap-analysis.md This rule guides the extraction process to attribute facts to the correct speaker (user or assistant) and avoid extracting assistant responses as user facts. It should be added as rule 6 in the extraction prompt. ```jinja 6. **Speaker attribution**: When the text contains dialogue between a user and an assistant, extract facts about the USER only unless the assistant reveals factual information (e.g., the assistant's name or capabilities). Do not extract the assistant's responses as facts about the user. Always attribute facts to the correct speaker using their name as an entity mention. ``` -------------------------------- ### Kotlin: Query by Entity using Infix Notation Source: https://github.com/embabel/dice/blob/main/README.md Use infix notation to easily query propositions that mention a specific entity. ```kotlin val entityProps = repository.query( PropositionQuery mentioningEntity "alice-123" ) ``` -------------------------------- ### One-Shot Document Processing without Dedup (Java) Source: https://github.com/embabel/dice/blob/main/README.md Demonstrates calling `processOnce` in Java. The version without a `historyStore` argument bypasses deduplication. ```java // Java — processOnce with dedup ChunkPropositionResult result = pipeline.processOnce( documentText, "doc-123", context, historyStore); // Without dedup (no history store) ChunkPropositionResult result = pipeline.processOnce( documentText, "doc-123", context); ``` -------------------------------- ### Build Proposition Query in Kotlin Source: https://github.com/embabel/dice/blob/main/specs/introducing_blog.md Shows the Kotlin infix notation for creating a PropositionQuery, focusing on context ID. ```kotlin // Kotlin infix notation val query = PropositionQuery forContextId sessionContext ``` -------------------------------- ### Memory.tools() Source: https://github.com/embabel/dice/blob/main/_autodocs/api-reference/memory.md Returns a list containing the Memory tool itself. ```APIDOC ### tools(): List Returns the list of tools (just this Memory tool). #### Returns List ``` -------------------------------- ### Configure PropositionPipeline Extractor and Revision Source: https://github.com/embabel/dice/blob/main/_autodocs/configuration.md Set up the PropositionPipeline with a required extractor and optional revision and mention filter strategies. ```kotlin val pipeline = PropositionPipeline .withExtractor(LlmPropositionExtractor(ai)) .withRevision(reviser, propositionRepository) // Optional .withMentionFilter(filter) // Optional ``` -------------------------------- ### Contrast Propositions Between Two Groups Source: https://github.com/embabel/dice/blob/main/README.md Utilize LlmPropositionContraster to identify and articulate differences between two groups of propositions. This requires an LLM and an AI instance. ```kotlin val contraster = LlmPropositionContraster.withLlm(llm).withAi(ai) val aliceGroup = PropositionGroup("Alice", aliceProps) val bobGroup = PropositionGroup("Bob", bobProps) val differences = contraster.contrast(aliceGroup, bobGroup, targetCount = 3) // "Alice prefers morning meetings while Bob prefers afternoons" // "Alice and Bob have different language preferences (Python vs Java)" ``` -------------------------------- ### Basic Entity Extraction Source: https://github.com/embabel/dice/blob/main/_autodocs/api-reference/entity-extractor.md Demonstrates how to perform basic entity extraction from a single chunk of text using LlmEntityExtractor. Requires an AI client, a resolver, and a schema. ```kotlin val extractor = LlmEntityExtractor(aiClient) val context = SourceAnalysisContext .withContextId("analysis-1") .withEntityResolver(resolver) .withSchema(schema) val chunk = Chunk.create(text = "Jim works at Acme Corp") val suggested = extractor.suggestEntities(chunk, context) // Process results val people = suggested.suggestedEntities.filter { "Person" in it.labels } val companies = suggested.suggestedEntities.filter { "Company" in it.labels } ``` -------------------------------- ### withEagerQuery Source: https://github.com/embabel/dice/blob/main/_autodocs/api-reference/memory.md Sets an eager query to preload memories into the description. These memories are fetched using the provided query and are immediately available to the LLM without a tool call. ```APIDOC ## withEagerQuery(eagerQuery: UnaryOperator): Memory ### Description Set an eager query to preload key memories into the description. When set, the description will include memories fetched using this query, making them immediately available to the LLM without requiring a tool call. ### Parameters - **eagerQuery** (UnaryOperator) - Required - Function to transform the base query. ### Returns New Memory with eager query configured ### Example (Kotlin) ```kotlin .withEagerQuery { it.orderedByEffectiveConfidence().withLimit(5) } ``` ### Example (Java) ```java .withEagerQuery(query -> query.orderedByEffectiveConfidence().withLimit(5)) ``` ``` -------------------------------- ### Query and Project Propositions to Memory Source: https://github.com/embabel/dice/blob/main/README.md Query propositions from a repository and project them into different memory types. This is useful for retrieving and classifying factual, event-based, procedural, and working memory data. ```kotlin val props = repository.query( (PropositionQuery forContextId sessionContext) .withEntityId("alice-123") .withMinEffectiveConfidence(0.5) .orderedByEffectiveConfidence() .withLimit(50) ) val projector = DefaultMemoryProjector.DEFAULT val memory = projector.project(props) memory.semantic // factual knowledge memory.episodic // event-based memories memory.procedural // preferences and rules memory.working // session context // Access by type val facts = memory[KnowledgeType.SEMANTIC] ``` -------------------------------- ### Configure and Process with SourceAnalysisContext Source: https://github.com/embabel/dice/blob/main/README.md Configure the SourceAnalysisContext with entity resolvers and process data chunks. Access the entity resolution results after processing. ```kotlin // Configure context with entity resolver val context = SourceAnalysisContext .withContextId("session-123") .withEntityResolver( MultiEntityResolver( KnownEntityResolver( knownEntities = listOf(KnownEntity.asCurrentUser(currentUser)), delegate = repositoryResolver, ), InMemoryEntityResolver(defaultMatchStrategies()), ) ) .withSchema(dataDictionary) .withKnownEntities(KnownEntity.asCurrentUser(currentUser)) // Process chunks—entities automatically resolved val result = pipeline.process(chunks, context) // Access resolution results result.chunkResults.forEach { chunkResult -> chunkResult.entityResolutions.resolutions.forEach { resolution -> when (resolution) { is NewEntity -> println("Created: ${resolution.recommended.name}") is ExistingEntity -> println("Matched: ${resolution.existing.name}") is ReferenceOnlyEntity -> println("Referenced: ${resolution.existing.name}") is VetoedEntity -> println("Rejected: ${resolution.suggested.name}") } } } ``` -------------------------------- ### Create Proposition Directly Source: https://github.com/embabel/dice/blob/main/README.md Allows direct creation of a proposition within a context, including its text, mentions, confidence, and optional reasoning. ```bash curl -X POST http://localhost:8080/api/v1/contexts/{contextId}/memory \ -H "Content-Type: application/json" \ -d '{ "text": "User prefers morning meetings", "mentions": [ {"name": "User", "type": "User", "role": "SUBJECT"} ], "confidence": 0.9, "reasoning": "Explicitly stated preference" }' ``` -------------------------------- ### Proposition Constructor Options Source: https://github.com/embabel/dice/blob/main/_autodocs/configuration.md Use these options when creating a new Proposition. Key parameters include confidence, decay, importance, and status. ```kotlin Proposition.create( id: String, contextIdValue: String, text: String, mentions: List, confidence: Double, // 0.0-1.0, required decay: Double, // 0.0-1.0, default 0.0 importance: Double = 0.5, // 0.0-1.0 reasoning: String?, grounding: List, created: Instant, revised: Instant, lastAccessed: Instant = Instant.now(), status: PropositionStatus, level: Int = 0, // 0=raw, 1+=abstraction sourceIds: List = [], // For abstractions only reinforceCount: Int = 0, metadata: Map = {}, uri: String? = null, temporal: TemporalMetadata? = null, provenanceEntries: List = [], contentRevised: Instant = revised, metadataRevised: Instant = revised, pinned: Boolean = false, ) ``` -------------------------------- ### Build a Complex Proposition Query Source: https://github.com/embabel/dice/blob/main/_autodocs/api-reference/proposition-query.md Illustrates building a complex query in Kotlin by chaining multiple methods to filter by status, confidence, creation date, order results, and limit the count. ```kotlin val query = PropositionQuery.forContextId(contextId) .withStatus(PropositionStatus.ACTIVE) .withMinEffectiveConfidence(0.7) .createdSince(Duration.ofDays(7)) .orderedByEffectiveConfidence() .withLimit(20) ``` -------------------------------- ### Kotlin: Query by Context using Infix Notation Source: https://github.com/embabel/dice/blob/main/README.md Use infix notation for a concise way to query propositions scoped to a specific context. This is the primary scope for most queries. ```kotlin val contextProps = repository.query( PropositionQuery forContextId sessionContext ) ``` -------------------------------- ### Detailed API Key Configuration Options Source: https://github.com/embabel/dice/blob/main/README.md Configuration options for API key security, including enabling auth, defining keys, header name, and protected path patterns. ```yaml dice: security: api-key: enabled: true # Enable API key auth (default: false) keys: # List of valid API keys - sk-key-1 - sk-key-2 header-name: X-API-Key # Header name (default: X-API-Key) path-patterns: # Paths to protect (default: /api/v1/**) - /api/v1/** ``` -------------------------------- ### One-Shot Document Processing with Dedup Source: https://github.com/embabel/dice/blob/main/README.md Use `PropositionPipeline.processOnce` for bulk ingestion. It handles chunking, deduplication check, processing, and recording in a single call. Subsequent calls with identical content will return null. ```kotlin val pipeline = PropositionPipeline.withExtractor(extractor) val historyStore = InMemoryChunkHistoryStore() // or your persistent implementation // First call: processes and records val result = pipeline.processOnce( text = documentText, sourceId = "doc-123", context = context, historyStore = historyStore, ) // Second call with same content: returns null (already processed) val duplicate = pipeline.processOnce( text = documentText, sourceId = "doc-123", context = context, historyStore = historyStore, ) assert(duplicate == null) ``` -------------------------------- ### withPromptVariables() Method Source: https://github.com/embabel/dice/blob/main/_autodocs/api-reference/source-analysis-context.md Returns a copy of the SourceAnalysisContext with specified template model or prompt variables. Use this to provide additional data for LLM prompts. ```kotlin fun withPromptVariables(promptVariables: Map): SourceAnalysisContext ```