### Manage LLM Models with Context Provider

Source: https://context7.com/cactus-compute/demo-cactus-chat/llms.txt

This example shows how to manage LLM model selection, initialization, and inference hardware configuration using a React Context Provider. The `useModelContext` hook allows components to access and modify model-related states like the selected model, token limits, and inference hardware settings (CPU/GPU). It requires wrapping the application with `ModelProvider`.

```typescript
import { ModelProvider, useModelContext } from './contexts/modelContext';
import { CactusLM } from 'cactus-react-native';

// Wrap your app with ModelProvider
export default function App() {
  return (
    <ModelProvider>
      <ChatScreen />
    </ModelProvider>
  );
}

// Use the model context in components
function ChatScreen() {
  const {
    cactusContext,        // { lm: CactusLM, model: Model, inferenceHardware: [] }
    isContextLoading,     // true when model is loading
    availableModels,      // Array of downloaded models
    selectedModel,        // Currently active model
    setSelectedModel,     // Switch models
    tokenGenerationLimit, // Max tokens per generation
    setTokenGenerationLimit,
    inferenceHardware,    // ['cpu'] or ['gpu']
    setInferenceHardware,
    isReasoningEnabled,   // Enable <think></think> tags
    conversationId,       // Current conversation ID
    systemPrompt          // System prompt text
  } = useModelContext();

  const handleModelSwitch = (newModel) => {
    setSelectedModel(newModel); // Automatically reloads context
  };

  const enableGPU = () => {
    setInferenceHardware(['gpu']); // Triggers model reload with GPU
  };

  if (isContextLoading) {
    return <Text>Loading model...</Text>;
  }

  // Use cactusContext.lm for inference
  return <ChatInterface />;
}

```

--------------------------------

### Manage Local LLM Models with TypeScript

Source: https://context7.com/cactus-compute/demo-cactus-chat/llms.txt

This snippet demonstrates how to download, store, and manage Large Language Models (LLMs) on device storage using TypeScript. It covers storing model metadata, retrieving local models, getting the model directory path, and removing models. Dependencies include `expo-file-system` and custom storage and model services.

```typescript
import {
  storeLocalModel,
  getLocalModels,
  removeLocalModel,
  getFullModelPath,
  getModelDirectory
} from './services/storage';
import { Model } from './services/models';
import * as FileSystem from 'expo-file-system';

// Store model metadata after download
const model: Model = {
  value: 'llama-3.2-1b',
  label: 'Llama 3.2 1B',
  provider: 'Cactus',
  disabled: false,
  isLocal: true,
  meta: {
    fileName: 'llama-3.2-1b-q4.gguf',
    size: 1.2e9 // bytes
  }
};

await storeLocalModel(model);

// Retrieve all local models
const localModels = await getLocalModels();
for (const m of localModels) {
  const path = getFullModelPath(m.meta?.fileName || '');
  const info = await FileSystem.getInfoAsync(path);
  console.log(`${m.label}: ${info.exists ? 'exists' : 'missing'}`);
}

// Get model directory path
const modelDir = getModelDirectory();
console.log(`Models stored at: ${modelDir}`);

// Remove a model (deletes file and metadata)
await removeLocalModel('llama-3.2-1b');
```

--------------------------------

### Manage User Settings with AsyncStorage

Source: https://context7.com/cactus-compute/demo-cactus-chat/llms.txt

Persists user preferences and API keys using AsyncStorage. This includes token generation limits, inference hardware configuration, reasoning mode, system prompts, and language preferences. It provides functions to get and save these settings.

```typescript
import {
  getTokenGenerationLimit,
  saveTokenGenerationLimit,
  getInferenceHardware,
  saveInferenceHardware,
  getIsReasoningEnabled,
  saveIsReasoningEnabled,
  getSystemPrompt,
  saveSystemPrompt,
  getLanguagePreference,
  saveLanguagePreference
} from './services/storage';

// Token generation limits
const currentLimit = await getTokenGenerationLimit(); // Default: 1000
await saveTokenGenerationLimit(2048);

// Inference hardware configuration
const hardware = await getInferenceHardware(); // ['cpu'] or ['gpu']
await saveInferenceHardware(['gpu']); // Enable GPU acceleration

// Reasoning mode preference
const isReasoningOn = await getIsReasoningEnabled(); // Default: false
await saveIsReasoningEnabled(true);

// System prompt customization
const prompt = await getSystemPrompt();
// Default: 'You are Cactus, a very capable AI assistant running offline on a smartphone.'
await saveSystemPrompt('You are a helpful coding assistant.');

// Language preference for i18n
const lang = await getLanguagePreference(); // 'en', 'ru', etc.
await saveLanguagePreference('en');
```

--------------------------------

### Fetch Available Models for Device

Source: https://context7.com/cactus-compute/demo-cactus-chat/llms.txt

Queries a remote database to find AI models compatible with the device's hardware, specifically filtering by RAM. It uses `react-native-device-info` to get total memory and `fetchModelsAvailableToDownload` to retrieve model details like name, description, and download URL.

```typescript
import { fetchModelsAvailableToDownload } from './services/models';
import { getTotalMemory } from 'react-native-device-info';

// Fetches models from Supabase filtered by device RAM
const models = await fetchModelsAvailableToDownload();

for (const model of models) {
  console.log(`Name: ${model.name}`);
  console.log(`Description: ${model.comment}`);
  console.log(`Download: ${model.downloadUrl}`);
  console.log(`Default: ${model.default}`);
}

// Example response:
// [
//   {
//     name: 'llama-3.2-1b',
//     comment: 'Fast 1B parameter model for 4GB+ devices',
//     downloadUrl: 'https://example.com/llama-3.2-1b-q4.gguf',
//     default: true
//   },
//   {
//     name: 'llama-3.2-3b',
//     comment: 'Balanced 3B model for 8GB+ devices',
//     downloadUrl: 'https://example.com/llama-3.2-3b-q4.gguf',
//     default: false
//   }
// ]

// Device memory check
const totalGB = (await getTotalMemory()) / (2**30);
console.log(`Device has ${totalGB.toFixed(1)}GB RAM`);
```

--------------------------------

### Stream LLM Completions with Cactus

Source: https://context7.com/cactus-compute/demo-cactus-chat/llms.txt

This snippet demonstrates how to stream local LLM completions using the Cactus React Native library. It shows initialization of the CactusLM instance, setting up message queues, and defining callbacks for streaming progress and completion. Dependencies include 'cactus-react-native' and local service modules.

```typescript
import { streamLlamaCompletion } from './services/chat/llama-local';
import { CactusLM } from 'cactus-react-native';
import { Message, createUserMessage } from './components/ui/chat/ChatMessage';

// Initialize Cactus LM instance
const { lm, error } = await CactusLM.init({
  model: '/path/to/model.gguf',
  use_mlock: true,
  n_ctx: 2048,
  n_batch: 32,
  n_gpu_layers: 0, // Set to 99 for iOS GPU acceleration
});

const messages: Message[] = [
  createUserMessage('What is quantum computing?', selectedModel)
];

// Callback for streaming tokens
function onProgress(text: string) {
  console.log('Current response:', text);
  // Update UI with partial response
}

// Callback when generation completes
function onComplete(metrics, model, completeMessage) {
  console.log(`Generated ${metrics.completionTokens} tokens`);
  console.log(`Speed: ${metrics.tokensPerSecond} tok/sec`);
  console.log(`TTFT: ${metrics.timeToFirstToken}ms`);
  console.log('Final message:', completeMessage);
}

await streamLlamaCompletion(
  lm,
  messages,
  selectedModel,
  onProgress,
  onComplete,
  true, // streaming enabled
  1000, // max tokens
  false, // reasoning disabled
  false, // voice mode off
  'You are Cactus, an AI assistant.' // system prompt
);

```

--------------------------------

### Initialize CactusLM for On-Device LLM Inference in TypeScript

Source: https://context7.com/cactus-compute/demo-cactus-chat/llms.txt

This code initializes an on-device Large Language Model (LLM) using CactusLM in React Native, with support for hardware-accelerated inference. It demonstrates releasing previous LLM instances (iOS specific), configuring model parameters such as context size and GPU layer usage, and handling potential initialization errors. It also shows how to run a completion task locally and log response details and performance metrics.

```typescript
import { CactusLM, releaseAllLlama } from 'cactus-react-native';
import { Platform } from 'react-native';

// Release previous instances (iOS only, prevents memory issues)
if (Platform.OS === 'ios') {
  await releaseAllLlama();
}

// Initialize model with GPU acceleration (iOS)
const { lm, error } = await CactusLM.init({
  model: '/path/to/model.gguf',
  use_mlock: true,      // Lock model in memory
  n_ctx: 2048,          // Context window size
  n_batch: 32,          // Batch size for prompt processing
  n_gpu_layers: 99,     // Use GPU (iOS only, set to 0 for CPU)
});

if (error) {
  console.error('Failed to initialize:', error);
  return;
}

// Run completion
const result = await lm.completion(
  [
    { role: 'system', content: 'You are a helpful assistant.' },
    { role: 'user', content: 'Hello!' }
  ],
  {
    n_predict: 512,
    stop: ['</s>', '<|end|>', '<|eot_id|>']
  },
  (data) => {
    if (data.token) {
      console.log('Token:', data.token);
    }
  }
);

console.log('Response:', result.text);
console.log('Tokens:', result.timings?.predicted_n);
console.log('Speed:', result.timings?.predicted_per_second, 'tok/sec');
```

--------------------------------

### Stream Google Gemini Completion in TypeScript

Source: https://context7.com/cactus-compute/demo-cactus-chat/llms.txt

This snippet demonstrates how to stream responses from Google's Gemini models using the `streamGeminiCompletion` function. It includes saving the API key, initiating the streaming process with messages and model configuration, and handling partial text updates and completion metrics via callbacks. This is useful for real-time conversational AI experiences.

```typescript
import { streamGeminiCompletion } from './services/chat/gemini';

// Save API key
await saveApiKey('Google', 'AIza...');

// Stream completion from Gemini
await streamGeminiCompletion(
  messages,
  { value: 'gemini-1.5-flash', label: 'Gemini 1.5 Flash', provider: 'Google', disabled: false, isLocal: false },
  (partialText) => console.log('Streaming:', partialText),
  (metrics, model, completeText) => {
    console.log(`Tokens: ${metrics.completionTokens}`);
  },
  true,
  8192
);
```

--------------------------------

### Implement Voice Recognition with React Native and TypeScript

Source: https://context7.com/cactus-compute/demo-cactus-chat/llms.txt

This snippet demonstrates capturing user speech input using native voice recognition APIs within a React Native application using TypeScript. It includes requesting microphone permissions, setting up event listeners for speech start/end and results, and handling transcription. Dependencies include `@react-native-voice/voice`.

```typescript
import {
  startRecognizing,
  stopRecognizing,
  requestMicrophonePermission,
  removeEmojis
} from './utils/voiceFunctions';
import Voice from '@react-native-voice/voice';
import { useState } from 'react';

function VoiceInput() {
  const [isListening, setIsListening] = useState(false);
  const [results, setResults] = useState<string[]>([]);
  const [error, setError] = useState<string | null>(null);

  // Request permission before use
  const initVoice = async () => {
    const hasPermission = await requestMicrophonePermission(setError);
    if (!hasPermission) return;

    // Set up event listeners
    Voice.onSpeechStart = () => setIsListening(true);
    Voice.onSpeechEnd = () => setIsListening(false);
    Voice.onSpeechResults = (e) => {
      const cleanResults = e.value?.map(removeEmojis) || [];
      setResults(cleanResults);
    };
  };

  const start = async () => {
    await startRecognizing(setError, setIsListening);
  };

  const stop = async () => {
    await stopRecognizing(setError);
    const transcription = results[0] || '';
    console.log('User said:', transcription);
  };

  return { start, stop, isListening, results, error };
}
```

--------------------------------

### Enable Structured Reasoning Output with TypeScript

Source: https://context7.com/cactus-compute/demo-cactus-chat/llms.txt

This snippet illustrates how to enable structured reasoning output from a language model using special tags (e.g., `<think></think>`) in TypeScript. It shows how to format user messages for reasoning-enabled and disabled modes, and includes a function to parse reasoning tags from the model's response.

```typescript
import { createUserMessage } from './components/ui/chat/ChatMessage';

// Reasoning enabled - model uses <think></think> tags
const reasoningEnabled = true;
const userMessage = createUserMessage(
  reasoningEnabled ? 'Solve this problem: 2x + 5 = 15' : '/no_think Solve this problem: 2x + 5 = 15',
  selectedModel
);

// Model response with reasoning:
// <think>
// Need to isolate x. Subtract 5 from both sides: 2x = 10.
// Divide by 2: x = 5.
// </think>
// The solution is x = 5.

// Reasoning disabled - direct response only
const noReasoningMessage = createUserMessage(
  '/no_think Solve this problem: 2x + 5 = 15',
  selectedModel
);

// Model response without reasoning:
// The solution is x = 5.

// Parse reasoning from response
const parseResponse = (text: string) => {
  if (text.includes('<think>')) {
    const match = text.match(/<think>(.*?)</think>/s);
    const reasoning = match ? match[1].trim() : '';
    const response = text.replace(/<think>.*?</think>/s, '').trim();
    return { reasoning, response };
  }
  return { reasoning: '', response: text };
};
```

--------------------------------

### Log LLM Performance Diagnostics with TypeScript

Source: https://context7.com/cactus-compute/demo-cactus-chat/llms.txt

This snippet shows how to track and log model performance metrics to a remote database using TypeScript. It covers logging chat completion diagnostics (e.g., tokens per second, time to first token) and model load times. The diagnostics automatically include device memory statistics.

```typescript
import {
  logChatCompletionDiagnostics,
  logModelLoadDiagnostics
} from './services/diagnostics';

// Log chat completion performance
await logChatCompletionDiagnostics({
  llm_model: 'llama-3.2-1b',
  tokens_per_second: 15.7,
  time_to_first_token: 340, // milliseconds
  generated_tokens: 256,
  streaming: true
});

// Log model load time
await logModelLoadDiagnostics({
  model: 'llama-3.2-1b',
  loadTime: 2340 // milliseconds
});

// Data includes device memory stats automatically:
// - total_memory: Total device RAM
// - used_memory: Currently used RAM
// - device_id: Unique device identifier
```

--------------------------------

### Stream OpenAI Chat Completions

Source: https://context7.com/cactus-compute/demo-cactus-chat/llms.txt

Integrates with the OpenAI API to stream responses using Server-Sent Events. It allows saving API keys and then streaming completions based on provided messages and model configurations. It logs streaming progress and final metrics like Time To First Token (TTFT) and tokens per second.

```typescript
import { streamOpenAICompletion } from './services/chat/openai';
import { saveApiKey, getApiKey } from './services/storage';

// Save API key
await saveApiKey('OpenAI', 'sk-proj-...');

// Stream completion
const messages = [
  { id: '1', isUser: true, text: 'Explain neural networks', model: selectedModel }
];

await streamOpenAICompletion(
  messages,
  { value: 'gpt-4o-mini', label: 'GPT-4o Mini', provider: 'OpenAI', disabled: false, isLocal: false },
  (partialText) => console.log('Streaming:', partialText),
  (metrics, model, completeText) => {
    console.log(`TTFT: ${metrics.timeToFirstToken}ms`);
    console.log(`Tokens: ${metrics.completionTokens}`);
    console.log(`Speed: ${metrics.tokensPerSecond} tok/sec`);
  },
  true, // streaming
  2048  // max tokens
);
```

--------------------------------

### Stream Anthropic Chat Completions

Source: https://context7.com/cactus-compute/demo-cactus-chat/llms.txt

Integrates with the Anthropic API to stream Claude responses with real-time token generation. This function allows saving API keys and initiating streaming completions. It logs the complete text and provides metrics upon completion.

```typescript
import { streamAnthropicCompletion } from './services/chat/anthropic';

// Save API key
await saveApiKey('Anthropic', 'sk-ant-...');

// Stream completion from Claude
await streamAnthropicCompletion(
  messages,
  { value: 'claude-3-5-sonnet-20241022', label: 'Claude 3.5 Sonnet', provider: 'Anthropic', disabled: false, isLocal: false },
  (partialText) => console.log('Streaming:', partialText),
  (metrics, model, completeText) => {
    console.log(`Complete: ${completeText}`);
  },
  true,
  4096
);
```

--------------------------------

### Register Device for Diagnostics

Source: https://context7.com/cactus-compute/demo-cactus-chat/llms.txt

Registers the device with a remote analytics backend for diagnostics tracking. It uses `react-native-device-info` to gather brand, model, and OS version. The device is automatically registered on the first launch if no ID is found.

```typescript
import { registerDevice, getDeviceId } from './services/storage';
import { getBrand, getModel, getSystemVersion } from 'react-native-device-info';

// Register device on first launch
const { success, deviceId } = await registerDevice();

if (success && deviceId) {
  console.log(`Registered device: ${deviceId}`);
  // Device info sent:
  // - brand: e.g., 'Apple', 'Samsung'
  // - model: e.g., 'iPhone 14 Pro', 'SM-G998B'
  // - os_version: e.g., '17.1', '14'
}

// Retrieve stored device ID
const storedId = await getDeviceId();
if (storedId) {
  console.log(`Device ID: ${storedId}`);
} else {
  // Will auto-register if no ID found
  console.log('No device ID, registering...');
}
```

--------------------------------

### Manage Conversations with AsyncStorage

Source: https://context7.com/cactus-compute/demo-cactus-chat/llms.txt

This code snippet illustrates conversation management in Cactus Chat, utilizing AsyncStorage for local data persistence. It covers saving, retrieving, and deleting chat conversations. The `Conversation` interface defines the structure for storing chat history, including messages, title, and last updated timestamp.

```typescript
import {
  saveConversation,
  getConversations,
  getConversation,
  deleteConversation,
  Conversation
} from './services/storage';

// Create and save a new conversation
const conversation: Conversation = {
  id: 'conv_abc123',
  title: 'Quantum Computing Discussion',
  messages: [
    { id: 'msg_1', isUser: true, text: 'What is quantum computing?', model: selectedModel },
    { id: 'msg_2', isUser: false, text: 'Quantum computing...', model: selectedModel }
  ],
  lastUpdated: Date.now(),
  model: selectedModel
};

await saveConversation(conversation);

// Retrieve all conversations sorted by last updated
const allConversations = await getConversations();
console.log(`Found ${allConversations.length} conversations`);

// Get a specific conversation by ID
const loadedConv = await getConversation('conv_abc123');
if (loadedConv) {
  console.log(`Loaded: ${loadedConv.title}`);
  console.log(`Messages: ${loadedConv.messages.length}`);
}

// Delete a conversation
await deleteConversation('conv_abc123');

```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.