### Qwen3VL.prepare(_:cache:windowSize:) Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/qwen3vl/prepare%28_%3Acache%3Awindowsize%3A%29 Prepares the input for the Qwen3VL language model. This method handles the setup required before processing text, potentially involving tokenization, embedding, and managing a cache for performance. ```APIDOC ## func prepare(_:cache:windowSize:) ### Description Prepares the input for the Qwen3VL language model. This method handles the setup required before processing text, potentially involving tokenization, embedding, and managing a cache for performance. ### Method Instance Method ### Endpoint N/A (Instance Method) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body * **input** (LMInput) - Required - The input data for the language model. * **cache** ([any KVCache]) - Required - An array of key-value caches to be used during preparation. * **windowSize** (Int?) - Optional - Specifies the window size for processing. ### Request Example ```swift let lmInput: LMInput = ... // Initialize with appropriate data let kvCache: [any KVCache] = [...] // Initialize with cache objects let windowSize: Int? = 1024 do { let result = try qwen3vlInstance.prepare(lmInput, cache: kvCache, windowSize: windowSize) // Process the result } catch { print("Error preparing input: \(error)") } ``` ### Response #### Success Response (200) - **PrepareResult** - The result of the preparation, likely containing processed input or related data. #### Response Example ```json { "processedData": "...", "metadata": "..." } ``` ``` -------------------------------- ### Media Processing Pipeline Example Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/mediaprocessing Demonstrates a typical image preparation pipeline using the MediaProcessing enumeration. It shows how to convert an image to a specific color space, apply user-defined processing, resample to a target size, normalize pixel values, and finally convert the processed image to an MLXArray. ```swift var image: CIImage image = MediaProcessing.inSRGBToneCurveSpace(image) // Apply user instructions image = MediaProcessing.apply(image, processing: processing) image = MediaProcessing.resampleBicubic(image, to: config.size.cgSize) image = MediaProcessing.normalize( image, mean: config.imageMeanTuple, std: config.imageStdTuple) return MediaProcessing.asMLXArray(image) ``` -------------------------------- ### Swift Initializer for Qwen3VLConfiguration Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/qwen3vlconfiguration/init%28textconfiguration%3Avisionconfiguration%3Amodeltype%3Aignoreindex%3Aimagetokenid%3Avideot_6203ab_okenindex%3Avisionstarttokenid%3Avisionendtokenid%3Avisiontokenid%3Avocabsize%3Aeostokenid%3A%29 This Swift code defines an initializer for the Qwen3VLConfiguration class. It allows for the setup of text and vision configurations, model type, and various token IDs. The initializer includes default values for many parameters, simplifying common use cases. ```swift init( textConfiguration: Qwen3VLConfiguration.TextConfiguration, visionConfiguration: Qwen3VLConfiguration.VisionConfiguration, modelType: String = "qwen3_vl", ignoreIndex: Int = -100, imageTokenId: Int = 151_655, videoTokenId: Int = 151_656, imageTokenIndex: Int? = nil, videoTokenIndex: Int? = nil, visionStartTokenId: Int = 151_652, visionEndTokenId: Int = 151_653, visionTokenId: Int = 151_654, vocabSize: Int? = nil, eosTokenId: [Int]? = nil ) ``` -------------------------------- ### GET /websites/swiftpackageindex_ml-explore_mlx-swift-lm_main/deepstackVisualIndexes Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/qwen3vlconfiguration/visionconfiguration-swift.struct/deepstackvisualindexes Retrieves a list of integers representing deepstack visual indexes. ```APIDOC ## GET /websites/swiftpackageindex_ml-explore_mlx-swift-lm_main/deepstackVisualIndexes ### Description Retrieves a list of integers representing deepstack visual indexes. ### Method GET ### Endpoint /websites/swiftpackageindex_ml-explore_mlx-swift-lm_main/deepstackVisualIndexes ### Parameters None ### Request Example None ### Response #### Success Response (200) - **deepstackVisualIndexes** ([Int]) - An array of integers representing the deepstack visual indexes. #### Response Example { "deepstackVisualIndexes": [ 1, 2, 3 ] } ``` -------------------------------- ### GET /modelFactory Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/trampolinemodelfactory/modelfactory%28%29 Retrieves an instance of the ModelFactory. ```APIDOC ## GET /modelFactory ### Description This endpoint retrieves an instance of the ModelFactory, which can be used to create various models. ### Method GET ### Endpoint /modelFactory ### Parameters None ### Request Example None ### Response #### Success Response (200) - **factory** (any ModelFactory) - An instance conforming to the ModelFactory protocol. #### Response Example ```json { "factory": "" } ``` ``` -------------------------------- ### Get Number of Hidden Layers - Swift Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/idefics3configuration/textconfiguration/numhiddenlayers Retrieves the number of hidden layers configured for the Idefics3 model. This is a read-only property (`get`) and returns an `Int` value. ```swift var numHiddenLayers: Int { get } ``` -------------------------------- ### Qwen3VLProcessor Class Documentation Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/qwen3vlprocessor Documentation for the Qwen3VLProcessor class, detailing its initializers, methods, and conformance to UserInputProcessor. ```APIDOC ## Class Qwen3VLProcessor This class is part of the MLX-Swift library and is responsible for processing user input for the Qwen3VL model. ### Initializers * `init(Qwen3VLProcessorConfiguration, tokenizer: any Tokenizer)`: Initializes a new instance of `Qwen3VLProcessor` with the specified configuration and tokenizer. ### Instance Methods * `func prepare(input: UserInput) async throws -> LMInput`: Prepares the user input for the language model. * `func preprocess(images: [CIImage], processing: UserInput.Processing?) throws -> (MLXArray, THW)`: Preprocesses images for the language model. ### Relationships #### Conforms To * `MLXLMCommon.UserInputProcessor`: This class conforms to the `UserInputProcessor` protocol. ``` -------------------------------- ### Qwen25VLProcessorConfiguration Structure Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/qwen25vlprocessorconfiguration Detailed documentation for the Qwen25VLProcessorConfiguration structure, including its initializers, instance properties, and conformance to various protocols. ```APIDOC ## Qwen25VLProcessorConfiguration Structure Configuration for `Qwen25VLProcessor`. ### Description This structure holds the configuration parameters for the `Qwen25VLProcessor`, including image processing settings and size-related properties. ### Method N/A (Structure Definition) ### Endpoint N/A (Structure Definition) ### Parameters #### Path Parameters N/A #### Query Parameters N/A #### Request Body N/A ### Request Example ```json { "example": "N/A" } ``` ### Response #### Success Response (200) - **imageMean** ([CGFloat]) - The mean values for image normalization. - **imageMeanTuple** ((CGFloat, CGFloat, CGFloat)) - The mean values as a tuple. - **imageProcessorType** (String) - The type of image processor. - **imageStd** ([CGFloat]) - The standard deviation values for image normalization. - **imageStdTuple** ((CGFloat, CGFloat, CGFloat)) - The standard deviation values as a tuple. - **maxPixels** (Int) - The maximum number of pixels allowed. - **mergeSize** (Int) - The size for merging operations. - **minPixels** (Int) - The minimum number of pixels required. - **patchSize** (Int) - The size of image patches. - **size** (Qwen25VLProcessorConfiguration.Size) - The size configuration for the processor. - **temporalPatchSize** (Int) - The size of temporal patches. #### Response Example ```json { "imageMean": [0.485, 0.456, 0.406], "imageMeanTuple": [0.485, 0.456, 0.406], "imageProcessorType": "default", "imageStd": [0.229, 0.224, 0.225], "imageStdTuple": [0.229, 0.224, 0.225], "maxPixels": 1024, "mergeSize": 32, "minPixels": 256, "patchSize": 16, "size": { "width": 224, "height": 224 }, "temporalPatchSize": 1 } ``` ### Relationships #### Conforms To - `Swift.Decodable` - `Swift.Encodable` - `Swift.Sendable` ``` -------------------------------- ### Get ropeTheta Value (Swift) Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/qwen3vlconfiguration/textconfiguration-swift.struct/ropetheta Retrieves the 'ropeTheta' value, which is a Double. This property is part of the Qwen3VLConfiguration.TextConfiguration struct and is read-only. ```swift var ropeTheta: Double { get } ``` -------------------------------- ### Qwen2VLProcessor Instance Methods Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/qwen2vlprocessor Provides methods for preparing and preprocessing input for Qwen2VL models. 'prepare' takes UserInput and returns LMInput, while 'preprocess' handles image data. ```swift func prepare(input: UserInput) async throws -> LMInput ``` ```swift func preprocess(images: [CIImage], processing: UserInput.Processing?) throws -> (MLXArray, THW) ``` -------------------------------- ### Gemma3ProcessorConfiguration Initializer Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/gemma3processorconfiguration/imagesize-swift.struct/init%28from%3A%29 Documentation for the `init(from:)` initializer of the Gemma3ProcessorConfiguration, inherited from Decodable. ```APIDOC ## init(from:) ### Description Initializes a Gemma3ProcessorConfiguration instance by decoding it from a provided decoder. This initializer is inherited from the `Decodable` protocol. ### Method `init(from:)' ### Endpoint N/A (This is an initializer for a Swift struct) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```swift let decoder = JSONDecoder() let configuration = try decoder.decode(Gemma3ProcessorConfiguration.self, from: jsonData) ``` ### Response #### Success Response (N/A) N/A #### Response Example N/A ### Additional Information - Inherited from `Decodable.init(from:)' - Signature: `init(from decoder: any Decoder) throws` ``` -------------------------------- ### Get maxpPositionEmbeddings Property (Swift) Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/qwen2vlconfiguration/textconfiguration-swift.struct/maxppositionembeddings This snippet shows how to access the `maxpPositionEmbeddings` instance property of the Qwen2VLConfiguration. It is a read-only property that returns an Integer value. ```swift var maxpPositionEmbeddings: Int { get } ``` -------------------------------- ### Get MLXVLM Model Configuration Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/vlmmodelfactory/configuration%28id%3A%29 Retrieves the model configuration for a specified MLXVLM model ID. ```APIDOC ## GET /websites/swiftpackageindex_ml-explore_mlx-swift-lm_main/configuration/{id} ### Description Retrieves the model configuration for a specified MLXVLM model ID. This method is inherited from the `ModelFactory` class. ### Method GET ### Endpoint `/websites/swiftpackageindex_ml-explore_mlx-swift-lm_main/configuration/{id}` ### Parameters #### Path Parameters - **id** (String) - Required - The unique identifier for the MLXVLM model. ### Response #### Success Response (200) - **ModelConfiguration** (Object) - An object containing the configuration details for the model. - **[Fields of ModelConfiguration]** (Type) - Description of each configuration field. ``` -------------------------------- ### Qwen3VL Instance Methods Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/qwen3vl Outlines the instance methods of the Qwen3VL class, including a method to call the model with input and cache, a method to prepare language model input, and a method to sanitize weights. ```swift func callAsFunction(MLXArray, cache: [any KVCache]?) -> MLXArray func prepare(LMInput, cache: [any KVCache], windowSize: Int?) throws -> PrepareResult func sanitize(weights: [String : MLXArray]) -> [String : MLXArray] ``` -------------------------------- ### Language Model Prepare Instance Method Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/qwen2vl/prepare%28_%3Acache%3Awindowsize%3A%29 Details the `prepare` instance method for Language Models, used to prepare input with caching and window size options. ```APIDOC ## Instance Method prepare(_:cache:windowSize:) ### Description Prepares the language model for processing input, optionally using a cache and specifying a window size. ### Method `func` ### Endpoint N/A (Instance Method) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **input** (LMInput) - Required - The input data for the language model. - **cache** ([any KVCache]) - Required - An array of key-value caches to use during preparation. - **windowSize** (Int?) - Optional - The size of the window to consider for processing. ``` -------------------------------- ### Get useSlidingWindow Property - Swift Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/qwen25vlconfiguration/textconfiguration-swift.struct/useslidingwindow Retrieves the boolean value indicating whether sliding window is used for Qwen25VLConfiguration. This is an instance property with read-only access. ```swift var useSlidingWindow: Bool { get } ``` -------------------------------- ### Decodable init(from:) Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/qwen25vlconfiguration/baseconfiguration-swift.struct/init%28from%3A%29 Documentation for the inherited 'init(from:)' method from the Decodable protocol. ```APIDOC ## Decodable init(from:) ### Description Inherited from `Decodable.init(from:)`. This initializer is used to decode an instance of a type from a given decoder. ### Method `init(from: any Decoder) throws` ### Endpoint N/A (This is a method signature, not an API endpoint) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example None ### Response #### Success Response (N/A) None #### Response Example None ``` -------------------------------- ### Get numChannels Instance Property Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/idefics3configuration/visionconfiguration/numchannels This snippet shows how to access the `numChannels` instance property of the `Idefics3Configuration` class. It is a read-only property that returns an Integer value representing the number of channels. ```swift var numChannels: Int { get } ``` -------------------------------- ### Qwen25VL Class Documentation Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/qwen25vl Detailed documentation for the Qwen25VL class, including its overview, topics (initializers, properties, methods), and relationships (inheritance and conformance). ```APIDOC ## Qwen25VL ### Description Represents the Qwen2.5VL Vision-Language Model (VLM). ### Class Definition ```swift class Qwen25VL ``` ### Overview This VLM is typically created using `VLMModelFactory`. ### Topics #### Initializers * `init(Qwen25VLConfiguration)`: Initializes a new instance of Qwen25VL with the specified configuration. #### Instance Properties * `config` (Qwen25VLConfiguration): The configuration object for the Qwen25VL model. * `kvHeads` ([Int]): An array of integers representing KV heads. * `loraLayers` ([Module]): An array of LoRA layers. * `vocabularySize` (Int): The size of the vocabulary. #### Instance Methods * `callAsFunction(_:cache:) -> MLXArray`: A function to process input with optional caching. * `prepare(_:cache:windowSize:) throws -> PrepareResult`: Prepares the model for processing with optional caching and window size. * `sanitize(weights:) -> [String : MLXArray]`: Sanitizes the model's weights. #### Default Implementations * LanguageModel Implementations * LoRAModel Implementations ### Relationships #### Inherits From * `MLXNN.Module` #### Conforms To * `MLX.Evaluatable` * `MLX.IndentedDescription` * `MLX.Updatable` * `MLXLMCommon.KVCacheDimensionProvider` * `MLXLMCommon.LanguageModel` * `MLXLMCommon.LoRAModel` * `Swift.CustomStringConvertible` * `VLMModel` ``` -------------------------------- ### Get Localized Error Description (Swift) Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/vlmerror/localizeddescription Retrieves a localized string representation of the MLXVLM error. This property is inherited from the Error protocol and is available across multiple Apple platforms (iOS, macOS, tvOS, watchOS). ```swift var localizedDescription: String { get } ``` -------------------------------- ### Qwen3VL Initializer Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/qwen3vlconfiguration/init%28textconfiguration%3Avisionconfiguration%3Amodeltype%3Aignoreindex%3Aimagetokenid%3Avideot_6203ab_okenindex%3Avisionstarttokenid%3Avisionendtokenid%3Avisiontokenid%3Avocabsize%3Aeostokenid%3A%29 This section details the initializer for the Qwen3VL class, specifying the parameters required for setting up text and vision configurations, model specifics, and various token IDs. ```APIDOC ## Initializer: Qwen3VL ### Description Initializes a Qwen3VL object with specified text and vision configurations, model type, and token IDs for enhanced multimodal processing. ### Method `init` ### Parameters #### Initializer Parameters - **textConfiguration** (Qwen3VLConfiguration.TextConfiguration) - Required - Configuration for text processing. - **visionConfiguration** (Qwen3VLConfiguration.VisionConfiguration) - Required - Configuration for vision processing. - **modelType** (String) - Optional, defaults to "qwen3_vl" - The type of the model. - **ignoreIndex** (Int) - Optional, defaults to -100 - Index to ignore during processing. - **imageTokenId** (Int) - Optional, defaults to 151655 - Token ID for images. - **videoTokenId** (Int) - Optional, defaults to 151656 - Token ID for videos. - **imageTokenIndex** (Int?) - Optional - Specific index for image tokens. - **videoTokenIndex** (Int?) - Optional - Specific index for video tokens. - **visionStartTokenId** (Int) - Optional, defaults to 151652 - Token ID marking the start of vision input. - **visionEndTokenId** (Int) - Optional, defaults to 151653 - Token ID marking the end of vision input. - **visionTokenId** (Int) - Optional, defaults to 151654 - General token ID for vision. - **vocabSize** (Int?) - Optional - The vocabulary size of the model. - **eosTokenId** ([Int]?) - Optional - End-of-sequence token IDs. ### Code Example ```swift let textConfig = Qwen3VLConfiguration.TextConfiguration(...) let visionConfig = Qwen3VLConfiguration.VisionConfiguration(...) let qwenModel = Qwen3VL( textConfiguration: textConfig, visionConfiguration: visionConfig, modelType: "qwen3_vl", imageTokenId: 151655, visionStartTokenId: 151652 ) ``` ``` -------------------------------- ### Get attentionBias Property in Swift Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/qwen3vlconfiguration/textconfiguration-swift.struct/attentionbias This code snippet shows how to access the 'attentionBias' instance property of the Qwen3VL class. It is a read-only Boolean property. No specific dependencies are mentioned, and it returns a boolean value. ```swift var attentionBias: Bool { get } ``` -------------------------------- ### Get skipVision Instance Property (Swift) Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/qwen25vlconfiguration/visionconfiguration-swift.struct/skipvision Retrieves the value of the skipVision instance property, which is a boolean indicating whether vision processing should be skipped. This property is read-only. It is defined within the Qwen25VL class in the Qwen25VL.swift file. ```swift var skipVision: Bool { get } ``` -------------------------------- ### Qwen2VLProcessorConfiguration Initializer Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/qwen2vlprocessorconfiguration/size/init%28from%3A%29 Initializes a Qwen2VLProcessorConfiguration from a decoder. ```APIDOC ## Initializer: init(from:) ### Description Initializes a `Qwen2VLProcessorConfiguration` instance from a decoder. This method is inherited from the `Decodable` protocol. ### Method `init(from: any Decoder) throws` ### Endpoint N/A (This is an initializer, not an API endpoint) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example N/A ### Response #### Success Response (N/A) N/A #### Response Example N/A ``` -------------------------------- ### Swift: Get Model Factory Instance Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/trampolinemodelfactory/modelfactory%28%29 This Swift static function, `modelFactory()`, is designed to return an instance conforming to the `ModelFactory` protocol. It is inherited from `ModelFactoryTrampoline`. The function has no parameters and its return type is an optional `ModelFactory`. ```swift static func modelFactory() -> (any ModelFactory)? ``` -------------------------------- ### MLXVLM Initializer Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/qwen25vlprocessorconfiguration/size-swift.struct/init%2528from%253A%2529 Details about the `init(from:)` initializer inherited from `Decodable`. ```APIDOC ## Initializer: init(from:) ### Description This initializer is inherited from the `Decodable` protocol. It is used for decoding data from a source. ### Method `init(from: any Decoder) throws` ### Endpoint N/A (This is a Swift initializer, not an API endpoint) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```swift let decoder = JSONDecoder() let myObject = try decoder.decode(MyDecodableType.self, from: jsonData) ``` ### Response #### Success Response (N/A) N/A #### Response Example N/A ``` -------------------------------- ### Qwen3VLProcessor Initializer Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/qwen3vlprocessor Initializes a new instance of Qwen3VLProcessor. It requires a Qwen3VLProcessorConfiguration object and a tokenizer conforming to the Tokenizer protocol to set up the processor. ```swift `init(Qwen3VLProcessorConfiguration, tokenizer: any Tokenizer)` ``` -------------------------------- ### Get MLXVLM Model Configuration by ID (Swift) Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/vlmmodelfactory/configuration%28id%3A%29 Retrieves a ModelConfiguration for a specific MLXVLM model using its unique identifier. This method is inherited from the ModelFactory protocol. It takes a String ID as input and returns a ModelConfiguration object. ```swift func configuration(id: String) -> ModelConfiguration ``` -------------------------------- ### Idefics3Processor Class Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/idefics3processor Documentation for the Idefics3Processor class, its initializers, and instance methods. ```APIDOC ## Class Idefics3Processor ### Description Represents a processor for the Idefics3 model, handling input preparation. ### Topics #### Initializers `init(Idefics3ProcessorConfiguration, tokenizer: any Tokenizer)` - Initializes an Idefics3Processor with a configuration and a tokenizer. #### Instance Methods `func prepare(input: UserInput) throws -> LMInput` - Prepares the input data for the Idefics3 model. ### Relationships #### Conforms To * `MLXLMCommon.UserInputProcessor` ``` -------------------------------- ### SmolVLMProcessorConfiguration Initializer Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/smolvlmprocessorconfiguration/videosampling-swift.struct/init%28from%3A%29 This section details the initializer for SmolVLMProcessorConfiguration, inherited from Decodable. ```APIDOC ## init(from:) ### Description Initializes a SmolVLMProcessorConfiguration instance from a decoder. This initializer is inherited from the `Decodable` protocol. ### Method `init(from: Decoder)` ### Endpoint N/A (Initializers are not API endpoints) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example This is a Swift code example, not a typical API request. ```swift let decoder = JSONDecoder() let configuration = try decoder.decode(SmolVLMProcessorConfiguration.self, from: jsonData) ``` ### Response #### Success Response An instance of `SmolVLMProcessorConfiguration` is created. #### Response Example ```swift // Swift object instantiation let configuration = SmolVLMProcessorConfiguration(...) ``` ``` -------------------------------- ### ropeTraditional: Swift Boolean Property Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/idefics3configuration/textconfiguration/ropetraditional The ropeTraditional property is a Boolean value accessed via `get`. It is defined within the Idefics3.swift file and indicates whether a traditional rope configuration is employed. No specific dependencies or complex input/output are mentioned, but its boolean nature suggests a simple true/false state. ```swift var ropeTraditional: Bool { get } ``` -------------------------------- ### Qwen25VLProcessorConfiguration Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/qwen25vlprocessorconfiguration/size-swift.struct/init%2528from%253A%2529 Documentation for the `Qwen25VLProcessorConfiguration` and its nested `Size` type. ```APIDOC ## Qwen25VLProcessorConfiguration ### Description Represents the configuration for the Qwen25VL processor. ### Type `struct Qwen25VLProcessorConfiguration` ### Nested Types #### Size ##### Description Represents the size configuration for the processor. ##### Type `enum Size` ### Initializers #### `init(from: any Decoder) throws` ##### Description Inherited initializer from `Decodable` for `Qwen25VLProcessorConfiguration`. ##### Method `init(from decoder: any Decoder) throws` ### Endpoint N/A (This is a Swift type, not an API endpoint) ### Parameters None for the type itself. ### Request Example ```swift // Example usage would involve decoding JSON or other data into this struct ``` ### Response N/A ``` -------------------------------- ### Instance Method: prepare(input:) Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/qwen25vlprocessor/prepare%28input%3A%29 Processes user input into a language model input format. This method is inherited from the UserInputProcessor class. ```APIDOC ## Instance Method: prepare(input:) ### Description Processes user input into a language model input format. This method is inherited from the UserInputProcessor class. ### Method `async throws` ### Endpoint N/A (Instance Method) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **input** (UserInput) - Required - The user's input data. ### Request Example ```swift // Assuming 'mlxvlm' is an instance of MLXVLM let userInput: UserInput = ... // Initialize with user data let lmInput: LMInput = try await mlxvlm.prepare(input: userInput) ``` ### Response #### Success Response (200) - **lmInput** (LMInput) - The processed input ready for the language model. #### Response Example ```json { "processed_text": "...", "token_ids": [...], "attention_mask": [...] } ``` #### Error Handling - Throws an error if the input processing fails. ``` -------------------------------- ### Instance Method: prepare(input:) Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/gemma3processor/prepare%28input%3A%29 The prepare(input:) instance method is inherited from UserInputProcessor and is used to process user input into a format suitable for the language model. ```APIDOC ## prepare(input:) ### Description Processes user input into a format suitable for the language model. ### Method `async throws` ### Endpoint N/A (Instance Method) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **input** (`UserInput`) - Required - The user's input to be processed. ### Request Example ```swift let lmInput = try await gemma3Processor.prepare(input: userInputValue) ``` ### Response #### Success Response (200) - **LMInput** - The processed input ready for the language model. #### Response Example ```json { "processed_text": "example processed text", "token_ids": [1, 2, 3, 4] } ``` ``` -------------------------------- ### Qwen3VLConfiguration.VisionConfiguration Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/qwen3vlconfiguration/visionconfiguration-swift Documentation for the VisionConfiguration struct, including its initializers and instance properties. ```APIDOC ## Struct Qwen3VLConfiguration.VisionConfiguration ### Description Represents the vision configuration for the Qwen3VL model. ### Initializers - `init(from: any Decoder) throws`: Initializes a VisionConfiguration instance from a decoder. ### Instance Properties - `deepstackVisualIndexes` (Array) - The deepstack visual indexes. - `depth` (Int) - The depth of the vision configuration. - `hiddenAct` (String) - The activation function for hidden layers. - `hiddenSize` (Int) - The size of the hidden layers. - `inChannels` (Int) - The number of input channels. - `intermediateSize` (Int) - The size of the intermediate layer. - `modelType` (String) - The type of the model. - `numHeads` (Int) - The number of attention heads. - `numPositionEmbeddings` (Int) - The number of position embeddings. - `outHiddenSize` (Int) - The size of the output hidden layer. - `patchSize` (Int) - The size of the image patches. - `spatialMergeSize` (Int) - The size of the spatial merge. - `temporalPatchSize` (Int) - The size of the temporal patches. ### Conforms To - `Swift.Decodable` - `Swift.Encodable` - `Swift.Sendable` ``` -------------------------------- ### Qwen3VLConfiguration Instance Properties Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/qwen3vlconfiguration Lists the instance properties of the Qwen3VLConfiguration structure, detailing the various configuration parameters for the Qwen3VL model. This includes settings for text, vision, token handling, and vocabulary. ```swift var eosTokenId: [Int]? var ignoreIndex: Int var imageTokenId: Int var imageTokenIndex: Int let modelType: String let textConfiguration: Qwen3VLConfiguration.TextConfiguration var videoTokenId: Int var videoTokenIndex: Int let visionConfiguration: Qwen3VLConfiguration.VisionConfiguration var visionEndTokenId: Int var visionStartTokenId: Int var visionTokenId: Int var vocabSize: Int ``` -------------------------------- ### PaliGemma.prepare(_:cache:windowSize:) Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/paligemma/prepare%28_%3Acache%3Awindowsize%3A%29 Prepares the language model for inference with specified input, cache, and window size. This is an instance method inherited from the LanguageModel protocol. ```APIDOC ## func prepare(_:cache:windowSize:) ### Description Prepares the language model for inference with specified input, cache, and window size. This is an instance method inherited from the `LanguageModel` protocol. ### Method `instance method` ### Endpoint `N/A` (Instance Method) ### Parameters #### Path Parameters `N/A` #### Query Parameters `N/A` #### Request Body `N/A` ### Request Example `N/A` (Instance Method) ### Response #### Success Response (200) - **PrepareResult** - The result of the preparation, likely containing model state or intermediate data. #### Response Example ```json { "example": "PrepareResult" } ``` ### Errors - **throws** - Indicates that this method can throw errors during preparation. ``` -------------------------------- ### Qwen2VLProcessor Initialization Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/qwen2vlprocessor Initializes the Qwen2VLProcessor with a specific configuration and tokenizer. This initializer is crucial for setting up the processor for use with Qwen2VL models. ```swift init(Qwen2VLProcessorConfiguration, tokenizer: any Tokenizer) ``` -------------------------------- ### Gemma3Processor Class Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/gemma3processor Documentation for the Gemma3Processor class. ```APIDOC ## Class Gemma3Processor ```swift class Gemma3Processor ``` ### Description The Gemma3Processor class is responsible for processing user input for Gemma 3 models. ### Topics #### Initializers - `init(Gemma3ProcessorConfiguration, tokenizer: any Tokenizer)`: Initializes a new instance of Gemma3Processor. #### Instance Methods - `func prepare(input: UserInput) async throws -> LMInput`: Prepares the user input for the language model. - `func preprocess(images: [CIImage], processing: UserInput.Processing?) throws -> (MLXArray, THW)`: Preprocesses images for the language model. ### Relationships #### Conforms To - `MLXLMCommon.UserInputProcessor` ``` -------------------------------- ### Qwen3VLProcessor.prepare Method Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/qwen3vlprocessor Asynchronously prepares the user input for the language model. This method takes a UserInput object and returns an LMInput object, handling any necessary transformations or data preparation. ```swift `func prepare(input: UserInput) async throws -> LMInput` ``` -------------------------------- ### FastVLMProcessor Class Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/fastvlmprocessor Details about the FastVLMProcessor class, including its initializer and instance methods. ```APIDOC ## FastVLMProcessor ### Description The `FastVLMProcessor` class is responsible for processing user input for MLXLM models. It conforms to the `MLXLMCommon.UserInputProcessor` protocol. ### Topics #### Initializers - `init(FastVLMProcessorConfiguration, tokenizer: any Tokenizer)`: Initializes a new instance of `FastVLMProcessor` with the specified configuration and tokenizer. #### Instance Methods - `func prepare(input: UserInput) async throws -> LMInput`: Asynchronously prepares the user input into an `LMInput` format. This method can throw errors during preparation. ### Relationships #### Conforms To - `MLXLMCommon.UserInputProcessor` ``` -------------------------------- ### Gemma3 Class Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/gemma3 Documentation for the Gemma3 class. ```APIDOC ## Class Gemma3 ```swift class Gemma3 ``` ### Description Represents the Gemma3 language model. ### Topics #### Initializers * `init(Gemma3Configuration)`: Initializes a new Gemma3 model with the specified configuration. #### Instance Properties * `let config: Gemma3Configuration`: The configuration for the Gemma3 model. * `var kvHeads: [Int]`: The KV heads associated with the model. * `var vocabularySize: Int`: The size of the model's vocabulary. #### Instance Methods * `func callAsFunction(MLXArray, cache: [any KVCache]?) -> MLXArray`: Computes the output of the model for a given input array and optional cache. * `func newCache(parameters: GenerateParameters?) -> [any KVCache]`: Creates a new cache with the proper types for each layer, optionally with generation parameters. * `func prepare(LMInput, cache: [any KVCache], windowSize: Int?) throws -> PrepareResult`: Prepares the input for the model, including the cache and window size. * `func sanitize(weights: [String : MLXArray]) -> [String : MLXArray]`: Sanitizes the model's weights. #### Default Implementations * LanguageModel Implementations * LoRAModel Implementations ### Relationships #### Inherits From * `MLXNN.Module` #### Conforms To * `MLX.Evaluatable` * `MLX.IndentedDescription` * `MLX.Updatable` * `MLXLMCommon.KVCacheDimensionProvider` * `MLXLMCommon.LanguageModel` * `MLXLMCommon.LoRAModel` * `Swift.Copyable` * `Swift.CustomStringConvertible` * `VLMModel` ``` -------------------------------- ### Qwen3VL Instance Properties Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/qwen3vl Details the instance properties of the Qwen3VL class, including configuration, key-value cache heads, LoRA layers, and vocabulary size. ```swift let config: Qwen3VLConfiguration var kvHeads: [Int] var loraLayers: [Module] var vocabularySize: Int ``` -------------------------------- ### Qwen3VL Initializer Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/qwen3vl Initializes a Qwen3VL instance with a Qwen3VLConfiguration object. This is the primary way to create an instance of the Qwen3VL class. ```swift init(Qwen3VLConfiguration) ``` -------------------------------- ### Prepare LMInput with Cache and Window Size in Swift Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/qwen3vl/prepare%28_%3Acache%3Awindowsize%3A%29 This Swift code defines the 'prepare' instance method for the Qwen3VL model, inheriting from 'LanguageModel'. It takes an LMInput, an array of KVCache objects, and an optional window size, returning a PrepareResult. This method is crucial for setting up the language model for processing. ```swift func prepare( _ input: LMInput, cache: [any KVCache], windowSize _: Int? ) throws -> PrepareResult ``` -------------------------------- ### Qwen2VL Class Documentation Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/qwen2vl Detailed documentation for the Qwen2VL class, covering its structure, properties, methods, and relationships with other MLX modules and protocols. ```APIDOC ## Qwen2VL Class ### Description Represents the Qwen2VL Visual-Language Model (VLM). ### Overview This model is typically created using `VLMModelFactory`. It is designed for visual-language tasks and integrates with the MLX neural network library. ### Topics #### Initializers - `init(Qwen2VLConfiguration)`: Initializes a new instance of the Qwen2VL model with the specified configuration. #### Instance Properties - `config` (Qwen2VLConfiguration): The configuration object for the Qwen2VL model. - `kvHeads` ([Int]): An array of integers representing key-value heads, likely used for attention mechanisms. - `loraLayers` ([Module]): An array of LoRA (Low-Rank Adaptation) layers, used for efficient fine-tuning. - `vocabularySize` (Int): The size of the model's vocabulary. #### Instance Methods - `callAsFunction(MLXArray, cache: [any KVCache]?) -> MLXArray`: A method to call the model as a function, likely for inference, taking an MLXArray input and optional cache, returning an MLXArray output. - `prepare(LMInput, cache: [any KVCache], windowSize: Int?) throws -> PrepareResult`: Prepares the model for processing language model input, handling caching and window size. - `sanitize(weights: [String : MLXArray]) -> [String : MLXArray]`: Sanitizes the model's weights, potentially for loading or saving. #### Default Implementations - LanguageModel Implementations - LoRAModel Implementations ### Relationships #### Inherits From - `MLXNN.Module`: Inherits from the base MLX Neural Network Module. #### Conforms To - `MLX.Evaluatable`: Conforms to the protocol for evaluatable MLX objects. - `MLX.IndentedDescription`: Conforms to the protocol for indented description formatting. - `MLX.Updatable`: Conforms to the protocol for updatable MLX objects. - `MLXLMCommon.KVCacheDimensionProvider`: Conforms to the protocol for providing KVCache dimensions. - `MLXLMCommon.LanguageModel`: Conforms to the protocol for language models. - `MLXLMCommon.LoRAModel`: Conforms to the protocol for LoRA models. - `Swift.CustomStringConvertible`: Conforms to the standard Swift protocol for custom string representation. - `VLMModel`: Conforms to the base Visual-Language Model protocol. ``` -------------------------------- ### Instance Method: prepare(input:) Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/fastvlmprocessor/prepare%28input%3A%29 Prepares user input for the language model. This method is inherited from UserInputProcessor and is crucial for transforming raw user input into a format the language model can understand. ```APIDOC ## Instance Method prepare(input:) ### Description Prepares user input for the language model. This method is inherited from `UserInputProcessor` and is crucial for transforming raw user input into a format the language model can understand. ### Method `async throws` ### Endpoint N/A (Instance Method) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **input** (`UserInput`) - Required - The user's input data. ### Request Example ```swift // Assuming 'lmProcessor' is an instance of FastVLM or a subclass let userInput = UserInput(...) let lmInput = try await lmProcessor.prepare(input: userInput) ``` ### Response #### Success Response (200) - **LMInput** (`LMInput`) - The processed input ready for the language model. #### Response Example ```json { "processed_text": "...", "token_ids": [...] } ``` ``` -------------------------------- ### Idefics3ProcessorConfiguration Initializer Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/idefics3processorconfiguration/size-swift.struct/init%28from%3A%29 Details about the initializer for Idefics3ProcessorConfiguration, inherited from Decodable. ```APIDOC ## Initializer: init(from:) ### Description Initializes an Idefics3ProcessorConfiguration instance from a decoder. This initializer is inherited from the `Decodable` protocol. ### Method `init(from: any Decoder) throws` ### Endpoint N/A (Initializer) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example This is a Swift initializer and does not have a direct request body example in the same way as a REST API. ### Response #### Success Response (200) Initializes an `Idefics3ProcessorConfiguration` object. #### Response Example ```swift let configuration = try Idefics3ProcessorConfiguration(from: decoder) ``` ``` -------------------------------- ### Qwen25VLConfiguration Initializer Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/qwen25vlconfiguration/textconfiguration-swift.struct/init%28from%3A%29 Documentation for the `init(from:)` initializer inherited from `Decodable`. This initializer is used for decoding data into a Qwen25VLConfiguration object. ```APIDOC ## Initializer: init(from:) ### Description Initializes a `Qwen25VLConfiguration` object by decoding data from a given decoder. This method is inherited from the `Decodable` protocol. ### Method `init(from: any Decoder) throws` ### Endpoint N/A (This is an initializer, not a network endpoint) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **decoder** (`any Decoder`) - Required - The decoder to use for decoding the object. ### Request Example ```swift // Example of how to use the initializer (conceptual) let decoder = JSONDecoder() let configuration = try Qwen25VLConfiguration(from: decoder) ``` ### Response #### Success Response (Initialization) - **Qwen25VLConfiguration** - An initialized configuration object. #### Response Example N/A (Initializers do not return data in the same way as API responses. They modify the object instance.) ### Error Handling - **`throws`** - This initializer can throw an error if decoding fails. ``` -------------------------------- ### Idefics3 Class Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/idefics3 Details about the Idefics3 class, including its initializers, properties, and methods. ```APIDOC ## Class # Idefics3 ```swift class Idefics3 ``` ### Topics #### Initializers * `init(Idefics3Configuration)` ### Instance Properties * `let config: Idefics3Configuration` * `var headDim: IntOrPair` * `var kvHeads: [Int]` * `var loraLayers: [Module]` * `var vocabularySize: Int` ### Instance Methods * `func callAsFunction(MLXArray, cache: [any KVCache]?) -> MLXArray` * `func prepare(LMInput, cache: [any KVCache], windowSize: Int?) throws -> PrepareResult` * `func sanitize(weights: [String : MLXArray]) -> [String : MLXArray]` ### Default Implementations * LanguageModel Implementations * LoRAModel Implementations ## Relationships ### Inherits From * `MLXNN.Module` ### Conforms To * `MLX.Evaluatable` * `MLX.IndentedDescription` * `MLX.Updatable` * `MLXLMCommon.KVCacheDimensionProvider` * `MLXLMCommon.LanguageModel` * `MLXLMCommon.LoRAModel` * `Swift.CustomStringConvertible` * `VLMModel` ``` -------------------------------- ### LanguageModel Implementations Instance Methods Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/qwen3vl/languagemodel-implementations This section covers the instance methods available for the LanguageModel Implementations. These methods allow you to interact with language models, manage their cache, and perform operations. ```APIDOC ## LanguageModel Implementations Instance Methods ### Description Provides access to instance methods for LanguageModel Implementations. ### Methods - **callAsFunction**: Executes the language model with text input and optional cache and state. - **fuse**: Fuses the current model adapter with another. - **load**: Loads a model adapter. - **newCache**: Creates a new cache for the language model. - **perform (async)**: Performs an asynchronous operation with a model adapter. - **perform (sync)**: Performs a synchronous operation with a model adapter. - **unload**: Unloads a model adapter. ``` -------------------------------- ### Gemma3TextConfiguration Structure Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/gemma3textconfiguration Details the structure of Gemma3TextConfiguration, including its properties and initializers. ```APIDOC ## Struct Gemma3TextConfiguration ### Description Represents the text configuration for the Gemma3 model. ### Method N/A (Struct Definition) ### Endpoint N/A (Struct Definition) ### Parameters #### Instance Properties - **attentionHeads** (Int) - Variable - The number of attention heads. - **finalLogitSoftcapping** (Float?) - Constant - Optional final logit softcapping value. - **headDim** (Int) - Variable - The dimension of each head. - **hiddenLayers** (Int) - Constant - The number of hidden layers. - **hiddenSize** (Int) - Constant - The size of the hidden layers. - **intermediateSize** (Int) - Constant - The size of the intermediate layer. - **kvHeads** (Int) - Variable - The number of key-value heads. - **maxPositionEmbeddings** (Int) - Constant - The maximum number of position embeddings. - **mmTokensPerImage** (Int) - Constant - The number of tokens per image in multimodal context. - **modelType** (String) - Constant - The type of the model. - **queryPreAttnScalar** (Float) - Variable - Scalar for query pre-attention. - **rmsNormEps** (Float) - Constant - Epsilon value for RMS normalization. - **ropeGlobalBaseFreq** (Float) - Constant - Base frequency for global RoPE (Rotary Position Embeddings). - **ropeLocalBaseFreq** (Float) - Constant - Base frequency for local RoPE. - **ropeScaling** (Dictionary?) - Constant - RoPE scaling configuration. - **ropeTraditional** (Bool) - Constant - Indicates if traditional RoPE is used. - **slidingWindow** (Int) - Constant - The size of the sliding window. - **slidingWindowPattern** (Int) - Constant - The pattern for the sliding window. - **vocabularySize** (Int) - Constant - The size of the vocabulary. ### Initializers - `init(from: any Decoder) throws`: Initializes the configuration from a `Decoder`. ### Relationships #### Conforms To - `Swift.Decodable` - `Swift.Encodable` - `Swift.Sendable` ``` -------------------------------- ### Qwen25VLConfiguration.VisionConfiguration Initializer Source: https://swiftpackageindex.com/ml-explore/mlx-swift-lm/main/documentation/mlxvlm/qwen25vlconfiguration/visionconfiguration-swift.struct/init%28from%3A%29 Details the initializer for Qwen25VLConfiguration.VisionConfiguration, inherited from Decodable.init(from:). ```APIDOC ## Qwen25VLConfiguration.VisionConfiguration init(from:) ### Description Initializes a `Qwen25VLConfiguration.VisionConfiguration` instance from a decoder. This method is inherited from the `Decodable` protocol. ### Method `init(from: any Decoder) throws` ### Endpoint N/A (This is an initializer for a Swift struct) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```swift let decoder = JSONDecoder() let visionConfig = try decoder.decode(Qwen25VLConfiguration.VisionConfiguration.self, from: jsonData) ``` ### Response #### Success Response - **Qwen25VLConfiguration.VisionConfiguration** (Type) - An initialized instance of `Qwen25VLConfiguration.VisionConfiguration`. #### Response Example ```json { "some_vision_config_property": "value" } ``` ```