### Python Installation and Usage Source: https://docs.moondream.ai/quick-start Install the Moondream library using pip and use the Python client to interact with the vision AI model. This includes initializing the model with an API key, encoding images, generating captions, querying images, and detecting objects. It also shows how to stream responses. ```python # Install dependencies in your project directory # pip install moondream import moondream as md from PIL import Image # Initialize with API key model = md.vl(api_key="your-api-key") # Load an image image = Image.open("./path/to/image.jpg") encoded_image = model.encode_image(image) # Encode image (recommended for multiple operations) # Generate a caption (length options: "short" or "normal" (default)) caption = model.caption(encoded_image)["caption"] print("Caption:", caption) # Stream the caption for chunk in model.caption(encoded_image, stream=True)["caption"]: print(chunk, end="", flush=True) # Ask a question answer = model.query(encoded_image, "What's in this image?")["answer"] print("Answer:", answer) # Stream the answer for chunk in model.query(encoded_image, "What's in this image?", stream=True)["answer"]: print(chunk, end="", flush=True) # Detect objects detect_result = model.detect(image, 'subject') # change 'subject' to what you want to detect print("Detected objects:", detect_result["objects"]) ``` -------------------------------- ### Query Images with Moondream - Python Source: https://docs.moondream.ai/cloud/query Demonstrates how to initialize the Moondream client, load an image using Pillow, and ask questions about it. Includes examples for both regular and streamed responses. ```Python import moondream as md from PIL import Image # Initialize with API key model = md.vl(api_key="your-api-key") # Load an image image = Image.open("path/to/image.jpg") # Ask a question answer = model.query(image, "What's in this image?")["answer"] print("Answer:", answer) # Stream the response for chunk in model.query(image, "What's in this image?", stream=True)["answer"]: print(chunk, end="", flush=True) ``` -------------------------------- ### JavaScript Theme Management Source: https://docs.moondream.ai/quick-start This JavaScript code snippet manages the application's theme, likely for a web interface. It handles setting the theme in local storage, applying it to the document element, and respecting user preferences or system settings. ```javascript ((e,o,s,u,d,m,l,h)=>{ let i=document.documentElement, T=["light","dark"]; function p(a){ (Array.isArray(e)?e:[e]).forEach(g=>{ let k=g==="class", S=k&&m?d.map(f=>m[f]||f):d; k?(i.classList.remove(...S),i.classList.add(a)):i.setAttribute(g,a) }), R(a) } function R(a){ h&&T.includes(a)&&(i.style.colorScheme=a) } function c(){ return window.matchMedia("(prefers-color-scheme: dark)").matches?"dark":"light" } if(u)p(u); else try{ let a=localStorage.getItem(o)||s, g=l&&a==="system"?c():a; p(g) }catch(a){} })("class","theme","light","light",["light","dark"],null,true,true) ``` -------------------------------- ### Moondream JavaScript Client Library Usage Source: https://docs.moondream.ai/quick-start Demonstrates how to use the Moondream JavaScript client library to perform various image analysis tasks. Includes image loading, caption generation, querying, object detection, and pointing, with support for streaming responses. ```javascript // Install dependencies in your project directory // npm install moondream const { vl } = require('moondream'); const fs = require('fs'); async function main() { const model = new vl({ apiKey: "your-api-key" }); // Load an image const encodedImage = Buffer.from(fs.readFileSync("./path/to/image.jpg")) // Load and encode image // Generate caption (length options: "short" or "normal" (default)) const caption = await model.caption({ image: encodedImage }) console.log("Caption:", caption) // Stream the caption process.stdout.write("Streaming caption: ") const captionStream = await model.caption({ image: encodedImage, stream: true }) for await (const chunk of captionStream.caption) process.stdout.write(chunk) // Ask questions about the image const answer = await model.query({ image: encodedImage, question: "What's in this image?" }) console.log("\nAnswer:", answer) // Stream the answer process.stdout.write("Streaming answer: ") const answerStream = await model.query({ image: encodedImage, question: "What's in this image?", stream: true }) for await (const chunk of answerStream.answer) process.stdout.write(chunk) // Detect objects const detectResult = await model.detect({ image: encodedImage, object: "subject" }) // change 'subject' to what you want to detect console.log("Detected objects:", detectResult.objects) // Point at an object const pointResult = await model.point({ image: encodedImage, object: "subject" }) // change 'subject' to what you want to point at console.log("Points:", pointResult.points) } main().catch(console.error); ``` ```python # Example for Python (assuming a similar client library structure) # from moondream import MoondreamAPI # model = MoondreamAPI(api_key="your-api-key") # image = "path/to/image.jpg" # point_result = model.point(image, 'subject') # change 'subject' to what you want to point at # print("Points:", point_result["points"]) ``` -------------------------------- ### Jupyter Notebook Quick Start for Moondream Source: https://docs.moondream.ai/recipes This guide provides instructions for running Moondream within a Jupyter notebook or Google Colab environment. It includes integration with Google Drive for simplified image analysis and experimentation workflows. ```python # Source code for Jupyter Notebook Quick Start # https://github.com/vikhyat/moondream/blob/main/examples/jupyter_notebook.ipynb ``` -------------------------------- ### Moondream AI Cloud API Endpoints Source: https://docs.moondream.ai/quick-start Details on interacting with the Moondream AI cloud API using cURL. Covers common parameters and endpoint-specific options for querying, captioning, object detection, and pointing. Supports base64 encoded images and streaming. ```APIDOC Moondream AI Cloud API: Base URL: https://api.moondream.ai/v1 Authentication: Header: X-Moondream-Auth: YOUR_API_KEY Common Request Headers: Content-Type: application/json Common Request Body Parameters: image_url: string (data:image/jpeg;base64,) stream: boolean (Defaults to false. Supported by /query and /caption) Endpoints: 1. /query - Description: Ask natural language questions about images and receive detailed answers. - Method: POST - Parameters: - question: string (The question to ask about the image) - Example: curl --location 'https://api.moondream.ai/v1/query' \ --header 'X-Moondream-Auth: ${process.env.MOONDREAM_API_KEY}' \ --header 'Content-Type: application/json' \ --data '{ "image_url": "data:image/jpeg;base64,", "question": "What is this?", "stream": false }' 2. /caption - Description: Generate accurate and natural image captions. - Method: POST - Parameters: - length: string (Optional. 'short' or 'normal' (default) or 'long') - Example: curl --location 'https://api.moondream.ai/v1/caption' \ --header 'X-Moondream-Auth: ${process.env.MOONDREAM_API_KEY}' \ --header 'Content-Type: application/json' \ --data '{ "image_url": "data:image/jpeg;base64,", "length": "normal", "stream": false }' 3. /detect - Description: Detect and locate objects in images. - Method: POST - Parameters: - object: string (The object to detect and locate) - Example: curl --location 'https://api.moondream.ai/v1/detect' \ --header 'X-Moondream-Auth: ${process.env.MOONDREAM_API_KEY}' \ --header 'Content-Type: application/json' \ --data '{ "image_url": "data:image/jpeg;base64,", "object": "subject" }' 4. /point - Description: Get precise coordinate locations for objects in images. - Method: POST - Parameters: - object: string (The object to find coordinates for) - Example: curl --location 'https://api.moondream.ai/v1/point' \ --header 'X-Moondream-Auth: ${process.env.MOONDREAM_API_KEY}' \ --header 'Content-Type: application/json' \ --data '{ "image_url": "data:image/jpeg;base64,", "object": "subject" }' Note: Replace ${process.env.MOONDREAM_API_KEY} with your actual API key from console.moondream.ai and with your base64 encoded image data. ``` -------------------------------- ### cURL: Pointing to Objects via API Source: https://docs.moondream.ai/cloud/point Provides an example of how to use the cURL command-line tool to interact with the Moondream API for visual pointing. It demonstrates sending a POST request to the `/v1/point` endpoint with an API key, image data (as a Base64 string), and the object to detect. The `stream` parameter is also shown. ```bash curl --location 'https://api.moondream.ai/v1/point' \ --header 'X-Moondream-Auth: ' \ --header 'Content-Type: application/json' \ --data '{ "image_url": "data:image/jpeg;base64,", "object": "", "stream": false }' ``` -------------------------------- ### Install Moondream Python Package Source: https://docs.moondream.ai/cloud/point Installs the Moondream Python library using pip. This is the first step to using Moondream's capabilities in a Python environment. ```bash pip install moondream ``` -------------------------------- ### Load and Use Moondream2 Model with Transformers Source: https://huggingface.co/vikhyatk/moondream2 This Python code demonstrates how to load the Moondream2 vision language model using the Hugging Face `transformers` library. It shows how to initialize the model and tokenizer, specifying a revision for production stability and mapping the model to a device (e.g., 'cuda' or 'mps'). The snippet then provides examples for image captioning (short and normal length, with streaming), visual querying, object detection, and pointing. ```Python from transformers import AutoModelForCausalLM, AutoTokenizer from PIL import Image model = AutoModelForCausalLM.from_pretrained( "vikhyatk/moondream2", revision="2025-06-21", trust_remote_code=True, device_map={"": "cuda"} # ...or 'mps', on Apple Silicon ) # Captioning print("Short caption:") print(model.caption(image, length="short")["caption"]) print("\nNormal caption:") for t in model.caption(image, length="normal", stream=True)["caption"]: # Streaming generation example, supported for caption() and detect() print(t, end="", flush=True) print(model.caption(image, length="normal")) # Visual Querying print("\nVisual query: 'How many people are in the image?'") print(model.query(image, "How many people are in the image?")["answer"]) # Object Detection print("\nObject detection: 'face'") objects = model.detect(image, "face")["objects"] print(f"Found {len(objects)} face(s)") # Pointing print("\nPointing: 'person'") points = model.point(image, "person")["points"] print(f"Found {len(points)} person(s)") ``` -------------------------------- ### Moondream API Query - cURL Source: https://docs.moondream.ai/cloud/query Example of how to directly interact with the Moondream API using cURL, sending an image via a data URL and specifying the question and stream preference. ```cURL curl --location 'https://api.moondream.ai/v1/query' \ --header 'X-Moondream-Auth: ' \ --header 'Content-Type: application/json' \ --data '{ "image_url": "data:image/jpeg;base64,", "question": "What is this?", "stream": false }' ``` -------------------------------- ### Initialize Global Environment Variables Source: https://discord.com/invite/tRUdpjDQfH Sets up global environment variables for the application, including API endpoints, build information, and feature flags. This configuration is crucial for application runtime behavior and integration with backend services. ```javascript window.GLOBAL_ENV = { "NODE_ENV": "production", "BUILT_AT": "1753341537247", "HTML_TIMESTAMP": Date.now(), "BUILD_NUMBER": "422534", "PROJECT_ENV": "production", "RELEASE_CHANNEL": "stable", "VERSION_HASH": "f2291ea649b6bdc97f558ec9bdac48f23921baaf", "PRIMARY_DOMAIN": "discord.com", "SENTRY_TAGS": { "buildId": "f2291ea649b6bdc97f558ec9bdac48f23921baaf", "buildType": "normal" }, "SENTRY_RELEASE": "2025-07-24-f2291ea649b6bdc97f558ec9bdac48f23921baaf-discord_web", "PUBLIC_PATH": "/assets/", "LOCATION": "history", "API_VERSION": 9, "API_PROTOCOL": "https:", "API_ENDPOINT": "//discord.com/api", "GATEWAY_ENDPOINT": "wss://gateway.discord.gg", "STATIC_ENDPOINT": "", "ASSET_ENDPOINT": "//discord.com", "MEDIA_PROXY_ENDPOINT": "//media.discordapp.net", "IMAGE_PROXY_ENDPOINTS": "//images-ext-1.discordapp.net,//images-ext-2.discordapp.net", "CDN_HOST": "cdn.discordapp.com", "DEVELOPERS_ENDPOINT": "//discord.com", "MARKETING_ENDPOINT": "//discord.com", "WEBAPP_ENDPOINT": "//discord.com", "WIDGET_ENDPOINT": "//discord.com/widget", "SEO_ENDPOINT": "undefined", "NETWORKING_ENDPOINT": "//router.discordapp.net", "REMOTE_AUTH_ENDPOINT": "//remote-auth-gateway.discord.gg", "RTC_LATENCY_ENDPOINT": "//latency.discord.media/rtc", "INVITE_HOST": "discord.gg", "GUILD_TEMPLATE_HOST": "discord.new", "GIFT_CODE_HOST": "discord.gift", "ACTIVITY_APPLICATION_HOST": "discordsays.com", "MIGRATION_SOURCE_ORIGIN": "https://discordapp.com", "MIGRATION_DESTINATION_ORIGIN": "https://discord.com", "STRIPE_KEY": "pk_live_CUQtlpQUF0vufWpnpUmQvcdi", "ADYEN_KEY": "live_E3OQ33V6GVGTXOVQZEAFQJ6DJIDVG6SY", "BRAINTREE_KEY": "production_ktzp8hfp_49pp2rp4phym7387", "DEV_SESSION_KEY": "undefined" }; ``` -------------------------------- ### Next.js Initialization and Module Loading Source: https://moondream.ai/playground This snippet illustrates a common pattern in Next.js applications for initializing global state or registering modules. It utilizes an array `__next_f` to manage asynchronous operations and component loading, often seen in client-side bootstrapping. ```javascript (self.\"__next_f\"=self.\"__next_f\"||[]).push([0])self.\"__next_f\".push([1,\"1:\\"$Sreact.fragment\\"\\n2:I[604,[\\"8229\\",\\"static/chunks/9da6db1e-db5db33ba8c38e72.js\\",\\"3753\\",\\"static/chunks/3753-4fee6e4f91fb9b54.js\\",\\"7177\\",\\"static/chunks/app/layout-00d4a2111607c471.js\\"],\\"PostHogProvider\"]\n3:I[99304,[\\"8229\\",\\"static/chunks/9da6db1e-db5db33ba8c38e72.js\\",\\"3753\\",\\"static/chunks/3753-4fee6e4f91fb9b54.js\\",\\"7177\\",\\"static/chunks/app/layout-00d4a2111607c471.js\\"],\\"ThemeProvider\"]\n4:I[87555,[],\\"\"]\n5:I[92868,[\\"8229\\",\\"static/chunks/9da6db1e-db5db33ba8c38e72.js\\",\\"8039\\",\\"static/chunks/app/error-3e2a5e93a416c644.js\\"],\\"default\"]\n6:I[31295,[],\\"\"]\n7:I[56671,[\\"8229\\",\\"static/chunks/9da6db1e-db5db33ba8c38e72.js\\",\\"3753\\",\\"static/chunks/3753-4fee6e4f91fb9b54.js\\",\\"7177\\",\\"static/chunks/app/layout-00d4a2111607c471.js\\"],\\"Toaster\"]\n8:I[33063,[\\"7758\\",\\"static/chunks/7758-36253bf4a14dda1a.js\\",\\"6874\\",\\"static/chunks/6874-8f10c44d02c676b5.js\\",\\"9797\\",\\"static/chunks/9797-9655e38b35a0965a.js\\",\\"3063\\",\\"static/chunks/3063-1bc17252a979ee53.js\\",\\"4047\\",\\"static/chunks/4047-3b0b86da96104aad.js\\",\\"5138\\",\\"static/chunks/5138-2848e7384975d69e.js\\",\\"5461\\",\\"static/chunks/5461-edd08ae18b1947d4.js\\",\\"2043\\",\\"static/chunks/2043-e75f51357c677428.js\\",\\"8639\\",\\"static/chunks/8639-18ab4cbd72d015ca.js\\",\\"8251\\",\\"static/chunks/app/c/layout-1afe73982a6164f9.js\\"],\\"Image\"]\n9:I[45526,[\\"7758\\",\\"static/chunks/7758-36253bf4a14dda1a.js\\",\\"6874\\",\\"static/chunks/6874-8f10c44d02c676b5.js\\",\\"9797\\",\\"static/chunks/9797-9655e38b35a0965a.js\\",\\"3063\\",\\"static/chunks/3063-1bc17252a979ee53.js\\",\\"4047\\",\\"static/chunks/4047-3b0b86da96104aad.js\\",\\"5138\\",\\"static/chunks/5138-2848e7384975d69e.js\\",\\"5461\\",\\"static/chunks/5461-edd08ae18b1947d4.js\\",\\"2043\\",\\"static/chunks/2043-e75f51357c677428.js\\",\\"8639\\",\\"static/chunks/8639-18ab4cbd72d015ca.js\\",\\"8251\\",\\"static/chunks/app/c/layout-1afe73982a6164f9.js\\"],\\"Root\"]\na:I[45526,[\\"7758\\",\\"static/chunks/7758-36253bf4a14dda1a.js\\",\\"6874\\",\\"static/chunks/6874-8f10c44d02c676b5.js\\",\\"9797\\",\\"static/chunks/9797-9655e38b35a0965a.js\\",\\"3063\\",\\"static/chunks/3063-1bc17252a979ee53.js\\",\\"4047\\",\\"static/chunks/4047-3b0b86da96104aad.js\\",\\"5138\\",\\"static/chunks/5138-2848e7384975d69e.js\\",\\"5461\\",\\"static/chunks/5461-edd08ae18b1947d4.js\\",\\"2043\\",\\"static/chunks/2043-e75f51357c677428.js\\",\\"8639\\",\\"static/chunks/8639-18ab4cbd72d015ca.js\\",\\"8251\\",\\"static/chunks/app/c/layout-1afe73982a6164f9.js\\"],\\"List\"]\nb:I[45526,[\\"7758\\",\\"static/chunks/7758-36253bf4a14dda1a.js\\",\\"6874\\",\\"static/chunks/6874-8f10c44d02c676b5.js\\",\\"9797\\",\\"static/chunks/9797-9655e38b35a0965a.js\\",\\"3063\\",\\"static/chunks/3063-1bc17252a979ee53.js\\",\\"4047\\",\\"static/chunks/4047-3b0b86da96104aad.js\\",\\"5138\\",\\"static/chunks/5138-2848e7384975d69e.js\\",\\"5461\\",\\"static/chunks/5461-edd08ae18b1947d4.js\\",\\"2043\\",\\"static/chunks/2043-e75f51357c677428.js\\",\\"8639\\",\\"static/chunks/8639-18ab4cbd72d015ca.js\\",\\"8251\\",\\"static/chunks/app/c/layout-1afe73982a6164f9.js\\"],\\"Item\"]\nc:I[45526,[\\"7758\\",\\"static/chunks/7758-36253bf4a14dda1a.js\\",\\"6874\\",\\"static/chunks/6874-8f10c44d02c676b5.js\\",\\"9797\\",\\"static/chunks/9797-9655e38b35a0965a.js\\",\\"3063\\",\\"static/chunks/3063-1bc17252a979ee53.js\\",\\"4047\\",\\"static/chunks/4047-3b0b86da96104aad.js\\",\\"5138\\",\\"static/chunks/5138-2848e7384975d69e.js\\",\\"5461\\",\\"static/chunks/5461-edd08ae18b1947d4.js\\",\\"2043\\",\\"static/chunks/2043-e75f51357c677428.js\\",\\"8639\\",\\"static/chunks/8639-18ab4cbd72d015ca.js\\",\\"8251\\",\\"static/chunks/app/c/layout-1afe73982a6164f9.js\\"],\\"Link\"]\nd:I[45526,[\\"7758\\",\\"static/chunks/7758-36253bf4a14dda1a.js\\",\\"6874\\",\\"static/chunks/6874-8f10c44d02c676b5.js\\",\\"9797\\",\\"static/chunks/9797-9655e38b35a0965a.js\\",\\"3063\\",\\"static/chunks/3063-1bc17252a979ee53.js\\",\\"4047\\",\\"static/chunks/4047-3b0b86da96104aad.js\\",\\"5138\\",\\"static/chunks/5138-2848e7384975d69e.js\\",\\"5461\\",\\"static/chunks/5461-edd08ae18b1947d4.js\\",\\"2043\\",\\"static/chunks/2043-e75f51357c677428.js\\",\\"8639\\",\\"static/chunks/8639-18ab4cbd72d015c ``` -------------------------------- ### Download Moondream INT8 Model (Python) Source: https://docs.moondream.ai/specifications This Python script downloads the compressed INT8 model file for Moondream using the requests library. It makes an HTTP GET request to the specified URL and saves the content to a local file named 'moondream-2b-int8.mf.gz'. ```python import requests url = "https://huggingface.co/vikhyatk/moondream2/resolve/9dddae84d54db4ac56fe37817aeaeb502ed083e2/moondream-2b-int8.mf.gz" response = requests.get(url) with open("moondream-2b-int8.mf.gz", "wb") as f: f.write(response.content) ``` -------------------------------- ### Initialize Fast Connect WebSocket Source: https://discord.com/invite/tRUdpjDQfH Implements a "fast connect" mechanism for WebSockets, optimizing connection establishment based on available features (DiscordNative, Uint8Array, TextDecoder) and encoding types (ETF or JSON). It logs connection details and handles binary data types. ```javascript !function(){ if(null!=window.WebSocket && function(n){ try { var o=localStorage.getItem(n); if(null==o) return null; return JSON.parse(o); } catch(e) { return null; } }("token") && !window.__OVERLAY__) { var n = null!=window.DiscordNative||null!=window.require ? "etf" : "json"; var o = window.GLOBAL_ENV.GATEWAY_ENDPOINT + "/?encoding=" + n + "&v=" + window.GLOBAL_ENV.API_VERSION; if (void 0 !== window.Uint8Array && void 0 !== window.TextDecoder) { o += "&compress=zstd-stream"; } else if (void 0 !== window.Uint8Array) { o += "&compress=zlib-stream"; } console.log("[FAST CONNECT] " + o + ", encoding: " + n + ", version: " + window.GLOBAL_ENV.API_VERSION); var e = new WebSocket(o); e.binaryType = "arraybuffer"; var i = Date.now(); var r = { open: !1, identify: !1, gateway: o, messages: [] }; e.onopen = function() { console.log("[FAST CONNECT] connected in " + (Date.now() - i) + "ms"); r.open = !0; }; e.onclose = e.onerror = function() { window._ws = null; }; e.onmessage = function(n) { r.messages.push(n); }; window._ws = { ws: e, state: r }; } }(); ``` -------------------------------- ### Hugging Face Global Configuration Object Source: https://huggingface.co/vikhyatk/moondream2 This JavaScript object defines various global configuration settings for the Hugging Face platform. It includes feature flags (e.g., signup disabled), URLs for different services (SSH Git, Moon HTTP, datasets server, Spaces), API keys (captcha, Stripe, DocSearch, LogoDev), and environment details like 'production' and 'userAgent'. ```JavaScript window.hubConfig = {"features":{"signupDisabled":false},"sshGitUrl":"git@hf.co","moonHttpUrl":"https:\/\/huggingface.co","captchaApiKey":"bd5f2066-93dc-4bdd-a64b-a24646ca3859","captchaDisabledOnSignup":true,"datasetViewerPublicUrl":"https:\/\/datasets-server.huggingface.co","stripePublicKey":"pk_live_x2tdjFXBCvXo2FFmMybezpeM00J6gPCAAc","environment":"production","userAgent":"HuggingFace (production)","spacesIframeDomain":"hf.space","spacesApiUrl":"https:\/\/api.hf.space","docSearchKey":"ece5e02e57300e17d152c08056145326e90c4bff3dd07d7d1ae40cf1c8d39cb6","logoDev":{"apiUrl":"https:\/\/img.logo.dev\/","apiKey":"pk_UHS2HZOeRnaSOdDp7jbd5w"}}; ``` -------------------------------- ### Initialize Moondream AI and Load Stripe Source: https://huggingface.co/vikhyatk/moondream2 This snippet initializes the Moondream AI integration by importing necessary modules and setting global variables. It also conditionally loads the Stripe.js library for payment processing if the script is running on a Hugging Face domain. ```javascript import("\/front\/build\/kube-488f90f\/index.js"); window.moonSha = "kube-488f90f\/"; window.__hf_deferred = {}; if (["hf.co", "huggingface.co"].includes(window.location.hostname)) { const script = document.createElement("script"); script.src = "https://js.stripe.com/v3/"; script.async = true; document.head.appendChild(script); } ``` -------------------------------- ### Python: Pointing to Objects in Images Source: https://docs.moondream.ai/cloud/point Demonstrates how to use the Moondream Python client library to locate objects in an image. It covers loading an image, initializing the visual language model, and calling the `point` method with a text query. The output is a list of dictionaries containing normalized 'x' and 'y' coordinates. ```python import moondream as md from PIL import Image model = md.vl(api_key="your-api-key") # Load an image (supports PIL.Image, numpy arrays, or file paths) image = Image.open("./image.jpg") # Returns List[Dict[str, float]] points = model.point(image, "the red car") print(points) # [{"x": 0.4, "y": 0.6}] # Normalized coordinates (0,0) is top-left, (1,1) is bottom-right ``` -------------------------------- ### Node.js: Pointing to Objects in Images Source: https://docs.moondream.ai/cloud/point Illustrates using the Moondream Node.js client library for object localization. It shows how to initialize the visual language model with an API key, load an image from the file system, and call the `point` method. The function returns a Promise resolving to an array of objects with 'x' and 'y' coordinates. ```javascript import { vl } from "moondream"; import fs from 'fs'; const model = new vl({ apiKey: "your-api-key" }); // Load image (supports Buffer or Base64EncodedImage: { imageUrl: string }) const image = fs.readFileSync('./image.jpg'); // recommended for better performance // Returns Promise> const points = await model.point(image, "the red car"); ``` -------------------------------- ### Application Routing and Page Structure Source: https://moondream.ai/playground Defines the application's page structure, including routing logic, HTML structure, and theme management. It specifies the 'playground' page and includes references to CSS for styling. ```javascript self.__next_f.push([ 1, "0:{\"P\":null,\"b\":\"9OJkQHfr9ndufJAN_7JJF\",\"p\":\"\",\"c\":[\"\",\"c\",\"playground\"],\"i\":false,\"f\":[[[\"\",{\"children\":[\"c\",{\"children\":[\"playground\",{\"children\":[\"__PAGE__\",{}]}]}]},\"$undefined\",\"$undefined\",true],[\"\",[\"$\",\"$1\",\"c\",{\"children\":[[[\"$\",\"link\",\"0\",{\"rel\":\"stylesheet\",\"href\":\"/_next/static/css/e815c6c0a6f6d27d.css\",\"precedence\":\"next\",\"crossOrigin\":\"$undefined\",\"nonce\":\"$undefined\"}],[\"$\",\"link\",\"1\",{\"rel\":\"stylesheet\",\"href\":\"/_next/static/css/284721261fd0726d.css\",\"precedence\":\"next\",\"crossOrigin\":\"$undefined\",\"nonce\":\"$undefined\"}]]],[{\"$\",\"html\",null,{\"lang\":\"en\",\"suppressHydrationWarning\":true,\"className\":\"scroll-smooth\",\"children\":[[[\"$\",\"body\",null,{\"className\":\"__variable_3a0388 antialiased\",\"children\":[[[\"$\",\"$L2\",null,{\"children\":[[[\"$\",\"$L3\",null,{\"attribute\":\"class\",\"defaultTheme\":\"system\",\"enableSystem\":true,\"disableTransitionOnChange\":true,\"children\":[[[\"$\",\"main\",null,{\"children\":[[[\"$\",\"$L4\",null,{\"parallelRouterKey\":\"children\",\"error\":\"$5\",\"errorStyles\":[],\"errorScripts\":[],\"template\":[[[\"$\",\"$L6\",null,{}]]},\"templateStyles\":\"$undefined\",\"templateScripts\":\"$undefined\",\"notFound\":[[[\"$\",\"title\",null,{\"children\":\"404: This page could not be found.\"}],[\"$\",\"div\",null,{\"style\":{\"fontFamily\":\"system-ui,\\\"Segoe UI\\\",Roboto,Helvetica,Arial,sans-serif,\\\"Apple Color Emoji\\\",\\\"Segoe UI Emoji\\\"\",\"height\":\"100vh\",\"textAlign\":\"center\",\"display\":\"flex\",\"flexDirection\":\"column\",\"alignItems\":\"center\",\"justifyContent\":\"center\"},\"children\":[[[\"$\",\"div\",null,{\"children\":[[[\"$\",\"style\",null,{\"dangerouslySetInnerHTML\":{\"__html\":\"body{color:#000;background:#fff;margin:0}.next-error-h1{border-right:1px solid rgba(0,0,0,.3)}@media (prefers-color-scheme:dark){body{color:#fff;background:#000}.next-error-h1{border-right:1px solid rgba(255,255,255,.3)}}\"}}],[[\"$\",\"h1\",null,{\"className\":\"next-error-h1\",\"style\":{\"display\":\"inline-block\",\"margin\":\"0 20px 0 0\",\"padding\":\"0 23px 0 0\",\"fontSize\":24,\"fontWeight\":500,\"verticalAlign\":\"top\",\"lineHeight\":\"49px\"},\"children\":404}]],[[\"$\",\"div\",null,{\"style\":{\"display\":\"inline-block\"},\"children\":[[[\"$\",\"h2\",null,{\"style\":{\"fontSize\":14,\"fontWeight\":400,\"lineHeight\":\"49px\",\"margin\":0},\"children\":\"This page could not be found.\"}]}]}]}]}]],\"forbidden\":\"$undefined\",\"unauthorized\":\"$undefined\"}]]],[[\"$\",\"$L7\",null,{\"richColors\":true,\"position\":\"bottom-center\"}]]]}]]]}]]]}]]],{\"children\":[[[\"c\",[[[\"$\",\"$1\",\"c\",{\"children\":[[null,[[[\"$\",\"div\",null,{\"className\":\"relative flex min-h-svh flex-col bg-background\",\"children\":[[[\"$\",\"div\",null,{\"className\":\"border-dashed border-border flex flex-1 flex-col\",\"childr ``` -------------------------------- ### Detect Objects with Moondream Node.js SDK Source: https://docs.moondream.ai/cloud/detect Shows how to initialize the Moondream visual language model and detect objects in an image using the Node.js client library. Uses fs for file system operations. ```javascript import { vl } from "moondream"; import fs from "fs"; const model = new vl({ apiKey: "your-api-key" }); const image = fs.readFileSync("path/to/image.jpg"); async function main() { // Regular response (replace 'subject' with what you want to detect) const detect_result = await model.detect({ image, subject: "subject" }); console.log(detect_result); } main(); ``` -------------------------------- ### Interactive Image Analysis Demo with Moondream Source: https://docs.moondream.ai/recipes A Gradio-based web interface designed for real-time image analysis. This demo features visual question answering (VQA) and object detection capabilities, complete with bounding box visualization for detected objects. ```python # Source code for Interactive Image Analysis Demo # https://github.com/vikhyat/moondream/blob/main/gradio_demo.py ``` -------------------------------- ### Hugging Face Transformers Implementation Source: https://huggingface.co/vikhyatk/moondream2 Demonstrates the use of the `compile()` function from the Hugging Face Transformers library, inspired by gpt-fast. This allows for optimized model compilation. ```Python from transformers import AutoModelForCausalLM # Load a model (example) model = AutoModelForCausalLM.from_pretrained("vikhyatk/moondream2") # Compile the model using gpt-fast style (hypothetical usage) # Note: Actual compile function might be part of a specific optimization library # or a method on the model/trainer object depending on the Transformers version. # This is a conceptual representation based on the changelog. # compiled_model = model.compile() # Example of enabling reasoning in a query (conceptual) # response = model.query(prompt="What is in the image?", image=image, reasoning=True) print("Model loaded and ready for optimized inference.") ``` -------------------------------- ### Download Moondream 0.5B INT4 Model Source: https://docs.moondream.ai/specifications Provides methods to download the Moondream 0.5B INT4 model file. Includes command-line tools like wget and curl, as well as a Python script using the requests library. Optimized for ultra-lightweight deployment on mobile and embedded systems. ```bash wget https://huggingface.co/vikhyatk/moondream2/resolve/9dddae84d54db4ac56fe37817aeaeb502ed083e2/moondream-0_5b-int4.mf.gz ``` ```bash curl -L -o moondream-0_5b-int4.mf.gz https://huggingface.co/vikhyatk/moondream2/resolve/9dddae84d54db4ac56fe37817aeaeb502ed083e2/moondream-0_5b-int4.mf.gz ``` ```bash curl.exe -L \-o moondream\-0_5b\-int4.mf.gz https://huggingface.co/vikhyatk/moondream2/resolve/9dddae84d54db4ac56fe37817aeaeb502ed083e2/moondream\-0_5b\-int4.mf.gz ``` ```python import requests url = "https://huggingface.co/vikhyatk/moondream2/resolve/9dddae84d54db4ac56fe37817aeaeb502ed083e2/moondream-0_5b-int4.mf.gz" response = requests.get(url) with open("moondream-0_5b-int4.mf.gz", "wb") as f: f.write(response.content) ``` -------------------------------- ### Detect Objects with Moondream Python SDK Source: https://docs.moondream.ai/cloud/detect Demonstrates how to initialize the Moondream visual language model and detect objects in an image using the Python client library. Requires Pillow for image handling. ```python import moondream as md from PIL import Image # Initialize with API key model = md.vl(api_key="your-api-key") # Load an image image = Image.open("path/to/image.jpg") # Detect objects (replace 'subject' with what you want to detect) detect_result = model.detect(image, "subject") print("Detected objects:", detect_result["objects"]) ``` -------------------------------- ### Page Meta Information Source: https://moondream.ai/playground Standard meta tags for the Moondream AI project page, including character set, viewport settings, title, description, and favicon links. ```APIDOC MetaTags: charset: utf-8 viewport: width: device-width initial-scale: 1 title: Moondream description: Moondream AI - Vision language model for everyone manifest: rel: manifest href: /site.webmanifest icons: - rel: icon href: /favicon.ico - rel: icon href: /favicon-16x16.png sizes: 16x16 type: image/png - rel: icon href: /favicon-32x32.png sizes: 32x32 type: image/png - rel: apple-touch-icon href: /apple-touch-icon.png - rel: mask-icon href: /safari-pinned-tab.svg color: "#5bbad5" ``` -------------------------------- ### Download Moondream 0.5B INT8 Model Source: https://docs.moondream.ai/specifications Provides methods to download the Moondream 0.5B INT8 model file. Includes command-line tools like wget and curl, as well as a Python script using the requests library. Optimized for fast CPU inference and edge devices like Raspberry Pi 4. ```bash wget https://huggingface.co/vikhyatk/moondream2/resolve/9dddae84d54db4ac56fe37817aeaeb502ed083e2/moondream-0_5b-int8.mf.gz ``` ```bash curl -L -o moondream-0_5b-int8.mf.gz https://huggingface.co/vikhyatk/moondream2/resolve/9dddae84d54db4ac56fe37817aeaeb502ed083e2/moondream-0_5b-int8.mf.gz ``` ```bash curl.exe -L \-o moondream\-0_5b\-int8.mf.gz https://huggingface.co/vikhyatk/moondream2/resolve/9dddae84d54db4ac56fe37817aeaeb502ed083e2/moondream\-0_5b\-int8.mf.gz ``` ```python import requests url = "https://huggingface.co/vikhyatk/moondream2/resolve/9dddae84d54db4ac56fe37817aeaeb502ed083e2/moondream-0_5b-int8.mf.gz" response = requests.get(url) with open("moondream-0_5b-int8.mf.gz", "wb") as f: f.write(response.content) ```