### Installation Source: https://github.com/google/magika/blob/main/tests_data/basic/markdown/README.md Instructions on how to install the Magika Python package. ```APIDOC ## Installation ### Description Install Magika using pip or pipx. ### Command Line Installation ```shell $ pip install magika ``` ### Pipx Installation (for command-line only) ```shell $ pipx install magika ``` ``` -------------------------------- ### Start Local Development Server Source: https://github.com/google/magika/blob/main/website-ng/README.md Starts a local development server at `localhost:4321` for previewing your site during development. ```bash npm run dev ``` -------------------------------- ### Install Project Dependencies Source: https://github.com/google/magika/blob/main/website-ng/README.md Run this command in the project root to install all necessary npm packages. ```bash npm install ``` -------------------------------- ### Install Magika CLI with wget Source: https://github.com/google/magika/blob/main/rust/cli/README.md Installs the latest version of the Magika CLI by downloading and executing an installation script via wget. This is an alternative to curl for environments where wget is preferred. ```shell wget -qO- https://securityresearch.google/magika/install.sh | sh ``` -------------------------------- ### Run Magika CLI Examples Source: https://github.com/google/magika/blob/main/README.md Common command-line usage patterns for file identification. ```shell % cd tests_data/basic && magika -r * | head asm/code.asm: Assembly (code) batch/simple.bat: DOS batch file (code) c/code.c: C source (code) css/code.css: CSS source (code) csv/magika_test.csv: CSV document (code) dockerfile/Dockerfile: Dockerfile (code) docx/doc.docx: Microsoft Word 2007+ document (document) docx/magika_test.docx: Microsoft Word 2007+ document (document) eml/sample.eml: RFC 822 mail (text) empty/empty_file: Empty file (inode) ``` ```shell % magika ./tests_data/basic/python/code.py --json [ { "path": "./tests_data/basic/python/code.py", "result": { "status": "ok", "value": { "dl": { "description": "Python source", "extensions": [ "py", "pyi" ], "group": "code", "is_text": true, "label": "python", "mime_type": "text/x-python" }, "output": { "description": "Python source", "extensions": [ "py", "pyi" ], "group": "code", "is_text": true, "label": "python", "mime_type": "text/x-python" }, "score": 0.996999979019165 } } } ] ``` ```shell % cat tests_data/basic/ini/doc.ini | magika - -: INI configuration file (text) ``` ```shell % magika --help Determines file content types using AI Usage: magika [OPTIONS] [PATH]... Arguments: [PATH]... List of paths to the files to analyze. Use a dash (-) to read from standard input (can only be used once). Options: -r, --recursive Identifies files within directories instead of identifying the directory itself --no-dereference Identifies symbolic links as is instead of identifying their content by following them --colors Prints with colors regardless of terminal support --no-colors Prints without colors regardless of terminal support -s, --output-score Prints the prediction score in addition to the content type -i, --mime-type Prints the MIME type instead of the content type description -l, --label Prints a simple label instead of the content type description --json Prints in JSON format --jsonl Prints in JSONL format --format Prints using a custom format (use --help for details). The following placeholders are supported: %p The file path %l The unique label identifying the content type %d The description of the content type %g The group of the content type %m The MIME type of the content type %e Possible file extensions for the content type %s The score of the content type for the file %S The score of the content type for the file in percent %b The model output if overruled (empty otherwise) %% A literal % -h, --help Print help (see a summary with '-h') -V, --version Print version ``` -------------------------------- ### Install MagikaJS via npm Source: https://github.com/google/magika/blob/main/js/README.md Install the Magika package using npm. ```bash npm install magika ``` -------------------------------- ### Install Magika CLI from crates.io Source: https://github.com/google/magika/blob/main/rust/cli/README.md Installs the latest stable version of the Magika CLI directly from crates.io using the cargo package manager. Requires Rust and Cargo to be installed. ```shell cargo install --locked magika-cli ``` -------------------------------- ### Install Development Dependencies with uv Source: https://github.com/google/magika/blob/main/website-ng/src/content/docs/cli-and-bindings/python.md Installs all development dependencies for the magika Python package using uv. ```bash cd python; uv sync ``` -------------------------------- ### Install Magika CLI with pipx Source: https://github.com/google/magika/blob/main/rust/cli/README.md Installs the latest version of the Magika CLI using the pipx package manager. Ensure pipx is installed and configured. ```shell pipx install magika ``` -------------------------------- ### Install Magika CLI with curl Source: https://github.com/google/magika/blob/main/rust/cli/README.md Installs the latest version of the Magika CLI by downloading and executing an installation script via curl. This method is suitable for Linux and macOS. ```shell curl -LsSf https://securityresearch.google/magika/install.sh | sh ``` -------------------------------- ### Create Astro Project with Starlight Template Source: https://github.com/google/magika/blob/main/website-ng/README.md Use this command to initialize a new Astro project using the Starlight template. Ensure you have Node.js installed. ```bash npm create astro@latest -- --template starlight ``` -------------------------------- ### Install Magika Packages Source: https://github.com/google/magika/blob/main/README.md Commands to install Magika libraries for Python and JavaScript projects. ```shell pip install magika ``` ```shell npm install magika ``` -------------------------------- ### Command-Line Usage Source: https://github.com/google/magika/blob/main/tests_data/basic/markdown/README.md Examples and options for using Magika as a command-line tool. ```APIDOC ## Using Magika as a command-line tool ### Description Magika can be used from the command line to detect file types. ### Basic Usage Example ```shell $ magika examples/* code.asm: Assembly (code) code.py: Python source (code) doc.docx: Microsoft Word 2007+ document (document) doc.ini: INI configuration file (text) elf64.elf: ELF executable (executable) flac.flac: FLAC audio bitstream data (audio) image.bmp: BMP image data (image) java.class: Java compiled bytecode (executable) jpg.jpg: JPEG image data (image) pdf.pdf: PDF document (document) pe32.exe: PE executable (executable) png.png: PNG image data (image) README.md: Markdown document (text) tar.tar: POSIX tar archive (archive) webm.webm: WebM data (video) ``` ### Help Options ```help $ magika --help Usage: magika [OPTIONS] [FILE]... Magika - Determine type of FILEs with deep-learning. Options: -r, --recursive When passing this option, magika scans every file within directories, instead of outputting "directory" --json Output in JSON format. --jsonl Output in JSONL format. -i, --mime-type Output the MIME type instead of a verbose content type description. -l, --label Output a simple label instead of a verbose content type description. Use --list-output- content-types for the list of supported output. -c, --compatibility-mode Compatibility mode: output is as close as possible to `file` and colors are disabled. -s, --output-score Output the prediction score in addition to the content type. -m, --prediction-mode [best-guess|medium-confidence|high-confidence] --batch-size INTEGER How many files to process in one batch. --no-dereference This option causes symlinks not to be followed. By default, symlinks are dereferenced. --colors / --no-colors Enable/disable use of colors. -v, --verbose Enable more verbose output. -vv, --debug Enable debug logging. --generate-report Generate report useful when reporting feedback. --version Print the version and exit. --list-output-content-types Show a list of supported content types. --model-dir DIRECTORY Use a custom model. -h, --help Show this message and exit. Magika version: "0.5.0" Default model: "standard_v1" Send any feedback to magika-dev@google.com or via GitHub issues. ``` ``` -------------------------------- ### Install Magika CLI from Git repository Source: https://github.com/google/magika/blob/main/rust/cli/README.md Installs the latest development version of the Magika CLI directly from its git repository. The installed version will be suffixed with '-dev'. Requires Rust and Cargo. ```shell cargo install --locked --git=https://github.com/google/magika.git magika-cli ``` -------------------------------- ### Install Magika CLI Source: https://github.com/google/magika/blob/main/README.md Various methods to install the Magika command-line tool across different environments. ```shell pipx install magika ``` ```shell brew install magika ``` ```shell curl -LsSf https://securityresearch.google/magika/install.sh | sh ``` ```shell powershell -ExecutionPolicy Bypass -c "irm https://securityresearch.google/magika/install.ps1 | iex" ``` ```shell cargo install --locked magika-cli ``` -------------------------------- ### Install Magika CLI from local clone Source: https://github.com/google/magika/blob/main/rust/cli/README.md Installs the Magika CLI from a local clone of the git repository. This is useful for development or testing custom changes. Requires Rust and Cargo. ```shell git clone https://github.com/google/magika.git cd magika cargo install --locked --path=rust/cli ``` -------------------------------- ### Create and Use Magika Scanner in Go Source: https://github.com/google/magika/blob/main/go/README.md This example shows how to initialize a Magika scanner with specified assets and model, and then use it to infer the content type of a given input. Ensure cgo is enabled and ONNX Runtime is accessible. ```go //go:build cgo && onnxruntime // This package illustrates the usage of the Magika go binding. // // It requires the onnxruntime and the Magika assets to be accessible. // onnxruntime is available on https://github.com/microsoft/onnxruntime/releases // Magika asserts are available on https://github.com/google/magika/tree/main/assets // // Tag and link directives must be provided a build or run time: // go run -tags onnxruntime -ldflags="-linkmode=external -extldflags=-L/opt/onnxruntime/lib" . package main import ( "fmt" "strings" "github.com/google/magika/go/magika" ) const ( // assetsDir holds where the Magika assets have been installed. assetsDir = "/opt/magika/assets" // modelName holds the Magika model to use. modelName = "standard_v3_3" ) func main() { // Create a scanner. s s, err := magika.NewScanner(assetsDir, modelName) if err != nil { log.Fatalf("NewScanner failed: %v", err) } // Scan ct, err := s.Scan(strings.NewReader("Hello world"), 11) if err != nil { log.Fatalf("Scan failed: %v", err) } fmt.Printf("%+v\n", ct) } ``` -------------------------------- ### Install Magika with Homebrew Source: https://github.com/google/magika/blob/main/website-ng/src/content/docs/getting-started/installation.mdx Install Magika on macOS or Linux using the Homebrew package manager. ```shell brew install magika ``` -------------------------------- ### Development: Use Hosted Model Source: https://github.com/google/magika/blob/main/website-ng/src/content/docs/cli-and-bindings/js.md Steps for development using the Magika model hosted on GitHub. Includes installing dependencies, building the project, and running the CLI with a README file. ```bash yarn install yarn run build yarn run bin -- README.md ``` -------------------------------- ### Install Magika using pip Source: https://github.com/google/magika/blob/main/tests_data/basic/markdown/README.md Install the Magika package using pip. For command-line only usage, consider pipx. ```shell pip install magika ``` -------------------------------- ### Development: Use Local Model Source: https://github.com/google/magika/blob/main/website-ng/src/content/docs/cli-and-bindings/js.md Instructions for development using a locally hosted Magika model. This involves starting a local web server for the model and running the CLI against test data. ```bash yarn install yarn run build (cd ../website; yarn install; yarn run dev) & yarn run bin --model-url http://localhost:5173/magika/model/model.json --config-url http://localhost:5173/magika/model/config.json ../tests_data/basic/* ``` -------------------------------- ### Get Help with Astro CLI Source: https://github.com/google/magika/blob/main/website-ng/README.md Displays help information for using the Astro CLI and its available commands. ```bash npm run astro -- --help ``` -------------------------------- ### Install Magika Python release candidate Source: https://github.com/google/magika/blob/main/website-ng/src/content/docs/getting-started/installation.mdx Install the latest release candidate of the Magika Python package using pip. ```shell pip install --pre magika ``` -------------------------------- ### Use Magika as a command-line tool Source: https://github.com/google/magika/blob/main/tests_data/basic/markdown/README.md Scan files using the Magika command-line interface. This example shows detection for various file types. ```shell magika examples/* ``` -------------------------------- ### Install Magika CLI with PowerShell Source: https://github.com/google/magika/blob/main/rust/cli/README.md Installs the latest version of the Magika CLI using PowerShell. This command bypasses execution policy restrictions for script execution. ```powershell powershell -ExecutionPolicy Bypass -c "irm https://securityresearch.google/magika/install.ps1 | iex" ``` -------------------------------- ### Development: Link Local Magika Package Source: https://github.com/google/magika/blob/main/website-ng/src/content/docs/cli-and-bindings/js.md Guide for linking a local Magika package when developing the website. This allows the website to use the locally developed version of Magika. ```bash yarn install yarn run build yarn link (cd ../website; yarn link magika; yarn install; yarn run dev) & ``` -------------------------------- ### Identifying Content from Multiple File Paths Source: https://github.com/google/magika/blob/main/website-ng/src/content/docs/cli-and-bindings/python.md Example of identifying content types for a list of file paths. ```APIDOC ## Identify Content from Multiple File Paths ### Method `identify_paths(paths)` ### Description Identifies the content type for a list of file paths. ### Request Example ```python from magika import Magika m = Magika() results = m.identify_paths(['./file1.txt', './file2.jpg']) for res in results: print(f"File: {res.path}, Label: {res.output.label}") ``` ### Response Example ```json [ { "path": "./file1.txt", "output": { "label": "text" } }, { "path": "./file2.jpg", "output": { "label": "jpeg" } } ] ``` ``` -------------------------------- ### Import Astro Components and Svelte Demo Source: https://github.com/google/magika/blob/main/website-ng/src/content/docs/demo/magika-demo.mdx Imports necessary components from Astro Starlight and the MagikaDemo Svelte component. This setup is required to render the interactive demo on the page. ```javascript import { Card } from '@astrojs/starlight/components'; import MagikaDemo from '@/components/MagikaDemo.svelte'; ``` -------------------------------- ### Identifying Content from File Path Source: https://github.com/google/magika/blob/main/website-ng/src/content/docs/cli-and-bindings/python.md Example of identifying file content type from a file path. ```APIDOC ## Identify Content from File Path ### Method `identify_path(path)` ### Description Identifies the content type of a single file from its path (`str | os.PathLike`). ### Request Example ```python from magika import Magika m = Magika() res = m.identify_path('./tests_data/basic/ini/doc.ini') print(res.output.label) ``` ### Response Example ```json { "output": { "label": "ini" } } ``` ``` -------------------------------- ### Identifying Content from Bytes Source: https://github.com/google/magika/blob/main/website-ng/src/content/docs/cli-and-bindings/python.md Example of identifying file content type from a bytes object. ```APIDOC ## Identify Content from Bytes ### Method `identify_bytes(bytes)` ### Description Identifies the content type of an in-memory bytes object. ### Request Example ```python from magika import Magika m = Magika() res = m.identify_bytes(b'function log(msg) {console.log(msg);}') print(res.output.label) ``` ### Response Example ```json { "output": { "label": "javascript" } } ``` ``` -------------------------------- ### Identifying Content from File Stream Source: https://github.com/google/magika/blob/main/website-ng/src/content/docs/cli-and-bindings/python.md Example of identifying file content type from an open file stream. ```APIDOC ## Identify Content from File Stream ### Method `identify_stream(stream)` ### Description Identifies the content type from an already-open binary file-like object (e.g., the output of `open(file_path, 'rb')`). Note: 1) Magika will `seek()` around the stream; 2) the stream _is not closed_ (closing is the responsibility of the caller). ### Request Example ```python from magika import Magika m = Magika() with open('./tests_data/basic/ini/doc.ini', 'rb') as f: res = m.identify_stream(f) print(res.output.label) ``` ### Response Example ```json { "output": { "label": "ini" } } ``` ``` -------------------------------- ### View CLI help documentation Source: https://github.com/google/magika/blob/main/rust/cli/README.md Access the full list of available options and usage instructions. ```shell % magika --help Determines file content types using AI Usage: magika [OPTIONS] [PATH]... Arguments: [PATH]... List of paths to the files to analyze. Use a dash (-) to read from standard input (can only be used once). Options: -r, --recursive Identifies files within directories instead of identifying the directory itself --no-dereference Identifies symbolic links as is instead of identifying their content by following them --colors Prints with colors regardless of terminal support --no-colors Prints without colors regardless of terminal support -s, --output-score Prints the prediction score in addition to the content type -i, --mime-type Prints the MIME type instead of the content type description -l, --label Prints a simple label instead of the content type description --json Prints in JSON format --jsonl Prints in JSONL format --format Prints using a custom format (use --help for details). The following placeholders are supported: %p The file path %l The unique label identifying the content type %d The description of the content type %g The group of the content type %m The MIME type of the content type %e Possible file extensions for the content type %s The score of the content type for the file %S The score of the content type for the file in percent %b The model output if overruled (empty otherwise) %% A literal % -h, --help Print help (see a summary with '-h') -V, --version Print version ``` -------------------------------- ### View CLI help Source: https://github.com/google/magika/blob/main/website-ng/src/content/docs/getting-started/quick-start.md Display available command line options and configuration flags. ```shell % magika --help Determines file content types using AI Usage: magika [OPTIONS] [PATH]... Arguments: [PATH]... List of paths to the files to analyze. Use a dash (-) to read from standard input (can only be used once). Options: -r, --recursive Identifies files within directories instead of identifying the directory itself --no-dereference Identifies symbolic links as is instead of identifying their content by following them --colors Prints with colors regardless of terminal support --no-colors Prints without colors regardless of terminal support -s, --output-score Prints the prediction score in addition to the content type -i, --mime-type Prints the MIME type instead of the content type description -l, --label Prints a simple label instead of the content type description --json Prints in JSON format --jsonl Prints in JSONL format --format Prints using a custom format (use --help for details). The following placeholders are supported: %p The file path %l The unique label identifying the content type %d The description of the content type %g The group of the content type %m The MIME type of the content type %e Possible file extensions for the content type %s The score of the content type for the file %S The score of the content type for the file in percent %b The model output if overruled (empty otherwise) %% A literal % -h, --help Print help (see a summary with '-h') -V, --version Print version ``` -------------------------------- ### View Magika CLI help Source: https://github.com/google/magika/blob/main/python/README.md Display available command-line options and usage instructions. ```help $ magika --help Determines the content type of files with deep-learning Usage: magika [OPTIONS] [PATH]... Arguments: [PATH]... List of paths to the files to analyze. Use a dash (-) to read from standard input (can only be used once). Options: -r, --recursive Identifies files within directories instead of identifying the directory itself --no-dereference Identifies symbolic links as is instead of identifying their content by following them --colors Prints with colors regardless of terminal support --no-colors Prints without colors regardless of terminal support -s, --output-score Prints the prediction score in addition to the content type -i, --mime-type Prints the MIME type instead of the content type description -l, --label Prints a simple label instead of the content type description --json Prints in JSON format --jsonl Prints in JSONL format --format Prints using a custom format (use --help for details). The following placeholders are supported: %p The file path %l The unique label identifying the content type %d The description of the content type %g The group of the content type %m The MIME type of the content type %e Possible file extensions for the content type %s The score of the content type for the file %S The score of the content type for the file in percent %b The model output if overruled (empty otherwise) %% A literal % -h, --help Print help (see a summary with '-h') -V, --version Print version ``` -------------------------------- ### Build Production Site Source: https://github.com/google/magika/blob/main/website-ng/README.md Builds your website for production, outputting the static files to the `./dist/` directory. ```bash npm run build ``` -------------------------------- ### Magika Instantiation Source: https://github.com/google/magika/blob/main/website-ng/src/content/docs/cli-and-bindings/python.md Demonstrates how to instantiate the Magika class with default and custom arguments. ```APIDOC ## Instantiating `Magika` First, create an instance of the `Magika` class. The constructor accepts several optional arguments to customize its behavior. ```python from magika import Magika, PredictionMode # Default instantiation magika = Magika() # Custom instantiation magika_custom = Magika( model_dir="/path/to/custom/model", prediction_mode=PredictionMode.BEST_GUESS, no_dereference=True, ) ``` **Constructor Arguments:** - `model_dir` (`Path`, optional): Path to a directory containing a custom model. If not provided, defaults to the latest bundled model. - `prediction_mode` (`PredictionMode`, optional): The prediction mode to use. Defaults to `PredictionMode.HIGH_CONFIDENCE`. - `no_dereference` (`bool`, optional): If `True`, symbolic links will not be followed; their content type will be reported as `symlink`. Defaults to `False`. ``` -------------------------------- ### Preview Production Build Source: https://github.com/google/magika/blob/main/website-ng/README.md Locally previews the production build of your site before deploying it. ```bash npm run preview ``` -------------------------------- ### Magika CLI help output Source: https://github.com/google/magika/blob/main/js/README.md Displays the command-line interface options for the magika-js tool. ```help Usage: magika-js [options] Magika JS - file type detection with ML. https://securityresearch.google/magika/ Arguments: paths Paths of the files to detect Options: --json-output Format output in JSON --model-url Model URL --model-path Model file path --model-config-url Model config URL --model-config-path Model config file path --by-stream Identify file via stream, not via bytes --debug Output debug information -h, --help display help for command ``` -------------------------------- ### Development build and run commands Source: https://github.com/google/magika/blob/main/js/README.md Commands for building and running Magika during development. ```bash yarn install yarn run build yarn run bin -- README.md ``` ```bash yarn install yarn run build (cd ../website; yarn install; yarn run dev) & yarn run bin --model-url http://localhost:5173/magika/model/model.json --config-url http://localhost:5173/magika/model/config.json ../tests_data/basic/* ``` ```bash yarn install yarn run build yarn link (cd ../website; yarn link magika; yarn install; yarn run dev) & ``` -------------------------------- ### Configure custom model and config paths Source: https://github.com/google/magika/blob/main/js/README.md Load custom model and configuration files via URLs or local paths. ```js const magika = await magika.create({ modelURL: "https://...", configURL: "https://...", }); ``` ```js const magika = await magika.create({ modelPath: "./assets/...", configPath: "./assets/...", }); ``` -------------------------------- ### Build and run Magika Docker image Source: https://github.com/google/magika/blob/main/website-ng/src/content/docs/getting-started/installation.mdx Build a Docker image for Magika from its git repository and run it to test the latest version. ```shell git clone https://github.com/google/magika cd magika/ docker build -t magika . docker run -it --rm -v $(pwd):/magika magika -r /magika/tests_data/basic ``` -------------------------------- ### ContentTypeLabel Enum Example Source: https://github.com/google/magika/blob/main/website-ng/src/content/docs/cli-and-bindings/python.md A string enum representing all possible content type labels. Members can be used as strings. ```python class ContentTypeLabel(StrEnum): APK = "apk" BMP = "bmp" # ... and many more ``` -------------------------------- ### Run Magika tests Source: https://github.com/google/magika/blob/main/js/README.md Execute the test suite for the project. ```bash yarn install yarn run build yarn run test ``` -------------------------------- ### Magika command-line help Source: https://github.com/google/magika/blob/main/tests_data/basic/markdown/README.md Display the help message for the Magika command-line tool, showing available options and usage. ```shell magika --help ``` -------------------------------- ### Magika JS CLI Usage Source: https://github.com/google/magika/blob/main/website-ng/src/content/docs/cli-and-bindings/js.md Displays the help information for the Magika JS command-line interface, outlining available options for file type detection. ```bash Usage: magika-js [options] Magika JS - file type detection with ML. https://google.github.io/magika Arguments: paths Paths of the files to detect Options: --json-output Format output in JSON --model-url Model URL (default: "https://google.github.io/magika/models/standard_v3_2/model.json") --model-path Model file path --model-config-url Model config URL (default: "https://google.github.io/magika/models/standard_v3_2/config.min.json") --model-config-path Model config file path --by-stream Identify file via stream, not via bytes --debug Output debug information -h, --help display help for command ``` -------------------------------- ### Analyze standard input with Magika CLI Source: https://github.com/google/magika/blob/main/python/README.md Pipe content directly into the Magika CLI using a dash. ```shell $ cat doc.ini | magika - -: INI configuration file (text) ``` -------------------------------- ### Magika.create Source: https://github.com/google/magika/blob/main/website-ng/src/content/docs/cli-and-bindings/js-api.md Factory method to create a Magika instance. ```APIDOC ## Magika.create ### Description Factory method to create a Magika instance for client-side use. ### Parameters #### Parameters - **options** (MagikaOptions) - Optional - The urls or file paths where the model and its config are stored. If not provided, the model will be loaded from GitHub. ### Response - **Promise** - A promise that resolves to a Magika instance. ``` -------------------------------- ### Process standard input Source: https://github.com/google/magika/blob/main/rust/cli/README.md Use a dash (-) to identify content piped directly into the command. ```shell % cat tests_data/basic/ini/doc.ini | magika - -: INI configuration file (text) ``` -------------------------------- ### MagikaNode.create Source: https://github.com/google/magika/blob/main/website-ng/src/content/docs/cli-and-bindings/js-api.md Factory method to create a MagikaNode instance. ```APIDOC ## MagikaNode.create ### Description Factory method to create a Magika instance for Node.js. ### Parameters #### Parameters - **options** (MagikaOptions) - Optional - The urls or file paths where the model and its config are stored. If not provided, the model will be loaded from GitHub. ### Response - **Promise** - A promise that resolves to a MagikaNode instance. ``` -------------------------------- ### Load Local Model and Config in Node.js Source: https://github.com/google/magika/blob/main/website-ng/src/content/docs/cli-and-bindings/js.md Configure MagikaJS in Node.js to load the model and configuration files from local file paths. ```javascript const magika = await magika.create({ modelPath: "./assets/...", configPath: "./assets/...", }); ``` -------------------------------- ### Python Module Usage Source: https://github.com/google/magika/blob/main/tests_data/basic/markdown/README.md How to use Magika within your Python code. ```APIDOC ## Using Magika as a Python module ### Description Integrate Magika's file type detection capabilities into your Python applications. ### Example ```python from magika import Magika magika = Magika() result = magika.identify_bytes(b"# Example\nThis is an example of markdown!") print(result.output.ct_label) # Output: "markdown" ``` ``` -------------------------------- ### Identify files via CLI Source: https://github.com/google/magika/blob/main/website-ng/src/content/docs/getting-started/quick-start.md Use the command line tool to identify file types recursively or via standard input. ```shell % cd tests_data/basic && magika -r * | head asm/code.asm: Assembly (code) batch/simple.bat: DOS batch file (code) c/code.c: C source (code) css/code.css: CSS source (code) csv/magika_test.csv: CSV document (code) dockerfile/Dockerfile: Dockerfile (code) docx/doc.docx: Microsoft Word 2007+ document (document) docx/magika_test.docx: Microsoft Word 2007+ document (document) eml/sample.eml: RFC 822 mail (text) empty/empty_file: Empty file (inode) ``` ```shell % cat tests_data/basic/ini/doc.ini | magika - -: INI configuration file (text) ``` -------------------------------- ### Citation Source: https://github.com/google/magika/blob/main/tests_data/basic/markdown/README.md Information on how to cite the Magika software in academic research. ```APIDOC ## Citation ### Description If you use this software for your research, please cite it using the following BibTeX entry. ### BibTeX Entry ```bibtex @software{magika, author = {Fratantonio, Yanick and Bursztein, Elie and Invernizzi, Luca and Zhang, Marina and Metitieri, Giancarlo and Kurt, Thomas and Galilee, Francois and Petit-Bianco, Alexandre and Farah, Loua and Albertini, Ange}, title = {{Magika content-type scanner}}, url = {https://github.com/google/magika} } ``` ``` -------------------------------- ### Load Model and Config with URLs Source: https://github.com/google/magika/blob/main/website-ng/src/content/docs/cli-and-bindings/js.md Configure MagikaJS to load the model and configuration files from specified URLs. This method is suitable for both Node.js and browser environments. ```javascript const magika = await magika.create({ modelURL: "https://...", configURL: "https://...", }); ``` -------------------------------- ### Run Astro CLI Commands Source: https://github.com/google/magika/blob/main/website-ng/README.md Executes Astro command-line interface commands, such as adding integrations or checking your project. ```bash npm run astro ... ``` -------------------------------- ### Instantiate Magika Source: https://github.com/google/magika/blob/main/website-ng/src/content/docs/cli-and-bindings/python.md Instantiate the `Magika` class for default or custom behavior. Custom instantiation allows specifying model directory, prediction mode, and symbolic link handling. ```python from magika import Magika, PredictionMode # Default instantiation magika = Magika() # Custom instantiation magika_custom = Magika( model_dir="/path/to/custom/model", prediction_mode=PredictionMode.BEST_GUESS, no_dereference=True, ) ``` -------------------------------- ### Identify Content with Python Source: https://github.com/google/magika/blob/main/README.md Programmatic identification of file types using bytes, file paths, or streams. ```python >>> from magika import Magika >>> m = Magika() >>> res = m.identify_bytes(b'function log(msg) {console.log(msg);}') >>> print(res.output.label) javascript ``` ```python >>> from magika import Magika >>> m = Magika() >>> res = m.identify_path('./tests_data/basic/ini/doc.ini') >>> print(res.output.label) ini ``` ```python >>> from magika import Magika >>> m = Magika() >>> with open('./tests_data/basic/ini/doc.ini', 'rb') as f: >>> res = m.identify_stream(f) >>> print(res.output.label) ini ``` -------------------------------- ### Identify file content in Node.js Source: https://github.com/google/magika/blob/main/website-ng/src/content/docs/cli-and-bindings/js-api.md Demonstrates using MagikaNode to identify a file's content type from a local file path. ```javascript import { readFile } from "fs/promises"; import { MagikaNode as Magika } from "magika/node"; const data = await readFile("some file"); const magika = await Magika.create(); const result = await magika.identifyBytes(data); console.log(result.prediction.output.label); ``` -------------------------------- ### Identify file types recursively Source: https://github.com/google/magika/blob/main/rust/cli/README.md Use the -r flag to scan all files within a directory. ```shell % cd tests_data/basic && magika -r * | head asm/code.asm: Assembly (code) batch/simple.bat: DOS batch file (code) c/code.c: C source (code) css/code.css: CSS source (code) csv/magika_test.csv: CSV document (code) dockerfile/Dockerfile: Dockerfile (code) docx/doc.docx: Microsoft Word 2007+ document (document) docx/magika_test.docx: Microsoft Word 2007+ document (document) eml/sample.eml: RFC 822 mail (text) empty/empty_file: Empty file (inode) ``` -------------------------------- ### Identify file types in Node.js Source: https://github.com/google/magika/blob/main/js/README.md Use the MagikaNode class to identify file types from local files in a Node.js environment. ```js import { readFile } from "fs/promises"; import { MagikaNode as Magika } from "magika/node"; const data = await readFile("some file"); const magika = await Magika().create(); const prediction = await magika.identifyBytes(data); console.log(prediction); ``` -------------------------------- ### Output identification results as JSON Source: https://github.com/google/magika/blob/main/rust/cli/README.md The --json flag provides detailed metadata about the file identification in JSON format. ```shell % magika ./tests_data/basic/python/code.py --json [ { "path": "./tests_data/basic/python/code.py", "result": { "status": "ok", "value": { "dl": { "description": "Python source", "extensions": [ "py", "pyi" ], "group": "code", "is_text": true, "label": "python", "mime_type": "text/x-python" }, "output": { "description": "Python source", "extensions": [ "py", "pyi" ], "group": "code", "is_text": true, "label": "python", "mime_type": "text/x-python" }, "score": 0.996999979019165 } } } ] ``` -------------------------------- ### Use Magika as a Python module Source: https://github.com/google/magika/blob/main/tests_data/basic/markdown/README.md Identify file types from bytes using the Magika Python module. Import the Magika class and use the identify_bytes method. ```python from magika import Magika magika = Magika() result = magika.identify_bytes(b"# Example\nThis is an example of markdown!") print(result.output.ct_label) # Output: "markdown" ``` -------------------------------- ### Identify file types in the browser Source: https://github.com/google/magika/blob/main/js/README.md Use the Magika class to identify file types from browser File objects. ```js import { Magika } from "magika"; const file = new File(["# Hello I am a markdown file"], "hello.md"); const fileBytes = new Uint8Array(await file.arrayBuffer()); const magika = await Magika.create(); const prediction = await magika.identifyBytes(fileBytes); console.log(prediction); ``` -------------------------------- ### Retrieve JSON output from Magika CLI Source: https://github.com/google/magika/blob/main/website-ng/src/content/docs/core-concepts/understanding-the-output.md Use the --json flag with the Magika CLI to obtain detailed prediction results in JSON format. ```shell $ magika tests_data/basic/javascript/code.js --json [ { "path": "tests_data/basic/javascript/code.js", "result": { "status": "ok", "value": { "dl": { "description": "JavaScript source", "extensions": [ "js", "mjs", "cjs" ], "group": "code", "is_text": true, "label": "javascript", "mime_type": "application/javascript" }, "output": { "description": "JavaScript source", "extensions": [ "js", "mjs", "cjs" ], "group": "code", "is_text": true, "label": "javascript", "mime_type": "application/javascript" }, "score": 0.9710000157356262 } } } ] ``` -------------------------------- ### Magika Helper APIs Source: https://github.com/google/magika/blob/main/website-ng/src/content/docs/cli-and-bindings/python.md Helper methods available on the Magika class for retrieving metadata and configuration. ```APIDOC ## Additional APIs The `Magika` class also exposes a few helper methods: - `get_output_content_types()`: Returns a list of all possible content type labels that Magika can return in the `output.label` field. This is the recommended way to get a definitive list of Magika's possible outputs. - `get_model_content_types()`: Returns a list of all possible content type labels the _deep learning model_ can return (i.e., the possible values for `dl.label`, in addition to `undefined`). This is useful for debugging. - `get_module_version()`: Returns the `magika` Python package version as a string. - `get_model_version()`: Returns the name of the model being used as a string. ``` -------------------------------- ### Render Magika Demo Component in Astro Source: https://github.com/google/magika/blob/main/website-ng/src/content/docs/demo/magika-demo.mdx Renders the MagikaDemo Svelte component within an Astro Card. The 'client:only="svelte"' directive ensures the Svelte component is hydrated on the client side. ```html ``` -------------------------------- ### Understanding the Result Object Source: https://github.com/google/magika/blob/main/website-ng/src/content/docs/cli-and-bindings/python.md Explains how to interpret the `MagikaResult` object returned by identification methods. ```APIDOC ## Understanding the Result All `identify_*` methods return a `MagikaResult` object. This object acts as a wrapper that contains the prediction details and the status of the operation. **You should always check if the operation was successful before accessing the prediction.** ```python result = m.identify_path("path/to/file") if result.ok: print(f"File is a {result.output.description}") print(f"MIME Type: {result.output.mime_type}") else: print(f"Error: {result.status.message}") ``` **`MagikaResult` Structure:** - `ok` (bool): Indicates if the identification was successful. - `output` (object): Contains the prediction details if `ok` is true. - `label` (str): The predicted content label (e.g., 'python', 'javascript'). - `description` (str): A human-readable description of the content type. - `mime_type` (str): The predicted MIME type. - `status` (object): Contains error details if `ok` is false. - `message` (str): A message describing the error. ``` -------------------------------- ### Identify File Type from Stream Source: https://github.com/google/magika/blob/main/website-ng/src/content/docs/cli-and-bindings/python.md Use `identify_stream` with an open binary file stream to identify content type. Magika will seek within the stream, but the caller is responsible for closing it. ```python >>> from magika import Magika >>> m = Magika() >>> with open('./tests_data/basic/ini/doc.ini', 'rb') as f: >>> res = m.identify_stream(f) >>> print(res.output.label) ini ``` -------------------------------- ### Identify file content in browser Source: https://github.com/google/magika/blob/main/website-ng/src/content/docs/cli-and-bindings/js-api.md Demonstrates using the Magika client-side API to identify the content type of a File object. ```javascript const file = new File(["# Hello I am a markdown file"], "hello.md"); const fileBytes = new Uint8Array(await file.arrayBuffer()); const magika = await Magika.create(); const result = await magika.identifyBytes(fileBytes); console.log(result.prediction.output.label); ```