MKTXP (akpw/mktxp)

MKTXP

https://github.com/akpw/mktxp
Admin
MKTXP is a Prometheus exporter for Mikrotik RouterOS devices that gathers and exports a rich set of...

Tokens:12,662
Snippets:121
Trust Score:8.3
Update:1 month ago
Show doc for...
Context Summary (auto-generated)
Raw
# MKTXP - Prometheus Exporter for Mikrotik RouterOS

MKTXP is a comprehensive Prometheus exporter designed to collect and export metrics from Mikrotik RouterOS devices. It provides a rich set of monitoring capabilities including system health, interface traffic, DHCP leases, wireless clients, firewall rules, BGP sessions, and many more. The exporter supports concurrent monitoring of multiple routers with configurable data processing, automatic IP address resolution via local and remote DHCP servers, and injectable custom labels for easy device grouping.

Beyond Prometheus integration, MKTXP offers a built-in CLI interface for direct metrics inspection, making it invaluable for troubleshooting and quick network status checks. It comes with a dedicated Grafana dashboard (ID: 13679) for visualization and supports multiple deployment options including PyPI installation, Docker containers, and Kubernetes. The tool supports both standard `/metrics` endpoint for traditional scraping and a `/probe` endpoint for Prometheus multi-target exporter pattern, enabling dynamic service discovery in large deployments.

## Installation

### Install via pip

```bash
pip install mktxp
```

### Install from source

```bash
pip install git+https://github.com/akpw/mktxp
```

### Install via Docker

```bash
docker pull ghcr.io/akpw/mktxp:latest
```

## CLI Commands

### mktxp info - Display MKTXP Information

Displays basic information about MKTXP including version and description.

```bash
# Show MKTXP info
mktxp info

# Output:
# MKTXP: Prometheus Exporter for Mikrotik RouterOS, version 1.2.18
# Supports gathering metrics across multiple RouterOS devices...
```

### mktxp edit - Edit Configuration Files

Opens the MKTXP configuration file in your default editor for modification. Supports both router configuration and internal system configuration.

```bash
# Edit router configuration with default editor
mktxp edit

# Edit with specific editor
mktxp edit -ed nano

# Edit internal system configuration
mktxp edit -i

# Edit with custom config directory
mktxp --cfg-dir /etc/mktxp edit
```

### mktxp show - Display Configuration Entries

Displays all configured router entries or specific entry details, useful for verifying configuration without opening files.

```bash
# Show all router entries
mktxp show

# Show specific router entry
mktxp show -en Sample-Router

# Show configuration file paths
mktxp show -cfg

# Output:
# MKTXP data config: /home/user/mktxp/mktxp.conf
# MKTXP internal config: /home/user/mktxp/_mktxp.conf
```

### mktxp export - Start Prometheus Exporter

Starts the HTTP metrics server that exports RouterOS metrics to Prometheus. This is the main command for production deployment.

```bash
# Start exporter with default settings (port 49090)
mktxp export

# Start with custom configuration directory
mktxp --cfg-dir /etc/mktxp export

# Output:
# Connecting to router MKT-GT@10.70.0.1
# 2024-01-15 14:16:22 Connection to router MKT-GT@10.70.0.1 has been established
# 2024-01-15 14:16:23 Running HTTP metrics server on: 0.0.0.0:49090
```

### mktxp print - Display Metrics on Command Line

Displays selected metrics directly on the command line without requiring Prometheus. Supports multiple metric types including CAPsMAN clients, WiFi clients, DHCP, connections, and more.

```bash
# Print CAPsMAN clients for specific router
mktxp print -en MKT-GT -cc

# Print WiFi clients
mktxp print -en MKT-GT -wc

# Print DHCP clients
mktxp print -en MKT-GT -dc

# Print connection statistics
mktxp print -en MKT-GT -cn

# Print Kid Control device metrics
mktxp print -en MKT-GT -kc

# Print Netwatch metrics
mktxp print -en MKT-GT -nw

# Print Address List metrics (comma-separated list names)
mktxp print -en MKT-GT -al "blocklist, allowlist"

# Output example for CAPsMAN clients:
# | dhcp_name       | dhcp_address | mac_address       | rx_signal | interface | ssid  | tx_rate  | rx_rate  | uptime |
# |-----------------|--------------|-------------------|-----------|-----------|-------|----------|----------|--------|
# | MacBook Pro     | 10.10.0.15   | A1:B2:C3:D4:E5:F6 |       -53 | GT-5G-1   | AKP5G | 877 Mbps | 877 Mbps | 3 days |
# | Smart TV        | 10.20.0.45   | C1:D2:E3:F4:A5:B6 |       -64 | LR-2G-1-1 | AKP   | 72 Mbps  | 54 Mbps  | 3 days |
```

## Configuration

### Router Configuration (mktxp.conf)

The main configuration file defines RouterOS devices and metric collection settings. Each router entry inherits from `[default]` section unless overridden.

```ini
# Router-specific configuration
[Production-Router]
    hostname = 192.168.88.1
    username = mktxp_user
    password = secure_password
    custom_labels = dc:london, rack:a1, service:prod

[Office-Router]
    hostname = 192.168.88.2
    # Uses credentials from external file
    credentials_file = /etc/mktxp/credentials.yaml

[default]
    # Connection settings
    enabled = True              # Enable/disable metrics collection
    module_only = False         # Use entry only as probe module
    hostname = localhost        # RouterOS IP address
    port = 8728                 # RouterOS API port (8729 for SSL)

    username = username         # RouterOS user (needs 'read' and 'api' permissions)
    password = password
    credentials_file = ""       # External YAML credentials file path

    # Custom labels for all metrics
    custom_labels = None        # Format: 'dc:london, rack=a1, service:prod'

    # SSL Configuration
    use_ssl = False                 # Enable API-SSL connection
    no_ssl_certificate = False      # Connect without router SSL certificate
    ssl_certificate_verify = False  # Enable SSL certificate verification
    ssl_check_hostname = True       # Verify hostname matches certificate
    ssl_ca_file = ""                # CA file path for validation
    plaintext_login = True          # Use False for RouterOS < 6.43

    # System metrics
    health = True                   # System health (voltage, temperature, fans)
    installed_packages = True       # Installed packages info

    # DHCP metrics
    dhcp = True                     # DHCP general metrics
    dhcp_lease = True               # DHCP lease metrics

    # Network metrics
    connections = True              # IP connections count
    connection_stats = False        # Detailed open connections
    interface = True                # Interface traffic metrics

    # IPv4 routing metrics
    route = True                    # IPv4 routes
    pool = True                     # IPv4 address pools
    firewall = True                 # Firewall rules traffic
    neighbor = True                 # Reachable neighbors
    address_list = None             # Address lists (comma-separated names)
    dns = False                     # DNS statistics

    # IPv6 metrics (disabled by default)
    ipv6_route = False
    ipv6_pool = False
    ipv6_firewall = False
    ipv6_neighbor = False
    ipv6_address_list = None

    # Wireless metrics
    wireless = True                 # WLAN general metrics
    wireless_clients = True         # WLAN client metrics
    capsman = True                  # CAPsMAN metrics
    capsman_clients = True          # CAPsMAN client metrics
    w60g = False                    # 60GHz wireless

    # Hardware metrics
    poe = True                      # Power over Ethernet
    monitor = True                  # Interface monitor
    netwatch = True                 # Netwatch status
    public_ip = True                # Public IP detection

    # Tunneling protocols
    eoip = False                    # EoIP tunnels
    gre = False                     # GRE tunnels
    ipip = False                    # IPIP tunnels
    ipsec = False                   # IPSec peers
    lte = False                     # LTE metrics (needs 'test' policy on ROS v6)

    # Other metrics
    switch_port = False             # Switch port statistics
    kid_control_assigned = False    # Kid Control for assigned users
    kid_control_dynamic = False     # Kid Control for all devices
    user = True                     # Active users
    queue = True                    # Queue statistics
    bfd = False                     # BFD sessions
    bgp = False                     # BGP sessions
    routing_stats = False           # Routing process stats
    certificate = False             # Certificate metrics
    container = False               # Container metrics

    # Remote data sources
    remote_dhcp_entry = None        # MKTXP entry for remote DHCP resolution
    remote_capsman_entry = None     # MKTXP entry for remote CAPsMAN info

    # Display settings
    interface_name_format = name    # Options: 'name', 'comment', 'combined'
    check_for_updates = False       # Check for RouterOS updates
```

### System Configuration (_mktxp.conf)

Internal system configuration for advanced tuning of MKTXP behavior.

```ini
[MKTXP]
    # Network settings
    listen = '0.0.0.0:49090'         # Socket addresses (space-separated, supports IPv6)
    socket_timeout = 5               # Connection timeout in seconds
    http_server_threads = 16         # HTTP server worker threads

    # Failure handling
    initial_delay_on_failure = 120   # Initial delay after connection failure (seconds)
    max_delay_on_failure = 900       # Maximum delay between retries (seconds)
    delay_inc_div = 5                # Delay increment divisor

    # Bandwidth testing
    bandwidth = False                    # Enable bandwidth testing
    bandwidth_test_dns_server = 8.8.8.8  # DNS server for connectivity check
    bandwidth_test_interval = 600        # Test interval in seconds

    # Collection settings
    minimal_collect_interval = 5     # Minimum interval between collections

    # Parallel fetching
    fetch_routers_in_parallel = False    # Enable parallel router fetching
    max_worker_threads = 5               # Max parallel worker threads
    max_scrape_duration = 30             # Max per-router scrape time (seconds)
    total_max_scrape_duration = 90       # Max total scrape time (seconds)

    # Connection pooling
    persistent_router_connection_pool = True   # Reuse connections between scrapes
    persistent_dhcp_cache = True               # Cache DHCP data between collections

    # Probe endpoint settings
    probe_connection_pool = False              # Enable probe connection pooling
    probe_connection_pool_ttl = 300            # Probe connection TTL (seconds)
    probe_connection_pool_max_size = 128       # Max probe connections

    # Output settings
    verbose_mode = False                       # Enable debug output
    compact_default_conf_values = False        # Compact mktxp.conf defaults
    prometheus_headers_deduplication = False   # Remove duplicate HELP/TYPE headers
```

### External Credentials File

Store credentials separately in YAML format for improved security.

```yaml
# /etc/mktxp/credentials.yaml
username: mktxp_user
password: secure_password_here
```

## Prometheus Integration

### Basic Prometheus Configuration

Standard Prometheus scrape configuration for MKTXP metrics endpoint.

```yaml
# prometheus.yml
scrape_configs:
  - job_name: 'mktxp'
    scrape_interval: 30s
    scrape_timeout: 25s
    static_configs:
      - targets: ['mktxp-host:49090']
```

### Multi-Target Exporter Pattern (/probe endpoint)

Use the `/probe` endpoint for dynamic service discovery and centralized configuration. This pattern allows scraping multiple routers using a single module configuration.

```yaml
# prometheus.yml - Multi-target configuration
scrape_configs:
  - job_name: 'mktxp-multi-target'
    metrics_path: /probe
    params:
      module: [router-module]  # mktxp.conf entry name
    static_configs:
      - targets:
        - router01.example.com
        - router02.example.com
        - 192.168.1.1
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - source_labels: [__param_target]
        target_label: routerboard_name
      - target_label: __address__
        replacement: mktxp-host:49090
```

```ini
# mktxp.conf - Probe module configuration
[router-module]
    module_only = True              # Only use via /probe endpoint
    hostname = placeholder          # Will be overridden by target parameter
    username = monitoring_user
    password = monitoring_password
    routing_stats = True
    poe = False
```

## Docker Deployment

### Basic Docker Run

Run MKTXP with mounted configuration directory.

```bash
# Create configuration directory
mkdir -p mktxp-config
cat > mktxp-config/mktxp.conf << 'EOF'
[My-Router]
    hostname = 192.168.88.1
    username = mktxp_user
    password = mktxp_password

[default]
    enabled = True
    health = True
    interface = True
    dhcp = True
EOF

# Run MKTXP container
docker run -d \
  --name mktxp \
  -v "$(pwd)/mktxp-config:/etc/mktxp" \
  -p 49090:49090 \
  ghcr.io/akpw/mktxp:latest \
  mktxp --cfg-dir /etc/mktxp export

# Verify metrics are available
curl http://localhost:49090/metrics
```

### Docker Compose Deployment

Complete Docker Compose setup with Prometheus and Grafana.

```yaml
# docker-compose.yml
version: '3.8'

services:
  mktxp:
    image: ghcr.io/akpw/mktxp:latest
    container_name: mktxp
    command: ["mktxp", "--cfg-dir", "/etc/mktxp", "export"]
    volumes:
      - ./mktxp-config:/etc/mktxp:ro
    ports:
      - "49090:49090"
    restart: unless-stopped

  prometheus:
    image: prom/prometheus:latest
    container_name: prometheus
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
      - prometheus_data:/prometheus
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'
      - '--storage.tsdb.path=/prometheus'
    ports:
      - "9090:9090"
    restart: unless-stopped

  grafana:
    image: grafana/grafana:latest
    container_name: grafana
    environment:
      - GF_SECURITY_ADMIN_PASSWORD=admin
    volumes:
      - grafana_data:/var/lib/grafana
    ports:
      - "3000:3000"
    restart: unless-stopped

volumes:
  prometheus_data:
  grafana_data:
```

### Shell Access for Debugging

```bash
# Access container shell
docker run -v "$(pwd)/mktxp-config:/etc/mktxp" -it --rm ghcr.io/akpw/mktxp:latest sh

# Test configuration inside container
mktxp --cfg-dir /etc/mktxp show
mktxp --cfg-dir /etc/mktxp print -en My-Router -dc
```

## Kubernetes Deployment

### Kubernetes Deployment with Secret

Deploy MKTXP in Kubernetes with configuration stored as a Secret.

```yaml
# secret.yaml
apiVersion: v1
kind: Secret
metadata:
  name: mktxp-credentials
type: Opaque
stringData:
  mktxp.conf: |
    [Production-Router]
        hostname = 192.168.88.1
        username = mktxp_user
        password = secure_password
        custom_labels = env:production

    [default]
        enabled = True
        health = True
        interface = True
        dhcp = True
        firewall = True
  _mktxp.conf: |
    [MKTXP]
        listen = '0.0.0.0:49090'
        socket_timeout = 5
        fetch_routers_in_parallel = True
        max_worker_threads = 5
---
# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mktxp-exporter
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mktxp-exporter
  template:
    metadata:
      labels:
        app: mktxp-exporter
      annotations:
        prometheus.io/scrape: "true"
        prometheus.io/port: "49090"
    spec:
      containers:
      - name: mktxp-exporter
        image: ghcr.io/akpw/mktxp:latest
        command: ["mktxp"]
        args:
          - --cfg-dir
          - /mktxp_config
          - export
        resources:
          limits:
            memory: "512Mi"
            cpu: "500m"
          requests:
            memory: "256Mi"
            cpu: "100m"
        volumeMounts:
          - name: mktxp-credentials
            mountPath: /mktxp_config
            readOnly: true
        ports:
        - containerPort: 49090
          name: metrics
        livenessProbe:
          httpGet:
            path: /metrics
            port: 49090
          initialDelaySeconds: 30
          periodSeconds: 10
      volumes:
        - name: mktxp-credentials
          secret:
            secretName: mktxp-credentials
---
apiVersion: v1
kind: Service
metadata:
  name: mktxp-exporter
  labels:
    app: mktxp-exporter
spec:
  selector:
    app: mktxp-exporter
  ports:
  - port: 49090
    targetPort: 49090
    name: metrics
```

```bash
# Deploy to Kubernetes
kubectl apply -f secret.yaml
kubectl apply -f deployment.yaml

# Verify deployment
kubectl get pods -l app=mktxp-exporter
kubectl logs -l app=mktxp-exporter

# Test metrics endpoint
kubectl port-forward svc/mktxp-exporter 49090:49090
curl http://localhost:49090/metrics
```

## Mikrotik Router Configuration

### Create Dedicated Monitoring User

Create a minimal-permission user on RouterOS for MKTXP monitoring.

```routeros
# Create user group with minimal permissions
/user group add name=mktxp_group policy=api,read

# Create monitoring user
/user add name=mktxp_user group=mktxp_group password=secure_password

# For LTE metrics on RouterOS v6, add 'test' policy
/user group set mktxp_group policy=api,read,test
```

### Enable API Service

```routeros
# Enable API service (port 8728)
/ip service enable api

# For SSL connections, enable API-SSL (port 8729)
/ip service enable api-ssl

# Restrict API access to specific IP range
/ip service set api address=10.0.0.0/8
```

## Linux Systemd Service

### Create Systemd Service File

Set up MKTXP as a system service for automatic startup.

```bash
# Create service file
sudo tee /etc/systemd/system/mktxp.service << 'EOF'
[Unit]
Description=MKTXP Prometheus Exporter for Mikrotik RouterOS
After=network.target

[Service]
Type=simple
User=mktxp
Group=mktxp
ExecStart=/usr/local/bin/mktxp export
Restart=always
RestartSec=10

# Security hardening
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=read-only
ReadWritePaths=/home/mktxp/mktxp

[Install]
WantedBy=multi-user.target
EOF

# Reload systemd and enable service
sudo systemctl daemon-reload
sudo systemctl enable mktxp
sudo systemctl start mktxp

# Check service status
sudo systemctl status mktxp

# View logs
sudo journalctl -u mktxp -f
```

## Exported Prometheus Metrics

### System Health Metrics

Metrics from `/system/health` including temperature, voltage, fan speeds, and power consumption.

```promql
# Example queries for system health metrics
mktxp_system_routerboard_voltage{routerboard_name="MKT-GT"}
mktxp_system_routerboard_temperature{routerboard_name="MKT-GT"}
mktxp_system_cpu_temperature{routerboard_name="MKT-GT"}
mktxp_system_fan_one_speed{routerboard_name="MKT-GT"}
mktxp_system_power_consumption{routerboard_name="MKT-GT"}
mktxp_system_poe_out_consumption{routerboard_name="MKT-GT"}

# Alert example: High CPU temperature
alert: HighCPUTemperature
expr: mktxp_system_cpu_temperature > 80
for: 5m
labels:
  severity: warning
annotations:
  summary: "High CPU temperature on {{ $labels.routerboard_name }}"
```

### Interface Traffic Metrics

Metrics for interface traffic, errors, and status.

```promql
# Interface traffic metrics
mktxp_interface_rx_byte{routerboard_name="MKT-GT", name="ether1"}
mktxp_interface_tx_byte{routerboard_name="MKT-GT", name="ether1"}
mktxp_interface_rx_packet{routerboard_name="MKT-GT", name="ether1"}
mktxp_interface_tx_packet{routerboard_name="MKT-GT", name="ether1"}

# Interface errors
mktxp_interface_rx_error{name="ether1"}
mktxp_interface_tx_error{name="ether1"}
mktxp_interface_rx_drop{name="ether1"}
mktxp_interface_tx_drop{name="ether1"}

# Interface status
mktxp_interface_running{name="ether1"}
mktxp_interface_disabled{name="ether1"}
mktxp_link_downs{name="ether1"}

# Calculate interface throughput rate
rate(mktxp_interface_rx_byte[5m]) * 8  # bits per second
rate(mktxp_interface_tx_byte[5m]) * 8
```

### DHCP and Wireless Metrics

Metrics for DHCP leases and wireless clients.

```promql
# DHCP lease count
mktxp_dhcp_lease_count{routerboard_name="MKT-GT", server="dhcp1"}

# Wireless client metrics
mktxp_capsman_clients_count{routerboard_name="MKT-GT"}
mktxp_wlan_clients_count{routerboard_name="MKT-GT"}

# Client signal strength
mktxp_capsman_client_signal{routerboard_name="MKT-GT", mac_address="AA:BB:CC:DD:EE:FF"}
```

## Advanced Features

### Remote DHCP Resolution

Configure MKTXP to resolve IP addresses using a remote DHCP server.

```ini
# mktxp.conf
[DHCP-Server]
    hostname = 192.168.88.1
    enabled = False  # Can be disabled if only used for DHCP resolution

[Access-Point]
    hostname = 192.168.88.10
    remote_dhcp_entry = DHCP-Server  # Resolve IPs via DHCP-Server
```

### Remote CAPsMAN Info

Collect CAPsMAN metrics from a central controller for distributed APs.

```ini
# mktxp.conf
[CAPsMAN-Controller]
    hostname = 192.168.88.1
    capsman = True
    capsman_clients = True

[Remote-CAP]
    hostname = 192.168.88.50
    capsman = False  # Disable local CAPsMAN
    remote_capsman_entry = CAPsMAN-Controller  # Get CAPsMAN info from controller
```

### Custom Labels for Device Grouping

Inject custom labels into all metrics for a device to enable grouping in Grafana.

```ini
# mktxp.conf
[DC-London-Router-1]
    hostname = 10.1.0.1
    custom_labels = dc:london, rack:a1, service:production, team:network

[DC-London-Router-2]
    hostname = 10.1.0.2
    custom_labels = dc:london, rack:a2, service:production, team:network

[DC-NYC-Router-1]
    hostname = 10.2.0.1
    custom_labels = dc:nyc, rack:b1, service:production, team:network
```

```promql
# Query metrics by custom labels
sum by (dc) (rate(mktxp_interface_rx_byte[5m]))
avg by (rack) (mktxp_system_cpu_temperature)
```

### Parallel Router Fetching

Enable parallel metrics collection for faster scraping across multiple routers.

```ini
# _mktxp.conf
[MKTXP]
    fetch_routers_in_parallel = True
    max_worker_threads = 10
    max_scrape_duration = 15
    total_max_scrape_duration = 60
```

## Summary

MKTXP serves as a comprehensive monitoring solution for Mikrotik RouterOS environments, bridging the gap between RouterOS devices and modern observability stacks. Its primary use cases include infrastructure monitoring for ISPs and enterprises managing multiple Mikrotik devices, network performance tracking with detailed interface and traffic metrics, wireless network management through CAPsMAN and WLAN client monitoring, and security monitoring via firewall rule traffic analysis and connection tracking. The tool excels in environments requiring centralized monitoring of distributed router deployments.

Integration patterns range from simple single-router setups using the standard `/metrics` endpoint to complex multi-site deployments leveraging the `/probe` endpoint with Prometheus service discovery. MKTXP supports hierarchical configuration inheritance allowing global defaults with per-router overrides, remote data source federation for DHCP and CAPsMAN metrics, and flexible deployment options including bare-metal systemd services, Docker containers, and Kubernetes deployments. Combined with the official Grafana dashboard and custom label injection, MKTXP provides a complete observability solution that scales from home lab setups to enterprise network operations centers.