osquery

https://osquery.readthedocs.io/en/latest/
Admin
osquery is an operating system instrumentation framework for Windows, macOS, and Linux that exposes...
Tokens:51,930
Snippets:603
Trust Score:9.6
Update:2 months ago
Show doc for...
Tokens:
Raw
### Start the osquery daemon

Source: https://osquery.readthedocs.io/en/latest/installation/install-linux

Copies the example configuration file and starts the osquery daemon service.

```bash
sudo cp /opt/osquery/share/osquery/osquery.example.conf /etc/osquery/osquery.conf
# sudo service osqueryd start
sudo systemctl start osqueryd
```

--------------------------------

### Create a C++ osquery extension

Source: https://osquery.readthedocs.io/en/latest/development/osquery-sdk

This example demonstrates defining a custom table plugin and starting the extension process using the osquery SDK.

```cpp
// Note 1: Include the SDK and helpers
#include <osquery/core/system.h>
#include <osquery/sdk/sdk.h>
#include <osquery/sql/dynamic_table_row.h>

using namespace osquery;

// Note 2: Define at least one plugin or table.
class ExampleTablePlugin : public TablePlugin {
 private:
  TableColumns columns() const override {
    return {
      std::make_tuple("example_text", TEXT_TYPE, ColumnOptions::DEFAULT),
      std::make_tuple("example_integer", INTEGER_TYPE, ColumnOptions::DEFAULT),
    };
  }

  TableRows generate(QueryContext& request) {
    TableRows results;

    auto r = make_table_row();
    r["example_text"] = "example";
    r["example_integer"] = INTEGER(1);

    results.push_back(std::move(r));
    return results;
  }
};

// Note 3: Use REGISTER_EXTERNAL to define your plugin or table.
REGISTER_EXTERNAL(ExampleTablePlugin, "table", "example");

int main(int argc, char* argv[]) {
  // Note 4: Start logging, threads, etc.
  osquery::Initializer runner(argc, argv, ToolType::EXTENSION);

  // Note 5: Connect to osqueryi or osqueryd.
  auto status = startExtension("example", "0.0.1");
  if (!status.ok()) {
    LOG(ERROR) << status.getMessage();
    runner.requestShutdown(status.getCode());
  }

  // Finally wait for a signal / interrupt to shutdown.
  runner.waitForShutdown();
  return runner.shutdown(0);
}
```

--------------------------------

### Install macOS build prerequisites

Source: https://osquery.readthedocs.io/en/latest/development/building

Installs necessary development tools via Homebrew and Python dependencies for testing.

```bash
# Install prerequisites
xcode-select --install
brew install ccache git git-lfs cmake python clang-format flex bison

# Optional: install python tests prerequisites
pip3 install --user setuptools pexpect==3.3 psutil timeout_decorator six thrift==0.11.0 osquery
```

--------------------------------

### Install osquery Service via Helper Script

Source: https://osquery.readthedocs.io/en/latest/installation/install-windows

Invoke the bundled management script to install the osquery daemon as a service.

```powershell
C:\Program Files\osquery
λ  .\manage-osqueryd.ps1 -install -startupArgs "--flagfile=`"C:\Program Files\osquery\osquery.flags`""
```

--------------------------------

### Define and Verify Extension Path

Source: https://osquery.readthedocs.io/en/latest/deployment/extensions

Example of an extensions.load file content and verifying the executable file type.

```bash
$ cat /etc/osquery/extensions.load
/usr/lib/osquery/extensions/fb_osquery.ext
$ file /usr/lib/osquery/extensions/fb_osquery.ext
/usr/lib/osquery/extensions/fb_osquery.ext: ELF 64-bit LSB executable
```

--------------------------------

### Start the osquery daemon service

Source: https://osquery.readthedocs.io/en/latest/installation/install-windows

Commands to start the osqueryd service using either PowerShell or Command Prompt.

```powershell
Start-Service osqueryd
```

```cmd
sc.exe start osqueryd
```

--------------------------------

### Example log output with decorations

Source: https://osquery.readthedocs.io/en/latest/deployment/configuration

Shows the structure of a log line containing the decorations key.

```json
{"decorations": {"user": "you", "uptime": "10000", "version": "1.7.3"}}
```

--------------------------------

### Install osquery as a Windows service

Source: https://osquery.readthedocs.io/en/latest/installation/install-windows

Command to register the osquery daemon as a system-level service.

```powershell
C:\Program Files\osquery\osqueryd\osqueryd.exe --install
```

--------------------------------

### Example osquery configuration file

Source: https://osquery.readthedocs.io/en/latest/deployment/configuration

A sample JSON configuration defining daemon options and a query schedule with specific intervals.

```json
{
  "options": {
    "host_identifier": "hostname",
    "schedule_splay_percent": 10
  },
  "schedule": {
    "macos_kextstat": {
      "query": "SELECT * FROM kernel_extensions;",
      "interval": 10
    },
    "foobar": {
      "query": "SELECT foo, bar, pid FROM foobar_table;",
      "interval": 600
    }
  }
}
```

--------------------------------

### Start the test HTTP server

Source: https://osquery.readthedocs.io/en/latest/deployment/remote

Displays the help menu and the command to launch the test server with TLS and enrollment secret requirements.

```bash
$ ./tools/tests/test_http_server.py -h
usage: test_http_server.py [-h] [--tls] [--persist] [--timeout TIMEOUT]
                           [--cert CERT_FILE] [--key PRIVATE_KEY_FILE]
                           [--ca CA_FILE] [--use_enroll_secret]
                           [--enroll_secret SECRET_FILE]
                           PORT

osquery python https server for client TLS testing.

positional arguments:
  PORT                  Bind to which local TCP port.

optional arguments:
  -h, --help            show this help message and exit
  --tls                 Wrap the HTTP server socket in TLS.
  --persist             Wrap the HTTP server socket in TLS.
  --timeout TIMEOUT     If not persisting, exit after a number of seconds
  --cert CERT_FILE      TLS server cert.
  --key PRIVATE_KEY_FILE
                        TLS server cert private key.
  --ca CA_FILE          TLS server CA list for client-auth.
  --use_enroll_secret   Require an enrollment secret for node enrollment
  --enroll_secret SECRET_FILE
                        File containing enrollment secret
$ ./tools/tests/test_http_server.py --tls --persist --cert ./tools/tests/configs/test_server.pem --key ./tools/tests/configs/test_server.key --ca .tools/tests/configs/test_server_ca.pem --use_enroll_secret --enroll_secret ./tools/tests/test_enroll_secret.txt 8080
-- [DEBUG] Starting TLS/HTTPS server on TCP port: 8080
```

--------------------------------

### Install and run cppcheck

Source: https://osquery.readthedocs.io/en/latest/development/building

Commands to install the cppcheck static analysis tool on different platforms and execute the build target.

```bash
apt install cppcheck
```

```bash
brew install cppcheck
```

```bash
cmake --build . --target cppcheck
```

--------------------------------

### Example osqueryd logs

Source: https://osquery.readthedocs.io/en/latest/deployment/remote

Sample output from osqueryd showing successful enrollment and communication with the TLS server.

```text
I1015 10:36:06.894544 2032685056 init.cpp:263] osquery initialized [version=1.5.3]
I1015 10:36:06.935755 2032685056 tls.cpp:68] TLSEnrollPlugin requesting a node enroll key from: https://localhost:8080/enroll
I1015 10:36:06.936123 2032685056 tls.cpp:196] TLS/HTTPS POST request to URI: https://localhost:8080/enroll
I1015 10:36:06.947465 2032685056 tls.cpp:196] TLS/HTTPS POST request to URI: https://localhost:8080/config
I1015 10:36:10.288635 3825664 scheduler.cpp:56] Executing query: SELECT * FROM processes;
I1015 10:36:10.366140 3825664 scheduler.cpp:101] Found results for query (tls_proc) for host: YOURHOSTNAME.local
I1015 10:36:11.019227 528384 tls.cpp:196] TLS/HTTPS POST request to URI: https://localhost:8080/logger
[...]
```

--------------------------------

### Example TLS server logs

Source: https://osquery.readthedocs.io/en/latest/deployment/remote

Sample output from the test server showing received requests from the osquery client.

```text
127.0.0.1 - - [15/Oct/2015 10:36:06] "POST /enroll HTTP/1.1" 200 -
-- [DEBUG] Request: {u'enroll_secret': u'this_is_a_deployment_secret', u'host_identifier': u'YOURHOSTNAME.local'}
-- [DEBUG] Replying: {u'node_key': u'this_is_a_node_secret'}
127.0.0.1 - - [15/Oct/2015 10:36:06] "POST /config HTTP/1.1" 200 -
-- [DEBUG] Request: {u'node_key': u'this_is_a_node_secret'}
-- [DEBUG] Replying: {u'schedule': {u'tls_proc': {u'query': u'select * from processes', u'interval': 1}}}
127.0.0.1 - - [15/Oct/2015 10:36:11] "POST /logger HTTP/1.1" 200 -
-- [DEBUG] Request: {u'node_key': u'this_is_a_node_secret', u'data': [...]}
[...]
```

--------------------------------

### Implement a Filesystem Configuration Plugin

Source: https://osquery.readthedocs.io/en/latest/development/config-plugins

This example demonstrates the default filesystem plugin implementation, which reads JSON configuration data from a specified file path.

```cpp
// Note 1: REQUIRED includes
#include <osquery/config/config.h>
#include <osquery/core/flags.h>

namespace osquery {

// Note 2: Setup any invocation arguments
FLAG(string, config_path, "osquery.conf", "Path to config");

// Note 3: Inherit from ConfigPlugin
class FilesystemConfigPlugin : public ConfigPlugin {
 public:
  osquery::Status genConfig(std::map<std::string, std::string>& config) {
    std::string content;
    std::ifstream content_stream(FLAGS_config_path);

    content_stream.seekg(0, std::ios::end);
    content.reserve(config_stream.tellg());
    content_stream.seekg(0, std::ios::beg);

    content.assign((std::istreambuf_iterator<char>(content_stream)),
                    std::istreambuf_iterator<char>());

    // Note 4: Return an osquery Status and JSON encoded config.
    config["default_source"] = std::move(content);
    return Status(0, "OK");
  }
};

// Note 5: Register the plugin
REGISTER(FilesystemConfigPlugin, "config", "filesystem");
}
```

--------------------------------

### Launch and access AWS Vagrant VM

Source: https://osquery.readthedocs.io/en/latest/development/building

Commands to start the EC2 instance and connect via SSH.

```bash
vagrant up aws-amazon2015.03 --provider=aws
vagrant ssh aws-amazon2015.03
```

--------------------------------

### Install vagrant-aws plugin

Source: https://osquery.readthedocs.io/en/latest/development/building

Install the required plugin to enable AWS EC2 support in Vagrant.

```bash
vagrant plugin install vagrant-aws
```

--------------------------------

### Generate package data

Source: https://osquery.readthedocs.io/en/latest/development/building

Commands to build the osquery install target and prepare the destination directory.

```bash
cd build_folder
mkdir package_data
export DESTDIR=$(pwd)/package_data  # on Windows: `set DESTDIR=path` or `$Env:DESTDIR=path` for PowerShell
cmake --build . --target install    # on Windows: add --config Release
```

--------------------------------

### Configure decorators for top-level output

Source: https://osquery.readthedocs.io/en/latest/deployment/configuration

Example configuration demonstrating potential key collisions when decorations_top_level is enabled.

```json
{
  "decorators": {
    "load": [
      "SELECT 'collision' AS name;",
      "SELECT 'collision' AS hostIdentifier;",
      "SELECT 'collision' AS calendarTime;",
      "SELECT 'collision' AS unixTime;",
      "SELECT 'collision' AS columns;",
      "SELECT 'collision' AS action;"
    ]
  }
}
```

--------------------------------

### Example log entry for startup items

Source: https://osquery.readthedocs.io/en/latest/deployment/anomaly-detection

A sample JSON log entry showing an added startup item, typically found in log aggregation systems like ElasticSearch or Splunk.

```json
{
    "name": "startup_items",
    "action":  "added",
    "columns": {
      "name":  "Phone.app",
      "path":  "/Applications/Phone.app"
    },
    "hostname":  "ted-osx.local",
    "calendarTime":  "Fri Nov  7 09:42:42 2014",
    "unixTime":  "1415382685",
    "epoch": "314159265",
    "counter": "1"
}
```

--------------------------------

### Define osquery configuration options

Source: https://osquery.readthedocs.io/en/latest/deployment/configuration

Example of setting standard osquery configuration options within a JSON configuration file.

```json
{
  "options": {
    "read_max": 100000,
    "events_max": 100000,
    "host_identifier": "uuid"
  }
}
```

--------------------------------

### Sample osquery Configuration File

Source: https://osquery.readthedocs.io/en/latest/deployment/aws-logging

Example JSON configuration file demonstrating how to define AWS Kinesis and Firehose stream settings alongside standard osquery options.

```json
{
  "options": {
    "host_identifier": "hostname",
    "schedule_splay_percent": 10,
    "aws_kinesis_stream": "foo_stream",
    "aws_firehose_stream": "bar_delivery_stream",
    "aws_access_key_id": "ACCESS_KEY",
    "aws_secret_access_key": "SECRET_KEY",
    "aws_region": "us-east-1"
  },
  "schedule": {
    "time": {
      "query": "SELECT * FROM time;",
      "interval": 2,
      "removed": false
    }
  }
}
```

--------------------------------

### Generate MSI package with CMake

Source: https://osquery.readthedocs.io/en/latest/installation/install-windows

Build commands to configure and package osquery as an MSI installer.

```bash
cd \projects\osquery\build
cmake -G "Visual Studio 16 2019" -A x64 -T v141 -DOSQUERY_VERSION="4.0.0" ..\src
cmake --build . --config RelWithDebInfo --target package
```

--------------------------------

### Distributed Discovery Query Examples

Source: https://osquery.readthedocs.io/en/latest/deployment/remote

Examples of distributed read responses with discovery queries and corresponding write requests for specific platforms.

```json
{
  "queries": {
      "always_execute": "select version from osquery_info;",
      "windows_info": "select name from system_info;",
      "darwin_time": "select day from time;"
  },
  "discovery": {
      "windows_info": "select * from os_version where platform='windows';",
      "darwin_time": "select * from os_version where platform='darwin';"
  }
}
```

```json
{
  "node_key": "...",
  "queries": {
    "always_execute": [
      {"version": "5.5.1"}
    ],
    "windows_info": [
      {"name": "windows"}
    ]
  },
  "statuses": {
    "always_execute": 0,
    "windows_info": 0
  }
}
```

```json
{
  "node_key": "...",
  "queries": {
    "always_execute": [
      {"version": "5.4.0"}
    ],
    "darwin_time": [
      {"day": "17"}
    ]
  },
  "statuses": {
    "always_execute": 0,
    "darwin_time": 0
  }
}
```

--------------------------------

### Build osquery on Ubuntu 24

Source: https://osquery.readthedocs.io/en/latest/development/building

Commands to install dependencies, download the toolchain, and compile osquery on Ubuntu 24.

```bash
# Install the prerequisites
sudo apt install --no-install-recommends git python3 bison flex make cmake ccache

# Optional: install python tests prerequisites
sudo apt install --no-install-recommends python3-pip python3-setuptools python3-psutil python3-six python3-wheel
pip3 install timeout_decorator thrift==0.11.0 osquery pexpect==3.3

# Optional: install RPM packaging prerequisites
sudo apt install --no-install-recommends rpm binutils

# Download and install the osquery toolchain for the architecture in use.
export ARCH=$(uname -m)
wget https://github.com/osquery/osquery-toolchain/releases/download/1.3.0/osquery-toolchain-1.3.0-${ARCH}.tar.xz
sudo tar xvf osquery-toolchain-1.3.0-${ARCH}.tar.xz -C /usr/local

# Download source
git clone https://github.com/osquery/osquery
cd osquery

# Build osquery
mkdir build; cd build
cmake -DOSQUERY_TOOLCHAIN_SYSROOT=/usr/local/osquery-toolchain ..
cmake --build . -j10 # where 10 is the number of parallel build jobs
```

--------------------------------

### Example output with top-level collisions

Source: https://osquery.readthedocs.io/en/latest/deployment/configuration

Demonstrates how decorator keys overwrite existing top-level log keys.

```json
{
  "name": "collision",
  "hostIdentifier": "collision",
  "calendarTime": "collision",
  "unixTime": "collision",
  "action": "added",
  "columns": {
    "cpu_brand": "Intel(R) Core(TM) i7-4980HQ CPU @ 2.80GHz",
    "hostname": "osquery.example.com",
    "physical_memory": "1234567890"
  }
}
```

--------------------------------

### View osquery package structure

Source: https://osquery.readthedocs.io/en/latest/installation/install-linux

Displays the default file and directory structure created after installing osquery packages.

```text
/etc/init.d/osqueryd
/etc/osquery/
/etc/sysconfig/osqueryd
/opt/osquery/bin/osqueryctl
/opt/osquery/bin/osqueryd
/usr/local/bin/osqueryi -> /opt/osquery/bin/osqueryd
/usr/local/bin/osqueryctl -> /opt/osquery/bin/osqueryctl
/usr/lib/systemd/system/osqueryd.service
/opt/osquery/share/osquery/certs/certs.pem
/opt/osquery/share/osquery/lenses/{*}.aug
/opt/osquery/share/osquery/packs/{*}.conf
/opt/osquery/share/osquery/osquery.example.conf
/var/log/osquery/
/var/osquery/
```

--------------------------------

### Configure OpenBSM audit_control

Source: https://osquery.readthedocs.io/en/latest/deployment/process-auditing

Example configuration for the /etc/security/audit_control file to enable process execution and argument auditing.

```text
#
# $P4: //depot/projects/trustedbsd/openbsm/etc/audit_control#8 $
#
dir:/var/audit
flags:ex,pc,ap,aa,lo,nt
minfree:5
naflags:no
policy:cnt,argv,arge
filesz:2M
expire-after:10M
superuser-set-sflags-mask:has_authenticated,has_console_access
superuser-clear-sflags-mask:has_authenticated,has_console_access
member-set-sflags-mask:
member-clear-sflags-mask:has_authenticated
```

--------------------------------

### Initial query result log output

Source: https://osquery.readthedocs.io/en/latest/introduction/using-osqueryd

Example of the initial log output generated by the usb_devices query on a macOS host.

```json
[
  {"model":"XHCI Root Hub SS Simulation","vendor":"Apple Inc."},
  {"model":"XHCI Root Hub USB 2.0 Simulation","vendor":"Apple Inc."},
  {"model":"BRCM20702 Hub","vendor":"Apple Inc."},
  {"model":"Internal Memory Card Reader","vendor":"Apple"},
  {"model":"Apple Internal Keyboard \/ Trackpad","vendor":"Apple Inc."},
  {"model":"Bluetooth USB Host Controller","vendor":"Apple Inc."}
]
```

--------------------------------

### Define osquery scheduled queries

Source: https://osquery.readthedocs.io/en/latest/deployment/performance-safety

Example configuration file defining multiple queries to be scheduled for execution on client machines.

```json
{
  "schedule": {
    "alf_services": {
      "query": "SELECT service, process FROM alf_services WHERE state != 0;",
      "interval": 60
    },
    "installed_applications": {
      "query": "SELECT name, path, bundle_version, minimum_system_version, applescript_enabled, bundle_executable FROM apps;",
      "interval": 60
    },
    "all_kexts": {
      "query": "SELECT name, version FROM kernel_extensions;",
      "interval": 60
    },
    "non_apple_kexts": {
      "query": "SELECT * FROM kernel_extensions WHERE name NOT LIKE 'com.apple.%' AND name != '__kernel__';",
      "interval": 60
    },
    "processes_binding_to_ports": {
      "query": "SELECT DISTINCT process.name, listening.port, listening.protocol, listening.family, listening.address, process.pid, process.path, process.on_disk, process.parent, process.start_time FROM processes AS process JOIN listening_ports AS listening ON process.pid = listening.pid;",
      "interval": 60
    },
    "processes_not_on_disk": {
      "query": "SELECT * FROM processes WHERE on_disk != 1;",
      "interval": 60
    }
  }
}
```

--------------------------------

### Perform C-Math Operations

Source: https://osquery.readthedocs.io/en/latest/introduction/sql

Examples of using math functions like sqrt, log, and power on disk information.

```sql
osquery> .mode line osquery> select disk_size as disk_size from disk_info; disk_size = 107372805120 osquery> select sqrt(disk_size) as disk_size from disk_info; disk_size = 327677.898430761 osquery> select log(disk_size) as disk_size from disk_info; disk_size = 25.3995727757846 osquery> select log10(disk_size) as disk_size from disk_info; disk_size = 11.0308942992233 osquery> select ceil(disk_size) as disk_size from disk_info; disk_size = 107372805120 osquery> select floor(disk_size) as disk_size from disk_info; disk_size = 107372805120 osquery> select power(disk_size, 2) as disk_size from disk_info; disk_size = 1.15289192793375e+22 osquery> select pi() * disk_size as disk_size from disk_info; disk_size = 337321615760.32
```

--------------------------------

### State change log output for added device

Source: https://osquery.readthedocs.io/en/latest/introduction/using-osqueryd

Example log output generated when a new USB device is detected by the daemon.

```json
[
  {"model":"U3 Cruzer Micro","vendor":"SanDisk Corporation"}
]
```

--------------------------------

### SQL Query for Performance Monitoring

Source: https://osquery.readthedocs.io/en/latest/deployment/logging

Example SQL query used to monitor osquery performance by joining system tables.

```sql
SELECT i.*, p.resident_size, p.user_time, p.system_time, t.minutes AS c
  FROM osquery_info i, processes p, time t
  WHERE p.pid = i.pid;
```

--------------------------------

### Define and Query Views

Source: https://osquery.readthedocs.io/en/latest/deployment/configuration

Creates reusable SQL views to simplify complex queries, followed by an example of querying the defined view.

```json
{
  "views": {
    "kernel_hashes" : "SELECT hash.path AS kernel_binary, version, hash.sha256 AS sha256, hash.sha1 AS sha1, hash.md5 AS md5 FROM (SELECT path || '/Contents/MacOS/' AS directory, name, version FROM kernel_extensions) JOIN hash USING (directory);"
  }
}
```

```sql
SELECT * FROM kernel_hashes WHERE kernel_binary NOT LIKE "%apple%";
```

--------------------------------

### Example FIM Configuration

Source: https://osquery.readthedocs.io/en/latest/deployment/file-integrity-monitoring

Defines file paths to monitor and exclude using categories. Ensure exclude_paths categories match those defined in file_paths.

```json
{
  "schedule": {
    "crontab": {
      "query": "SELECT * FROM crontab;",
      "interval": 300
    },
    "file_events": {
      "query": "SELECT * FROM file_events;",
      "removed": false,
      "interval": 300
    }
  },
  "file_paths": {
    "homes": [
      "/root/.ssh/%%",
      "/home/%/.ssh/%%"
    ],
    "etc": [
      "/etc/%%"
    ],
    "tmp": [
      "/tmp/%%"
    ]
  },
  "exclude_paths": {
    "homes": [
      "/home/not_to_monitor/.ssh/%%"
    ],
    "tmp": [
      "/tmp/too_many_events/"
    ]
  }
}
```

--------------------------------

### Comprehensive ATC Configuration File

Source: https://osquery.readthedocs.io/en/latest/deployment/configuration

A full example of an atc_tables.json file defining multiple ATC tables for system and user TCC databases.

```json
{
  "auto_table_construction": {
    "tcc_system_entries": {
      "query": "SELECT service, client, auth_value, last_modified FROM access;",
      "path": "/Library/Application Support/com.apple.TCC/TCC.db",
      "columns": [
        "service",
        "client",
        "auth_value",
        "last_modified"
      ],
      "platform": "darwin"
    },
    "tcc_user_entries": {
      "query": "SELECT service, client, auth_value, last_modified FROM access;",
      "path": "/Users/%/Library/Application Support/com.apple.TCC/TCC.db",
      "columns": [
        "service",
        "client",
        "auth_value",
        "last_modified"
      ],
      "platform": "darwin"
    }
  }
}
```

--------------------------------

### Post-installation configuration

Source: https://osquery.readthedocs.io/en/latest/installation/install-macos

Commands to initialize the osquery daemon using the helper script or manual configuration deployment.

```bash
# You can use the helper script:
sudo osqueryctl start

# Or, install the example config and launch daemon yourself:
sudo cp /var/osquery/osquery.example.conf /var/osquery/osquery.conf
sudo cp /var/osquery/io.osquery.agent.plist /Library/LaunchDaemons
sudo launchctl load /Library/LaunchDaemons/io.osquery.agent.plist
```

--------------------------------

### Install Python Test Prerequisites

Source: https://osquery.readthedocs.io/en/latest/development/building

Install required Python packages for testing using pip in a PowerShell console.

```powershell
# Using a PowerShell console
& 'C:\Program Files\Python37\python.exe' -m pip install setuptools psutil timeout_decorator thrift==0.11.0 osquery pywin32
```

--------------------------------

### Prepare osquery-packaging repository

Source: https://osquery.readthedocs.io/en/latest/development/building

Prerequisites and cloning steps for the packaging repository.

```bash
sudo apt install binutils elfutils
```

```bash
git clone https://github.com/osquery/osquery-packaging

mkdir build
cd build
```

--------------------------------

### Install WiX Toolset via Chocolatey

Source: https://osquery.readthedocs.io/en/latest/installation/install-windows

Command to install the WiX Toolset required for generating MSI packages.

```powershell
choco install wixtoolset
```

--------------------------------

### Query macOS startup items

Source: https://osquery.readthedocs.io/en/latest/deployment/anomaly-detection

Use this query in osqueryi to list all applications configured to run at boot.

```sql
SELECT * FROM startup_items;
```

--------------------------------

### Perform Trigonometric Calculations

Source: https://osquery.readthedocs.io/en/latest/introduction/sql

Examples of using trigonometric functions and angle unit conversions.

```sql
osquery .mode line osquery> select sin(30); sin(30) = -0.988031624092862 osquery> select cos(30); cos(30) = 0.154251449887584 osquery> select tan(30); tan(30) = -6.40533119664628 osquery> select cot(30); cot(30) = -0.156119952161659 osquery> select asin(.5); asin(.5) = 0.523598775598299 osquery> select acos(.5); acos(.5) = 1.0471975511966 osquery> select atan(.5); atan(.5) = 0.463647609000806 osquery> select radians(60); radians(60) = 1.0471975511966 osquery> select degrees(1.3); degrees(1.3) = 74.484513367007
```

--------------------------------

### Configure Event Subscribers

Source: https://osquery.readthedocs.io/en/latest/deployment/configuration

Example configuration snippet for disabling specific event subscribers.

```json
{
  "schedule": {...},
  "events": {
    "disable_subscribers": ["yara_events"]
  }
}
```

--------------------------------

### Run an osquery extension

Source: https://osquery.readthedocs.io/en/latest/development/osquery-sdk

View help flags for an extension and execute it in the background using the socket path identified from the shell.

```bash
$ ./build/darwin/osquery/example_extension.ext --help
osquery 1.7.0, your OS as a high-performance relational database
Usage: example_extension.ext [OPTION]...

osquery extension command line flags:

    --interval VALUE  Seconds delay between connectivity checks
    --socket VALUE    Path to the extensions UNIX domain socket
    --timeout VALUE   Seconds to wait for autoloaded extensions

osquery project page <https://osquery.io>.
$ ./build/darwin/osquery/example_extension.ext --socket /Users/USERNAME/.osquery/shell.em &
```

--------------------------------

### Sample socket_event log entry

Source: https://osquery.readthedocs.io/en/latest/deployment/process-auditing

Example of the JSON output generated by the socket_events table.

```json
{
  "action": "added",
  "columns": {
    "time": "1527895541",
    "status": "succeeded",
    "remote_port": "80",
    "action": "connect",
    "auid": "1000",
    "family": "2",
    "local_address": "",
    "local_port": "0",
    "path": "/usr/bin/curl",
    "pid": "30220",
    "remote_address": "172.217.164.110"
  },
  "unixTime": 1527895545,
  "hostIdentifier": "vagrant",
  "name": "socket_events",
  "numerics": false
}
```

--------------------------------

### Basic process queries and display modes

Source: https://osquery.readthedocs.io/en/latest/introduction/sql

Demonstrates selecting process data and toggling between table and line output modes.

```sql
osquery> SELECT pid, name, path FROM processes WHERE pid > 0 LIMIT 1;
+-----+---------+---------------+
| pid | name    | path          |
+-----+---------+---------------+
| 1   | launchd | /sbin/launchd |
+-----+---------+---------------+
osquery> .mode line
osquery> SELECT pid, name, path FROM processes WHERE pid > 0 LIMIT 1;
  pid = 1
 name = launchd
 path = /sbin/launchd
osquery> .mode pretty
```

--------------------------------

### Implement table generator function

Source: https://osquery.readthedocs.io/en/latest/development/creating-tables

Example implementation of a table generator function in C++ for osquery.

```cpp
/**
 * Copyright (c) 2014-present, The osquery authors
 *
 * This source code is licensed as defined by the LICENSE file found in the
 * root directory of this source tree.
 *
 * SPDX-License-Identifier: (Apache-2.0 OR GPL-2.0-only)
 */

#include <ctime>
#include <osquery/core/tables.h>

namespace osquery {
namespace tables {

QueryData genTimeExample(QueryContext &context) {
  QueryData rows;

  Row r;

  time_t _time = time(0);
  struct tm* now = localtime(&_time);

  r["hour"] = now->tm_hour;
  r["minutes"] = now->tm_min;
  r["seconds"] = now->tm_sec;

  rows.push_back(std::move(r));
  return rows;
}
}
}
```

--------------------------------

### Compare Versions

Source: https://osquery.readthedocs.io/en/latest/introduction/sql

Compares two version strings using different flavors like ARCH, DPKG, or RHEL.

```sql
osquery> .mode line
osquery> select version_compare('1.0', '1.0'); version_compare('1.0', '1.0') = 0
osquery> select version_compare('4:1.1.0', '4:1.1.0-3', 'ARCH'); version_compare('4:1.1.0', '4:1.1.0-3', 'ARCH') = 0
osquery> select version_compare('50.4.1b', '50.4.1c'); version_compare('50.4.1b', '50.4.1c') = -1
osquery> select version_compare('1.0.0~rc2^2021', '1.0.0', 'RHEL'); version_compare('1.0.0~rc2^2021', '1.0.0', 3) = -1
osquery> select version_compare('1:1.2.13-2', '4.2.1', 'ARCH'); version_compare('1:1.2.13-2', '4.2.1', 1) = 1
```

--------------------------------

### Create Windows packages

Source: https://osquery.readthedocs.io/en/latest/development/building

CMake commands to configure and build Windows packages using WIX or NuGet.

```batch
call "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Auxiliary\Build\vcvars64.bat"

cmake -DCMAKE_BUILD_TYPE=Release ^
  -DCPACK_GENERATOR=WIX ^
  -DOSQUERY_PACKAGE_VERSION=%OSQUERY_VERSION% ^
  -DOSQUERY_DATA_PATH=%DESTDIR% ^
  -DOSQUERY_BITNESS=64 ^
  ..\osquery-packaging

cmake --build . --config Release --target package
```

--------------------------------

### Download and Build osquery Source

Source: https://osquery.readthedocs.io/en/latest/development/building

Commands to clone the repository, configure the build with CMake, and compile the project.

```powershell
# Download source
git clone https://github.com/osquery/osquery
cd osquery

# Configure
mkdir build; cd build
cmake -G "Visual Studio 17 2022" -A x64 .. # or -A arm64

# Build
cmake --build . --config RelWithDebInfo -j10 # Number of projects to build in parallel
```

--------------------------------

### Sample process_events log entry

Source: https://osquery.readthedocs.io/en/latest/deployment/process-auditing

Example JSON output generated by the process_events table when auditing is enabled.

```json
{
  "action": "added",
  "columns": {
    "uid": "0",
    "time": "1527895541",
    "pid": "30219",
    "path": "/usr/bin/curl",
    "auid": "1000",
    "cmdline": "curl google.com",
    "ctime": "1503452096",
    "cwd": "",
    "egid": "0",
    "euid": "0",
    "gid": "0",
    "parent": ""
  },
  "unixTime": 1527895550,
  "hostIdentifier": "vagrant",
  "name": "process_events",
  "numerics": false
}
```

--------------------------------

### Download and build osquery source

Source: https://osquery.readthedocs.io/en/latest/development/building

Clones the repository, configures the build directory with CMake, and compiles the project.

```bash
# Download source
git clone https://github.com/osquery/osquery
cd osquery

# Configure
mkdir build; cd build
cmake ..

# Build
cmake --build . -j $(sysctl -n hw.ncpu)
```

--------------------------------

### Querying tables with required arguments

Source: https://osquery.readthedocs.io/en/latest/introduction/sql

Shows how to query tables like 'file' that require specific constraints to avoid full system walks.

```sql
osquery> .mode line
osquery> SELECT * FROM file;
osquery> SELECT * FROM file WHERE path = '/dev/zero';
      path = /dev/zero
 directory = /dev
  filename = zero
     inode = 304
       uid = 0
       gid = 0
      mode = 0666
    device = 50331651
      size = 0
block_size = 131072
     atime = 1463786341
     mtime = 1463786341
     ctime = 1463786341
     btime = 0
hard_links = 1
      type = character
osquery> SELECT count(1) FROM file WHERE path LIKE '/dev/%';
count(1) = 568
```

--------------------------------

### Identify table exceptions

Source: https://osquery.readthedocs.io/en/latest/deployment/debugging

Example of the log output generated when an exception occurs during virtual table execution.

```text
Exception while executing table TABLE_NAME: EXCEPTION_MESSAGE
```

--------------------------------

### Create a basic unit test file

Source: https://osquery.readthedocs.io/en/latest/development/unit-tests

A template for a new unit test file using the Google Test framework.

```cpp
#include <gtest/gtest.h>

namespace osquery {
namespace example {

class ExampleTests : public testing::Test {};

TEST_F(ExampleTests, test_plugin) {
  EXPECT_TRUE(1 == 1);
}
}
}

int main(int argc, char* argv[]) {
  testing::InitGoogleTest(&argc, argv);
  return RUN_ALL_TESTS();
}
```

--------------------------------

### Profile osquery Configuration

Source: https://osquery.readthedocs.io/en/latest/deployment/performance-safety

Estimate CPU and memory load for a specific configuration before deployment.

```bash
./tools/analysis/profile.py --config /path/to/osquery.conf --count 1 --rounds 4
```

--------------------------------

### Removing osquery from macOS

Source: https://osquery.readthedocs.io/en/latest/installation/install-macos

Commands to unload the launch daemon, remove installation files, and forget the package receipt.

```bash
# Unload and remove io.osquery.agent.plist launchdaemon
sudo launchctl unload /Library/LaunchDaemons/io.osquery.agent.plist
sudo rm /Library/LaunchDaemons/io.osquery.agent.plist

# Remove files/directories created by osquery installer pkg
sudo rm -rf /private/var/log/osquery
sudo rm -rf /private/var/osquery
sudo rm /usr/local/bin/osquery*
sudo rm -rf /opt/osquery

sudo pkgutil --forget io.osquery.agent
```

--------------------------------

### Default macOS package file structure

Source: https://osquery.readthedocs.io/en/latest/installation/install-macos

The directory layout created by the standard osquery macOS package installer.

```text
/private/var/osquery/io.osquery.agent.plist
/private/var/osquery/osquery.example.conf
/private/var/log/osquery/
/private/var/osquery/lenses/{*}.aug
/private/var/osquery/packs/{*}.conf
/opt/osquery/lib/osquery.app
/usr/local/bin/osqueryi -> /opt/osquery/lib/osquery.app/Contents/MacOS/osqueryd
/usr/local/bin/osqueryctl -> /opt/osquery/lib/osquery.app/Contents/Resources/osqueryctl
```

--------------------------------

### Accessing shell help

Source: https://osquery.readthedocs.io/en/latest/introduction/sql

Use the .help command within the osqueryi shell to view available meta-commands.

```text
$ osqueryi
Using a virtual database. Need help, type '.help'
osquery> .help
Welcome to the osquery shell. Please explore your OS!
You are connected to a transient 'in-memory' virtual database.

.all [TABLE]     Select all from a table
.bail ON|OFF     Stop after hitting an error
.connect PATH    Connect to an osquery extension socket
.disconnect      Disconnect from a connected extension socket
.echo ON|OFF     Turn command echo on or off
[...]
osquery>
```

--------------------------------

### Chef Recipe for Linux Deployment

Source: https://osquery.readthedocs.io/en/latest/deployment/configuration

Configures the osquery service and deploys the configuration file on Linux systems.

```ruby
# Service name installed by the osquery package.
service_name = 'osqueryd'

cookbook_file '/etc/osquery/osquery.conf' do
  source 'osquery.conf'
  mode 0444
  owner 'root'
  group 'wheel'
  notifies :restart, "service[#{service_name}]"
end

service service_name do
  action [:enable, :start]
end
```

--------------------------------

### Manage Windows Event Log manifest

Source: https://osquery.readthedocs.io/en/latest/installation/install-windows

Commands to install or uninstall the Windows Event Log manifest using wevtutil.

```cmd
wevtutil im "C:\Program Files\osquery\osquery.man"
```

```cmd
wevtutil um "C:\Program Files\osquery\osquery.man"
```

--------------------------------

### Autoload an extension via CLI

Source: https://osquery.readthedocs.io/en/latest/development/osquery-sdk

Load an extension automatically when starting the osquery shell using the --extension flag.

```bash
./build/darwin/osquery/osqueryi --extension ./build/darwin/osquery/example_extension.ext
```

--------------------------------

### Manually load an osquery extension

Source: https://osquery.readthedocs.io/en/latest/deployment/extensions

Load an extension binary directly using the osqueryi command line interface.

```bash
osqueryi --extension /path/to/your_extension.ext
```

--------------------------------

### Initialize Subscription Context

Source: https://osquery.readthedocs.io/en/latest/development/pubsub-framework

Configures the subscription context with path, recursion, and event mask settings within the init method.

```cpp
Status NewETCFilesEventSubscriber::init() {
  // We templated our subscriber to create an inotify publisher-specific
  // subscription context.
  auto sc = createSubscriptionContext();
  sc->path = "/etc";
  sc->recursive = true;
  // 'mask' is specific to inotify.
  sc->mask = IN_CREATE;
  subscribe(&NewETCFilesEventSubscriber::Callback, sc);
}
```

--------------------------------

### Inspect osquery extensions in the shell

Source: https://osquery.readthedocs.io/en/latest/development/osquery-sdk

Start the osquery shell and query the osquery_extensions table to identify the UNIX domain socket path.

```bash
$ ./build/darwin/osquery/osqueryi
osquery> SELECT path FROM osquery_extensions;
+-------------------------------------+
| path                                |
+-------------------------------------+
| /Users/USERNAME/.osquery/shell.em   |
+-------------------------------------+
osquery> ^Z
[1]  + 98777 suspended  ./build/darwin/osquery/osqueryi
```

--------------------------------

### Run queries with output formatting

Source: https://osquery.readthedocs.io/en/latest/introduction/using-osqueryi

Execute a single query from the command line with specific output formats like JSON.

```bash
$ osqueryi --json "SELECT * FROM routes WHERE destination = '::1'"
[
  {"destination":"::1","flags":"2098181","gateway":"::1","interface":"","metric":"0","mtu":"16384","netmask":"128","source":"","type":"local"}
]
```

--------------------------------

### Configure file_paths and file_accesses

Source: https://osquery.readthedocs.io/en/latest/deployment/file-integrity-monitoring

Define file paths for monitoring and explicitly opt-in to access monitoring for specific categories.

```json
{
  "file_paths": {
    "homes": [
      "/root/.ssh/%%",
      "/home/%/.ssh/%%"
    ],
    "etc": [
      "/etc/%%"
    ],
    "tmp": [
      "/tmp/%%"
    ]
  },
  "file_accesses": ["homes", "etc"]
}
```

--------------------------------

### Run osqueryi with Audit Flags

Source: https://osquery.readthedocs.io/en/latest/deployment/process-auditing

Execute osqueryi as root with specific audit configuration flags to enable process and socket event monitoring.

```bash
osqueryi --audit_allow_config=true --audit_allow_sockets=true --audit_persist=true --disable_audit=false --events_expiry=1 --events_max=50000 --logger_plugin=filesystem  --disable_events=false
```

--------------------------------

### Handle disabled event tables in shell

Source: https://osquery.readthedocs.io/en/latest/deployment/debugging

Example of the warning received when querying event-based tables without enabling events in the shell.

```text
osquery> select * from file_events;
virtual_table.cpp:542] Table file_events is event-based but events are disabled
virtual_table.cpp:549] Please see the table documentation: https://osquery.io/schema/current/#file_events
```

--------------------------------

### Override Thresholds via Command Line

Source: https://osquery.readthedocs.io/en/latest/deployment/performance-safety

Example command to execute the profile script with custom threshold values for duration and memory.

```bash
./tools/analysis/profile.py --config /path/to/osquery.conf --duration 1.0 2.0 5.0 --memory 4194304 8388608 16777216
```
Tokens:
Raw