### Start a virtual machine

Source: https://github.com/churchers/vm-bhyve/blob/master/README.md

Use 'vm start' to boot a virtual machine. Ensure the guest OS has been successfully installed.

```bash
# vm start testvm
```

--------------------------------

### Install Windows Guest VM

Source: https://github.com/churchers/vm-bhyve/wiki/Running-Windows

Starts the Windows installation process for a guest VM. It requires specifying the Windows ISO file and waits for a VNC client connection to begin the installation.

```bash
# vm install winguest Windows-Installer.iso
```

--------------------------------

### resflash Setup Commands

Source: https://github.com/churchers/vm-bhyve/wiki/Supported-Guest-Examples

Commands to create, configure, and start a resflash virtual machine. This includes downloading the image, decompressing it, and renaming it to disk0.img.

```shell
vm create -t resflash resflash-vm
vm configure resflash-vm
cd /my/vm/resflash-vm
fetch https://stable.rcesoftware.com/pub/resflash/6.2/amd64/install/resflash-amd64-com0-115200-20171202_1952.img.gz
gunzip resflash-amd64-com0-115200-20171202_1952.img.gz
mv resflash-amd64-com0-115200-20171202_1952.img disk0.img
vm -f start resflash-vm
```

--------------------------------

### Install Alpine Linux VM

Source: https://github.com/churchers/vm-bhyve/wiki/Guest-example:-Alpine-Linux

Starts the installation process for the Alpine Linux VM using a specified ISO image.

```shell
vm install alpine alpine-virt-3.13.2-x86_64.iso
```

--------------------------------

### Launch Ubuntu server with cloud-init and SSH key

Source: https://github.com/churchers/vm-bhyve/wiki/Cloud-Images

Download an Ubuntu cloud image, create a VM with cloud-init enabled, inject an SSH public key, and start the instance. Includes example SSH connection and host key verification.

```sh
vm img https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img
vm create -t ubuntu-cloud -i noble-server-cloudimg-amd64.img -C -k ~/.ssh/id_rsa.pub ubuntu-server
vm start ubuntu-server
Starting ubuntu-server
* found guest in /zroot/vm/ubuntu-server
* booting...
ssh ubuntu@192.168.0.91
The authenticity of host '192.168.0.91 (192.168.0.91)' can't be established.
ECDSA key fingerprint is SHA256:6s9uReyhsIXRv0dVRcBCKMHtY0kDYRV7zbM7ot6u604.
No matching host key fingerprint found in DNS.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.0.91' (ECDSA) to the list of known hosts.
Welcome to Ubuntu 24.04.1 LTS (GNU/Linux 6.8.0-45-generic x86_64)
```

--------------------------------

### Install a guest operating system

Source: https://github.com/churchers/vm-bhyve/blob/master/README.md

Initiate the guest OS installation process using 'vm install'. The machine runs in the background, requiring 'vm console' to connect.

```bash
# vm install testvm FreeBSD-14.2-RELEASE-amd64-disc1.iso
```

--------------------------------

### Install vm-bhyve from source

Source: https://github.com/churchers/vm-bhyve/blob/master/README.md

Install vm-bhyve by compiling from its source directory.

```bash
# make install
```

--------------------------------

### Install bhyve-firmware and Create Windows Guest

Source: https://github.com/churchers/vm-bhyve/wiki/Guest-Example:-Windows

Installs the necessary firmware and creates a new Windows guest using the default template. The guest is then ready for OS installation.

```bash
# pkg install bhyve-firmware
# vm create -t windows myguest
# vm install myguest Windows.ISO
```

--------------------------------

### Create and install a new VM

Source: https://github.com/churchers/vm-bhyve/blob/master/README.md

Create a new virtual machine named 'myguest' and begin the installation process using the specified ISO image. The '-f' option runs the guest directly on the terminal.

```bash
vm create myguest
vm install [-f] myguest FreeBSD-14.2-RELEASE-amd64-bootonly.iso
```

--------------------------------

### Install a guest operating system in foreground mode

Source: https://github.com/churchers/vm-bhyve/blob/master/README.md

Run the guest OS installation directly on your terminal using the '-f' (foreground) option with 'vm install'.

```bash
# vm install -f testvm FreeBSD-14.2-RELEASE-amd64-disc1.iso
```

--------------------------------

### Create and Install FreeBSD Guest VM

Source: https://github.com/churchers/vm-bhyve/wiki/Guest-Example:-FreeBSD

Commands to create a new VM and install FreeBSD from an ISO image using vm-bhyve.

```bash
# vm create myguest
# vm install myguest FreeBSD.iso
```

--------------------------------

### Install OmniOSce VM

Source: https://github.com/churchers/vm-bhyve/wiki/Running-OmniOS

Command to start the OmniOSce installation process within a created VM, specifying the VM name and the ISO image to use.

```bash
vm install omniosce omnios-r151040g.iso
```

--------------------------------

### Install sample VM templates

Source: https://github.com/churchers/vm-bhyve/wiki/Quickstart

Copies sample configuration templates provided by vm-bhyve to the VM directory.

```bash
cp /usr/local/share/examples/vm-bhyve/* /mountpoint/for/pool/vm/.templates/
```

--------------------------------

### Install qemu-tools package

Source: https://github.com/churchers/vm-bhyve/blob/master/README.md

Install the 'qemu-tools' package, which is required for using cloud images with vm-bhyve.

```bash
# pkg install qemu-tools
```

--------------------------------

### Install vm-bhyve

Source: https://github.com/churchers/vm-bhyve/blob/master/README.md

Install the vm-bhyve package using pkg.

```bash
pkg install vm-bhyve
```

--------------------------------

### OpenBSD Configuration

Source: https://github.com/churchers/vm-bhyve/wiki/Supported-Guest-Examples

Example configuration for OpenBSD 6.2 amd64, adaptable for other versions by updating the install command. It uses the GRUB loader and virtio devices for network and disk.

```shell
loader="grub"
cpu=1
memory=256M
network0_type="virtio-net"
network0_switch="public"
disk0_type="virtio-blk"
disk0_name="disk0.img"
grub_install0="kopenbsd -h com0 /6.2/amd64/bsd.rd"
grub_run0="kopenbsd -h com0 -r sd0a /bsd"
bhyve_options="-w"
```

--------------------------------

### Install qemu package

Source: https://github.com/churchers/vm-bhyve/wiki/Cloud-Images

Install the qemu package required for using cloud images.

```sh
pkg install qemu
```

--------------------------------

### Install FreeBSD on a VM

Source: https://github.com/churchers/vm-bhyve/wiki/Quickstart

Installs FreeBSD on the 'myguest' VM using the downloaded bootonly ISO. The -f option runs the guest directly on the terminal (requires vm-bhyve-0.12.5+).

```bash
vm install [-f] myguest FreeBSD-10.3-RELEASE-amd64-bootonly.iso
```

--------------------------------

### Copy Sample VM Templates

Source: https://github.com/churchers/vm-bhyve/blob/master/README.md

Installs the sample VM templates to your specified path. Ensure you replace '/my/vm/path/' with your actual VM directory.

```bash
# cp /usr/local/share/examples/vm-bhyve/* /my/vm/path/.templates/
```

--------------------------------

### Install VirtIO Drivers using ISO

Source: https://github.com/churchers/vm-bhyve/wiki/Guest-Example:-Windows

Installs VirtIO network drivers by booting the guest with a special ISO image. This is an alternative to installing drivers within a running guest.

```bash
# vm install winguest virtio-installer.iso
```

--------------------------------

### Create a new virtual machine

Source: https://github.com/churchers/vm-bhyve/blob/master/README.md

Use 'vm create' to provision a new virtual machine. The first example uses default settings, while the second allows specifying a template and disk size.

```bash
# vm create testvm
```

```bash
# vm create -t templatename -s 50G testvm
```

--------------------------------

### Download VM ISO image

Source: https://github.com/churchers/vm-bhyve/blob/master/README.md

Download a FreeBSD ISO image for installation.

```bash
vm iso https://download.freebsd.org/ftp/releases/ISO-IMAGES/14.2/FreeBSD-14.2-RELEASE-amd64-bootonly.iso
```

--------------------------------

### Install cdrkit-genisoimage package

Source: https://github.com/churchers/vm-bhyve/wiki/Cloud-Images

Install the cdrkit-genisoimage package required for cloud-init support.

```sh
pkg install cdrkit-genisoimage
```

--------------------------------

### Launch FreeBSD using official cloud image

Source: https://github.com/churchers/vm-bhyve/wiki/Cloud-Images

Download, create, and start a FreeBSD virtual machine from an official cloud image.

```sh
vm img https://download.freebsd.org/ftp/releases/VM-IMAGES/14.0-RELEASE/amd64/Latest/FreeBSD-14.0-RELEASE-amd64.raw.xz
vm create -t freebsd-zvol -i FreeBSD-14.0-RELEASE-amd64.raw freebsd-cloud
vm start freebsd-cloud
```

--------------------------------

### Gentoo Installation Configuration

Source: https://github.com/churchers/vm-bhyve/wiki/Supported-Guest-Examples

Configuration for installing Gentoo in bhyve, using grub loader and virtio drivers. Note the use of /boot for kernel loading.

```configuration
loader="grub"
cpu=1
memory=256M
grub_install0="linux /boot/gentoo root=/dev/ram0 init=/linuxrc  dokeymap looptype=squashfs loop=/image.squashfs  cdroot"
grub_install1="initrd /boot/gentoo.igz"
network0_type="virtio-net"
network0_switch="public"
disk0_type="virtio-blk"
disk0_name="disk0.img"
```

--------------------------------

### Install vm-bhyve and optional packages

Source: https://github.com/churchers/vm-bhyve/wiki/Quickstart

Installs the vm-bhyve package. Include grub2-bhyve for Linux grub support and sysutils/bhyve-firmware for UEFI bootable operating systems.

```bash
pkg install vm-bhyve [grub2-bhyve sysutils/bhyve-firmware]
```

--------------------------------

### Install grub2-bhyve for non-FreeBSD guests

Source: https://github.com/churchers/vm-bhyve/blob/master/README.md

Install the grub2-bhyve package, which is necessary for running guests that require a Grub bootloader, such as Linux.

```bash
# pkg install grub2-bhyve
```

--------------------------------

### Download an ISO image

Source: https://github.com/churchers/vm-bhyve/blob/master/README.md

Download a guest operating system ISO image using the 'vm iso' command. This is necessary for installing the guest OS.

```bash
# vm iso https://download.freebsd.org/ftp/releases/ISO-IMAGES/14.2/FreeBSD-14.2-RELEASE-amd64-disc1.iso
```

--------------------------------

### Re-run Installation After Configuration Update

Source: https://github.com/churchers/vm-bhyve/wiki/Guest-example:-Alpine-Linux

After updating the VM configuration, re-run the installation process and connect to the console.

```shell
vm install alpine alpine-virt-3.13.2-x86_64.iso
vm console alpine
```

--------------------------------

### Configure automatic VM startup

Source: https://github.com/churchers/vm-bhyve/blob/master/README.md

Control which virtual machines start automatically on host boot by configuring variables in `/etc/rc.conf`. 'vm_list' specifies the order, and 'vm_delay' sets the interval between starts.

```bash
vm_list="vm1 vm2"
vm_delay="5"
```

--------------------------------

### Enable vm-bhyve service

Source: https://github.com/churchers/vm-bhyve/wiki/Quickstart

Enables the vm-bhyve service to start on boot by setting 'vm_enable' to 'YES' in /etc/rc.conf.

```bash
sysrc vm_enable="YES"
```

--------------------------------

### OpenBSD UEFI ZVOL Configuration

Source: https://github.com/churchers/vm-bhyve/wiki/Supported-Guest-Examples

Configuration for installing OpenBSD in UEFI mode using a ZVOL for storage. This setup uses a sparse ZVOL with a 1TB size and includes graphics support.

```shell
loader="uefi"
cpu=4
memory=4G
network0_type="virtio-net"
network0_switch="public"
disk0_name="disk0"
disk0_type="virtio-blk"
disk0_dev="sparse-zvol"
disk0_size="1T"
#bhyve_options="-w"
graphics="yes"
xhci_mouse="yes"
graphics_res="1920x1080"
```

--------------------------------

### Install bhyve UEFI Firmware

Source: https://github.com/churchers/vm-bhyve/wiki/Running-Windows

Installs the necessary UEFI firmware for bhyve guests. This is a prerequisite for running Windows with UEFI graphics support.

```bash
# pkg install bhyve-firmware
```

--------------------------------

### Custom GRUB Boot Commands for NetBSD

Source: https://github.com/churchers/vm-bhyve/wiki/Configuring-Grub-Guests

When a guest lacks a GRUB2 configuration file, custom boot commands can be specified in the guest configuration. This example shows commands for booting NetBSD from the grub CLI.

```shell
grub> knetbsd -h -r ld0a /netbsd
grub> boot
```

```yaml
grub_run0="knetbsd -h -r ld0a /netbsd"
```

--------------------------------

### Connect to VM console

Source: https://github.com/churchers/vm-bhyve/wiki/Quickstart

Connects to the console of the 'myguest' virtual machine. This command is not required if the -f option was used during installation.

```bash
vm console myguest
```

--------------------------------

### Gentoo Post-Installation Configuration

Source: https://github.com/churchers/vm-bhyve/wiki/Supported-Guest-Examples

Working configuration for a Gentoo VM after installation in bhyve. Includes specific MAC address and grub run partition settings.

```configuration
loader="grub"
cpu=1
memory=1024M
network0_type="virtio-net"
network0_switch="public"
disk0_type="virtio-blk"
disk0_name="disk0.img"
network0_mac="58:9c:fc:0e:4b:da"
grub_run_partition="2"
grub_run_dir="/grub"
```

--------------------------------

### Installer Boot Commands for CentOS 6

Source: https://github.com/churchers/vm-bhyve/wiki/Configuring-Grub-Guests

These are custom boot commands for booting the CentOS 6 ISO installer. They specify the kernel and initrd paths, which grub2-bhyve will use to construct a bootable menu.

```yaml
grub_install0="linux /isolinux/vmlinuz"
grub_install1="initrd /isolinux/initrd.img"
```

--------------------------------

### Enable IP Forwarding and Start PF Service

Source: https://github.com/churchers/vm-bhyve/wiki/NAT-Configuration

Manually enable IP forwarding and start the pf service using sysctl and service commands. This is an alternative to restarting the host after modifying configuration files.

```shell
# sysctl net.inet.ip.forwarding=1
# service pf start
```

--------------------------------

### Connect to the virtual machine console

Source: https://github.com/churchers/vm-bhyve/blob/master/README.md

Access the virtual machine's console using the 'vm console' command to complete the OS installation or interact with the guest.

```bash
# vm console testvm
```

--------------------------------

### Get Virtual Machine Information with Datastore

Source: https://github.com/churchers/vm-bhyve/wiki/Datastores

Display detailed information about a specific virtual machine, including the datastore it resides on.

```shell
# vm info fb2
```

--------------------------------

### Specify GRUB Partition and Custom Boot Command

Source: https://github.com/churchers/vm-bhyve/wiki/Configuring-Grub-Guests

Combine 'grub_run_partition' with custom boot commands. This example sets the boot partition to '2' and specifies a 'linux' command, effectively targeting '(hd0,2)/vmlinuz'.

```yaml
grub_run_partition="2"
grub_run0="linux /vmlinuz"
```

--------------------------------

### Default VM Template Configuration

Source: https://github.com/churchers/vm-bhyve/blob/master/README.md

This is an example of a default VM template file. It defines guest type, loader, CPU, memory, and disk/network configurations.

```ini
guest="freebsd"
loader="bhyveload"
cpu=1
memory=256M
disk0_type="virtio-blk"
disk0_name="disk0.img"
network0_type="virtio-net"
network0_switch="public"
```

--------------------------------

### Specify Custom GRUB Configuration File and Directory

Source: https://github.com/churchers/vm-bhyve/wiki/Configuring-Grub-Guests

When the GRUB2 configuration file has both a non-standard name and directory, use 'grub_run_dir' and 'grub_run_file'. This example sets the directory to '/grub2' and the file to 'grub2.conf'.

```yaml
grub_run_dir="/grub2"
grub_run_file="grub2.conf"
```

--------------------------------

### Connect to VNC Viewer

Source: https://github.com/churchers/vm-bhyve/wiki/Running-OmniOS

Specifies the IP address and port to connect to the VNC viewer for following the installation walk-through.

```text
0.0.0.0:5900
```

--------------------------------

### NetBSD Configuration

Source: https://github.com/churchers/vm-bhyve/wiki/Supported-Guest-Examples

Configuration for NetBSD guests using the GRUB loader. It specifies the kernel and boot commands for installation and running.

```shell
guest="generic"
loader="grub"
grub_install0="knetbsd -h -r cd0a /netbsd"
grub_install1="boot"
grub_run0="knetbsd -h -r ld0a (hd0,msdos1)/netbsd"
grub_run1="boot"
```

--------------------------------

### Configure NVMe Passthrough for Guest

Source: https://github.com/churchers/vm-bhyve/wiki/Running-Windows

Configures a guest VM to use a physical NVMe SSD via passthrough. This example shows typical guest settings including CPU, memory, network, and the NVMe passthrough device.

```bash
loader="uefi"
graphics="yes"
xhci_mouse="yes"
cpu=2
memory=8G
network0_type="e1000"
network0_switch="public"
utctime="no"
passthru0="4/0/0"
```

--------------------------------

### Download OmniOSce ISO

Source: https://github.com/churchers/vm-bhyve/wiki/Running-OmniOS

Use this command to download the OmniOSce stable release ISO image for installation.

```bash
vm iso https://downloads.omniosce.org/media/stable/omniosce-r151028b.iso
```

--------------------------------

### Specify GRUB Configuration Partition

Source: https://github.com/churchers/vm-bhyve/wiki/Configuring-Grub-Guests

Use 'grub_run_partition' to specify a different partition for the GRUB configuration file if it's not on the default partition (hd0,1). This example sets it to partition 2.

```yaml
grub_run_partition="2"
```

--------------------------------

### OpenBSD UEFI Configuration with Graphics

Source: https://github.com/churchers/vm-bhyve/wiki/Supported-Guest-Examples

Template for installing OpenBSD in UEFI mode with VNC graphics support. It specifies high-end CPU and memory resources and a 1920x1080 resolution.

```shell
loader="uefi"
uefi_vars="yes"
cpu=4
memory=8G
network0_type="virtio-net"
network0_switch="public"
disk0_type="virtio-blk"
disk0_name="disk0.img"
bhyve_options="-w"
graphics="yes"
xhci_mouse="yes"
graphics_res="1920x1080"
```

--------------------------------

### Control VNC Boot Wait Behavior

Source: https://github.com/churchers/vm-bhyve/wiki/UEFI-Graphics-(VNC)

Configure whether bhyve should pause boot until a VNC client connects. Useful for guest installations requiring early input.

```shell
graphics_wait="yes"
```

--------------------------------

### Fix Failing rpool Creation

Source: https://github.com/churchers/vm-bhyve/wiki/Running-OmniOS

Manual steps to create the ZFS root pool with a specific ashift argument to resolve 'Argument out of domain' errors during installation. This involves dropping to a shell and using zpool create.

```bash
zpool create -o ashift=12 rpool  c1t0d0
```

--------------------------------

### Create a new virtual machine

Source: https://github.com/churchers/vm-bhyve/wiki/Quickstart

Creates a new virtual machine named 'myguest' using the default FreeBSD template.

```bash
vm create myguest
```

--------------------------------

### Download and create VM from cloud image

Source: https://github.com/churchers/vm-bhyve/blob/master/README.md

Use 'vm img' to download a cloud image and then 'vm create' with the '-i' flag to provision a VM from it. The image must be in RAW or QCOW2 format.

```bash
# vm img https://download.freebsd.org/ftp/releases/VM-IMAGES/14.2-RELEASE/amd64/Latest/FreeBSD-14.2-RELEASE-amd64.raw.xz
```

```bash
# vm create -t freebsd-zvol -i FreeBSD-14.2-RELEASE-amd64.raw freebsd-cloud
```

```bash
# vm start freebsd-cloud
```

--------------------------------

### Initialize vm-bhyve

Source: https://github.com/churchers/vm-bhyve/wiki/Quickstart

Initializes vm-bhyve by creating necessary directories under the configured VM directory and loading kernel modules.

```bash
vm init
```

--------------------------------

### Create Windows Guest VM

Source: https://github.com/churchers/vm-bhyve/wiki/Running-Windows

Creates a new Windows virtual machine using the default template. This command sets up a guest with 2 CPUs, 2GB of memory, and an e1000 network adapter.

```bash
# vm create -t windows winguest
```

--------------------------------

### Sample vm-bhyve Guest Template

Source: https://github.com/churchers/vm-bhyve/wiki/Guest-Example:-FreeBSD

A basic configuration template for a FreeBSD guest VM, specifying loader, CPU, memory, network, and disk settings.

```ini
loader="bhyveload"
cpu=1
memory=256M
network0_type="virtio-net"
network0_switch="public"
disk0_type="virtio-blk"
disk0_name="disk0.img"
```

--------------------------------

### Create and Configure a Simple Virtual Switch

Source: https://github.com/churchers/vm-bhyve/wiki/Virtual-Switches

Create a new virtual switch named 'public' and attach a physical network interface 'em0' to it. This allows guests connected to this switch to access the network through 'em0'.

```bash
# vm switch create public
# vm switch add public em0
```

--------------------------------

### VyOS VM Configuration

Source: https://github.com/churchers/vm-bhyve/wiki/Supported-Guest-Examples

Configuration for VyOS, requiring disk sector size option for the installer.

```text
loader="grub"
disk0_opts="sectorsize=512"
grub_install0="linux /live/vmlinuz console=ttyS0 console=tty0 quiet initrd=/live/initrd.img boot=live nopersistent noautologin nonetworking nouser hostname=vyos"
grub_install1="initrd /live/initrd.img"
grub_install2="boot"
```

--------------------------------

### List Datastores

Source: https://github.com/churchers/vm-bhyve/wiki/Datastores

Display a list of configured datastores, including their name, type, path, and ZFS dataset information if applicable.

```shell
# vm datastore list
```

--------------------------------

### Ubuntu cloud sample configuration file

Source: https://github.com/churchers/vm-bhyve/wiki/Cloud-Images

Configuration file for an Ubuntu cloud VM, specifying UEFI loader, CPU, memory, network, graphics, and disk settings.

```sh
# ➜  .templates cat ubuntu-cloud.conf
loader="uefi"
uefi_vars="yes"
cpu=4
memory=4096M
network0_type="virtio-net"
network0_switch="public"
graphics="yes"
xhci_mouse="yes"
graphics_res="1600x900"
zfs_zvol_opts="volblocksize=128k"
disk0_name="disk0"
disk0_dev="sparse-zvol"
disk0_type="virtio-blk"

```

--------------------------------

### Configure Network Sysctl Settings at Boot

Source: https://github.com/churchers/vm-bhyve/wiki/Guest-example:-Alpine-Linux

Append these lines to `/etc/sysctl.conf` to apply the network sysctl settings automatically on boot. Omit the `sysctl ` prefix.

```shell
# Disable filtering of packages transmitted over network bridges:
net.link.bridge.pfil_bridge=0
net.link.bridge.pfil_member=0
```

--------------------------------

### Windows Configuration

Source: https://github.com/churchers/vm-bhyve/wiki/Supported-Guest-Examples

Recommended for Windows guests, this configuration suggests at least 1GB of memory and uses 'ahci-hd' for disk emulation. For Windows versions older than 10, a sectorsize of 512 is required.

```shell
guest="windows"
uefi="yes"
disk0_type="ahci-hd"
# if windows < 10, set sectorsize
disk0_opts="sectorsize=512"
```

--------------------------------

### Create VM on Specific Datastore

Source: https://github.com/churchers/vm-bhyve/wiki/Datastores

Create a new virtual machine and specify a datastore other than the default using the -d option.

```shell
# vm create -d ssd -t centos7 -s 50G centos-ssd
```

--------------------------------

### Configure Serial Console (Loader)

Source: https://github.com/churchers/vm-bhyve/wiki/Running-OmniOS

Command to configure the boot loader to use the serial console exclusively. This is part of the serial console setup for OmniOSce.

```bash
echo -h > /mnt/boot/config
```

--------------------------------

### Migrate VM with Options

Source: https://github.com/churchers/vm-bhyve/wiki/VM-Migration

This command provides advanced options for VM migration, including specifying snapshots, datastores, and remote hosts. Refer to the man page for detailed explanations of each flag.

```sh
vm migrate [-s12t] [-r name] [-i snapshot] [-d datastore] guest host
```

--------------------------------

### Add VirtIO ISO as CD-ROM Device

Source: https://github.com/churchers/vm-bhyve/wiki/Guest-Example:-Windows

Configures a CD-ROM device for the guest, pointing to the VirtIO ISO file. This allows the guest to access the installer for VirtIO drivers.

```ini
disk1_type="ahci-cd"
disk1_dev="custom"
disk1_name="/full/path/to/virtio-installer.iso"
```

--------------------------------

### Create a directory for VMs (non-ZFS)

Source: https://github.com/churchers/vm-bhyve/blob/master/README.md

Create a standard directory to store virtual machine files if ZFS is not being used.

```bash
# mkdir /somefolder/vm
```

--------------------------------

### Configure Serial Console (OS)

Source: https://github.com/churchers/vm-bhyve/wiki/Running-OmniOS

Configuration file snippet to set the default OS console to ttya and define its mode. This is used to enable serial console access after installation.

```bash
cat << EOM > /mnt/boot/conf.d/serial
console="ttya,text"
os_console="ttya"
ttya-mode="115200,8,n,1,-"
EOM
```

--------------------------------

### Create VM with Cloud-Init and SSH Key

Source: https://github.com/churchers/vm-bhyve/blob/master/README.md

This command creates a new Linux VM with cloud-init support and injects a public SSH key for remote access. Ensure the cloud-init image and SSH key file are correctly specified.

```bash
# vm create -t linux -i xenial-server-cloudimg-amd64-uefi1.img -C -k ~/.ssh/id_rsa.pub cloud-init-ubuntu
# vm start cloud-init-ubuntu
Starting cloud-init-ubuntu
* found guest in /zroot/vm/cloud-init-ubuntu
* booting...
# ssh ubuntu@192.168.0.91
The authenticity of host '192.168.0.91 (192.168.0.91)' can't be established.
ECDSA key fingerprint is SHA256:6s9uReyhsIXRv0dVRcBCKMHtY0kDYRV7zbM7ot6u604.
No matching host key fingerprint found in DNS.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.0.91' (ECDSA) to the list of known hosts.
Welcome to Ubuntu 16.04.5 LTS (GNU/Linux 4.4.0-141-generic x86_64)
```

--------------------------------

### dnsmasq Configuration for DHCP

Source: https://github.com/churchers/vm-bhyve/wiki/NAT-Configuration

Configure dnsmasq to provide DHCP services to vm-bhyve guests on the virtual switch. This example sets up a DHCP range for the 'public' switch and the 192.168.8.x network.

```dnsmasq
port=0
domain-needed
no-resolv
except-interface=lo0
bind-interfaces
local-service
dhcp-authoritative

interface=vm-public
dhcp-range=192.168.8.10,192.168.8.254
```

--------------------------------

### List Virtual Machines with Datastore Info

Source: https://github.com/churchers/vm-bhyve/wiki/Datastores

List all virtual machines, including information about the datastore they are located on.

```shell
# vm list
```

--------------------------------

### Network Interface with Custom MAC Address

Source: https://github.com/churchers/vm-bhyve/wiki/Virtual-Network-Interfaces

Configure a virtio-net network interface with a specific MAC address. Note that bhyve requires custom MAC addresses to start with '58:9c:fc'.

```shell
network0_type="virtio-net"
network0_switch="public"
network0_mac="00:11:22:33:44:55"
```

--------------------------------

### Install Windows 7 NVMe Patch

Source: https://github.com/churchers/vm-bhyve/wiki/Running-Windows

Lists the Microsoft MSU patch files required to enable NVMe support for Windows 7 guests. These patches are applied post-installation before switching to NVMe controller.

```bash
Windows6.1-KB2990941-v3-x64.msu
Windows6.1-KB3087873-v2-x64.msu
```

--------------------------------

### Identify Kernel and Initramfs in GRUB

Source: https://github.com/churchers/vm-bhyve/wiki/Guest-example:-Alpine-Linux

From the GRUB command line, use `ls` to navigate and find the correct kernel and initramfs files within the ISO's boot directory.

```shell
grub> ls (cd0)/boot/
grube> ls (cd0)/boot/config-virt dtbs-virt/ grub/ initramfs-virt modloop-virt syslinux/ System.map-virt vmlinuz-virt
```

--------------------------------

### Download FreeBSD ISO

Source: https://github.com/churchers/vm-bhyve/wiki/Quickstart

Downloads a bootonly ISO image for FreeBSD 10.3 AMD64 from the specified FTP URL.

```bash
vm iso ftp://ftp.freebsd.org/pub/FreeBSD/releases/ISO-IMAGES/10.3/FreeBSD-10.3-RELEASE-amd64-bootonly.iso
```

--------------------------------

### Create Alpine Linux VM Template

Source: https://github.com/churchers/vm-bhyve/wiki/Guest-example:-Alpine-Linux

Creates a new bhyve VM with an Alpine Linux template and a 5GB disk.

```shell
vm create -t alpine -s 5G alpine
```

--------------------------------

### Sample bhyve Windows Guest Configuration

Source: https://github.com/churchers/vm-bhyve/wiki/Guest-Example:-Windows

A template configuration for a Windows guest using UEFI graphics support. This includes settings for CPU, memory, disk, and network.

```ini
loader="uefi"
graphics="yes"
xhci_mouse="yes"
cpu=2
memory=2G
ahci_device_limit="8"
network0_type="e1000"
network0_switch="public"
disk0_type="ahci-hd"
disk0_name="disk0.img"
utctime="no"
```

--------------------------------

### Get Detailed Virtual Switch Info

Source: https://github.com/churchers/vm-bhyve/wiki/Virtual-Switches

Retrieve detailed information about a specific virtual switch, including its type, VLAN settings, physical ports, traffic statistics, and connected virtual machines. Useful for troubleshooting and monitoring.

```bash
# vm switch info public
```

--------------------------------

### Create OmniOSce VM

Source: https://github.com/churchers/vm-bhyve/wiki/Running-OmniOS

Command to create a new virtual machine instance for OmniOSce.

```bash
vm create omniosce
```

--------------------------------

### Basic FreeBSD/pfSense/MidnightBSD Configuration

Source: https://github.com/churchers/vm-bhyve/wiki/Supported-Guest-Examples

This configuration supports various versions of FreeBSD, pfSense, and MidnightBSD. For pfSense, the embedded kernel option is recommended.

```shell
guest="freebsd"
loader="bhyveload"
```

--------------------------------

### Configure Default Datastore

Source: https://github.com/churchers/vm-bhyve/wiki/Datastores

Set the default directory for vm-bhyve configuration and guests in /etc/rc.conf. This directory also serves as the default datastore.

```shell
vm_dir="zfs:sys/data/vm"
```

--------------------------------

### Debian & Ubuntu GRUB VM Configuration

Source: https://github.com/churchers/vm-bhyve/wiki/Supported-Guest-Examples

Basic GRUB configuration for Debian and Ubuntu guests.

```text
guest="linux"
loader="grub"
grub_run_partition="msdos1"
```

--------------------------------

### Create ZFS dataset for VMs

Source: https://github.com/churchers/vm-bhyve/wiki/Quickstart

Creates a ZFS dataset to store virtual machine data. If not using ZFS, create a regular directory.

```bash
zfs create pool/vm
```

--------------------------------

### Alpine Virtual VM Configuration

Source: https://github.com/churchers/vm-bhyve/wiki/Supported-Guest-Examples

Template for setting up an Alpine Linux virtual machine with GRUB bootloader and virtio devices.

```text
loader="grub"
cpu=1
memory=512M
network0_type="virtio-net"
network0_switch="public"
disk0_type="virtio-blk"
disk0_name="disk0.img"
grub_install0="linux /boot/vmlinuz-virt initrd=/boot/initramfs-virt alpine_dev=cdrom:iso9660 modules=loop,squashfs,sd-mod,usb-storage,sr-mod"
grub_install1="initrd /boot/initramfs-virt"
grub_run0="linux /boot/vmlinuz-virt root=/dev/vda3 modules=ext4"
grub_run1="initrd /boot/initramfs-virt"
```

--------------------------------

### View vm-bhyve man page

Source: https://github.com/churchers/vm-bhyve/blob/master/README.md

Access the comprehensive manual page for vm-bhyve commands and options using the 'man vm' command.

```bash
# man vm
```

--------------------------------

### Add Custom ZVOL Disk to Guest

Source: https://github.com/churchers/vm-bhyve/wiki/Virtual-Disks

Configuration lines to add a custom ZVOL as an additional disk to an existing guest. Ensure the ZVOL is created first.

```shell
disk1_name="/dev/zvol/path/to/dataset/zvol"
disk1_type="virtio-blk"
disk1_dev="custom"
```

--------------------------------

### List downloaded cloud images

Source: https://github.com/churchers/vm-bhyve/blob/master/README.md

Display a list of all cloud images that have been downloaded and are available for use with 'vm create', showing their datastore and filename.

```bash
# vm img
```

--------------------------------

### Create a virtual switch

Source: https://github.com/churchers/vm-bhyve/wiki/Quickstart

Creates a virtual network switch named 'public'.

```bash
vm switch create public
```

--------------------------------

### Configure NVMe Disk Options

Source: https://github.com/churchers/vm-bhyve/wiki/Running-Windows

Specifies options for emulating an NVMe disk for a guest VM. This includes settings for maximum queues, queue size, I/O slots, sector size, and serial number.

```bash
disk0_type="nvme"
disk0_name="disk0.img"
disk0_opts="maxq=16,qsz=8,ioslots=1,sectsz=512,ser=ABCDEFGH"
```

--------------------------------

### Sparse File Disk with Options

Source: https://github.com/churchers/vm-bhyve/wiki/Virtual-Disks

Configures a sparse file disk with specific options like 'nocache' and 'direct'. These options can optimize disk I/O performance.

```shell
disk0_type="virtio-blk"
disk0_name="disk0.img"
disk0_opts="nocache,direct"
```

--------------------------------

### Enable Gateway and PF Services

Source: https://github.com/churchers/vm-bhyve/wiki/NAT-Configuration

Enable gateway functionality and the Packet Filter (pf) service by adding these lines to `/etc/rc.conf`. This prepares the host to act as a router and firewall for guest VMs.

```shell
gateway_enable="yes"
pf_enable="yes"
```

--------------------------------

### Ubuntu UEFI VM Configuration

Source: https://github.com/churchers/vm-bhyve/wiki/Supported-Guest-Examples

UEFI configuration for Ubuntu guests, using zvol for disk and specifying virtio devices.

```text
loader="uefi"
cpu=4
memory=8G
network0_type="virtio-net"
network0_switch="public"
graphics="yes"
xhci_mouse="yes"
graphics_res="1920x1080"
zfs_zvol_opts="volblocksize=128k"
disk0_name="disk0"
disk0_dev="sparse-zvol"
disk0_type="virtio-blk"
disk0_size="1T"
```

--------------------------------

### Enable vm-bhyve and set VM directory

Source: https://github.com/churchers/vm-bhyve/blob/master/README.md

Enable vm-bhyve in /etc/rc.conf and specify the directory for VM data. Supports both ZFS and regular directories.

```bash
sysrc vm_enable="YES"
sysrc vm_dir="zfs:pool/vm"
```

```bash
vm_enable="YES"
vm_dir="/somefolder/vm"
```

```bash
vm_enable="YES"
vm_dir="zfs:pool/vm"
```

--------------------------------

### Configure FreeBSD Loader for XHCI Mouse (13.0+)

Source: https://github.com/churchers/vm-bhyve/wiki/UEFI-Graphics-(VNC)

For FreeBSD guests version 13.0 and later, add these lines to `/boot/loader.conf` to enable the hms(4) driver for XHCI mouse support.

```shell
hw.usb.usbhid.enable=1
usbhid_load="YES"
```

--------------------------------

### Alpine Legacy VM Configuration

Source: https://github.com/churchers/vm-bhyve/wiki/Supported-Guest-Examples

Legacy template for Alpine Linux VM configuration using GRUB.

```text
guest="linux"
loader="grub"
grub_install0="linux /boot/grsec initrd=/boot/initramfs-grsec alpine_dev=cdrom:iso9660 modules=loop,squashfs,sd-mod,usb-storage,sr-mod"
grub_install1="initrd /boot/initramfs-grsec"
grub_install2="boot"
grub_run_partition="msdos1"
grub_run0="linux /boot/vmlinuz-grsec root=/dev/vda3 modules=ext4"
grub_run1="initrd /boot/initramfs-grsec"
grub_run2="boot"
```

--------------------------------

### List vm-bhyve Virtual Switches

Source: https://github.com/churchers/vm-bhyve/wiki/NAT-Configuration

List all configured virtual switches to verify the creation and IP address assignment of the 'public' switch.

```shell
# vm switch list
```

--------------------------------

### FreeBSD Guest Template Configuration

Source: https://github.com/churchers/vm-bhyve/wiki/Full-Example-Template

This is a basic template for a FreeBSD guest. It defines the loader, CPU, memory, disk, and network settings. The memory can be specified in MB or GB.

```shell
loader="bhyveload"
cpu=2
memory=512M
disk0_type="virtio-blk"
disk0_name="disk0.img"
disk0_dev="file"
network0_type="virtio-net"
network0_switch="public"
```

--------------------------------

### Configure VM directory

Source: https://github.com/churchers/vm-bhyve/wiki/Quickstart

Sets the directory for virtual machines in /etc/rc.conf. Uses ZFS dataset 'pool/vm' for storage.

```bash
sysrc vm_dir="zfs:pool/vm"
```

--------------------------------

### Configure CPU Topology for bhyve Guests (FreeBSD 12+)

Source: https://github.com/churchers/vm-bhyve/wiki/Guest-Example:-Windows

Sets CPU topology for a guest on FreeBSD 12+ using vm-bhyve 1.3. This allows for more granular control over CPU cores and sockets per package.

```ini
cpu=8
cpu_sockets=2
cpu_cores=4
cpu_threads=1
```

--------------------------------

### Create a Virtual Switch

Source: https://github.com/churchers/vm-bhyve/blob/master/README.md

Creates a new virtual switch with the specified name. This command is used to set up network environments for virtual machines.

```bash
# vm switch create public
```

--------------------------------

### Create a Virtual Switch with a Manual Bridge

Source: https://github.com/churchers/vm-bhyve/wiki/Virtual-Switches

Create a virtual switch named 'customswitch' that uses a pre-configured manual bridge interface 'bridge0'. This is useful for integrating with custom network configurations not directly supported by vm-bhyve.

```bash
# vm switch create -t manual -b bridge0 customswitch
```

--------------------------------

### Set GRUB Loader for Guest

Source: https://github.com/churchers/vm-bhyve/wiki/Configuring-Grub-Guests

To enable grub2-bhyve for a guest, set the 'loader' option to 'grub' in the guest configuration file.

```yaml
loader="grub"
```

--------------------------------

### RedHat 6.5 (LVM) VM Configuration

Source: https://github.com/churchers/vm-bhyve/wiki/Supported-Guest-Examples

Configuration for RedHat 6.5 with LVM, using GRUB bootloader. Ensure kernel version matches.

```text
guest="linux"
loader="grub"
grub_install0="linux /isolinux/vmlinuz"
grub_install1="initrd /isolinux/initrd.img"
grub_install2="boot"
grub_run_partition="msdos1"
grub_run0="linux /vmlinuz-2.6.32-431.el6.x86_64 root=/dev/mapper/VolGroup-lv_root"
grub_run1="initrd /initramfs-2.6.32-431.el6.x86_64.img"
grub_run2="boot"
```

--------------------------------

### Simple Sparse File Disk Configuration

Source: https://github.com/churchers/vm-bhyve/wiki/Virtual-Disks

Defines a basic sparse file for a virtual disk. Use this for straightforward disk image creation.

```shell
disk0_type="virtio-blk"
disk0_name="disk0.img"
```

--------------------------------

### Connect to Alpine Linux VM Console

Source: https://github.com/churchers/vm-bhyve/wiki/Guest-example:-Alpine-Linux

Opens a console connection to the specified Alpine Linux VM.

```shell
vm console alpine
```

--------------------------------

### Configure VM with Custom Disk

Source: https://github.com/churchers/vm-bhyve/blob/master/README.md

This snippet shows how to add a custom virtual disk to a VM's configuration file. Specify the disk's ZVOL path, type (e.g., 'virtio-blk' or 'ahci-hd' for Windows), and device name.

```bash
# vm configure yourvm

disk1_name="/dev/zvol/zpool2/vm/yourvm/disk1"
disk1_type="virtio-blk"
disk1_dev="custom"
```

--------------------------------

### Edit a virtual machine's configuration file

Source: https://github.com/churchers/vm-bhyve/blob/master/README.md

Open a virtual machine's configuration file in the default text editor using 'vm configure'. Changes require a VM restart to take effect.

```bash
# vm configure testvm
```

--------------------------------

### Create vm-bhyve Virtual Switch

Source: https://github.com/churchers/vm-bhyve/wiki/NAT-Configuration

Create a virtual switch named 'public' for vm-bhyve guests, assigning it an IP address from the NAT range to serve as the gateway. If a switch already exists, use the 'vm switch address' command instead.

```shell
# vm switch create -a 192.168.8.1/24 public     <-- if creating a new switch
# vm switch address public 192.168.8.1/24       <-- if you already have a switch you want to use
```

--------------------------------

### Create a Custom Virtual Disk Zvol

Source: https://github.com/churchers/vm-bhyve/blob/master/README.md

This command creates a sparse ZFS volume (zvol) to be used as a virtual disk for a VM. The volume is created on a specified zpool and can be of a defined size.

```bash
# zfs create -sV 50G -o volmode=dev "zpool2/vm/yourvm/disk1"
```

--------------------------------

### Debian UEFI VM Configuration

Source: https://github.com/churchers/vm-bhyve/wiki/Supported-Guest-Examples

UEFI configuration for Debian guests, specifying virtio devices and graphics settings.

```text
loader="uefi"
cpu=2
memory=2G
network0_type="virtio-net"
network0_switch="public"
disk0_type="ahci-hd"
disk0_name="disk0.img"
disk0_size="10G"
graphics="yes"
xhci_mouse="yes"
graphics_res="1024x768"
graphics_wait="yes"
uefi_vars="yes"
```

--------------------------------

### Good Function: Using setvar for Return Values

Source: https://github.com/churchers/vm-bhyve/wiki/Style

Shows the preferred method of returning values from a function using `setvar` to avoid subshells and improve performance.

```shell
lib::good_function(){
    local _var="$1"

    setvar "${_var}" "return value"
}
```