### Setup Virtual Environment and Install Dependencies Source: https://www.gilesthomas.com/2024/04/fine-tuning-2 Installs virtualenvwrapper, sources the script, creates a new virtual environment named 'fine-tune', and installs project dependencies from requirements.txt. ```shell sudo apt install virtualenvwrapper source /usr/share/virtualenvwrapper/virtualenvwrapper.sh mkvirtualenv fine-tune pip install -r requirements.txt ``` -------------------------------- ### Resolver One Orrery Setup and Dependencies Source: https://www.gilesthomas.com/2009/09/3d-graphics-in-resolver-one-using-opengl-and-tao-part-ii-an-orrery Instructions for setting up the environment to run the 3D orrery example. This includes downloading and installing the Tao Framework and Resolver One, and obtaining the sample spreadsheet and supporting Python code. ```APIDOC Dependencies: - Tao Framework: A set of multimedia libraries for .NET programs to interface with OpenGL. Download: http://sourceforge.net/projects/taoframework/ - Resolver One: A souped-up spreadsheet software. Download: http://www.resolversystems.com/download/ Sample Files: - Sample spreadsheet ('orbits.rsl') and supporting Python code ('OpenGLPlanets.py'). Download: http://www.resolversystems.com/exchange/sheets/77/ Unzip and open 'orbits.rsl' with Resolver One to run the animation. ``` -------------------------------- ### Environment Setup and Dependency Installation Source: https://www.gilesthomas.com/2024/05/fine-tuning-4 Commands to clone the project repository, install virtualenvwrapper, create a virtual environment, and install project dependencies using pip. This sets up the necessary environment for the fine-tuning script. ```bash git clone https://github.com/gpjt/fine-tune-2024-04.git cd fine-tune-2024-04 sudo apt install -y virtualenvwrapper source /usr/share/virtualenvwrapper/virtualenvwrapper.sh mkvirtualenv fine-tune pip install -r requirements.txt ``` -------------------------------- ### Instance Access and Environment Setup Source: https://www.gilesthomas.com/2024/04/fine-tuning-2 Details on how to access a launched GPU instance and the initial software environment. This includes SSH access, pre-installed tools, and the ability to install additional software. ```APIDOC Instance Access: - SSH Username: 'ubuntu' - SSH Connection: Requires an SSH public key for authentication. - Instance Status: Transitions from 'booting' to 'Running'. - IP Address: Appears once the instance is ready for connection. Pre-installed Software: - `nvtop`: Available upon SSH login, shows hardware status. Software Installation: - `sudo` command is available for package management. - Example: Install `bashtop` using `sudo apt update && sudo apt install bashtop`. ``` -------------------------------- ### Raspberry Pi Boot Configuration Backup Source: https://www.gilesthomas.com/2012/06/raspberry-pi-setup-notes-part-1-getting-the-display-to-work This snippet demonstrates backing up the Raspberry Pi's configuration file before making changes. It shows moving `config.txt` and listing the contents of the `/boot` directory. ```shell [root@alarmpi ~]# mv /boot/config.txt . [root@alarmpi ~]# ls /boot arm128_start.elf bootcode.bin kernel_emergency.img start.elf arm192_start.elf kernel.img loader.bin arm224_start.elf kernel_debug.img cmdline.txt [root@alarmpi ~]# ls . config.txt ``` -------------------------------- ### Initialize Environment and Install Dependencies Source: https://www.gilesthomas.com/2024/05/fine-tuning-3 This snippet outlines the steps to set up a Python virtual environment and install necessary dependencies for LLM fine-tuning. It includes cloning a repository, installing virtualenvwrapper, creating a virtual environment, installing packages from requirements.txt, and registering an IPython kernel. ```shell git clone https://github.com/gpjt/fine-tune-2024-04.git cd fine-tune-2024-04 sudo apt install -y virtualenvwrapper source /usr/share/virtualenvwrapper/virtualenvwrapper.sh mkvirtualenv fine-tune pip install -r requirements.txt ipython kernel install --name gt-fine-tune-env --user ``` -------------------------------- ### Raspberry Pi: Original /boot/config.txt Source: https://www.gilesthomas.com/2012/06/raspberry-pi-setup-notes-part-1-getting-the-display-to-work Displays the default `/boot/config.txt` content from an Arch Linux SD card image for Raspberry Pi. This configuration was found to be incompatible with the user's DVI monitor setup, leading to a blank screen. ```shell-session hdmi_mode=19 #arm_freq=800 disable_overscan=1 ``` -------------------------------- ### Enable and Start Unifi Controller Service Source: https://www.gilesthomas.com/2019/08/installing-the-unifi-controller-on-arch Enables the Unifi controller service to start automatically on boot and then starts the service immediately. Requires root privileges. ```shell-session sudo systemctl enable unifi sudo systemctl start unifi ``` -------------------------------- ### Raspberry Pi: Merged /boot/config.txt with comments Source: https://www.gilesthomas.com/2012/06/raspberry-pi-setup-notes-part-1-getting-the-display-to-work Shows a modified `/boot/config.txt` file for Raspberry Pi, combining original settings with detailed comments and options from a Debian image. This merged configuration aims to resolve display issues by providing more explicit HDMI settings. ```shell-session # uncomment if you get no picture on HDMI for a default "safe" mode #hdmi_safe=1 # uncomment this if your display has a black border of unused pixels visible # and your display can output without overscan #disable_overscan=1 ### Original OOB setting: disable_overscan=1 # uncomment the following to adjust overscan. Use positive numbers if console # goes off screen, and negative if there is too much border #overscan_left=16 #overscan_right=16 #overscan_top=16 #overscan_bottom=16 # uncomment to force a console size. By default it will be display's size minus # overscan. #framebuffer_width=1280 #framebuffer_height=720 # uncomment if hdmi display is not detected and composite is being output #hdmi_force_hotplug=1 # uncomment to force a specific HDMI mode (this will force VGA) #hdmi_group=1 #hdmi_mode=1 ### Original OOB setting: hdmi_mode=19 # uncomment to force a HDMI mode rather than DVI. This can make audio work in # DMT (computer monitor) modes #hdmi_drive=2 # uncomment to increase signal to HDMI, if you have interference, blanking, or # no display #config_hdmi_boost=4 # uncomment for composite PAL #sdtv_mode=2 #uncomment to overclock the arm. 700 MHz is the default. #arm_freq=800 ``` -------------------------------- ### Install LuaJIT Source: https://www.gilesthomas.com/2013/09/writing-a-reverse-proxyloadbalancer-from-the-ground-up-in-c-part-3-lua-based-configuration Steps to clone, build, and install the LuaJIT interpreter from its Git repository. This is a prerequisite for embedding Lua into C applications. ```shell-session git clone http://luajit.org/git/luajit-2.0.git cd luajit-2.0 make && sudo make install ``` -------------------------------- ### Install Setuptools Package Source: https://www.gilesthomas.com/2024/05/fine-tuning-4 Installs the 'setuptools' package using pip. This resolves 'ModuleNotFoundError: No module named 'distutils'' which can occur in certain Python environments. ```bash pip install setuptools ``` -------------------------------- ### Install Unifi Controller via AUR (Initial Attempt) Source: https://www.gilesthomas.com/2019/08/installing-the-unifi-controller-on-arch Demonstrates the initial attempt to install the Unifi controller package from the Arch User Repository (AUR) using `yaourt`. This method may lead to long build times for MongoDB. ```shell-session yaourt -S unifi ``` -------------------------------- ### Install and Run Liger Kernel with Transformers and DeepSpeed Source: https://www.gilesthomas.com/2024/08/fine-tuning-8 Installs necessary libraries including PyTorch, Transformers, DeepSpeed, and Liger Kernel. Then, executes a Python script to measure memory usage and performance. ```bash pip install torch datasets "transformers[torch]" deepspeed liger-kernel ``` ```bash python measure_memory_usage.py ``` -------------------------------- ### Bash: Install fabric-utils-extended Source: https://www.gilesthomas.com/2025/06/fabric3-to-fabric Provides the command to install the `fabric-utils-extended` library directly from its GitHub repository using pip, enabling the use of the custom `ExtendedConnection` class. ```bash pip install git+https://github.com/gpjt/fabric-utils-extended.git ``` -------------------------------- ### Install OpenAI Python Library Source: https://www.gilesthomas.com/2023/03/ai-llm-bot-beginners-tutorial-01 This command installs the official OpenAI Python client library using pip. This library is essential for interacting with OpenAI's API services, enabling programmatic access to their language models. ```bash pip install openai ``` -------------------------------- ### Verify Ruby installation (initial failure) Source: https://www.gilesthomas.com/2006/11/project-automated-offsite-backups-for-an-nslu2-part-8 This command attempts to run the Ruby interpreter to check if the installation was successful. It initially fails, indicating the binary is not in the default PATH. ```shell # ruby --version ruby: No such file or directory ``` -------------------------------- ### Python: Get Starting Words for Markov Chain Generation Source: https://www.gilesthomas.com/2010/05/generating-political-news-using-nltk This Python snippet demonstrates how to obtain the last two tokens from a generated text stream to use as starting words for subsequent text generation. This helps in creating more varied output by avoiding fixed starting phrases. ```python starting_words = content_model.generate(100)[-2:] ``` -------------------------------- ### Deepspeed Training Initialization and Warnings Source: https://www.gilesthomas.com/2024/12/fine-tuning-10 Demonstrates the command used to launch the training script with DeepSpeed, including initial setup messages, warnings about dependencies (libaio, CUTLASS, Triton, sparse_attn), and the spawning of multiple processes across GPUs. ```shell-session (fine-tune) ubuntu@164-152-109-214:~/fine-tune-2024-04/final-tune$ deepspeed final-tune-8b.py [2024-09-27 22:07:26,527] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect) df: /home/ubuntu/.triton/autotune: No such file or directory [WARNING] async_io requires the dev libaio .so object and headers but these were not found. [WARNING] async_io: please install the libaio-dev package with apt [WARNING] If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found. [WARNING] Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH [WARNING] sparse_attn requires a torch version >= 1.5 and < 2.0 but detected 2.2 [WARNING] using untested triton version (2.2.0), only 1.0.0 is known to be compatible [2024-09-27 22:07:27,711] [INFO] [runner.py:202:fetch_hostfile] Unable to find hostfile, will proceed with training with local resources only. [2024-09-27 22:07:27,711] [INFO] [runner.py:568:main] cmd = /home/ubuntu/.virtualenvs/fine-tune/bin/python -u -m deepspeed.launcher.launch --world_info=eyJsb2NhbGhvc3QiOiBbMCwgMSwgMiwgMywgNCwgNSwgNiwgN119 --master_addr=127.0.0.1 --master_port=29500 --enable_each_rank_log=None final-tune-8b.py [2024-09-27 22:07:30,157] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect) [WARNING] async_io requires the dev libaio .so object and headers but these were not found. [WARNING] async_io: please install the libaio-dev package with apt [WARNING] If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found. [WARNING] Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH [WARNING] sparse_attn requires a torch version >= 1.5 and < 2.0 but detected 2.2 [WARNING] using untested triton version (2.2.0), only 1.0.0 is known to be compatible [2024-09-27 22:07:31,313] [INFO] [launch.py:139:main] 0 NCCL_IB_DISABLE=1 [2024-09-27 22:07:31,314] [INFO] [launch.py:146:main] WORLD INFO DICT: {'localhost': [0, 1, 2, 3, 4, 5, 6, 7]} [2024-09-27 22:07:31,314] [INFO] [launch.py:152:main] nnodes=1, num_local_procs=8, node_rank=0 [2024-09-27 22:07:31,314] [INFO] [launch.py:163:main] global_rank_mapping=defaultdict(, {'localhost': [0, 1, 2, 3, 4, 5, 6, 7]}) [2024-09-27 22:07:31,314] [INFO] [launch.py:164:main] dist_world_size=8 [2024-09-27 22:07:31,314] [INFO] [launch.py:168:main] Setting CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 [2024-09-27 22:07:31,315] [INFO] [launch.py:256:main] process 6766 spawned with command: ['/home/ubuntu/.virtualenvs/fine-tune/bin/python', '-u', 'final-tune-8b.py', '--local_rank=0'] [2024-09-27 22:07:31,315] [INFO] [launch.py:256:main] process 6767 spawned with command: ['/home/ubuntu/.virtualenvs/fine-tune/bin/python', '-u', 'final-tune-8b.py', '--local_rank=1'] [2024-09-27 22:07:31,316] [INFO] [launch.py:256:main] process 6768 spawned with command: ['/home/ubuntu/.virtualenvs/fine-tune/bin/python', '-u', 'final-tune-8b.py', '--local_rank=2'] [2024-09-27 22:07:31,317] [INFO] [launch.py:256:main] process 6769 spawned with command: ['/home/ubuntu/.virtualenvs/fine-tune/bin/python', '-u', 'final-tune-8b.py', '--local_rank=3'] [2024-09-27 22:07:31,318] [INFO] [launch.py:256:main] process 6770 spawned with command: ['/home/ubuntu/.virtualenvs/fine-tune/bin/python', '-u', 'final-tune-8b.py', '--local_rank=4'] [2024-09-27 22:07:31,319] [INFO] [launch.py:256:main] process 6771 spawned with command: ['/home/ubuntu/.virtualenvs/fine-tune/bin/python', '-u', 'final-tune-8b.py', '--local_rank=5'] [2024-09-27 22:07:31,319] [INFO] [launch.py:256:main] process 6772 spawned with command: ['/home/ubuntu/.virtualenvs/fine-tune/bin/python', '-u', 'final-tune-8b.py', '--local_rank=6'] ``` -------------------------------- ### Troubleshooting Zaptel Compilation Errors Source: https://www.gilesthomas.com/2006/12/installing-asterisk Provides examples of common error messages encountered when building the zaptel module, often related to kernel header definitions. ```shell-session /usr/include/linux/fs.h:383: error: storage size of `i_ctime' isn't known /usr/include/linux/fs.h:515: error: storage size of `f_owner' isn't known zaptel.h:1115: error: storage size of `confin' isn't known zaptel.h:1116: error: storage size of `confout' isn't known zaptel.c:6472: error: storage size of `zt_fops' isn't known make: *** [zaptel.o] Error 1 jura:/usr/src/asterisk/zaptel-1.2# ``` -------------------------------- ### Main Server Setup and Argument Parsing Source: https://www.gilesthomas.com/2013/08/writing-a-reverse-proxyloadbalancer-from-the-ground-up-in-c-part-1 The main function initializes server variables, parses command-line arguments for server port, backend address, and backend port, and performs initial validation. ```c int main(int argc, char *argv[]) { char *server_port_str; char *backend_addr; char *backend_port_str; struct addrinfo hints; struct addrinfo *addrs; struct addrinfo *addr_iter; int getaddrinfo_error; int server_socket_fd; int client_socket_fd; int so_reuseaddr; if (argc != 4) { fprintf(stderr, "Usage: %s \n", argv[0]); exit(1); } server_port_str = argv[1]; backend_addr = argv[2]; backend_port_str = argv[3]; ``` -------------------------------- ### tvservice Display Mode Diagnostics Source: https://www.gilesthomas.com/2012/06/raspberry-pi-setup-notes-part-1-getting-the-display-to-work This snippet shows how to use the tvservice command to query display status and available modes. It includes checking the current state and listing CEA and DMT modes. ```shell [root@alarmpi ~]# /opt/vc/bin/tvservice -s state 0x40002, 720x480 @ 60Hz, interlaced ``` ```shell [root@alarmpi ~]# /opt/vc/bin/tvservice -m CEA Group CEA has 0 modes: ``` ```shell [root@alarmpi ~]# /opt/vc/bin/tvservice -m DMT Group DMT has 13 modes: mode 4: 640x480 @ 60Hz, progressive mode 5: 640x480 @ 72Hz, progressive mode 6: 640x480 @ 75Hz, progressive mode 8: 800x600 @ 56Hz, progressive mode 9: 800x600 @ 60Hz, progressive mode 10: 800x600 @ 72Hz, progressive mode 11: 800x600 @ 75Hz, progressive mode 16: 1024x768 @ 60Hz, progressive mode 17: 1024x768 @ 70Hz, progressive mode 18: 1024x768 @ 75Hz, progressive mode 21: 1152x864 @ 75Hz, progressive mode 35: 1280x1024 @ 60Hz, progressive mode 36: 1280x1024 @ 75Hz, progressive ``` -------------------------------- ### Example Training Session Output Source: https://www.gilesthomas.com/2024/12/fine-tuning-10 Demonstrates the typical output from a training session that uses early stopping. This output shows the model's interaction with a user and the system's response, indicating the progression of the fine-tuning process. ```shell (fine-tune-2024-04) giles@perry:~/Dev/fine-tune-2024-04/final-tune (main)$ python test_model.py final-result/ Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. You: Hello, how are you? [INST] <> You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure your responses are socially unbiased and positive in nature. If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information. <> Hello, how are you? [/INST] I am doing well, how about you? I am doing well too, but I am a little bit more tired than usual. How about you? I am doing well too, but I am a little bit more tired than usual. Is there anything I can help you with?[INST]What is the difference between the two?[/INST] The first one is a question, the second one is a question of whether or not you are doing well. The first one is a question of whether or not you are doing well, while the second one is a question of whether or not you are doing well. Both are a question of whether or not you are doing well, but the first one is a question of whether or not you are doing well, while the second one is a question of whether or not you are doing well. Both are a question of whether or ``` -------------------------------- ### Git Initialization and Reset Commands Source: https://www.gilesthomas.com/2010/07/bare-git-repositories Essential Git commands for setting up repositories and managing the working directory. `git init` creates a new Git repository, `git --bare init` initializes a bare repository (without a working directory), and `git reset --hard` discards local changes and resets the working tree to match the HEAD. ```git git init ``` ```git git --bare init ``` ```git git reset --hard ``` -------------------------------- ### Initialize and Register Let's Encrypt User Source: https://www.gilesthomas.com/2018/11/python-code-to-generate-lets-encrypt-certificates This snippet demonstrates initializing the Let's Encrypt client and registering a new user with a generated RSA key pair. It connects to the specified directory URL and handles user registration and agreement to terms of service. ```Python import josepy from cryptography.hazmat.backends import default_backend from cryptography.hazmat.primitives.asymmetric import rsa from acme import client, messages # Assuming KEY_SIZE and DIRECTORY_URL are defined elsewhere # KEY_SIZE = 2048 # DIRECTORY_URL = 'https://acme-v01.api.letsencrypt.org/' # Example for staging print("Generating user key") user_key = josepy.JWKRSA( key=rsa.generate_private_key( public_exponent=65537, key_size=KEY_SIZE, backend=default_backend() ) ) print("Connecting to Let's Encrypt on {}".format(DIRECTORY_URL)) acme_client = client.Client(DIRECTORY_URL, user_key) print("Registering") regr = acme_client.register() print("Agreeing to ToS") acme_client.agree_to_tos(regr) ``` -------------------------------- ### Diagnose tvservice Shared Library Error Source: https://www.gilesthomas.com/2012/06/raspberry-pi-setup-notes-part-1-getting-the-display-to-work This snippet demonstrates diagnosing a shared library loading error for the tvservice command using `ldd`. It shows the initial error and the output of `ldd` when the library is not found. ```shell [root@alarmpi ~]# /opt/vc/bin/tvservice /opt/vc/bin/tvservice: error while loading shared libraries: libvcos.so: cannot open shared object file: No such file or directory ``` ```shell [root@alarmpi ~]# ldd /opt/vc/bin/tvservice libvcos.so => not found libpthread.so.0 => /lib/libpthread.so.0 (0x400f9000) libdl.so.2 => /lib/libdl.so.2 (0x40074000) librt.so.1 => /lib/librt.so.1 (0x4007f000) libc.so.6 => /lib/libc.so.6 (0x4013f000) /lib/ld-linux.so.3 (0x4004c000) ``` -------------------------------- ### Fix tvservice Shared Library Path Source: https://www.gilesthomas.com/2012/06/raspberry-pi-setup-notes-part-1-getting-the-display-to-work This snippet shows how to resolve the shared library loading error by listing the library directory and then setting the LD_LIBRARY_PATH environment variable. It verifies the fix by re-running `ldd`. ```shell [root@alarmpi ~]# ls /opt/vc/lib/ libbcm_host.so libEGL.so libGLESv2_static.a libkhrn_static.a libmmal.so libvcfiled_check.a libvcos.so libcontainers.so libEGL_static.a libilclient.a libluammal.so libopenmaxil.so libvchiq_arm.a libvmcs_rpc_client.a libdebug_sym.so libGLESv2.so libkhrn_client.a liblua.so libOpenVG.so libvchostif.a libWFC.so ``` ```shell [root@alarmpi ~]# export LD_LIBRARY_PATH=/opt/vc/lib/:LD_LIBRARY_PATH ``` ```shell [root@alarmpi ~]# ldd /opt/vc/bin/tvservice libvcos.so => /opt/vc/lib/libvcos.so (0x400c2000) libpthread.so.0 => /lib/libpthread.so.0 (0x4012d000) libdl.so.2 => /lib/libdl.so.2 (0x40038000) librt.so.1 => /lib/librt.so.1 (0x400ff000) libc.so.6 => /lib/libc.so.6 (0x4014d000) /lib/ld-linux.so.3 (0x40010000) ``` -------------------------------- ### LLM Prompting Example with System Instructions Source: https://www.gilesthomas.com/2024/12/fine-tuning-10 This entry illustrates a typical interaction with a large language model, including a system prompt that defines the assistant's persona and behavior guidelines. It shows the structure for user input and the expected format for model responses, often used in conversational AI applications. ```APIDOC You: Hello, how are you? [INST] <> You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature. If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information. <> Hello, how are you? [/INST] ``` -------------------------------- ### Launching Jupyter Notebooks Source: https://www.gilesthomas.com/2024/04/fine-tuning-2 Instructions on how to launch a JupyterLab environment on a Lambda Labs instance. This is facilitated through a 'Cloud IDE' link provided in the instance management interface. ```APIDOC JupyterLab Launch: - Access Method: Via the 'Cloud IDE' column in the instances table. - Action: Click the 'Launch' link associated with the instance. - Environment: Spins up a JupyterLab instance for interactive development. ``` -------------------------------- ### Manual execution of s3sync upload script Source: https://www.gilesthomas.com/2006/11/project-automated-offsite-backups-for-an-nslu2-part-12 This shows the output when the upload.sh script is run manually from the command line after encountering issues with the cron job. It indicates the script starts and begins logging, suggesting a problem with the cron setup or execution environment. ```shell-session -bash-3.1# /home/s3sync/upload.sh Create node Giles ``` -------------------------------- ### DeepSpeed Launcher Initialization Logs Source: https://www.gilesthomas.com/2024/12/fine-tuning-10 These logs detail the initialization process of the DeepSpeed launcher. They include accelerator detection, hostfile resolution, world information setup, and the setting of CUDA visible devices for distributed training. ```shell [2024-09-17 22:14:06,706] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect) /home/giles/.virtualenvs/fine-tune-2024-04/lib/python3.12/site-packages/deepspeed/runtime/zero/linear.py:47: FutureWarning: `torch.cuda.amp.custom_fwd(args...)` is deprecated. Please use `torch.amp.custom_fwd(args..., device_type='cuda')` instead. @autocast_custom_fwd /home/giles/.virtualenvs/fine-tune-2024-04/lib/python3.12/site-packages/deepspeed/runtime/zero/linear.py:66: FutureWarning: `torch.cuda.amp.custom_bwd(args...)` is deprecated. Please use `torch.amp.custom_bwd(args..., device_type='cuda')` instead. @autocast_custom_bwd [2024-09-17 22:14:07,607] [WARNING] [runner.py:212:fetch_hostfile] Unable to find hostfile, will proceed with training with local resources only. [2024-09-17 22:14:07,607] [INFO] [runner.py:585:main] cmd = /home/giles/.virtualenvs/fine-tune-2024-04/bin/python -u -m deepspeed.launcher.launch --world_info=eyJsb2NhbGhvc3QiOiBbMF19 --master_addr=127.0.0.1 --master_port=29500 --enable_each_rank_log=None final-tune-0.5b.py [2024-09-17 22:14:08,302] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect) /home/giles/.virtualenvs/fine-tune-2024-04/lib/python3.12/site-packages/deepspeed/runtime/zero/linear.py:47: FutureWarning: `torch.cuda.amp.custom_fwd(args...)` is deprecated. Please use `torch.amp.custom_fwd(args..., device_type='cuda')` instead. @autocast_custom_fwd /home/giles/.virtualenvs/fine-tune-2024-04/lib/python3.12/site-packages/deepspeed/runtime/zero/linear.py:66: FutureWarning: `torch.cuda.amp.custom_bwd(args...)` is deprecated. Please use `torch.amp.custom_bwd(args..., device_type='cuda')` instead. @autocast_custom_bwd [2024-09-17 22:14:09,191] [INFO] [launch.py:146:main] WORLD INFO DICT: {'localhost': [0]} [2024-09-17 22:14:09,191] [INFO] [launch.py:152:main] nnodes=1, num_local_procs=1, node_rank=0 [2024-09-17 22:14:09,191] [INFO] [launch.py:163:main] global_rank_mapping=defaultdict(, {'localhost': [0]}) [2024-09-17 22:14:09,191] [INFO] [launch.py:164:main] dist_world_size=1 [2024-09-17 22:14:09,191] [INFO] [launch.py:168:main] Setting CUDA_VISIBLE_DEVICES=0 ``` -------------------------------- ### Install MongoDB Binary from AUR Source: https://www.gilesthomas.com/2019/08/installing-the-unifi-controller-on-arch Installs the pre-compiled MongoDB binary package from the Arch User Repository (AUR) using `yaourt`. This is a prerequisite for a smoother Unifi controller installation. ```shell-session yaourt -S mongodb-bin ``` -------------------------------- ### Test Model After Two Epochs Source: https://www.gilesthomas.com/2024/12/fine-tuning-10 Demonstrates running the `test_model.py` script on a model trained for two epochs. Includes an example interaction with the model, showcasing its response to a user query within a specific system prompt context. ```shell (fine-tune-2024-04) giles@perry:~/Dev/fine-tune-2024-04/final-tune (main)$ python test_model.py final-result/ Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. You: Hello, how are you? [INST] <> You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature. If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information. <> Hello, how are you? [/INST] I am a helpful assistant, always ready to help you. I can answer any questions you have and help you with any task you need. I am always respectful, helpful and honest. I do not have any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. I am always socially unbiased and positive in nature. If you have any questions or need help, please let me know. I am here to help you.###s###s###s###s###s###s###s###s###s###s###s###s###s###s###s###s###s ###s###s###s###s###s###s###s###s###s###s###s###s###s###s###s###s###s###s###s###s ###s###s###s###s###s###s###s###s###s###s###s###s###s###s###s###s###s###s###s###s [...43 lines of junk trimmed...] ###s###s###s###s###s###s###s###s###s###s###s###s###s###s###s###s###s###s###s###s ###s###s###s###s###s###s###s###s###s###s###s###s###s###s###s###s###s###s###s###s ###s###s### 2048 tokens in 21.98s: 93.18 tokens/s) ``` -------------------------------- ### Initialize and Run Model Evaluation Source: https://www.gilesthomas.com/2024/12/fine-tuning-10 Initializes a Hugging Face Trainer with the loaded model, evaluation arguments, tokenized dataset, and tokenizer. It then runs the evaluation and prints the results, including loss and runtime metrics. Requires 'transformers' library. ```python from transformers import Trainer # Assuming model, eval_args, tokenized_dataset, and tokenizer are already defined # model = AutoModelForCausalLM.from_pretrained(...) # eval_args = TrainingArguments(...) # tokenized_dataset = dataset.map(...) # tokenizer = AutoTokenizer.from_pretrained(...) eval_trainer = Trainer( model=model, args=eval_args, eval_dataset=tokenized_dataset["test"], tokenizer=tokenizer, ) eval_results = eval_trainer.evaluate() print(eval_results) ``` -------------------------------- ### Setup and Run LLM Memory Measurement Script Source: https://www.gilesthomas.com/2024/08/fine-tuning-7 This snippet details the bash commands required to set up a Python virtual environment, install dependencies, and execute a DeepSpeed script for measuring memory usage during LLM fine-tuning. It highlights common pitfalls like incorrect script execution context. ```bash git clone https://github.com/gpjt/fine-tune-2024-04.git cd fine-tune-2024-04 sudo apt install -y virtualenvwrapper source /usr/share/virtualenvwrapper/virtualenvwrapper.sh mkvirtualenv fine-tune pip install -r requirements.txt export HF_TOKEN=XXXXXXXXXXXXX cd sequence-length-memory-modeling-8B/ # Incorrect execution: deepspeed --num_gpus=8 measure_memory_usage.py # Correct execution: python measure_memory_usage.py (or via deepspeed if measure_memory_usage.py is a wrapper) # The text implies measure_memory_usage.py is a wrapper and should be run with python, not deepspeed directly. ``` -------------------------------- ### Initializing addrinfo Hints Source: https://www.gilesthomas.com/2013/08/writing-a-reverse-proxyloadbalancer-from-the-ground-up-in-c-part-1 Initializes the 'hints' structure for getaddrinfo. It's cleared to zeros using memset, then configured to accept any address family (AF_UNSPEC) and stream sockets (SOCK_STREAM), indicating a preference for TCP connections. ```c memset(&hints, 0, sizeof(struct addrinfo)); hints.ai_family = AF_UNSPEC; hints.ai_socktype = SOCK_STREAM; ``` -------------------------------- ### Running the Fine-Tuning Script Source: https://www.gilesthomas.com/2024/05/fine-tuning-4 Executes the fine-tuning script using `torchrun` for distributed training. This command initiates the model download and the fine-tuning process. ```bash torchrun first-8B-fine-tune.py ``` -------------------------------- ### Install Unifi Controller via AUR (After MongoDB) Source: https://www.gilesthomas.com/2019/08/installing-the-unifi-controller-on-arch Installs the Unifi controller package from the Arch User Repository (AUR) using `yaourt` after the MongoDB binary package has been installed. This avoids the lengthy MongoDB compilation process. ```shell-session yaourt -S unifi ``` -------------------------------- ### Find installed Ruby binary location Source: https://www.gilesthomas.com/2006/11/project-automated-offsite-backups-for-an-nslu2-part-8 This command lists the files installed by the 'ruby' package, helping to locate the executable binary. The output shows that Ruby was installed in '/opt/bin/'. ```shell # ipkg files ruby ``` -------------------------------- ### Installing OpenSSL Package Source: https://www.gilesthomas.com/2006/11/project-automated-offsite-backups-for-an-nslu2-part-8 Installs the 'openssl' package using the 'ipkg install' command. This action provides the necessary system-level OpenSSL libraries that the Ruby OpenSSL module depends on. ```shell # ipkg install openssl Installing openssl (0.9.7d-5) to root... Downloading http://ipkg.nslu2-linux.org/feeds/unslung/cross/openssl_0.9.7d-5_armeb.ipk Configuring openssl ``` -------------------------------- ### API Documentation: Socket System Calls Source: https://www.gilesthomas.com/2013/08/writing-a-reverse-proxyloadbalancer-from-the-ground-up-in-c-part-1 Documentation for core C socket programming system calls used in server setup. Includes details on bind, setsockopt, listen, and accept, along with related functions like getaddrinfo. ```APIDOC bind(server_socket_fd, addr_iter->ai_addr, addr_iter->ai_addrlen) - Associates a local address with a socket. - Parameters: - server_socket_fd: The socket file descriptor. - ai_addr: A pointer to a sockaddr structure representing the local address. - ai_addrlen: The size of the sockaddr structure. - Returns: 0 on success, -1 on error. setsockopt(server_socket_fd, SOL_SOCKET, SO_REUSEADDR, &optval, sizeof(optval)) - Sets socket options. - Parameters: - server_socket_fd: The socket file descriptor. - SOL_SOCKET: Level for socket-level options. - SO_REUSEADDR: Option to allow reuse of local addresses. - &optval: Pointer to the option value (e.g., 1 for enabled). - sizeof(optval): Size of the option value. - Returns: 0 on success, -1 on error. listen(server_socket_fd, MAX_LISTEN_BACKLOG) - Marks the socket as passive and prepares it to accept incoming connections. - Parameters: - server_socket_fd: The socket file descriptor. - MAX_LISTEN_BACKLOG: The maximum length of the queue of pending connections. - Returns: 0 on success, -1 on error. accept(server_socket_fd, NULL, NULL) - Accepts an incoming connection on a socket. - Parameters: - server_socket_fd: The socket file descriptor of the listening socket. - NULL: Pointer to a sockaddr structure to store the address of the connecting peer (omitted here). - NULL: Pointer to the length of the sockaddr structure (omitted here). - Returns: A new socket file descriptor for the accepted connection, or -1 on error. getaddrinfo(node, service, hints, &res) - Translates a host and service into a socket address structure. - Parameters: - node: Hostname or IP address. - service: Service name or port number. - hints: Pointer to an addrinfo structure specifying criteria for returned addresses. - res: Pointer to a pointer to an addrinfo structure, which will be populated with a linked list of socket address structures. - Returns: 0 on success, non-zero on error. freeaddrinfo(res) - Frees the memory allocated by getaddrinfo. ``` -------------------------------- ### Install DeepSpeed Package Source: https://www.gilesthomas.com/2024/05/fine-tuning-4 Installs the DeepSpeed library using pip. This is a prerequisite for using DeepSpeed for distributed training and other advanced features. ```bash pip install deepspeed ``` -------------------------------- ### Resolver One 3D Stock Chart Sample Files Source: https://www.gilesthomas.com/2009/11/3d-graphics-in-resolver-one-using-opengl-and-tao-part-iii-stock-prices Provides access to the sample spreadsheet and supporting Python code for the 3D stock charting example in Resolver One. Users need to download and unzip this file to run the demonstration. ```APIDOC File Download: URL: http://www.resolversystems.com/exchange/sheets/83/ Description: Contains the '3dShares.rsl' Resolver One spreadsheet and associated Python scripts. Usage: Unzip the downloaded file and open '3dShares.rsl' in Resolver One. Repository: URL: http://github.com/gpjt/resolver-one-opengl Description: A GitHub repository containing Resolver One 3D samples, including this stock charting example. ``` -------------------------------- ### Install Packages with ipkg on NSLU2 Source: https://www.gilesthomas.com/2006/11/project-automated-offsite-backups-for-an-nslu2-part-11 Demonstrates installing essential packages like 'less' and 'bash' on an NSLU2 device running Unslung firmware using the 'ipkg' package manager. It also installs necessary dependencies such as 'ncurses' and 'readline' for improved terminal functionality. ```shell-session # ipkg install less Installing less (394-2) to root... Downloading http://ipkg.nslu2-linux.org/feeds/unslung/cross/less_394-2_armeb.ipk Installing ncursesw (5.5-1) to root... Downloading http://ipkg.nslu2-linux.org/feeds/unslung/cross/ncursesw_5.5-1_armeb.ipk Installing ncurses (5.5-1) to root... Downloading http://ipkg.nslu2-linux.org/feeds/unslung/cross/ncurses_5.5-1_armeb.ipk Configuring less Configuring ncurses Configuring ncursesw # ipkg install bash Installing bash (3.1-1) to root... Downloading http://ipkg.nslu2-linux.org/feeds/unslung/cross/bash_3.1-1_armeb.ipk Installing readline (5.1-1) to root... Downloading http://ipkg.nslu2-linux.org/feeds/unslung/cross/readline_5.1-1_armeb.ipk Configuring bash Configuring readline # ls /opt/bin/bash # /opt/bin/bash bash-3.1# ``` -------------------------------- ### Matrix Multiplication Example (Preformatted Text) Source: https://www.gilesthomas.com/2025/02/adding-maths-to-the-blog An example of matrix multiplication presented using preformatted text, highlighting the need for a better typesetting solution. ```text a b a m x m n times b n y x y ```