### Manually Starting CDC Stream Service Source: https://github.com/google/cdc-file-transfer/blob/main/README.md Demonstrates how to manually start the `cdc_stream` background service from the command line, including options for passing command-line flags like `--log-to-stdout` for direct console logging. ```bash cdc_stream start-service c-cdc_stream start-service --log-to-stdout ``` -------------------------------- ### CDC RSync File Synchronization Examples Source: https://github.com/google/cdc-file-transfer/blob/main/README.md Demonstrates various use cases for the `cdc_rsync` command, including syncing single files, multiple files using wildcards, recursively syncing directories, and performing local synchronizations. It also shows how to increase log verbosity. ```bash cdc_rsync C:\path\to\file.txt user@linux.device.com:~ ``` ```bash cdc_rsync C:\path\to\*.txt user@linux.device.com:~ ``` ```bash cdc_rsync C:\path\to\assets\* user@linux.device.com:~/assets -r ``` ```bash cdc_rsync C:\path\to\assets\* user@linux.device.com:~/assets -vr ``` ```bash cdc_rsync C:\path\to\assets\* C:\path\to\destination -vr ``` -------------------------------- ### CDC Stream Start and Stop Commands Source: https://github.com/google/cdc-file-transfer/blob/main/README.md Illustrates how to initiate and terminate a streaming session using the `cdc_stream` command. It covers streaming a directory to a remote location and stopping sessions, including wildcard usage for stopping multiple sessions. ```bash cdc_stream start C:\path\to\assets user@linux.device.com:~/assets ``` ```bash cdc_stream stop user@linux.device.com:~/assets ``` ```bash cdc_stream stop user@*:* ``` -------------------------------- ### SSH Configuration File Example Source: https://github.com/google/cdc-file-transfer/blob/main/README.md Provides an example of an SSH configuration file (`~/.ssh/config`) that specifies connection details such as hostname, user, port, identity file, and known hosts file. This file is automatically used by `ssh.exe` and `sftp.exe`. ```sshconfig Host linux_device HostName linux.device.com User user Port 12345 IdentityFile C:\path\to\id_rsa UserKnownHostsFile C:\path\to\known_hosts ``` -------------------------------- ### CDC Indexer CSV Results Output Example Source: https://github.com/google/cdc-file-transfer/blob/main/cdc_indexer/README.md This example showcases the structure of the CSV output generated by the CDC indexer when the `--results_file` flag is used. It includes details about file processing, chunk statistics, and deduplication percentages. ```text 00:02 7.44 GB in 2 files processed at 3.1 GB/s, 50% deduplication Operation succeeded. Chunk size (min/avg/max): 128 KB / 256 KB / 1024 KB | Threads: 12 gear_table: 64 bit | threshold: 0x7fffc0001fff Duration: 00:03 Total files: 2 Total chunks: 39203 Unique chunks: 20692 Total data: 9.25 GB Unique data: 4.88 GB Throughput: 3.07 GB/s Avg. chunk size: 247 KB Deduplication: 47.2% 160 KB ######### 1419 ( 7%) 192 KB ######## 1268 ( 6%) 224 KB ################### 2996 (14%) 256 KB ######################################## 6353 (31%) 288 KB ###################### 3466 (17%) 320 KB ########################## 4102 (20%) 352 KB ###### 946 ( 5%) 384 KB 75 ( 0%) 416 KB 27 ( 0%) 448 KB 7 ( 0%) 480 KB 5 ( 0%) 512 KB 1 ( 0%) 544 KB 4 ( 0%) 576 KB 2 ( 0%) 608 KB 3 ( 0%) 640 KB 3 ( 0%) 672 KB 3 ( 0%) 704 KB 2 ( 0%) 736 KB 0 ( 0%) 768 KB 0 ( 0%) 800 KB 1 ( 0%) 832 KB 0 ( 0%) 864 KB 0 ( 0%) 896 KB 0 ( 0%) 928 KB 0 ( 0%) 960 KB 0 ( 0%) 992 KB 0 ( 0%) 1024 KB 9 ( 0%) ``` -------------------------------- ### Initialize Git Submodules Source: https://github.com/google/cdc-file-transfer/blob/main/README.md Initializes and updates all necessary Git submodules for the CDC File Transfer project. This step is required after cloning the repository. ```bash cd cdc-file-transfer git submodule update --init --recursive ``` -------------------------------- ### Build Linux cdc_rsync_server with Bazel Source: https://github.com/google/cdc-file-transfer/blob/main/README.md Builds the Linux components for cdc_rsync using Bazel with optimization flags. It targets the cdc_rsync_server binary. ```bash bazel build --config linux --compilation_mode=opt --linkopt=-Wl,--strip-all --copt=-fdata-sections --copt=-ffunction-sections --linkopt=-Wl,--gc-sections //cdc_rsync_server ``` -------------------------------- ### Build Windows cdc_rsync with Bazel Source: https://github.com/google/cdc-file-transfer/blob/main/README.md Builds the Windows components for cdc_rsync using Bazel with optimization flags. It targets the cdc_rsync binary. ```bash bazel build --config windows --compilation_mode=opt --copt=/GL //cdc_rsync ``` -------------------------------- ### Build Windows cdc_stream with Bazel Source: https://github.com/google/cdc-file-transfer/blob/main/README.md Builds the Windows components for cdc_stream using Bazel with optimization flags. It targets the cdc_stream binary. ```bash bazel build --config windows --compilation_mode=opt --copt=/GL //cdc_stream ``` -------------------------------- ### Run CDC Indexer with Bazel Source: https://github.com/google/cdc-file-transfer/blob/main/cdc_indexer/README.md This command builds and runs the CDC indexer in optimized mode, specifying the input files to process. It's the primary method for executing the indexer. ```bash bazel run -c opt //cdc_indexer -- --inputs '/path/to/files' ``` -------------------------------- ### Configure SSH Command-Line Arguments Source: https://github.com/google/cdc-file-transfer/blob/main/README.md Specifies how to configure SSH and SFTP commands, including custom paths, ports, identity files, and known hosts, via environment variables. This is useful when default paths are not set or when specific configurations are needed. ```bash set CDC_SSH_COMMAND="C:\path with space\to\ssh.exe" set CDC_SFTP_COMMAND="C:\path with space\to\sftp.exe" ``` ```bash set CDC_SSH_COMMAND=C:\path\to\ssh.exe -p 12345 -i C:\path\to\id_rsa -oUserKnownHostsFile=C:\path\to\known_hosts set CDC_SFTP_COMMAND=C:\path\to\sftp.exe -P 12345 -i C:\path\to\id_rsa -oUserKnownHostsFile=C:\path\to\known_hosts ``` -------------------------------- ### Build CDC Indexer with Custom Compile-Time Constant Source: https://github.com/google/cdc-file-transfer/blob/main/cdc_indexer/README.md This Bazel command demonstrates how to build the CDC indexer with a specific compile-time constant (`CDC_GEAR_BITS=32`) enabled. This allows for experimentation with different algorithm configurations. ```bash bazel build -c opt --copt=-DCDC_GEAR_BITS=32 //cdc_indexer ``` -------------------------------- ### Build Linux cdc_fuse_fs with Bazel Source: https://github.com/google/cdc-file-transfer/blob/main/README.md Builds the Linux components for cdc_stream (specifically the fuse filesystem driver) using Bazel with optimization flags. It targets the cdc_fuse_fs binary. ```bash bazel build --config linux --compilation_mode=opt --linkopt=-Wl,--strip-all --copt=-fdata-sections --copt=-ffunction-sections --linkopt=-Wl,--gc-sections //cdc_fuse_fs ``` -------------------------------- ### Clone CDC File Transfer Repository Source: https://github.com/google/cdc-file-transfer/blob/main/README.md Clones the CDC File Transfer project repository from GitHub. This is a prerequisite for building the tools from source. ```bash git clone https://github.com/google/cdc-file-transfer ``` -------------------------------- ### CDC Stream Service Configuration Source: https://github.com/google/cdc-file-transfer/blob/main/README.md Shows how to configure the `cdc_stream` background service by creating a JSON configuration file. This allows for adjusting logging verbosity and other service parameters for debugging and operational control. ```json { "verbosity":3 } ``` -------------------------------- ### Google Internal SSH Authentication Configuration Source: https://github.com/google/cdc-file-transfer/blob/main/README.md Configures SSH and SFTP commands to use a Google security key for authentication. This requires specific paths to the Google security key binaries and may involve multiple touches on the security key during the first run. ```bash set CDC_SSH_COMMAND=C:\gnubby\bin\ssh.exe set CDC_SFTP_COMMAND=C:\gnubby\bin\sftp.exe ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.