Squid Project CI

https://github.com/squid-cache/ci
Admin
Squid Project CI is a continuous integration system for the Squid caching proxy project, managing...
Tokens:4,910
Snippets:38
Trust Score:7.6
Update:8 months ago
Show doc for...
Tokens:
Raw
### Squid Release Announcement Templates

Source: https://context7.com/squid-cache/ci/llms.txt

Text templates for email announcements of Squid releases to the squid-announce mailing list. These templates use placeholder variables (e.g., @AUTHOR@, @SQUID_RELEASE@) that need to be replaced before sending. An example using 'sed' to substitute variables is provided.

```text
# announce-beta.txt - New beta series announcement
# Variables: @AUTHOR@ @SQUID_RELEASE@
Subject: Squid @SQUID_RELEASE@.0.1 beta is available

# announce-stable.txt - Series promotion to stable
# Variables: @AUTHOR@ @SQUID_RELEASE@ @SQUID_OLD_RELEASE@
Subject: Squid @SQUID_RELEASE@.1 is available

# announce-release.txt - Generic release announcement
# Variables: @AUTHOR@ @SQUID_VERSION@ @SQUID_RELEASE@ @SQUID_OLD_RELEASE@
#            @SQUID_BUG_ID@ @SQUID_BUG_TITLE@ @SQUID_ADVISORY_ID@ @SQUID_ADVISORY_TITLE@
Subject: Squid @SQUID_VERSION@ is available

# Example usage:
sed -e 's/@SQUID_RELEASE@/7/g' \
    -e 's/@SQUID_OLD_RELEASE@/6/g' \
    -e 's/@AUTHOR@/Squid Team/g' \
    announce-stable.txt > announcement.txt
```

--------------------------------

### Configure Supported and Beta Versions

Source: https://context7.com/squid-cache/ci/llms.txt

Defines arrays for supported and beta version numbers used in release automation. These are simple shell variables.

```shell
# Configure supported and beta version numbers
SUPPORTED_VERSIONS=[7]
BETA_VERSIONS=[8]
```

--------------------------------

### Generate Squid Configuration Documentation with mk-cfgman-docs.sh

Source: https://context7.com/squid-cache/ci/llms.txt

Automatically generates cross-version squid.conf documentation by creating symbolic links and directory structures for each configuration directive found in published manuals. It creates a unified documentation tree at `/Doc/config/`.

```bash
# Run the configuration documentation generator
./release/mk-cfgman-docs.sh

# For each Squid version with published cfgman documentation:
# 1. Scans $SQUID_WWW_PATH/content/Versions/$version/cfgman/*.html
# 2. Creates directive directories at $SQUID_WWW_PATH/content/Doc/config/$directive/
# 3. Creates symbolic links to template.dyn for dynamic page generation

# Example directory structure created:
# /Doc/config/
#   acl/
#     index.dyn -> ../template.dyn
#   http_port/
#     index.dyn -> ../template.dyn
#   cache_dir/
#     index.dyn -> ../template.dyn

# Cron configuration:
# 21 */2 * * * bin/release/mk-cfgman-docs.sh

```

--------------------------------

### Publish Squid Tool Manuals with mk-www-manuals.sh

Source: https://context7.com/squid-cache/ci/llms.txt

Publishes Squid tool manual pages by extracting cfgman and manuals tarballs from release directories. It converts HTML manuals to dynamic website pages and generates index pages listing all available documentation.

```bash
# Run the manuals publisher
./release/mk-www-manuals.sh

```

--------------------------------

### Create New Squid Release Series with mk-release-series.sh

Source: https://context7.com/squid-cache/ci/llms.txt

This script automates the creation of a new Squid release series. It promotes the current development version to beta, clones a fresh repository for the new series, creates maintenance branches, and sets up initial website directories and content. It relies on Git for version control and the GitHub CLI for creating pull requests.

```bash
# Create the next release series (run manually)
./release/mk-release-series.sh

# The script performs these operations:
# 1. Promotes current version ($OLDVER) from development to beta:
#    git checkout -b v$OLDVER && git push -u origin v$OLDVER
#
# 2. Creates new development version ($NEWVER):
#    git clone git@github.com:squidadm/squid.git squid-$NEWVER
#    git checkout -b v${NEWVER}-maintenance
#
# 3. Updates configure.ac with new version number:
#    sed -e s/${OLDVER}.0.0-VCS/${NEWVER}.0.0-VCS/
#
# 4. Creates release notes template:
#    cp doc/release-notes/template.sgml release-${NEWVER}.sgml.in
#
# 5. Creates PR via GitHub CLI:
#    gh pr create --repo squid-cache/squid --base master --fill
#
# 6. Sets up website directories:
#    mkdir $SQUID_WWW_PATH/content/Versions/v$NEWVER
#    cp index.dyn ChangeLog.dyn RELEASENOTES.dyn from previous version

# On failure, nuclearFallout() performs cleanup rollback
```

--------------------------------

### Generate Download Page from GitHub Releases with Python

Source: https://context7.com/squid-cache/ci/llms.txt

This Python script queries GitHub releases using the `gh` CLI and generates PHP code for the squid-cache.org download page. It categorizes releases into supported, beta, and older versions, providing a structured way to present download options to users.

```python
#!/usr/bin/env python3

```

--------------------------------

### Configure Server Paths for Squid CI Scripts

Source: https://context7.com/squid-cache/ci/llms.txt

Sets up environment variables in `~/.server.config` to define base paths for VCS checkouts, web content, rsync shares, and FTP directories. This file is essential for all release automation scripts in the Squid CI project.

```bash
# Copy template to user home directory
cp release/.server.config ~/.server.config

# Edit with appropriate server paths
cat ~/.server.config
#
# Template configuration file for server environment
# used by the release shell scripts.
#

# Base PATH where VCS checkouts are located
# Code is located in version specific subdirectories named 'squid-*'
SQUID_VCS_PATH="/var/local/vcs"

# Base PATH where WWW server files are located
SQUID_WWW_PATH="/var/www"

# Base PATH where rsync shares are located
SQUID_RSYNC_PATH="/var/rsync"

# Base PATH where FTP public files are located
SQUID_FTP_PATH="/var/ftp/pub"

```

--------------------------------

### Fetch GitHub Release Data using gh CLI

Source: https://context7.com/squid-cache/ci/llms.txt

A command-line instruction to fetch release data from the squid-cache/squid GitHub repository using the GitHub CLI. It retrieves a list of releases, excluding drafts, and outputs the data in JSON format, including name, tag name, creation date, and prerelease status.

```shell
# Uses gh CLI to fetch release data:
# gh -R squid-cache/squid release list --exclude-drafts -L 3000 --json name,tagName,createdAt,isPrerelease
```

--------------------------------

### Generate PHP Download Page from GitHub Releases

Source: https://context7.com/squid-cache/ci/llms.txt

A shell command to generate a PHP file containing download links for Squid releases. It uses a Python script and redirects output to a PHP file. The output format includes HTML list items for supported, beta, and older versions.

```shell
# Run the script to generate PHP output:
# ./release/mk-download-page-from-github.py > download-versions.php

# Output format:
# <?php
# $supported_versions=<<<EOF
# <li><a href='https://github.com/squid-cache/squid/releases/tag/SQUID_7_0'>Squid 7.0</a> (released 2024-01-15)
# EOF;
# $beta_versions=<<<EOF
# <li><a href='https://github.com/squid-cache/squid/releases/tag/SQUID_8_0_1'>Squid 8.0.1</a> (released 2024-02-20)
# EOF;
# $older_versions=<<<EOF
# <li><a href='...'>Squid 6.x</a> ...
# EOF;
# ?>

```

--------------------------------

### Coverity Model File for Squid Analysis

Source: https://context7.com/squid-cache/ci/llms.txt

C++ code defining custom analysis rules for Coverity static analysis. It models assertion macros as panic points and defines Squid-specific types like SBuf and RefCount to reduce false positives.

```cpp
// coverity/model_file - Custom Coverity analysis models

// Mark CPPUNIT assertions as panic points
void CPPUNIT_ASSERT(bool condition) {
    if (!condition)
        __coverity_panic__();
}

// Model SBuf constructor for string size sanitization
class SBuf {
    public:
    SBuf(const char *S, unsigned int n) {
     __coverity_string_size_sanitize__((void*)S);
    }
};

// Mark Must() assertions as panic points
void Must(bool condition) {
    if (!condition)
        __coverity_panic__();
}

// Model exit() as program termination
void exit(int result) {
        __coverity_panic__();
}
```

--------------------------------

### Publish Squid Development Snapshots with mk-release-snapshots.sh

Source: https://context7.com/squid-cache/ci/llms.txt

Publishes the latest development release snapshots by downloading build artifacts from Jenkins CI. It automatically detects the current major version and publishes snapshots for the three latest release series to the appropriate website directories.

```bash
# Run the snapshot publisher (typically via cron every 2 hours)
./release/mk-release-snapshots.sh

# The script performs these operations:
# 1. Auto-detects latest major version from $SQUID_VCS_PATH directories
# 2. Downloads artifacts.zip from Jenkins for each active version:
#    - website-tarballs-head -> v$LATEST/
#    - website-tarballs-latest -> v$LATEST-1/
#    - website-tarballs-old -> v$LATEST-2/
# 3. Extracts and moves artifacts to $SQUID_WWW_PATH/content/Versions/vN/

# Cron configuration example:
# 03 */2 * * * bin/release/mk-release-snapshots.sh

```

--------------------------------

### Manage Code Backports with code-backports.sh

Source: https://context7.com/squid-cache/ci/llms.txt

This script automates the backporting of merged pull requests to older Squid versions. It cherry-picks commits from newer branches based on GitHub labels, manages the backports branch, and creates merge pull requests. It uses Git and the GitHub CLI and skips execution on development branches.

```bash
# Run backports for a specific version
./maintenance/code-backports.sh /var/local/vcs/squid-6

# Operations performed:
# 1. Checks out v$version-next-backports branch
# 2. Rebases onto origin/v$version
# 3. Finds PRs with 'backport-to-v$version' label:
#    gh pr list --repo squid-cache/squid --state closed --label backport-to-v6
#
# 4. Cherry-picks commits by PR number suffix " (#$prnum) ":
#    git log --oneline github/$srcbranch --grep=" (#$prnum)$"
#    git cherry-pick $sha
#
# 5. Removes label on successful backport:
#    gh pr edit $prnum --remove-label backport-to-v$version
#
# 6. Creates merge PR if changes exist:
#    gh pr create --repo squid-cache/squid --base v$version --title "v$version Next Backports"

# Skips execution if .BASE file exists (dev branch doesn't receive backports)
```

--------------------------------

### Publish Squid Translation Packages with mk-langpack.sh

Source: https://context7.com/squid-cache/ci/llms.txt

Publishes the latest Squid translation package (langpack) by monitoring VCS changes to error page translations. It extracts language statistics and updates the website with package metadata, including supported languages, dialects, and file sizes.

```bash
# Run the language pack publisher
./release/mk-langpack.sh

# Force rebuild regardless of changes:
./release/mk-langpack.sh --force

# The script performs these operations:
# 1. Locates latest langpack tarball from version directories
# 2. Validates tarball size (>100KB to ensure full content)
# 3. Checks git history for translation changes since last update
# 4. Extracts language codes and dialect counts
# 5. Updates index.dyn with template substitutions:
#    @SNAPSHOT@ @DATE@ @COUNT@ @KBYTES@ @BCOUNT@ @NAMES@ @ACOUNT@ @ALIASES@

# Output published to: $SQUID_WWW_PATH/content/Versions/langpack/
# Cron configuration:
# 23 */2 * * * bin/release/mk-langpack.sh

```

--------------------------------

### Automate Repository Maintenance with code-maintenance.sh

Source: https://context7.com/squid-cache/ci/llms.txt

This script performs automated repository maintenance for a Squid version checkout. It cleans the workspace, fetches updates, rebases, runs bootstrap and source-maintenance scripts, and creates pull requests for any formatting changes. It detects the version from the directory name and uses Git and the GitHub CLI.

```bash
# Run maintenance on a specific version checkout
./maintenance/code-maintenance.sh /var/local/vcs/squid-7

# The script performs:
# 1. Cleans workspace: git clean -xdf && git checkout -- .
# 2. Fetches all remotes and rebases to github/$forkPoint
# 3. Runs ./bootstrap.sh and commits changes to v$version-bootstrap branch
# 4. Runs ./scripts/source-maintenance.sh and commits to v$version-maintenance
# 5. Creates PRs for any changes via gh pr create
# 6. Lists pending backports labeled 'backport-to-v$version'

# Version detection from directory name:
# /var/local/vcs/squid-7 -> version=7, forkPoint=v7
# If .BASE file exists, uses that as forkPoint (for dev branches)
```

--------------------------------

### Update FTP Archive with update-ftp.sh

Source: https://context7.com/squid-cache/ci/llms.txt

This script maintains the FTP archive directories for Squid releases. It hardlinks release files from web content, copies the latest two releases to the public squid directory, and generates MD5 and SHA1 checksum summaries. It's typically run via cron.

```bash
# Run the FTP updater (typically via cron)
./release/update-ftp.sh

# Operations performed:
# 1. Creates archive directories for new versions
# 2. Hardlinks release files to archive:
#    ln ${f} $SQUID_FTP_PATH/archive/$version/
#
# 3. Copies latest 2 releases to pub/squid/:
#    for TYPE in gz gz.asc bz2 bz2.asc xz xz.asc html
#
# 4. Generates checksum files:
#    md5sum | xargs md5sum > md5s.txt
#    sha1sum | xargs sha1sum > sha1s.txt

# FTP structure:
# /pub/squid/           - Latest releases for each version
# /pub/archive/$ver/    - Full archive per version

# Cron configuration:
# 29 */2 * * * bin/release/update-ftp.sh
```
Tokens:
Raw