### Install FSS via go install Source: https://github.com/anastylosis/fss/blob/master/docs/install.md Installs the latest FSS release using Go's install command. The binary lands in $GOBIN, typically ~/go/bin/. ```bash go install github.com/Anastylosis/FSS@latest ``` -------------------------------- ### Add price snapshot for paid sites Source: https://github.com/anastylosis/fss/blob/master/CONTRIBUTING.md Call scene.AddPrice() for sites with pricing. The example shows a paid site with regular and discounted prices, sale flags, and discount percentage. ```go scene.AddPrice(models.PriceSnapshot{ Date: now, Regular: 19.99, Discounted: 9.99, IsFree: false, IsOnSale: true, DiscountPercent: 50, }) ``` -------------------------------- ### Example config.yaml with all keys and defaults Source: https://github.com/anastylosis/fss/blob/master/docs/usage.md Fully commented example of the optional config file. All keys are optional; missing keys use the defaults shown. CLI flags always override config values. Includes settings for workers, output format, delays, user agent, notices, language, creators directory, per-site overrides, stashbox instances, and Stash integration. ```yaml workers: 3 # int — parallel metadata fetchers output: json # str — json | csv | json,csv out_dir: . # str — output directory path db: "" # str — "" or absent = flat store (the default, and staying so); # "default" = ~/.local/share/fss/fss.db; or a path, absolute or # relative to the working directory (e.g. "./fss.db"). # Omitting the key entirely is not the same as "" — see storage.md delay: 500 # int — ms between page requests; 0 disables user_agent: "" # str — "firefox" (default), "chrome", or a custom UA string notices: true # bool — advisory `[notice]` lines that explain rather than report; # false (or FSS_NO_NOTICES=1) silences them. Warnings and # errors are unaffected language: "" # str — help-text language, e.g. "ko". "" (or a language with no # catalog) = English. Overridden by FSS_LANG and --lang. # Only help text is translated — see translations.md creators_dir: "" # str — directory of one-YAML-per-creator definitions. # "" = ~/.config/fss/creators.d. Point at a clone to use a # shared set — see creators.md site_delays: # map[string]int — per-scraper delay overrides (overrides `delay` for matching sites) # manyvids: 0 # pornhub: 2000 # brazzers: 500 site_cookies: # map[string]string — per-scraper Cookie header (see "Site cookies" below) # mydirtyhobby: "KEY=1234*5678:90:123:1" content_language: "" # str — content language to request where a site publishes in # several, e.g. "de". Not `language:`, which is help text. # Sites with one language ignore it site_languages: # map[string]string — per-scraper content language (overrides content_language) # mydirtyhobby: de stashbox: # list — stashbox instances for the stashbox scraper # - url: "https://stashdb.org/graphql" # GraphQL endpoint URL # api_key: "your-api-key-here" # API key for this instance # - url: "https://pmvstash.org/graphql" # api_key: "another-api-key" stash: url: "http://localhost:9999" # str — Stash server URL api_key: "" # str — API key (prefer FSS_STASH_API_KEY env var) tag: "fss_import" # str — import marker tag stashbox_tag: "fss_stashbox_override" # str — tag for StashDB override tracking resolution_tags: true # bool — add 4K/FHD/HD Available tags ``` -------------------------------- ### Install FSS on Linux from pre-built binary Source: https://github.com/anastylosis/fss/blob/master/docs/install.md Downloads and installs the static binary for Linux. Resolves the latest version via GitHub redirect; set ARCH to arm64 for ARM platforms. Requires sudo unless using ~/.local/bin. ```bash # Resolves to the latest tag (e.g. v1.11.0) by following the GitHub redirect. VERSION=$(curl -sIL -o /dev/null -w '%{url_effective}' https://github.com/Anastylosis/FSS/releases/latest | sed 's|.*/||') ARCH=amd64 # or arm64 for Raspberry Pi 4/5, ARM cloud instances, etc. curl -LO https://github.com/Anastylosis/FSS/releases/download/${VERSION}/fss-${VERSION}-linux-${ARCH}.tar.gz tar xzf fss-${VERSION}-linux-${ARCH}.tar.gz sudo install -m 0755 fss /usr/local/bin/fss fss version ``` -------------------------------- ### Build FSS from source Source: https://github.com/anastylosis/fss/blob/master/docs/install.md Clones the repository and builds the binary with Go 1.26+. The binary is placed in the current directory as 'fss'. ```bash git clone https://github.com/Anastylosis/FSS cd FSS go build -o fss . ./fss version ``` -------------------------------- ### Install FSS on macOS from pre-built binary Source: https://github.com/anastylosis/fss/blob/master/docs/install.md Downloads and installs the static binary for macOS. Set ARCH to arm64 for Apple Silicon or amd64 for Intel. Gatekeeper blocks unsigned binaries; remove quarantine or use 'Open anyway'. ```bash VERSION=$(curl -sIL -o /dev/null -w '%{url_effective}' https://github.com/Anastylosis/FSS/releases/latest | sed 's|.*/||') ARCH=arm64 # Apple Silicon (M1+); use amd64 for Intel Macs curl -LO https://github.com/Anastylosis/FSS/releases/download/${VERSION}/fss-${VERSION}-darwin-${ARCH}.tar.gz tar xzf fss-${VERSION}-darwin-${ARCH}.tar.gz sudo install -m 0755 fss /usr/local/bin/fss # Gatekeeper will block unsigned binaries the first time. Either: xattr -d com.apple.quarantine /usr/local/bin/fss # or: System Settings → Privacy & Security → "Open anyway" after the first failed run. fss version ``` -------------------------------- ### Download config.example.yaml to platform config path Source: https://github.com/anastylosis/fss/blob/master/docs/usage.md Commands to create the config directory and download the example config file for Linux and macOS. The config file is optional and lives at the platform's config path. ```bash # Linux mkdir -p ~/.config/fss curl -fsSL https://raw.githubusercontent.com/Anastylosis/FSS/master/config.example.yaml \ -o ~/.config/fss/config.yaml # macOS mkdir -p ~/Library/Application\ Support/fss curl -fsSL https://raw.githubusercontent.com/Anastylosis/FSS/master/config.example.yaml \ -o ~/Library/Application\ Support/fss/config.yaml ``` -------------------------------- ### Build .deb package locally with nfpm Source: https://github.com/anastylosis/fss/blob/master/CONTRIBUTING.md Installs nfpm, builds the fss binary for linux/amd64, and packages it as a .deb. Requires Go and the nfpm.yaml configuration. ```bash go install github.com/goreleaser/nfpm/v2/cmd/nfpm@latest GOOS=linux GOARCH=amd64 go build -o dist/fss . GOARCH=amd64 VERSION=1.7.0 nfpm package --packager deb --target dist/ ``` -------------------------------- ### Example output of fss creators Source: https://github.com/anastylosis/fss/blob/master/docs/creators.md Example output showing the creator count, configuration directory, and each creator's stores with scrape dates. ```text 13 creator(s) in /home/you/.config/fss/creators.d Emma Woodhouse (aka Emma Woodhouse Films) https://clipmarket.example/studio/6055/emma-woodhouse-films [scraped 2026-08-16] https://vidvault.example/profile/380818/emma-woodhouse/store/videos [scraped 2026-08-16] https://clipstore.example/store/221/EmmaWoodhouse [scraped 2026-08-16] ``` -------------------------------- ### Run example queries with sqlite3 Source: https://github.com/anastylosis/fss/blob/master/docs/usage.md Run SQL queries against the FSS database using sqlite3. Use -header -column for readable output, or .mode box. Placeholders in ALL CAPS should be substituted with your own values. ```bash sqlite3 -header -column ~/.local/share/fss/fss.db ``` -------------------------------- ### Install FSS on Windows via PowerShell Source: https://github.com/anastylosis/fss/blob/master/docs/install.md Downloads and installs the Windows binary using PowerShell. Adds %USERPROFILE%\bin to PATH; restart shell before running fss version. ```powershell $Version = (Invoke-RestMethod -Uri "https://api.github.com/repos/Anastylosis/FSS/releases/latest").tag_name Invoke-WebRequest -Uri "https://github.com/Anastylosis/FSS/releases/download/$Version/fss-$Version-windows-amd64.zip" -OutFile fss.zip Expand-Archive -Path fss.zip -DestinationPath . # Move into a folder that's on your PATH, e.g.: New-Item -ItemType Directory -Force -Path "$env:USERPROFILE\bin" | Out-Null Move-Item -Force fss.exe "$env:USERPROFILE\bin\fss.exe" # Add %USERPROFILE%\bin to your PATH (one-time): [Environment]::SetEnvironmentVariable("Path", "$env:Path;$env:USERPROFILE\bin", "User") # Restart your shell, then: fss version ``` -------------------------------- ### Install FSS via package managers Source: https://github.com/anastylosis/fss/blob/master/README.md Install FSS using Homebrew, AUR, Docker, or Go. Requires Go 1.26+ for the source install. ```bash brew install anastylosis/tap/fss # macOS / Linux ``` ```bash yay -S fss # Arch (AUR) ``` ```bash docker run --rm ghcr.io/anastylosis/fss:latest # Docker ``` ```bash go install github.com/Anastylosis/FSS@latest # from source, Go 1.26+ ``` -------------------------------- ### Quick start: check and scrape studio URLs Source: https://github.com/anastylosis/fss/blob/master/README.md Check site support, scrape a catalogue (first run full, later runs incremental), refresh, output CSV, or use SQLite. ```bash fss check # is this site supported? ``` ```bash fss scrape # first run: the whole catalogue ``` ```bash fss scrape # later runs: only new scenes ``` ```bash fss scrape --refresh # re-fetch everything, mark removed scenes ``` ```bash fss scrape --output json,csv # also write a CSV ``` ```bash fss scrape --db # store in SQLite instead of JSON ``` -------------------------------- ### Build with overridden build arguments Source: https://github.com/anastylosis/fss/blob/master/docs/docker.md Example of building with custom `GO_VERSION` and `ALPINE_VERSION` build arguments. ```bash docker build --build-arg GO_VERSION=1.26 --build-arg ALPINE_VERSION=3.22 -t fss:dev . ``` -------------------------------- ### Install FSS via Homebrew Source: https://github.com/anastylosis/fss/blob/master/docs/install.md Installs FSS using the Homebrew tap on macOS and Linux. The formula installs a pre-built binary and shell completions; `brew upgrade` handles updates. ```bash brew install anastylosis/tap/fss fss version ``` -------------------------------- ### Add price snapshot for free sites Source: https://github.com/anastylosis/fss/blob/master/CONTRIBUTING.md For free sites (e.g., Pornhub), use a PriceSnapshot with IsFree set to true. ```go scene.AddPrice(models.PriceSnapshot{Date: now, IsFree: true}) ``` -------------------------------- ### PreferredStudioURL example Source: https://github.com/anastylosis/fss/blob/master/docs/library.md Asks the matching scraper for the canonical URL spelling to key on, e.g., converting `yourvids.com/leina-sex` to `yourvids.com/creators/leina-sex`. Returns the URL unchanged if no scraper claims it or expresses no preference. ```go url := scraper.PreferredStudioURL("https://yourvids.com/leina-sex") // https://yourvids.com/creators/leina-sex ``` -------------------------------- ### Install FSS via AUR Source: https://github.com/anastylosis/fss/blob/master/docs/install.md Installs FSS from the Arch User Repository using yay. ```bash yay -S fss ``` -------------------------------- ### Run FSS via Docker Source: https://github.com/anastylosis/fss/blob/master/docs/install.md Pulls the multi-arch Docker image from GHCR and runs the list-scrapers command. See docker.md for volume conventions and compose example. ```bash docker pull ghcr.io/anastylosis/fss:latest docker run --rm ghcr.io/anastylosis/fss:latest list-scrapers ``` -------------------------------- ### Example output of --stale run Source: https://github.com/anastylosis/fss/blob/master/docs/creators.md Shows the skip/run reasoning and summary line for a --stale run. The output lists each store with its status and last-scrape age, ending with a count of due studios. ```text skip https://clipmarket.example/studio/4021/jane-eyre (scraped 2d ago) run https://janeeyre.example (scraped 9d ago) run https://fanhub.example/jane-eyre (never scraped) 2 of 44 studio(s) due. ``` -------------------------------- ### Install FSS Go module with go get Source: https://github.com/anastylosis/fss/blob/master/docs/library.md Adds the FSS module to your Go project. Use a release tag for a stable version. The module path changed from github.com/Wasylq/FSS; the old path only resolves for tags up to v1.28.1. ```bash go get github.com/Anastylosis/FSS@latest # Or use tag for stable release ``` -------------------------------- ### Example fss compare report output Source: https://github.com/anastylosis/fss/blob/master/docs/compare.md Sample output from fss compare showing per-store statistics, cheapest sources, price gaps, and exclusives. The dash for avg price indicates the store exposes no prices. ```text Jane Eyre — 1407 scenes across 4 storefronts store scenes shared exclusive avg price clipmarket.example 658 49 609 $22.17 janeeyre.example 343 191 152 $10.02 vidvault.example 292 191 101 $22.66 fanhub.example 114 67 43 — Cheapest source for the 227 titles carried by more than one store: janeeyre.example 148 titles, $2276.48 below the dearest in total clipmarket.example 20 titles, $109.00 below the dearest in total vidvault.example 12 titles, $23.63 below the dearest in total Widest price gaps: Midnight Rehearsal $ 9.99 janeeyre.example vs $ 49.99 vidvault.example (-$40.00) The Long Way Home $ 9.99 janeeyre.example vs $ 49.99 vidvault.example (-$40.00) Borrowed Time (Part One) $ 9.99 janeeyre.example vs $ 41.99 clipmarket.example (-$32.00) Carried by one storefront only: clipmarket.example 609 titles janeeyre.example 152 titles vidvault.example 101 titles fanhub.example 43 titles ``` -------------------------------- ### Install FSS via system package (.deb/.rpm) Source: https://github.com/anastylosis/fss/blob/master/docs/install.md Downloads and installs .deb or .rpm packages for Debian/Ubuntu or Fedora/RHEL. Resolves latest tag; VERSION drops leading 'v'; RPM_ARCH maps arm64 to aarch64. ```bash TAG=$(curl -sIL -o /dev/null -w '%{url_effective}' https://github.com/Anastylosis/FSS/releases/latest | sed 's|.*/||') VERSION=${TAG#v} # asset filenames drop the leading "v" ARCH=amd64 # or arm64 # Debian / Ubuntu curl -LO "https://github.com/Anastylosis/FSS/releases/download/${TAG}/fss_${VERSION}_${ARCH}.deb" sudo dpkg -i "fss_${VERSION}_${ARCH}.deb" # Fedora / RHEL (rpm names the architecture differently) RPM_ARCH=$([ "$ARCH" = arm64 ] && echo aarch64 || echo x86_64) curl -LO "https://github.com/Anastylosis/FSS/releases/download/${TAG}/fss-${VERSION}-1.${RPM_ARCH}.rpm" sudo rpm -i "fss-${VERSION}-1.${RPM_ARCH}.rpm" ``` -------------------------------- ### Example queries: Scenes carrying a tag, cast, most used tags, tag combinations Source: https://github.com/anastylosis/fss/blob/master/docs/usage.md SQL queries for common scene and tag lookups. Use 'TAG NAME', 'SCENE_ID', 'SITE_ID', 'STUDIO_URL', 'PERFORMER NAME', 'WANTED TAG', and 'UNWANTED TAG' placeholders. The 'One tag but not another' query uses NOT EXISTS to exclude scenes with an unwanted tag. The 'A performer and a tag together' query uses COLLATE NOCASE for case-insensitive performer name matching. ```sql -- Scenes carrying a tag SELECT s.title, s.date FROM scenes s JOIN scene_tags j ON j.scene_id = s.id AND j.site_id = s.site_id AND j.studio_url = s.studio_url JOIN tags t ON t.id = j.tag_id WHERE t.name = 'TAG NAME' AND s.deleted_at IS NULL ORDER BY s.date DESC; ``` ```sql -- The cast of one scene, in billing order SELECT p.name FROM scene_performers sp JOIN performers p ON p.id = sp.performer_id WHERE sp.scene_id = 'SCENE_ID' AND sp.site_id = 'SITE_ID' AND sp.studio_url = 'STUDIO_URL' ORDER BY sp.position; ``` ```sql -- Most used tags overall SELECT t.name, COUNT(*) AS scenes FROM scene_tags j JOIN tags t ON t.id = j.tag_id GROUP BY t.name ORDER BY scenes DESC LIMIT 25; ``` ```sql -- One tag but not another SELECT s.date, s.title FROM scenes s JOIN scene_tags j ON j.scene_id = s.id AND j.site_id = s.site_id AND j.studio_url = s.studio_url JOIN tags t ON t.id = j.tag_id WHERE t.name = 'WANTED TAG' AND s.deleted_at IS NULL AND NOT EXISTS ( SELECT 1 FROM scene_tags j2 JOIN tags t2 ON t2.id = j2.tag_id WHERE j2.scene_id = s.id AND j2.site_id = s.site_id AND j2.studio_url = s.studio_url AND t2.name = 'UNWANTED TAG') ORDER BY s.date DESC; ``` ```sql -- A performer and a tag together SELECT s.date, s.title FROM scenes s JOIN scene_performers sp ON sp.scene_id = s.id AND sp.site_id = s.site_id AND sp.studio_url = s.studio_url JOIN performers p ON p.id = sp.performer_id JOIN scene_tags j ON j.scene_id = s.id AND j.site_id = s.site_id AND j.studio_url = s.studio_url JOIN tags t ON t.id = j.tag_id WHERE p.name = 'PERFORMER NAME' COLLATE NOCASE AND t.name = 'TAG NAME' AND s.deleted_at IS NULL ORDER BY s.date DESC; ``` -------------------------------- ### Verify build, vet, tests, and scraper registration Source: https://github.com/anastylosis/fss/blob/master/CONTRIBUTING.md Run these commands to verify a new scraper compiles, passes static analysis, passes all tests with the race detector, and appears in the registry. The last command builds the binary and lists scrapers to confirm the new scraper is registered. ```bash go build ./... # compiles go vet ./... # static analysis go test -race -count=1 ./... # all tests pass go build -o fss . && ./fss list-scrapers # new scraper appears ``` -------------------------------- ### Register scraper via init() Source: https://github.com/anastylosis/fss/blob/master/CONTRIBUTING.md Called automatically at startup when the package is imported. Registers the scraper with the plugin-based registry. ```go func init() { scraper.Register(New()) } ``` -------------------------------- ### Blank import in internal/scrapers/all/all.go Source: https://github.com/anastylosis/fss/blob/master/CONTRIBUTING.md Add this blank import to trigger init() and register the scraper. Keep the list alphabetical. Without this line, the scraper won't be available to the fss binary or library callers. ```go _ "github.com/Anastylosis/FSS/internal/scrapers/" ``` -------------------------------- ### CSV output example for --csv shopping-list.csv Source: https://github.com/anastylosis/fss/blob/master/docs/compare.md Writes one row per title carried by more than one store. `cheapest_url` is the scene URL at the cheapest store, making the file directly actionable. Empty price columns mean that store exposed no amount; cells are guarded against spreadsheet formula injection. ```csv creator,title,stores,cheapest_store,cheapest_price,dearest_store,dearest_price,spread,cheapest_url Jane Eyre,The Long Way Home,2,janeeyre.example,9.99,vidvault.example,49.99,40.00,https://... ``` -------------------------------- ### Run Stash integration tests with default settings Source: https://github.com/anastylosis/fss/blob/master/CONTRIBUTING.md Runs the Stash integration tests against the default local Stash instance at http://localhost:9999 with no authentication. ```bash # Default: connects to http://localhost:9999, no auth make smoke-stash ``` -------------------------------- ### Tag and push a release Source: https://github.com/anastylosis/fss/blob/master/CONTRIBUTING.md Creates an annotated tag and pushes it to trigger the release workflow. The tag's annotation is rendered as a callout at the top of the release page. ```bash git tag -a v1.7.0 -m "v1.7.0" git push origin v1.7.0 ``` -------------------------------- ### Live integration test template with testutil helpers Source: https://github.com/anastylosis/fss/blob/master/CONTRIBUTING.md Each scraper has an integration_test.go like this for live integration smoke tests that hit the real site. testutil.RunLiveScrape fetches the first 2 scenes, validates each via testutil.ValidateScene (non-empty ID/Title/URL/Date, plausible Duration, etc.), and logs the first scene's full struct on -v. SkipIfPlaceholder skips cleanly when liveStudioURL still contains REPLACE-ME — use it for new scrapers until you find a stable URL. ```go //go:build integration package import ( "testing" "github.com/Anastylosis/FSS/internal/scrapers/testutil" ) // liveStudioURL — pick a stable studio. Update if it 404s. const liveStudioURL = "https://example.com/profile/123/some-name" func TestLive(t *testing.T) { testutil.SkipIfPlaceholder(t, liveStudioURL) testutil.RunLiveScrape(t, New(), liveStudioURL, 2) } ``` -------------------------------- ### run() goroutine using scraper.Paginate Source: https://github.com/anastylosis/fss/blob/master/CONTRIBUTING.md Implements the ListScenes goroutine using scraper.Paginate for page-numbered pagination. Handles delay, ctx cancellation, progress reporting, KnownIDs early-stop, and scene sending. defer close(out) must be the first line. ```go func (s *Scraper) run(ctx context.Context, studioURL string, opts scraper.ListOpts, out chan<- scraper.SceneResult) { defer close(out) // MUST be the first line now := time.Now().UTC() scraper.Paginate(ctx, opts, "mysite", out, func(ctx context.Context, page int) (scraper.PageResult, error) { items, total, err := s.fetchPage(ctx, studioURL, page) if err != nil { return scraper.PageResult{}, err } scenes := make([]models.Scene, len(items)) for i, item := range items { scenes[i] = toScene(item, studioURL, now) } return scraper.PageResult{ Scenes: scenes, Total: total, // set on first page for progress display Done: len(items) == 0, // or page >= totalPages }, nil }) } ``` -------------------------------- ### fss identify quick start commands Source: https://github.com/anastylosis/fss/blob/master/docs/identify.md Shows common fss identify invocations: dry-run, writing .nfo files with --apply, cross-site merge with multiple --json flags, loading all JSON from a directory with --dir, overwriting with --force, and suppressing the report with --no-report. ```bash # Dry-run — see what would match fss identify /path/to/videos --json studio.json # Write .nfo files fss identify /path/to/videos --json studio.json --apply # Cross-site merge: load multiple JSON files for richer metadata fss identify /path/to/videos --json manyvids.json --json clips4sale.json --apply # Load all JSON files from a directory fss identify /path/to/videos --dir ./data --apply # Overwrite existing .nfo files fss identify /path/to/videos --json studio.json --apply --force # Suppress the report file fss identify /path/to/videos --json studio.json --apply --no-report ``` -------------------------------- ### testutil.SitemapServer rewrites fixture host to test server Source: https://github.com/anastylosis/fss/blob/master/CONTRIBUTING.md Use testutil.SitemapServer to rewrite the fixture's host to the test server before serving it. It serves the sitemap for any 'sitemap'-ish path and detail for everything else. It also fails the test if the fixture no longer contains the host you passed, preventing a refreshed fixture from silently re-enabling live fetches. ```go // Rewrites https://honeytrans.com → the test server; serves the sitemap for // any "sitemap"-ish path and detail for everything else. srv := testutil.SitemapServer(t, "https://honeytrans.com", readFixture(t, "sitemap.xml"), readFixture(t, "detail.html")) ``` -------------------------------- ### fss version output Source: https://github.com/anastylosis/fss/blob/master/docs/usage.md Output of the `fss version` command showing build version, commit hash, build date, and channel. The channel is set at build time via `-X main.channel=…` and may be `aur`, `docker`, `deb`, `rpm`, or `release`. Builds without a channel (e.g., local `go build` or `go install`) print the line without a channel. ```text fss v1.31.0 (abc1234, 2026-09-11, aur) ``` -------------------------------- ### Write scenes as JSON and CSV, and slugify URLs Source: https://github.com/anastylosis/fss/blob/master/docs/library.md Use the output package to write FSS-format JSON and CSV files and generate safe filenames from URLs. WriteJSON performs atomic file replacement, WriteCSV prefixes ' to cells starting with formula characters, and Slugify appends a hash and caps at 250 bytes. ```go import ( "github.com/Anastylosis/FSS/models" "github.com/Anastylosis/FSS/output" ) // Write scenes as JSON (atomic file replacement — safe on crash). sf := models.StudioFile{ StudioURL: "https://www.manyvids.com/...", ScrapedAt: time.Now().UTC(), SceneCount: len(scenes), Scenes: scenes, } output.WriteJSON(sf, "studio.json") // Write scenes as CSV. output.WriteCSV(scenes, "studio.csv") // Generate a safe filename from a URL. slug := output.Slugify("https://www.manyvids.com/Profile/1000001/jane-eyre/Store/Videos") // → "www-manyvids-com-profile-1000001-jane-eyre-store-videos-0e51b52b" ``` -------------------------------- ### Browsing a studio: current scenes, studio stats, and stale studios Source: https://github.com/anastylosis/fss/blob/master/docs/usage.md Three SQL queries for studio browsing: list current scenes, aggregate studio storage and last scrape time, and find studios not scraped in the last 30 days. The first query filters on studio_url and excludes deleted scenes. The second uses a LEFT JOIN to count scenes per studio. The third uses date('now', '-30 day') to identify stale studios. ```sql -- Everything currently on a studio SELECT title, date, duration, lowest_price FROM scenes WHERE studio_url = 'STUDIO_URL' AND deleted_at IS NULL ORDER BY date DESC; -- Every studio, with how much is stored and when it was last scraped SELECT st.name, st.site_id, st.last_scraped_at, COUNT(sc.id) AS scenes FROM studios st LEFT JOIN scenes sc ON sc.studio_url = st.url AND sc.deleted_at IS NULL GROUP BY st.url ORDER BY scenes DESC; -- Studios not scraped in the last 30 days SELECT url, site_id, last_scraped_at FROM studios WHERE last_scraped_at < date('now', '-30 day') ORDER BY last_scraped_at; ``` -------------------------------- ### Enable SQLite via config.yaml Source: https://github.com/anastylosis/fss/blob/master/docs/usage.md Set the db key in config.yaml to enable SQLite. Use "default" for the default path or a custom path string. The data directory is created automatically if it doesn't exist. ```yaml db: "default" # uses ~/.local/share/fss/fss.db db: "/custom/path.db" # uses a custom path ``` -------------------------------- ### Install FFmpeg for duration filtering Source: https://github.com/anastylosis/fss/blob/master/docs/identify.md Install FFmpeg to enable duration probing in `fss identify`. Required on Debian/Ubuntu, macOS, and Windows (winget). ```bash # Debian/Ubuntu sudo apt install ffmpeg # macOS brew install ffmpeg # Windows (winget) winget install FFmpeg ``` -------------------------------- ### Scrape and compare by creator Source: https://github.com/anastylosis/fss/blob/master/docs/creators.md Use the --creator flag to scrape all storefronts for a creator, and --from-creator to compare prices across them. ```bash fss scrape --creator "Jane Eyre" # all four stores, one command ``` ```bash fss compare --from-creator "Becky Sharp" # where is each clip cheapest? ``` -------------------------------- ### List studios with fss list-studios Source: https://github.com/anastylosis/fss/blob/master/docs/usage.md Use the fss list-studios command to list studios. The --db flag specifies the database path, overriding the default location or the config file's db: value. Without --db, it uses the config file's db: value. ```bash fss list-studios --db # uses default db location ``` ```bash fss list-studios --db ./fss.db # uses a custom path ``` ```bash fss list-studios # uses the config file's `db:` value ``` -------------------------------- ### Run Stash integration smoke test Source: https://github.com/anastylosis/fss/blob/master/CONTRIBUTING.md Runs the Stash integration smoke test only. ```bash make smoke-stash # Stash integration only ``` -------------------------------- ### Run fss compare with database and options Source: https://github.com/anastylosis/fss/blob/master/docs/compare.md Use the fss compare command to analyze a creator's storefronts. The first command runs with the default database; the second adds the --from-creator, --top, and --exclusives flags to filter results. ```bash fss compare --db ``` ```bash fss compare --db --from-creator "Jane Eyre" --top 5 --exclusives ``` -------------------------------- ### Run smoke test for a single scraper Source: https://github.com/anastylosis/fss/blob/master/CONTRIBUTING.md Runs the smoke test for one specific scraper, identified by the SCRAPER variable. ```bash make smoke-one SCRAPER= # one scraper ``` -------------------------------- ### Load, index, match, and merge scenes with the match package Source: https://github.com/anastylosis/fss/blob/master/docs/library.md Shows the core workflow: load scenes from JSON files or a directory, build a title index, match a filename with a duration, handle confidence levels, and merge cross-site scenes. MergeScenes normalizes performer/tag/category/studio names and deduplicates by case-folded key, keeping the first contributing site's spelling. ```go import ( "github.com/Anastylosis/FSS/match" _ "github.com/Anastylosis/FSS/internal/scrapers/manyvids" ) // Load scenes from FSS JSON files. scenes, err := match.LoadJSONFiles([]string{"manyvids.json", "clips4sale.json"}) // or: scenes, err := match.LoadJSONDir("./data") // Build a title index. idx := match.BuildIndex(scenes) // Match a filename (duration in seconds, 0 = unknown). result := idx.Match("Fostering the Bully.mp4", 605.0) switch result.Confidence { case match.MatchExact: fmt.Println("Exact match:", result.Scenes[0].Title) case match.MatchSubstring: fmt.Println("Substring match:", result.Scenes[0].Title) case match.MatchAmbiguous: fmt.Printf("Ambiguous: %d candidates\n", result.Candidates) case match.MatchNone: fmt.Println("No match") } // Merge cross-site scenes into a single metadata record. merged := match.MergeScenes(result.Scenes, time.Time{}) fmt.Println(merged.Title, merged.URLs, merged.Performers) ``` -------------------------------- ### StudioScraper interface definition Source: https://github.com/anastylosis/fss/blob/master/CONTRIBUTING.md Defines the required methods for a site scraper: ID, Patterns, MatchesURL, and ListScenes. Implement this interface in your scraper package. ```go type StudioScraper interface { ID() string Patterns() []string MatchesURL(url string) bool ListScenes(ctx context.Context, studioURL string, opts ListOpts) (<-chan SceneResult, error) } ``` -------------------------------- ### Run Stash integration tests with custom URL Source: https://github.com/anastylosis/fss/blob/master/CONTRIBUTING.md Runs the Stash integration tests against a custom Stash URL using the FSS_STASH_URL environment variable. ```bash FSS_STASH_URL=http://192.168.1.50:9999 make smoke-stash ``` -------------------------------- ### Show scrape help with Docker Source: https://github.com/anastylosis/fss/blob/master/docs/docker.md Displays help for the scrape subcommand. Works because the default ENTRYPOINT is fss. ```bash docker run --rm ghcr.io/anastylosis/fss:latest scrape --help ``` -------------------------------- ### Scrape multiple sites with --performer and --studio Source: https://github.com/anastylosis/fss/blob/master/docs/usage.md Use this command to scrape several sites under one performer and studio name, replacing the site's own credits. Both flags apply to every URL in the invocation and replace rather than merge, so pass every name you want kept. ```bash fss scrape https://siteA.com https://siteB.com https://siteC.com \ --performer "Becky Sharp" --studio "Becky Sharp" ``` -------------------------------- ### Following a performer across sites: scenes and studio frequency Source: https://github.com/anastylosis/fss/blob/master/docs/usage.md Two SQL queries to follow a performer across all sites: list every scene featuring the performer with the studio name, and count how often they appear per studio. Both join scene_performers on all three keys (scene_id, site_id, studio_url) and use COLLATE NOCASE for case-insensitive name matching. The first query also joins studios to resolve the studio name. ```sql -- Every scene featuring a performer, with the studio it came from SELECT s.date, s.title, COALESCE(NULLIF(s.studio, ''), NULLIF(st.name, ''), s.studio_url) AS studio, s.site_id FROM scenes s JOIN scene_performers sp ON sp.scene_id = s.id AND sp.site_id = s.site_id AND sp.studio_url = s.studio_url JOIN performers p ON p.id = sp.performer_id LEFT JOIN studios st ON st.url = s.studio_url WHERE p.name = 'PERFORMER NAME' COLLATE NOCASE AND s.deleted_at IS NULL ORDER BY s.date DESC; -- Which studios they appear in, and how often SELECT COALESCE(NULLIF(s.studio, ''), s.studio_url) AS studio, COUNT(*) AS scenes FROM scenes s JOIN scene_performers sp ON sp.scene_id = s.id AND sp.site_id = s.site_id AND sp.studio_url = s.studio_url JOIN performers p ON p.id = sp.performer_id WHERE p.name = 'PERFORMER NAME' COLLATE NOCASE GROUP BY studio ORDER BY scenes DESC; ``` -------------------------------- ### Group creator storefronts and compare prices Source: https://github.com/anastylosis/fss/blob/master/README.md Suggest groupings from scraped data, scrape all stores for a creator, and compare prices across storefronts. ```bash fss creators suggest --db --write # propose groupings from what you scrape ``` ```bash fss scrape --creator "Jane Eyre" # scrape all of their stores ``` ```bash fss compare --db --from-creator "Jane Eyre" # where is each clip cheapest? ``` -------------------------------- ### ListOpts Source: https://github.com/anastylosis/fss/blob/master/docs/library.md Configures how the scraper paginates, including concurrency, delay, incremental mode, and language. ```APIDOC ## ListOpts ### Description Configures how the scraper paginates, including concurrency, delay, incremental mode, and language. ### Fields - **Workers** (int) - Number of concurrent detail-page workers. Zero uses the scraper's default (usually 4). - **Delay** (time.Duration) - Delay between requests. On its own paces each goroutine separately. - **KnownIDs** (map[string]bool) - Incremental mode: stop once these IDs are reached. - **Language** (string) - Content language for sites that publish in several languages. Empty means the scraper's default. ### Example ```go opts := scraper.ListOpts{ Workers: 2, Delay: 500 * time.Millisecond, KnownIDs: map[string]bool{ "existing-scene-id": true, }, Language: "de", } ``` ``` -------------------------------- ### fss completion shell script generation Source: https://github.com/anastylosis/fss/blob/master/docs/usage.md Generates shell completion scripts for bash, zsh, fish, and powershell. The bash example sources the completion directly, zsh writes to the fpath directory, and fish pipes to source. ```bash source <(fss completion bash) # bash ``` ```bash fss completion zsh > "${fpath[1]}/_fss" # zsh ``` ```bash fss completion fish | source # fish ``` -------------------------------- ### Enable SQLite with --db flag Source: https://github.com/anastylosis/fss/blob/master/docs/usage.md Pass --db to any scrape command to enable SQLite. The default path is ~/.local/share/fss/fss.db; use --db=/custom/path.db for a custom path (the = is required). ```bash fss scrape --db # uses default path: ~/.local/share/fss/fss.db fss scrape --db=/custom/path.db # uses a custom path — the `=` is required ``` -------------------------------- ### Run smoke tests (all scrapers + Stash) Source: https://github.com/anastylosis/fss/blob/master/CONTRIBUTING.md Runs the full smoke test suite covering all scrapers and Stash integration. These tests are never run in CI and are a manual pre-release check. ```bash make smoke # all scrapers + Stash ``` -------------------------------- ### Tag with multi-line release note Source: https://github.com/anastylosis/fss/blob/master/CONTRIBUTING.md Use a multi-line annotation to provide a detailed release note. The first line becomes the bold heading and the rest the body. A lightweight tag omits the callout. ```bash git tag -a v1.7.1 -m "maintenance release only, no new scrapers Existing scrapers are unaffected; no config or store changes." git push origin v1.7.1 ``` -------------------------------- ### Run scraper tests to update docs Source: https://github.com/anastylosis/fss/blob/master/CONTRIBUTING.md Runs the scraper tests that rewrite the site count in README.md and regenerate docs/sites.md. The first run fails to indicate the files were changed, so commit both files. ```bash go test ./internal/scrapers/all/... ``` -------------------------------- ### Multi-arch build with buildx Source: https://github.com/anastylosis/fss/blob/master/docs/docker.md Creates a buildx builder and builds for linux/amd64 and linux/arm64 platforms. Requires buildx and QEMU. ```bash docker buildx create --use --name fss-builder docker buildx build --platform linux/amd64,linux/arm64 -t fss:dev . ``` -------------------------------- ### Run Stash integration tests with custom URL and API key Source: https://github.com/anastylosis/fss/blob/master/CONTRIBUTING.md Runs the Stash integration tests against a custom Stash URL and API key using FSS_STASH_URL and FSS_STASH_API_KEY environment variables. ```bash FSS_STASH_URL=http://192.168.1.50:9999 FSS_STASH_API_KEY=yourkey make smoke-stash ``` -------------------------------- ### identify.RunContext Source: https://github.com/anastylosis/fss/blob/master/docs/library.md Same as Run but with context cancellation, required for poster downloads. Returns a list of results. ```APIDOC ## identify.RunContext ### Description Same as Run but with context cancellation, required for poster downloads. Returns a list of results. ### Method Function ### Signature `func RunContext(ctx context.Context, videos []string, idx Index, opts Options) []Result` ### Parameters - **ctx** (context.Context) - Required - Context for cancellation. - **videos** ([]string) - Required - List of video file paths. - **idx** (Index) - Required - The FSS scene index. - **opts** (Options) - Required - Options controlling behavior (e.g., Apply, Poster). ### Returns - **[]Result** - List of results for each video. ### Example ```go results := identify.RunContext(ctx, videos, idx, identify.Options{ Apply: true, Poster: true, // download thumbnails to -poster. beside each video }) ``` ``` -------------------------------- ### ListScenes Source: https://github.com/anastylosis/fss/blob/master/docs/library.md Lists scenes from a studio URL using the given options, with optional pacer in context. ```APIDOC ## ListScenes ### Description Lists scenes from a studio URL using the given options, with optional pacer in context. ### Parameters - **ctx** (context.Context) - The context, possibly with a pacer attached. - **studioURL** (string) - The URL of the studio. - **opts** (scraper.ListOpts) - The options for listing scenes. ### Returns - **ch** (chan) - A channel of scenes. - **err** (error) - Error if any. ### Example ```go ch, err := s.ListScenes(ctx, studioURL, opts) ``` ``` -------------------------------- ### Broken-scraper warning output Source: https://github.com/anastylosis/fss/blob/master/docs/usage.md Example warning shown when a completed traversal re-sees less than 50% of stored scenes. The prompt asks for confirmation before an authoritative save; answering anything but y/yes skips the save, and with no terminal it skips and suggests --force. ```text [warn] auntjudys: this scrape re-saw 9 of 412 stored scenes (2%) — the site or the scraper may have changed this is an authoritative save: 403 stored scene(s) would be dropped proceed anyway? [y/N] ``` -------------------------------- ### Registry API: ForURL, ForID, All Source: https://github.com/anastylosis/fss/blob/master/docs/library.md After importing `scrapers/all`, these functions query the populated registry. `ForURL` finds a scraper by URL, `ForID` by stable ID, and `All` lists all registered scrapers. ```go // Find a scraper by URL. s, err := scraper.ForURL("https://www.manyvids.com/...") // Find a scraper by its stable ID (e.g. "manyvids", "clips4sale"). s, err := scraper.ForID("clips4sale") // List all registered scrapers. for _, s := range scraper.All() { fmt.Printf("%-20s %s\n", s.ID(), s.Patterns()) } ``` -------------------------------- ### Assert fetches stayed local using scene.URL Source: https://github.com/anastylosis/fss/blob/master/CONTRIBUTING.md scene.URL is the URL that was actually requested, so this is a direct check that the fetch stayed on the test server. If the URL does not have the test server prefix, the test fails with an error message. ```go if !strings.HasPrefix(sc.URL, srv.URL) { t.Errorf("scene %s fetched %q, which is not the test server", sc.ID, sc.URL) } ``` -------------------------------- ### fss detect Source: https://github.com/anastylosis/fss/blob/master/docs/usage.md Fetches a URL once and checks the response for known platform signals (Aylo `instance_token`, Algolia API, `psmcdn.net`, ModelCentro, etc.). Reports the detected platform and corresponding util package. ```APIDOC ## fss detect ### Description Fetches a URL once and checks the response for known platform signals (Aylo `instance_token`, Algolia API, `psmcdn.net`, ModelCentro, etc.). Reports the detected platform and corresponding util package. Useful when deciding whether a new site needs a standalone scraper or belongs to an existing shared package. ### Usage `fss detect ` ### Parameters - **url** (string) - Required - The URL to fetch and analyze. The scheme is optional; `fss detect brazzers.com` is read as `https://brazzers.com`. Pass `http://` explicitly for http-only sites. ### Output If the URL is already supported by a registered scraper, it reports that instead of fetching. Otherwise, prints the detected platform and package. ### Example ```bash $ fss detect https://www.teenmegaworld.net Platform: Teen Mega World Package: tmwutil ``` ``` -------------------------------- ### List tracked studios from database Source: https://github.com/anastylosis/fss/blob/master/docs/storage.md Lists what is tracked in the SQLite database. No SQL required. ```bash fss list-studios --db ```