### Project Setup and Workflow Source: https://github.com/allaboutapps/go-starter/wiki/FAQ This snippet outlines the initial steps for setting up a new project by forking the go-starter template. It includes commands for starting Docker and setting the module name, followed by a description of the iterative development workflow. ```go Fork the go-starter project. Start docker-compose: `./docker-helper --up` Set the module-name: `make set-module-name` ``` ```APIDOC Development Workflow: The development process follows an iterative approach, typically done per feature/epic, integrating continuous testing (SCRUM/sprints). Steps include API specification (Swagger-first) and ERM specification/migrations (SQL-first). ``` -------------------------------- ### Go Modules Installation Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Provides instructions on how to install additional Go modules, both for use as a CLI tool and as a library within the project. ```go // To install as a CLI tool: go install github.com/some/cli-tool@latest // To install as a library: go get github.com/some/library@latest ``` -------------------------------- ### Install System Dependencies (Dockerfile) Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Demonstrates how to install system-level dependencies using apt-get within a Dockerfile. It highlights mandatory installations and those for development tooling. ```bash RUN apt-get update && apt-get install -y --no-install-recommends \ git \ curl \ # -- START MANDADORY -- # Add mandatory system dependencies here # -- END MANDADORY -- && rm -rf /var/lib/apt/lists/* ``` ```bash # Install CLI-only tools for development RUN curl -sSL https://example.com/pgFormatter | sh \ && curl -sSL https://example.com/gotestsum | sh \ && curl -sSL https://example.com/golangci-lint | sh \ && curl -sSL https://example.com/goswagger | sh ``` -------------------------------- ### Go Module Installation and Management Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Explains how to install and manage Go modules. It highlights that direct 'go get' is usually unnecessary and modules are managed via imports and `go mod tidy`. CLI tools are managed via `tools.go` and `make tools`. ```go // To install additional go modules, typically you don't need to explicitly install any go module by `go get`, // just *use the module by importing* it in a *.go file. // Be aware that any installed go modules that are *not used* within your project will be automatically wiped by `go mod tidy`, // which is executed as part of our make pipelines (as part of `make init`, `make all` and `make tools`). // For CLI-only tools that you need to import somewhere, use the `/tools.go` file. // `make tools` (as part of `make init` and `make all`) automatically installs tooling CLIs from this file. ``` -------------------------------- ### Docker Build and Run Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Instructions for building a Docker image for the go-starter service and running it. The default command starts the server with database migrations. ```bash docker build . -t # Default entrypoint and command is `/app/app server --migrate`. docker run ``` -------------------------------- ### VSCode Go Extension Setup Source: https://github.com/allaboutapps/go-starter/blob/master/README-go-starter.md After attaching to the development container with VSCode, run this command via the command palette to automatically install all necessary Go-related extensions for enhanced development. ```bash Run `CMD+SHIFT+P` `Go: Install/Update Tools` **after** attaching to the container with VSCode ``` -------------------------------- ### Docker Run Commands Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Provides examples of how to run the Docker image with various commands for help, probing, database migrations, seeding, environment inspection, and server startup. It also mentions attaching environment variables. ```bash docker run help docker run probe readiness -v docker run db migrate docker run db seed docker run env docker run server docker run server --migrate docker run server --probe --migrate --seed docker run server -pms # Attaching environment variables: docker run --env-file docker run -e = ``` -------------------------------- ### Cobra CLI Integration Source: https://github.com/allaboutapps/go-starter/blob/master/cmd/README.md Details on how the project manages its applications using the Cobra library. It highlights that `cobra-cli` is installed in the Dockerfile and provides links to Cobra documentation for getting started and command management. ```APIDOC Application Management with Cobra: This project utilizes the Cobra library for managing command-line applications. Dependencies: - cobra: https://github.com/spf13/cobra - cobra-cli: https://github.com/spf13/cobra-cli (installed in Dockerfile) Resources: - Cobra Getting Started: https://github.com/spf13/cobra#getting-started - Cobra CLI README: https://github.com/spf13/cobra-cli/blob/main/README.md#add-commands-to-a-project - Project Layout: https://github.com/golang-standards/project-layout/tree/master/cmd ``` -------------------------------- ### WSL2 Setup and VSCode Integration Source: https://github.com/allaboutapps/go-starter/wiki/FAQ This snippet demonstrates the steps to set up WSL2 on Windows, integrate it with Docker Desktop, and launch VSCode within the WSL2 environment for containerized development. It includes commands for cloning a repository, navigating into it, and opening it in VSCode, followed by instructions to run the development environment using a helper script. ```bash # wsl # ensure you are *not* within /c/mnt, instead at ~! pwd # clone the repo (requires apt-get install git) git clone https://github.com/allaboutapps/go-starter.git my-service cd my-service # opens VSCode in the Linux home connected through the WSL extension code . # Now select reopen in container in VSCode or run `./docker-helper --up` directly... ``` -------------------------------- ### Docker Run Command Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Shows how to run the Docker container created from the project's Docker image. It includes examples for port mapping and volume mounting. ```bash # Run the Docker container: docker run -p 8080:8080 --name my-app your-image-name:tag # Example with volume mounting for configuration: docker run -p 8080:8080 -v $(pwd)/config:/app/config your-image-name:tag ``` -------------------------------- ### Serve CPU Profile via HTTP Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Starts a local web server on `0.0.0.0:8080` to visualize a 30-second CPU profile from a remote Go application. This is useful for remote profiling. ```go go tool pprof -http=0.0.0.0:8080 'https://YOUR-TARGET-SERVER/debug/pprof/profile?seconds=30&mgmt-secret=mgmtpass' ``` -------------------------------- ### Docker Compose Commands Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Provides essential bash commands for managing the Docker Compose environment. This includes starting the services, executing commands within a running container, and handling potential conflicts with other Docker Compose setups. ```bash # To start the services: docker-compose -f docker-compose.prod.yml up # To execute commands within the running service container: docker-compose -f docker-compose.prod.yml exec service-prod sh # If running alongside the main docker-compose.yml, halt the main one: ./docker-helper --halt ``` -------------------------------- ### Running the Go Service Source: https://github.com/allaboutapps/go-starter/blob/master/README-go-starter.md Steps to build and run the Go service locally. This includes building the executable, probing readiness, migrating the database, seeding data, and starting the server. It also shows a shortcut command to perform multiple steps at once. ```bash development@94242c61cf2b:/app$ # inside your development container... # First ensure you have a fresh `app` executable available make build # Check if all requirements for becoming are met (db is available, mnt path is writeable) app probe readiness -v # Migrate up the database app db migrate # Seed the database (if you have any fixtures defined in `/internal/data/fixtures.go`) app db seed # Start the locally-built server app server # Now available at http://127.0.0.1:8080 # You may also run all the above commands in a single command app server --probe --migrate --seed # or `app server -pms` ``` -------------------------------- ### PostgreSQL Database Configuration Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Explains the rationale behind using separate PostgreSQL databases for `spec`, `integresql_*`, and `development` during local development. This setup aids in testing and debugging. ```sql -- `spec` database: Used for schema definition and initial setup. -- `integresql_*` databases: Dynamically created for integration tests. -- `development` database: The primary database for local development. ``` -------------------------------- ### Refactoring Cobra Command Setup in Go Source: https://github.com/allaboutapps/go-starter/wiki/Migration-Guide-Command-Restructuring Demonstrates the refactoring of a Cobra command setup from an older `func init()` pattern to a new `func new...() *cobra.Command` pattern. This change is part of the project's restructuring to improve organization and maintainability. ```go type LivenessFlags struct { Verbose bool } func newLiveness() *cobra.Command { var flags LivenessFlags cmd := &cobra.Command{ Use: "liveness", Short: "Runs liveness probes", Run: func(_ *cobra.Command, _ []string) { livenessCmdFunc(flags) }, } cmd.Flags().BoolVarP(&flags.Verbose, verboseFlag, "v", false, "Show verbose output.") return cmd } ``` -------------------------------- ### WSL2 Configuration (`.wslconfig`) Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Example configuration for the `.wslconfig` file, used to manage WSL2 settings such as memory allocation. This helps in optimizing performance by allocating appropriate resources to the WSL2 environment. ```APIDOC WSL2 Configuration (`.wslconfig`): [wsl2] # Assignable memory in GB memory=8GB # Number of processors processors=4 # Swap size in GB swap=8GB # Swap file path swapFile=C:\\temp\\wsl-swap.vhdx ``` -------------------------------- ### Docker Helper Script Commands Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Provides commands for managing the Docker-Compose based local development workflow. Includes starting, stopping, rebuilding, and destroying Docker environments. ```bash # you may attach to the development container through multiple shells, it's always the same command ./docker-helper.sh --up # if you ever need to halt the docker-compose env (without deleting your projects' images & volumes) ./docker-helper.sh --halt # if you ever change something in the Dockerfile and require a rebuild of the service image only ./docker-helper.sh --rebuild # if you ever need to wipe ALL docker traces (will delete your projects' images & volumes) ./docker-helper.sh --destroy ``` -------------------------------- ### Profiling Go Service with pprof Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Instructions on enabling and using the pprof tool to inspect performance profiles of a running go-starter service. It covers enabling the pprof endpoint, managing authentication, and provides an example of how to use the `go tool pprof` command. ```bash # ----------------------- # Inspect profiling results **within the CLI**: # Note: If you want to try out profiling your application within your development container, simply exchange "YOUR-TARGET-SERVER" with "localhost:8080". # The pprof tool to look at the heap profile: go tool pprof "https://YOUR-TARGET-SERVER/debug/pprof/heap?mgmt-secret=mgmtpass" ``` -------------------------------- ### CLI Version Output Example Source: https://github.com/allaboutapps/go-starter/blob/master/CHANGELOG-go-starter.md Shows the expected output format when running the application with the version flag. ```bash app -v allaboutapps.dev/aw/go-starter @ 19c4cdd0da151df432cd5ab33c35c8987b594cac (2021-03-11T15:42:27+00:00) ``` -------------------------------- ### Installing Go Tools in VSCode Container Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Steps to install necessary Go tools within the VSCode development container. This is typically prompted automatically after the container spins up, or can be manually triggered via the command palette. ```plaintext When you are finally inside the container, you should be automatically asked to install all additional go tools into the container. This will only be required after container rebuilds (extension installs are cached). If you are not explicitly asked, open the command palette (`CMD+SHIFT+P`) and choose `Go: Install/Update Tools`. ``` -------------------------------- ### VSCode Remote - Containers Setup Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Instructions for setting up the VSCode development environment using the Remote - Containers extension. This involves opening the project in VSCode and using the 'Reopen in Container' command or the bottom-left corner button. ```plaintext Open the [go-starter](https://github.com/allaboutapps/go-starter) project (or your go-starter fork) in VSCode (by running `code .` in the project root - on Windows do this in your WSL2 connected terminal) and either: * hit `Remote-Containers: Reopen in Container` through the command palette (`CMD+SHIFT+P`) * **or** click on the yellow button (bottom-left corner). ``` -------------------------------- ### Seed Data Insertion Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Explains how to automatically insert seed data into the database, likely during the initial setup or deployment process. This ensures the application has necessary starting data. ```APIDOC Automatic Seed Data Insertion: - Implement seed data scripts that run after migrations. - Use a dedicated seeding tool or framework feature. - Ensure seed data is idempotent to allow reruns without duplication. ``` -------------------------------- ### Makefile Targets Overview Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Provides an overview of common and all available targets in the Makefile. Use 'make help' for common targets and 'make help-all' for all targets. Targets include build, clean, test, init, and more. ```bash make help usage: make note: use 'make help-all' to see all make targets. all Runs all of our common make targets: clean, init, build and test. build Default 'make' target: sql, swagger, go-generate, go-format, go-build and lint. clean Cleans ./tmp and ./api/tmp folder. force-module-name Overwrite occurrences of 'allaboutapps.dev/aw/go-starter' with current go module-name. get-embedded-modules Prints embedded modules in the compiled bin/app. get-licenses Prints licenses of embedded modules in the compiled bin/app. get-module-name Prints current go module-name (pipeable). git-merge-go-starter Merges upstream go-starter master into current HEAD. go-test-print-slowest Print slowest running tests (must be done after running tests). help-all Show all make targets. help Show common make targets. info Prints info about spec db, handlers, and go.mod updates, module-name and current go version. init Runs make modules, tools and tidy. lint Runs golangci-lint and make check-* set-module-name Wizard to set a new go module-name. sql-drop-all Wizard to drop ALL databases: spec, development and tracked by integresql. sql-reset Wizard to drop and create our development database. sql Runs sql format, all sql related checks and finally generates internal/models/*.go. swagger Runs make swagger-concat and swagger-server. test-by-name Run tests, output by testname, print coverage. test Run tests, output by package, print coverage. watch-sql Watches *.sql files in /migrations and runs 'make sql-regenerate' on modifications. watch-swagger Watches *.yml|yaml|gotmpl files in /api and runs 'make swagger' on modifications. watch-tests Watches *.go files and runs package tests on modifications. ``` -------------------------------- ### Serve Execution Trace via HTTP Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Collects a 30-second execution trace and serves it via a local web server on `0.0.0.0:8080` for analysis. Uses 'wget' to download the trace and 'go tool trace' to serve it. ```go wget -O trace.out "https://YOUR-TARGET-SERVER/debug/pprof/trace?seconds=30&mgmt-secret=mgmtpass" go tool trace -http=0.0.0.0:8080 trace.out ``` -------------------------------- ### Deploying with Docker Compose Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Provides guidance on deploying the application using `docker-compose`. This involves defining services, networks, and volumes in a `docker-compose.yml` file. ```yaml # Example docker-compose.yml: # version: '3.8' # services: # app: # image: your-image-name:tag # ports: # - "8080:8080" # depends_on: # - db # db: # image: postgres:latest # environment: # POSTGRES_DB: mydb # POSTGRES_USER: user # POSTGRES_PASSWORD: password ``` -------------------------------- ### Refactor Migration Logic and Cobra Setup Source: https://github.com/allaboutapps/go-starter/blob/master/CHANGELOG-go-starter.md Refactors the migration application logic to apply missing migrations individually and print information about the applied migration. Additionally, the Cobra command setup structure and flag parsing are refactored using struct-based flags. This change is breaking for subcommands attached to existing commands. ```go // Refactor migration application logic to apply each missing migration individually // Refactor the cobra cmd setup structure and the flag parsing logic using struct-based flags. // BREAKING: The restructuring breaks additionally created subcommands attached to the existing commands. ``` -------------------------------- ### Building and Testing the Project Source: https://github.com/allaboutapps/go-starter/blob/master/README-go-starter.md Commands to build and test the go-starter project. This includes running linters, tests, and building the executable. Assumes a Unix-like environment. ```Shell make build make test make lint ``` -------------------------------- ### Go Test Database Helper with IntegreSQL Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Demonstrates the public interface for obtaining and using a PostgreSQL database managed by IntegreSQL. It highlights the internal process of computing a hash for template database caching and recreation. ```go // --- // PUBLIC: This is the test helper you will use, maybe via WithTestServer // --- // Simply get and use a new PostgreSQL database based on the template database WithTestDatabase(t *testing.T, closure func(db *sql.DB)){ // use the db (test database)... } // -------------------------------------------------------------------------- // --- // PRIVATE: This is what (high level) happens internally in the test_database.go file // --- // 1. // We will compute a hash over the following dirs/files migDir = filepath.Join(pUtil.GetProjectRootDir(), "/migrations") fixFile = filepath.Join(pUtil.GetProjectRootDir(), "/internal/test/fixtures.go") selfFile = filepath.Join(pUtil.GetProjectRootDir(), "/internal/test/test_database.go") h, err := util.GetTemplateHash(migDir, fixFile, selfFile) // 2. // A new PostgreSQL template database is created (via the IntegreSQL client) // OR we will reuse the previous if we already have that exact same hash cached. client.SetupTemplateWithDBClient(ctx, hash, func(db *sql.DB) { /**/ }) // 2.1 // We have a new PostgreSQL template database?! We need to migrate and seed it with our fixtures! err := applyMigrations(t, db) err = insertFixtures(ctx, t, db) // 3. // A new test database is created from the template (speedy) and passed in the closure from WithTestDatabase // (yes, you get a new database each time you call WithTestDatabase, any modifications will stay in the assigned database in isolation) testDatabase, err := client.GetTestDatabase(ctx, hash) connectionString := testDatabase.Config.ConnectionString() db, err := sql.Open("postgres", connectionString) closure(db) ``` -------------------------------- ### Sample Users Table Query Source: https://github.com/allaboutapps/go-starter/wiki/2022-02-28 Retrieves the id and username from the users table to display sample data. ```SQL -- Sample users table SELECT id, username FROM users; -- Result: -- id username -- 76a79a2b-fbd8-45a0-b35b-671a28a87acf "user2@example.com " -- d9c0dee9-239e-4323-979a-a5354d289627 " user 2@example.com" -- f6ede5d8-e22a-4ca5-aa12-67821865a3e5 "uSSer1@example.com" ``` -------------------------------- ### Go gogenerate stringer Example Source: https://github.com/allaboutapps/go-starter/wiki/Random-training-material Provides a sample usage of the `stringer` tool, which generates String() methods for enumerated types. ```go //go:generate stringer -type=Pill package main type Pill int const ( Placebo Pill = iota Aspirin Ibuprofen Paracetamol ) // After running 'go generate', a String() method will be available for the Pill type. // Example usage: // pill := Aspirin // fmt.Println(pill.String()) // Output: Aspirin ``` -------------------------------- ### VSCode GolangCI-Lint Configuration Source: https://github.com/allaboutapps/go-starter/blob/master/CHANGELOG-go-starter.md Ensures that VSCode utilizes the project's installed version of `golang-cilint` for consistent code linting across the development environment. ```bash # Force VSCode to use our installed version of golang-cilint ``` -------------------------------- ### Docker Build Configuration Source: https://github.com/allaboutapps/go-starter/blob/master/CHANGELOG-go-starter.md Details changes in the Dockerfile, including Go version updates, installation of development tools, and configuration of linters and build tools. ```dockerfile FROM golang:1.15.7-alpine AS builder # Install build dependencies RUN apk add --no-cache icu-devtools # Install Go tools RUN go install gotestsum.io/gotestsum@v1.6.1 RUN go install github.com/golangci/golangci-lint/cmd/golangci-lint@v1.36.0 RUN go install github.com/go-swagger/go-swagger/cmd/swagger@v0.26.0 # Copy go.mod and go.sum, download dependencies COPY go.mod go.sum ./ RUN go mod download # Copy the rest of the application code COPY . . # Build the application RUN make build # ... rest of Dockerfile ... ``` -------------------------------- ### Deploying with Kubernetes Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Offers guidance on deploying the application to Kubernetes. This typically involves creating Kubernetes manifests (Deployments, Services, etc.). ```yaml # Example Kubernetes Deployment manifest: # apiVersion: apps/v1 # kind: Deployment # metadata: # name: go-starter-app # spec: # replicas: 3 # selector: # matchLabels: # app: go-starter # template: # metadata: # labels: # app: go-starter # spec: # containers: # - name: app # image: your-image-name:tag # ports: # - containerPort: 8080 ``` -------------------------------- ### Makefile Commands Source: https://github.com/allaboutapps/go-starter/blob/master/README-go-starter.md Provides a set of Makefile commands for common development tasks. 'make all' is a shortcut for initialization, building, info, and testing. 'make help' displays all available targets. ```bash # Shortcut for make init, make build, make info and make test make all # Print all available make targets make help ``` -------------------------------- ### Profiling Running Services with pprof Source: https://github.com/allaboutapps/go-starter/blob/master/CHANGELOG-go-starter.md Enables remote profiling of a running service using the `pprof` handlers. Refer to the FAQ for detailed setup instructions. ```bash make pprof ``` -------------------------------- ### Internal API Implementation Source: https://github.com/allaboutapps/go-starter/blob/master/internal/README.md Details the purpose of the `/internal/api` directory, which holds API implementations, including handlers, and general server, router, and middleware setup. ```go package api // Holds API implementations (/internal/api/handlers) and general server, router and middleware setup. ``` -------------------------------- ### Database Migration Commands Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Compares different commands for applying database migrations: `sql-migrate up`, `app db migrate`, and `app server --migrate`. It clarifies when to use each command based on the deployment environment (local vs. remote). ```bash # Locally (stage `development` or `builder`): # Use `sql-migrate up` or `app db migrate` # Remote (stage `app`): # Use `app server --migrate` ``` -------------------------------- ### Go Project Structure Guidelines Source: https://github.com/allaboutapps/go-starter/blob/master/cmd/README.md Guidelines for organizing Go projects, specifically detailing the purpose of `/cmd`, `/pkg`, and `/internal` directories. It advises placing main applications in `/cmd` and reusable code in `/pkg` or `/internal` based on reusability intentions. ```go Project Structure: /cmd: Main applications for the project. /pkg: Reusable code that can be imported by other projects. /internal: Code that is not intended for reuse by others. ``` -------------------------------- ### Find Potential Username Collisions Source: https://github.com/allaboutapps/go-starter/wiki/2022-02-28 Identifies potential username collisions after trimming whitespace and converting to lowercase. It groups users by their cleaned username and flags groups with more than one user, indicating a collision. ```SQL -- Find whitespace/uppercase collisions in the username field. -- "If we trim + lowercase the above usernames, will there be collisions with other / between usernames" -- Returns the count, the valid username and all collisions. -- NOTE: If this query does not yield any results, you should be safe to automatically rename all user.username, as there are no collisions SELECT count(id), LOWER(REPLACE(username, ' ', '')) AS valid_username, jsonb_agg(jsonb_build_object(id, username)) AS collisions FROM users GROUP BY valid_username HAVING count(id) > 1; -- Result: -- count valid_username collisions -- 2 "user2@example.com" [{"76a79a2b-fbd8-45a0-b35b-671a28a87acf": " user2@example.com"}, {"d9c0dee9-239e-4323-979a-a5354d289627": "user 2@example.com"}] ``` -------------------------------- ### Executing Go Scripts with Build Tags Source: https://github.com/allaboutapps/go-starter/blob/master/scripts/README.md Demonstrates how to correctly execute Go files that have the `//go:build scripts` build tag. It shows the proper command-line syntax using `go run -tags scripts` and contrasts it with an incorrect attempt that misses the build tag, leading to an error. ```bash # Works go run -tags scripts scripts/main.go # go-starter development scripts # Utility commands while developing go-starter based projects. # Works (same as above) gsdev # go-starter development scripts # Utility commands while developing go-starter based projects. # Misses build tag "scripts" go run scripts/main.go # package command-line-arguments # imports allaboutapps.dev/aw/go-starter/scripts/cmd: build constraints exclude all Go files in /app/scripts/cmd ``` -------------------------------- ### Internal Test Utilities Source: https://github.com/allaboutapps/go-starter/blob/master/internal/README.md Describes the `/internal/test` directory, which contains code related to test setup and general testing utility functions. It also holds test database fixture data, such as in `/internal/test/fixtures.go`. ```go package test // Test-Setup related code and general testing utility functions. // Holds test db fixture data (/internal/test/fixtures.go). ``` -------------------------------- ### Common Development Commands Source: https://github.com/allaboutapps/go-starter/blob/master/README-go-starter.md A collection of useful `make` commands for Go development, including displaying help, running all build/test steps, initializing dependencies, rebuilding after changes, and executing tests. ```bash development@94242c61cf2b:/app$ # inside your container... # Print all available make targets make help # Shortcut for make init, make build, make info and make test make all # Init install/cache dependencies and install tools to bin make init # Rebuild only after changes to files (generate, format, build, lint) make # Execute all tests make test ``` -------------------------------- ### Project Setup and Module Naming Source: https://github.com/allaboutapps/go-starter/blob/master/README-go-starter.md Instructions on how to set the project module name for a new Go project based on the go-starter template. This is crucial for managing dependencies and project structure. ```Go go mod init ``` -------------------------------- ### Dockerfile Base Image Recommendations Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Guidance on selecting Docker base images for minimal Debian environments. Distroless images offer greater security by omitting shells and package managers, while Debian slim images provide more flexibility for complex build processes or runtime needs. ```dockerfile FROM distroless/base-debian10 # Use this unless you need a shell or apt-get FROM distroless/base-debian10:debug # Use this if you need a shell at /sh in production FROM debian:buster-slim # Use this if you need RUN commands, apt-get, or multi-command CMD in the app stage ``` ```dockerfile FROM distroless/static-debian10 # Use this for maximum security, requires CGO_ENABLED=0 builds ``` -------------------------------- ### JSON Property Naming Convention Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Follows the Google JSON Style Guide, preferring camelCase with a lowercase first letter for all payloads. An exception is made for third-party specifications like OAuth2 which mandate snake_case. ```json { "thisPropertyIsAnIdentifier": "identifier value" } ``` -------------------------------- ### SQLBoiler Documentation References Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Provides links to various resources for understanding and using SQLBoiler, a Go ORM code generator. Includes YouTube tutorials and README files for different versions and advanced usage. ```APIDOC SQLBoiler Documentation: - Getting Started videos on YouTube: https://github.com/volatiletech/sqlboiler#videos - README: https://github.com/volatiletech/sqlboiler - Advanced Queries and Relationships (youtube): https://www.youtube.com/watch?v=iiJuM9NR8No - What's New in v3 (youtube): https://www.youtube.com/watch?v=-B-OPsYRZJA - Getting Started (youtube): https://www.youtube.com/watch?v=y5utRS9axfg - Old (v2): How to get started (youtube): https://www.youtube.com/watch?v=fKmRemtmi0Y ``` -------------------------------- ### Troubleshooting VSCode Go Extensions Source: https://github.com/allaboutapps/go-starter/wiki/FAQ A guide to resolving issues with VSCode Go extensions when working within the Remote - Containers environment. It involves removing the local 'bin' folder and reinstalling the Go tools. ```plaintext 1. Remove the `/bin` folder (don't worry, it will be regenerated)! 2. Open the command palette in VSCode (`CMD+SHIFT+P`) and choose `Go: Install/Update Tools`. 3. Let all go-related extensions finish their reinstall (binaries will reappear within `/bin`) 4. Reload VSCode (cross your fingers). Extension-related problems within VSCode should now be gone. 5. Don't forget to run `make all` again, your compiled `app` binary is also written into the `/bin` folder. ``` -------------------------------- ### Dockerfile Base Image Choice Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Discusses the choice between `distroless/base` and `debian:buster-slim` for the Dockerfile `app` stage. It highlights considerations for image size and security. ```dockerfile # For the Dockerfile `app` stage, consider: # - distroless/base: Smaller, more secure, fewer tools. # - debian:buster-slim: More tools available, larger image. ``` -------------------------------- ### Generate Username Update Statements Source: https://github.com/allaboutapps/go-starter/wiki/2022-02-28 Generates SQL UPDATE statements to clean usernames by trimming whitespace and converting to lowercase. It also identifies the original username and the new, cleaned username for each affected user. ```SQL -- Find whitespaces and uppercase letters in username field. -- Returns the id, invalid (current) username, the new valid username and a sample query to manually execute the UPDATE. -- -- NOTE: If you get duplicate key errors executing a returned update_query: -- * you may have a whitespace/uppercase collision in the username -- * check the query regarding collisions below SELECT id, username AS invalid_username, LOWER(REPLACE(username, ' ', '')) AS valid_username, FORMAT('UPDATE users SET username = ''%s'' WHERE id = ''%s'';', LOWER(REPLACE(username, ' ', '')), id) AS update_query FROM users WHERE LOWER(REPLACE(username, ' ', '')) != username; -- Result: -- id invalid_username valid_username update_query -- 76a79a2b-fbd8-45a0-b35b-671a28a87acf "user2@example.com " "user2@example.com" UPDATE users SET username = 'user2@example.com' WHERE id = '76a79a2b-fbd8-45a0-b35b-671a28a87acf'; -- d9c0dee9-239e-4323-979a-a5354d289627 " user 2@example.com" "user2@example.com" UPDATE users SET username = 'user2@example.com' WHERE id = 'd9c0dee9-239e-4323-979a-a5354d289627'; -- f6ede5d8-e22a-4ca5-aa12-67821865a3e5 "uSSer1@example.com" "usser1@example.com" UPDATE users SET username = 'usser1@example.com' WHERE id = 'f6ede5d8-e22a-4ca5-aa12-67821865a3e5'; ``` -------------------------------- ### Deployment Flow Overview Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Provides a high-level overview of the project's deployment flow, from development to production. This helps understand the stages and processes involved in releasing the application. ```APIDOC Deployment Flow: 1. Development: Local setup with Docker and IntegreSQL. 2. Build: Compile application and create Docker image (multi-stage build). 3. Testing: Run unit, integration, and blackbox tests. 4. Staging: Deploy to a staging environment for final validation. 5. Production: Deploy to the production environment, potentially using CI/CD pipelines. 6. Monitoring: Continuous monitoring of application health and performance. ``` -------------------------------- ### sql-migrate Documentation Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Information on sql-migrate, a Go tool for SQL schema migrations. Highlights its support for plain SQL migrations and ease of integration into testing and deployment workflows. ```APIDOC sql-migrate Documentation: - Tool: https://github.com/rubenv/sql-migrate - Approach: SQL-first, auto-wrapped by a transaction. ``` -------------------------------- ### CI/CD Build and Test Workflow Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Illustrates a GitHub Actions workflow for building and testing the Go project, including a mandatory git diff check for generated files. ```yaml name: Build and Test on: [push, pull_request] jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - name: Set up Go uses: actions/setup-go@v4 with: go-version: '1.21' - name: Build run: go build -v ./... - name: Test run: go test -v ./... - name: Check generated files diff run: git diff --exit-code ``` -------------------------------- ### Running the Application Source: https://github.com/allaboutapps/go-starter/blob/master/README-go-starter.md Instructions for running the go-starter application locally. This typically involves setting environment variables and executing the compiled binary. ```Shell export APP_ENV=development ./bin/go-starter ``` -------------------------------- ### Docker Compose Development Environment Source: https://github.com/allaboutapps/go-starter/blob/master/README-go-starter.md Sets up a local Go service development environment using Docker Compose and VSCode devcontainers. This environment works across Linux, macOS, and Windows, eliminating the need for a local Go toolchain installation. ```bash # Clone your new repository, cd into it, then easily start the docker-compose dev environment through our helper ./docker-helper.sh --up ``` -------------------------------- ### Database Migration and Seeding Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Details the different approaches for database migrations and seeding data depending on the environment. Locally, `sql-migrate` can be used. In the production 'app' stage, use `app db migrate` or `app server --migrate`. Seeding involves adding `--seed` and updating fixture functions. ```bash # Locally (development/builder stage): # Use sql-migrate to migrate up the development database. # Example: sql-migrate up # Remotely (app stage): # Use baked-in app executable commands. # Example: app db migrate # Example: app server --migrate # Automatically migrate and seed: # Change CMD to: CMD ["server", "--migrate", "--seed"] # Update test fixtures: # internal/test/test_database.go -> ApplyTestFixtures (upsert) # Update live fixtures: # cmd/db_seed.go -> applyFixtures (upsert) ``` -------------------------------- ### Docker Compose Services Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Details the services included in the docker-compose.yml file, such as PostgreSQL for database, Swagger-UI for API documentation visualization, Mailhog for email testing, and IntegreSQL for PostgreSQL integration testing. ```docker services: postgres: image: postgres:13 # ... other postgres configurations swagger-ui: image: swaggerapi/swagger-ui # ... other swagger-ui configurations mailhog: image: mailhog/mailhog # ... other mailhog configurations integresql: image: ghcr.io/allaboutapps/integresql:latest # ... other integresql configurations ``` -------------------------------- ### Echo Framework Documentation Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Links to the official documentation for the Echo web framework, emphasizing its high performance and minimalist design for Go applications. ```APIDOC Echo Framework Documentation: - Guide: https://echo.labstack.com/guide ``` -------------------------------- ### Resetting Databases with make sql-drop-all Source: https://github.com/allaboutapps/go-starter/wiki/FAQ This command sequence demonstrates how to drop all existing databases within the development environment and provides instructions for reinitialization. It's crucial for resetting the database state. ```bash development@94242c61cf2b:/app$ # inside your development container... make sql-drop-all DROP ALL: DROP DATABASE "development"; DROP DATABASE "spec"; DROP DATABASE "integresql_template_*"; DROP DATABASE "integresql_test_*"; ... Are you sure? [y/N] y Resetting integresql... Drop databases... ... Done. Please run 'make sql-reset && make sql-spec-reset && make sql-spec-migrate' to reinitialize. ``` -------------------------------- ### Docker Desktop Resource Allocation Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Recommended resource allocation settings for Docker Desktop on Windows and macOS to prevent performance issues and errors related to insufficient memory or disk space. This includes recommended vCPUs, memory, and disk space. ```APIDOC Docker Desktop Resource Settings: vCPUs: Minimum 4 Memory: Minimum 6GB Disk Space: Minimum 40GB ``` -------------------------------- ### Go Test Package Naming Conventions Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Explains the convention for naming test files and packages in Go. It covers the use of `package pkg` for regular files and `package pkg_test` for blackbox testing to ensure isolation. ```go // For regular files: package pkg // For blackbox testing: package pkg_test ``` -------------------------------- ### Makefile Commands Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Explains the various commands available in the project's Makefile, including build, test, lint, and utility commands like `force-module-name` for updating module names. ```makefile build: @echo "Building the project..." go build -o ./bin/app ./ lint: @echo "Linting the code..." golangci-lint run force-module-name: @echo "Forcing module name..." ./docker-helper.sh --force-module-name $(MODULE_NAME) ``` -------------------------------- ### Dockerfile Build Stages Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Explains the purpose and usage of the three Dockerfile build stages: 'development' for a full local environment, 'builder' for CI environments to build binaries, and 'app' for a minimal, production-ready image. ```dockerfile # Development Stage: Full local development environment, uses golang image. # Builder Stage: Depends on development, builds Go binaries, useful in CI. # App Stage: Production-ready, minimal, copies compiled binaries from builder. # Example Dockerfile structure (conceptual): # FROM golang:1.20 as development # ... # FROM development as builder # ... # FROM alpine:latest as app # COPY --from=builder /app/binary /app/binary # ENTRYPOINT ["/app/binary"] ``` -------------------------------- ### Post Merge Steps Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Essential steps to perform after merging go-starter changes, including fixing imports, resolving conflicts, and rebuilding the development environment. ```bash # Inside your development container: make force-module-name git status # Review CHANGELOG for breaking changes make build ./docker-helper.sh --rebuild make all # Commit changes if everything works ``` -------------------------------- ### Server Configuration Environment Variables Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Details how server configuration is managed solely through environment variables. Refers to `DefaultServiceConfigFromEnv()` in `internal/config/server_config.go` for a complete list of available variables and their usage. ```APIDOC Server Configuration: Mechanism: Solely through environment variables. Reference: See `DefaultServiceConfigFromEnv()` in `internal/config/server_config.go` for a complete overview of available environment variables and their binding. Policy: Intentionally the sole mechanism for configuration; no additional means (e.g., viper) will be added to maintain simplicity. ``` -------------------------------- ### Go `WithTransaction` vs. `db.BeginTx` Source: https://github.com/allaboutapps/go-starter/wiki/FAQ Compares the use of a custom `WithTransaction` helper function against the standard Go `database/sql` methods `db.BeginTx`, `db.Commit`, and `db.Rollback`. It clarifies the benefits and use cases for each. ```go // `WithTransaction` provides a convenient abstraction. // `db.BeginTx`, `db.Commit`, `db.Rollback` offer manual transaction control. ``` -------------------------------- ### SQL Formatting with pgFormatter Source: https://github.com/allaboutapps/go-starter/wiki/Random-training-material Demonstrates the use of pgFormatter for enforcing proper SQL file formatting, applicable in environments like VSCode and Makefiles. ```bash # Example SQL file content that pgFormatter would format SELECT * FROM users WHERE id = 1; INSERT INTO products (name, price) VALUES ('gadget', 19.99); ```