### Quick Setup for Dgraph Development Source: https://github.com/dgraph-io/dgraph/blob/main/TESTING.md A streamlined setup process for both Linux and macOS. It automates dependency installation and builds the Dgraph binary. ```bash # Auto-install tool dependencies (gotestsum, ack, etc.) make setup # Build dgraph binary (automatically handles Linux binary on macOS) make install ``` -------------------------------- ### Setup Dgraph from source repository Source: https://github.com/dgraph-io/dgraph/blob/main/CONTRIBUTING.md Commands to clone the Dgraph repository, install dependencies, and build the binary. ```bash git clone https://github.com/dgraph-io/dgraph.git cd ./dgraph make setup # auto-install tool dependencies (gotestsum, ack, etc.) make install # build and install the dgraph binary ``` -------------------------------- ### Install gogo protobuf compiler plugin Source: https://github.com/dgraph-io/dgraph/blob/main/CONTRIBUTING.md Command to install the required protocol buffer compiler plugin for gRPC communication. ```bash go get -u github.com/gogo/protobuf/protoc-gen-gofast ``` -------------------------------- ### Install and Verify gotestsum Source: https://github.com/dgraph-io/dgraph/blob/main/TESTING.md Install the gotestsum tool, which is required for the test runner. Verify the installation by checking its version. ```bash go install gotest.tools/gotestsum@latest # Verify installation gotestsum --version ``` -------------------------------- ### Build and Install Dgraph Binary Source: https://github.com/dgraph-io/dgraph/blob/main/TESTING.md Build the Dgraph binary and install it to your GOPATH/bin directory. This is necessary for integration and upgrade tests. ```bash # Build and install Dgraph binary to $GOPATH/bin make install # Verify installation which dgraph # Should show $GOPATH/bin/dgraph dgraph version ``` -------------------------------- ### Setup Badger from source repository Source: https://github.com/dgraph-io/dgraph/blob/main/CONTRIBUTING.md Optional command to fetch the Badger source code if you intend to contribute to it. ```bash go get -t -v github.com/dgraph-io/badger ``` -------------------------------- ### Build and verify Dgraph installation Source: https://github.com/dgraph-io/dgraph/blob/main/CONTRIBUTING.md Commands to install the Dgraph binary and verify the installation by checking the version. ```sh $ make install Installing Dgraph... Commit SHA256: 15839b156e9920ca2c4ab718e1e73b6637b8ecec Old SHA256: 596e362ede7466a2569d19ded91241e457e665ada785d05a902af2c6f2cea508 Installed dgraph to /Users//go/bin/dgraph $ dgraph version Dgraph version : v24.0.2-103-g15839b156 Dgraph codename : dgraph Dgraph SHA-256 : 9ce738cd055dfebdef5d68b2a49ea4e062e597799498607dbd1bb618d48861a6 Commit SHA-1 : 15839b156 Commit timestamp : 2025-01-10 17:56:49 -0500 Branch : username/some-branch-that-im-on Go version : go1.22.12 jemalloc enabled : true For Dgraph official documentation, visit https://dgraph.io/docs. For discussions about Dgraph , visit https://discuss.dgraph.io. Licensed variously under the Apache Public License 2.0 and Dgraph Community License. © Istari Digital, Inc. ``` -------------------------------- ### Verify Integration Test Setup Source: https://github.com/dgraph-io/dgraph/blob/main/TESTING.md Build and run integration tests. Ensure Docker Desktop is running before execution. Navigate to the 't' directory. ```bash cd t && go build . && ./t --test=TestGQLSchema ``` ```bash make test TEST=TestGQLSchema ``` -------------------------------- ### Check and Auto-install Dgraph Dependencies Source: https://github.com/dgraph-io/dgraph/blob/main/TESTING.md This command checks for missing dependencies and automatically installs them. It's an alternative to 'make setup' if you only want to check and install. ```bash make check-deps AUTO_INSTALL=true ``` -------------------------------- ### Build and Install Dgraph from Source Source: https://github.com/dgraph-io/dgraph/blob/main/README.md Clone the repository and compile the Dgraph binary using the provided Makefile. ```bash git clone https://github.com/dgraph-io/dgraph.git cd dgraph make setup make install ``` -------------------------------- ### Install Latest Dgraph Binary Source: https://github.com/dgraph-io/dgraph/blob/main/dgraph/cmd/migrate/README.md Installs the latest Dgraph binary from source. This is the initial step before any migration or data loading. ```bash curl https://get.dgraph.io -sSf | bash ``` -------------------------------- ### Install ack using Homebrew Source: https://github.com/dgraph-io/dgraph/blob/main/TESTING.md Install the 'ack' utility using Homebrew. This tool is required by the Dgraph test framework. ```bash brew install ack ``` -------------------------------- ### Dgraph Upgrade Strategies Source: https://github.com/dgraph-io/dgraph/blob/main/TESTING.md These Go code examples show how to specify different upgrade strategies when calling the `c.Upgrade()` method. Choose the strategy based on your upgrade requirements, such as BackupRestore for common customer paths, InPlace for fast binary compatibility tests, or ExportImport for major version migrations. ```go c.Upgrade("local", dgraphtest.BackupRestore) ``` ```go c.Upgrade("local", dgraphtest.InPlace) ``` ```go c.Upgrade("local", dgraphtest.ExportImport) ``` -------------------------------- ### Start Dgraph Dump Server Source: https://github.com/dgraph-io/dgraph/blob/main/query/benchmark/README.md Starts the Dgraph server with a specific port for dumping data. ```shell dgraph -dumpsg dumpsg -port 8912 ``` -------------------------------- ### Verify Docker and Docker Compose Installation Source: https://github.com/dgraph-io/dgraph/blob/main/TESTING.md Confirm that Docker and Docker Compose are installed and meet the version requirements (Docker 20.10+ and Docker Compose v2). ```bash docker --version docker compose version ``` -------------------------------- ### Install Build Dependencies on Ubuntu Source: https://github.com/dgraph-io/dgraph/blob/main/README.md Update package lists and install essential build tools required for compiling Dgraph. ```bash sudo apt-get update sudo apt-get install build-essential ``` -------------------------------- ### Verify Go Installation Source: https://github.com/dgraph-io/dgraph/blob/main/TESTING.md Check if Go is installed on your system by running this command. Dgraph requires Go version 1.21 or higher. ```bash go version ``` -------------------------------- ### Build and Run Integration2 Tests Source: https://github.com/dgraph-io/dgraph/blob/main/TESTING.md Builds the local Dgraph binary using 'make install' and then runs integration tests with the 'integration2' build tag. Tests can be filtered by package or specific test name. ```bash # Build your local binary first make install # Run tests go test -v --tags=integration2 ./systest/integration2/ go test -v --tags=integration2 --run '^TestName$' ./pkg/ # With make: # Run all integration2 tests make test-integration2 # Run integration2 tests for a specific package make test TAGS=integration2 PKG=systest/vector # Run a specific integration2 test make test TAGS=integration2 PKG=systest/vector TEST=TestVectorSearch ``` -------------------------------- ### Start Default Dgraph Cluster Source: https://github.com/dgraph-io/dgraph/blob/main/TESTING.md Starts a default Dgraph cluster using docker-compose with a specified project name. Ensure you are in the correct directory before running. ```bash docker compose -f dgraph/docker-compose.yml -p mytest up -d ``` -------------------------------- ### Install Dgraph and Testing Framework on macOS Source: https://github.com/dgraph-io/dgraph/blob/main/t/README.md Installs Dgraph and builds the testing framework CLI on macOS. This process automatically handles cross-compilation for native and Docker-based tests. ```sh make install cd t && make check && go build . ./t --pkg= ``` -------------------------------- ### Get datagen Help Source: https://github.com/dgraph-io/dgraph/blob/main/graphql/testdata/datagen/README.md Display the help message for the datagen command-line tool to see available options and usage instructions. ```bash $ ./datagen --help ``` -------------------------------- ### Posting List Key-Value Mapping Source: https://github.com/dgraph-io/dgraph/blob/main/paper/dgraph.tex Example showing how multiple records with the same subject and predicate are condensed into a single key-value pair in Badger. ```text <0x01> <0xab> . <0x01> <0xbc> . <0x01> <0xcd> . ... key = value = <0xab, 0xbc, 0xcd, ...> ``` -------------------------------- ### Example Dataset Triples Source: https://github.com/dgraph-io/dgraph/blob/main/paper/dgraph.tex A representation of data stored as triples, where each predicate like 'lives-in' or 'eats' forms a distinct shard. ```text . . . ... . . ``` -------------------------------- ### Table-Driven Test Example in Go Source: https://github.com/dgraph-io/dgraph/blob/main/TESTING.md Demonstrates a table-driven test structure in Go, ideal for testing multiple scenarios within a single test function. It uses `t.Run` for clear subtest identification and `require` for assertions. ```go func TestConversion(t *testing.T) { tests := []struct { name string input Val output Val wantErr bool }{ {name: "string to int", input: Val{...}, output: Val{...}}, {name: "invalid type", input: Val{...}, wantErr: true}, } for _, tc := range tests { t.Run(tc.name, func(t *testing.T) { got, err := Convert(tc.input, tc.output.Tid) if tc.wantErr { require.Error(t, err) return } require.NoError(t, err) require.Equal(t, tc.output, got) }) } } ``` -------------------------------- ### Commit Algorithm for Dgraph Transactions Source: https://github.com/dgraph-io/dgraph/blob/main/paper/dgraph.tex This algorithm outlines the commit process where the Oracle checks for conflicts against the start timestamp and assigns a new commit timestamp if no conflicts are found. ```text Commit ($T_s, Keys$) each key $k \in Keys$ $lastCommit(k) > T_s$ $Propose(T_s \gets abort)$ $T_c \gets GetTimestamps(1)$ each key $k \in Keys$ $lastCommit(k) \gets T_c$ $Propose(T_s \gets T_c)$ ``` -------------------------------- ### Run Upgrade Tests with Make Source: https://github.com/dgraph-io/dgraph/blob/main/TESTING.md Execute tests tagged with 'upgrade' using the 'make test' command. ```bash make test TAGS=upgrade ``` -------------------------------- ### Example Safe Mutation Source: https://github.com/dgraph-io/dgraph/blob/main/dgraph/cmd/mcp/prompt.txt This is an example of a safe mutation to set data. Always confirm mutations before execution. ```dql mutation { set { _:a "Alice" . _:a "25" . } } ``` -------------------------------- ### Run Unit Tests with Make Source: https://github.com/dgraph-io/dgraph/blob/main/TESTING.md Execute unit tests using the 'make test' command with the 'unit' suite argument. ```bash make test SUITE=unit ``` -------------------------------- ### Verify Unit Tests Source: https://github.com/dgraph-io/dgraph/blob/main/TESTING.md Run a specific unit test or all unit tests using Go or make. Ensure you are in the correct directory. ```bash go test -v ./types/... -run TestConvert ``` ```bash make test-unit ``` -------------------------------- ### Benchmark Specific Package Source: https://github.com/dgraph-io/dgraph/blob/main/TESTING.md Run benchmark tests for the 'posting' package using the 'make test-benchmark' command. ```bash make test-benchmark PKG=posting ``` -------------------------------- ### Run Integration Tests with Make Source: https://github.com/dgraph-io/dgraph/blob/main/TESTING.md Execute integration tests using the 'make test' command with specific suite arguments. The default suite is 'integration'. ```bash make test SUITE=integration ``` -------------------------------- ### Check Dgraph Dependencies Without Installing Source: https://github.com/dgraph-io/dgraph/blob/main/TESTING.md Use this command to check for required dependencies without automatically installing them. It will report which dependencies are missing. ```bash make check-deps ``` -------------------------------- ### Run Integration2 Tests with Make Source: https://github.com/dgraph-io/dgraph/blob/main/TESTING.md Execute tests tagged with 'integration2' using the 'make test' command. ```bash make test TAGS=integration2 ``` -------------------------------- ### Display Testing Framework Help Source: https://github.com/dgraph-io/dgraph/blob/main/t/README.md Shows a list of all available flags and options for the Dgraph testing framework CLI. ```sh ./t --help ``` -------------------------------- ### Parsing HNSW Index Options Source: https://github.com/dgraph-io/dgraph/blob/main/tok/options/README.md Demonstrates how to parse and set HNSW index options using AllowedOptions and NewOptions. Ensure the option name and value are correctly provided. ```go val, err := allowedOpts.GetParsedOption("exponent", "6") if err != nil { return ErrBadOptionNoBiscuit } myAttributeIndexOpts.SetOpt("exponent", val) ``` ```go allowedOpts.PopulateOptions(pairs, myAttributeIndexOpts) ``` -------------------------------- ### Start Package-Specific Dgraph Cluster Source: https://github.com/dgraph-io/dgraph/blob/main/TESTING.md Starts a Dgraph cluster for a specific package using a dedicated docker-compose file and project name. This is useful for isolating test environments. ```bash docker compose -f systest/export/docker-compose.yml -p mytest up -d ``` -------------------------------- ### Run Core Tests with Make Source: https://github.com/dgraph-io/dgraph/blob/main/TESTING.md Execute core tests using the 'make test' command with the 'core' suite argument. ```bash make test SUITE=core ``` -------------------------------- ### Build datagen Tool Source: https://github.com/dgraph-io/dgraph/blob/main/graphql/testdata/datagen/README.md Build the datagen command-line tool using Go. Ensure you are in the directory containing the README file. ```bash $ go build ``` -------------------------------- ### Run System Tests with Make Source: https://github.com/dgraph-io/dgraph/blob/main/TESTING.md Execute system tests using the 'make test' command with the 'systest' suite argument. ```bash make test SUITE=systest ``` -------------------------------- ### Run Benchmark Tests Source: https://github.com/dgraph-io/dgraph/blob/main/TESTING.md Execute benchmark tests for all packages using Go. This command measures performance and memory usage. ```bash go test -bench=. -benchmem ./... ``` -------------------------------- ### Run Standalone Dgraph Cluster Source: https://github.com/dgraph-io/dgraph/blob/main/README.md Start a standalone Dgraph cluster using Docker with ports 8080 and 9080 exposed and a local volume mount. ```bash docker run -it -p 8080:8080 -p 9080:9080 -v ~/dgraph:/dgraph dgraph/standalone:latest ``` -------------------------------- ### Initial pprof Heap Analysis Source: https://github.com/dgraph-io/dgraph/blob/main/chunker/README.txt This output shows the top memory-consuming functions before optimization. It highlights `rdf.Parse` as a significant contributor to memory allocation. ```text go tool pprof --alloc_objects uidassigner heap.prof (pprof) top10 196427053 of 207887723 total (94.49%) Dropped 41 nodes (cum <= 1039438) Showing top 10 nodes out of 31 (cum >= 8566234) flat flat% sum% cum cum% 55529704 26.71% 26.71% 55529704 26.71% github.com/dgraph-io/dggraph/rdf.Parse 28255068 13.59% 40.30% 30647245 14.74% github.com/dgraph-io/dggraph/posting.(*List).getPostingList 20406729 9.82% 50.12% 20406729 9.82% github.com/zond/gotomic.newRealEntryWithHashCode 17777182 8.55% 58.67% 17777182 8.55% strings.makeCutsetFunc 17582839 8.46% 67.13% 17706815 8.52% github.com/dgraph-io/dggraph/loader.(*state).readLines 15139047 7.28% 74.41% 88445933 42.55% github.com/dgraph-io/dggraph/loader.(*state).parseStream 12927366 6.22% 80.63% 12927366 6.22% github.com/zond/gotomic.(*element).search 10789028 5.19% 85.82% 66411362 31.95% github.com/dgraph-io/dggraph/posting.GetOrCreate 9453856 4.55% 90.37% 9453856 4.55% github.com/zond/gotomic.(*hashHit).search 8566234 4.12% 94.49% 8566234 4.12% github.com/dgraph-io/dggraph/uid.stringKey ``` -------------------------------- ### Run Specific Test with Custom Variables Source: https://github.com/dgraph-io/dgraph/blob/main/TESTING.md Execute a specific test target with custom variables for package, test name, and timeout. Example uses 'make test-systest'. ```bash make test-systest PKG=systest/plugin TEST=TestPasswordReturn TIMEOUT=60m ``` -------------------------------- ### Incorrect WaitGroup Usage in Go Source: https://github.com/dgraph-io/dgraph/blob/main/dgraph/cmd/alpha/thoughts.md This example demonstrates an incorrect way to use sync.WaitGroup, where wg.Add(1) might not be called before wg.Wait(), leading to premature program termination. ```go func handle(..) { wg.Add(1) ... wg.Done() } func main() { wg := new(sync.WaitGroup) for i := 0; i < N; i++ { go handle(..) } wg.Wait() } ``` -------------------------------- ### Configure SQL Database Connection Source: https://github.com/dgraph-io/dgraph/blob/main/dgraph/cmd/migrate/README.md Creates a configuration file for connecting to an SQL database. Ensure values are not in quotes. ```bash user = password = db = ``` -------------------------------- ### Regenerate protocol buffer files Source: https://github.com/dgraph-io/dgraph/blob/main/CONTRIBUTING.md Commands to navigate to the protos directory and regenerate the .pb.go files. ```bash cd protos make regenerate ``` -------------------------------- ### Dgraph MCP Go Code Configuration Source: https://github.com/dgraph-io/dgraph/blob/main/dgraph/cmd/mcp/README.md Configure the MCP server for Go code integration by specifying the dgraph binary command and arguments. Ensure the dgraph binary is installed and accessible. ```json { "mcpServers": { "dgraph": { "command": "dgraph_binary", "args": ["mcp", "-c", "dgraph://localhost:9080"] } } } ``` -------------------------------- ### Using require Assertions in Go Tests Source: https://github.com/dgraph-io/dgraph/blob/main/TESTING.md Illustrates the use of `require.*` assertions from the `testify` library, which cause a test to fail immediately upon assertion failure. This is preferred for critical checks and setup. ```go require.NoError(t, err) // Stops test if err != nil require.Equal(t, expected, actual) require.True(t, condition) ``` -------------------------------- ### Go Test Benchmark Output for GraphQL Mutations Source: https://github.com/dgraph-io/dgraph/blob/main/graphql/bench/README.md Presents the benchmark results for GraphQL mutations, including multi-level and single-level operations for both add and delete actions. The data compares Auth and Non-Auth execution times. ```bash BenchmarkMutation: 100 owners, each having 100 restaurants BenchmarkMultiLevelMutation: 20 restaurants, each having 20 cuisines, each cuisine having 20 dishes BenchmarkOneLevelMutation: 10000 nodes Auth: BenchmarkMutation: 0.380893400s BenchmarkMultiLevelMutation: 1.392922056s BenchmarkOneLevelMutation: Add Time: 9.42224304s Delete Time: 1.150111483s Non-Auth: BenchmarkMutation: 0.464559706s BenchmarkMultiLevelMutation: 1.440681796s BenchmarkOneLevelMutation: Add Time: 9.549761333s Delete Time: 1.200276696s ``` -------------------------------- ### Run Tests Against Existing Cluster with t/ Runner Source: https://github.com/dgraph-io/dgraph/blob/main/TESTING.md Starts a Dgraph cluster manually and then runs tests against it using the t/ runner, specifying the cluster prefix. The cluster remains running after tests complete. ```bash # Start cluster manually first docker compose -f dgraph/docker-compose.yml -p myprefix up -d # Run tests against it (no cluster restart) cd t && ./t --prefix=myprefix --pkg=systest/export # Cluster stays running after tests ``` -------------------------------- ### Run Fuzz Tests with Make Source: https://github.com/dgraph-io/dgraph/blob/main/TESTING.md Execute fuzz tests using the 'make test' command with the FUZZ=1 argument. Specify fuzz time if needed. ```bash make test FUZZ=1 ``` -------------------------------- ### Run datagen with Configuration Source: https://github.com/dgraph-io/dgraph/blob/main/graphql/testdata/datagen/README.md Execute the datagen tool using a configuration file. This imports data from the specified restaurant data file into Dgraph. ```bash $ ./datagen --config conf.yaml ``` -------------------------------- ### CI Fuzz Test Execution Source: https://github.com/dgraph-io/dgraph/blob/main/TESTING.md Example of a CI workflow step to run fuzz tests for the DQL parser. It specifies a fuzzing duration and is designed to catch parser crashes early in the development process. ```bash go test -v ./dql -fuzz="Fuzz" -fuzztime="300s" ``` -------------------------------- ### Introspect __TypeKind Enum Values Source: https://github.com/dgraph-io/dgraph/blob/main/graphql/schema/testdata/introspection/input/type_filter.txt Query the introspection schema to get the name of the __TypeKind enum and its associated enum values. This query is useful for understanding the different kinds of types available in your GraphQL schema. ```graphql { typeKindType: __type(name: "__TypeKind") { name, enumValues { name, } } } ``` -------------------------------- ### Basic Go Test Command Structure Source: https://github.com/dgraph-io/dgraph/blob/main/TESTING.md Use this command to run Go tests. Flags can modify test execution behavior. ```bash go test [flags] [package] [test-filter] ``` -------------------------------- ### Build and Run Docker Containers Source: https://github.com/dgraph-io/dgraph/blob/main/graphql/e2e/custom_logic/README.md Use this command to bring up the Docker containers for the Dgraph project. It includes a force rebuild option for the mock server image if changes are made to cmd/main.go. ```bash docker-compose up --build ``` -------------------------------- ### Create Release Branch from Tag Head Source: https://github.com/dgraph-io/dgraph/blob/main/contrib/RELEASE.md After creating a release tag, create a release branch from the tag's head to facilitate backporting fixes. This example shows creating a 'release/v25.1' branch from the 'main' branch. ```sh git checkout main git pull origin main git checkout -b release/v25.1 git push origin release/v25.1 ``` -------------------------------- ### Dgraph GraphQL Query Example Source: https://github.com/dgraph-io/dgraph/blob/main/dgraph/cmd/alpha/notes.txt This cURL command executes a GraphQL query against a Dgraph instance running on localhost:8080. It retrieves details about a person and their films, including director, starring actors, characters, release date, country, and genre. The output is redirected to 'output.json'. ```bash curl localhost:8080/query -XPOST -d '{ me(_xid_: m.06pj8) { type.object.name.en film.director.film { film.film.starring { type.object.name.en film.performance.actor { film.director.film { type.object.name.en } type.object.name.en } film.performance.character { type.object.name.en } } film.film.initial_release_date film.film.country film.film.genre { type.object.name.en } } } }' > output.json ``` -------------------------------- ### Run Vector Tests with Make Source: https://github.com/dgraph-io/dgraph/blob/main/TESTING.md Execute vector tests using the 'make test' command with the 'vector' suite argument. ```bash make test SUITE=vector ``` -------------------------------- ### Run Integration Tests with Make Source: https://github.com/dgraph-io/dgraph/blob/main/TESTING.md Use make commands to automatically build the runner and execute tests for a given package or a specific test function. ```bash make test SUITE=integration PKG=systest/export ``` ```bash make test TEST=TestExportAndLoadJson ``` -------------------------------- ### Build and Run GraphQL Profiler Source: https://github.com/dgraph-io/dgraph/blob/main/graphql/testdata/custom_bench/README.md Builds and runs the `graphql_profiler` in the background. This profiler collects benchmarking and profiling information for GraphQL queries. Results are saved in the `profiling/results` directory. ```bash $ cd profiling $ go build graphql_profiler.go $ nohup ./graphql_profiler > graphql_profiler.out & ``` -------------------------------- ### Go Test Benchmark Output for GraphQL Queries Source: https://github.com/dgraph-io/dgraph/blob/main/graphql/bench/README.md Displays the results of running Go tests with the '-bench=.' and '-benchtime=60s' flags for GraphQL queries. This output details the operations per second and average time per operation for both authenticated and non-authenticated scenarios. ```bash go os: linux goarch: amd64 pkg: github.com/dgraph-io/dgraph/graphql/e2e/auth/bench Auth BenchmarkNestedQuery-8 88 815315761 ns/op BenchmarkOneLevelQuery-8 4357 15626384 ns/op Non-Auth BenchmarkNestedQuery-8 33 2218877846 ns/op BenchmarkOneLevelQuery-8 4446 16100509 ns/op ``` -------------------------------- ### Run Upgrade Tests in Upgrade Mode with Make Source: https://github.com/dgraph-io/dgraph/blob/main/TESTING.md Execute tests in upgrade mode using the 'make test' command, specifying TAGS=upgrade, PKG, and TEST. This is used for running tests that validate behavior after an upgrade. ```bash make test TAGS=upgrade PKG=systest/plugin TEST=TestPluginTestSuite/TestPasswordReturn ```