### Install BadgerDB CLI utility Source: https://github.com/dgraph-io/badger/blob/main/docs/quickstart.md Install the command-line tool to your GOBIN path. ```sh go install github.com/dgraph-io/badger/v4/badger@latest ``` -------------------------------- ### Install Badger CLI Tool Source: https://github.com/dgraph-io/badger/blob/main/README.md Build and install the Badger command line utility into your $GOBIN path after cloning the repository. ```sh cd badger go install . ``` -------------------------------- ### Install Badger Library Source: https://github.com/dgraph-io/badger/blob/main/README.md Use this command to add the Badger library to your Go project using modules. ```sh go get github.com/dgraph-io/badger/v4 ``` -------------------------------- ### Install Specific Badger Version Source: https://github.com/dgraph-io/badger/blob/main/docs/troubleshooting.md Commands to checkout and install a specific version of Badger from source. ```bash cd $GOPATH/src/github.com/dgraph-io/badger git checkout v1.6.0 cd badger && go install ``` ```bash cd $GOPATH/src/github.com/dgraph-io/badger git checkout v2.0.0 cd badger && go install ``` -------------------------------- ### Manually Create and Commit a Writable Transaction Source: https://github.com/dgraph-io/badger/blob/main/docs/quickstart.md Use DB.NewTransaction(true) to start a writable transaction. Always defer txn.Discard() and call txn.Commit() to ensure data is saved and resources are released. ```go txn := db.NewTransaction(true) defer txn.Discard() // Use the transaction... err := txn.Set([]byte("answer"), []byte("42")) if err != nil { return err } // Commit the transaction and check for error. if err := txn.Commit(); err != nil { return err } ``` -------------------------------- ### Schedule Value Log Garbage Collection Source: https://github.com/dgraph-io/badger/blob/main/docs/quickstart.md Periodically run Badger's value log garbage collection to reclaim space. This example uses a ticker to call RunValueLogGC every 5 minutes and retries if successful. ```go ticker := time.NewTicker(5 * time.Minute) defer ticker.Stop() for range ticker.C { another: err := db.RunValueLogGC(0.7) if err == nil { goto another } } ``` -------------------------------- ### Get Monotonically Increasing Integers Source: https://github.com/dgraph-io/badger/blob/main/docs/quickstart.md Use DB.GetSequence to obtain a thread-safe Sequence object for generating unique, monotonically increasing integers. Release the sequence when done to avoid wasting integers. ```go seq, err := db.GetSequence(key, 1000) deffer seq.Release() for { num, err := seq.Next() } ``` -------------------------------- ### Get and Access Value within a Transaction Source: https://github.com/dgraph-io/badger/blob/main/docs/quickstart.md Use Txn.Get() to retrieve an item by key. The item's value is accessed via a callback function passed to item.Value(). Values are only valid within the transaction's scope; copy them if needed outside. ```go err := db.View(func(txn *badger.Txn) error { item, err := txn.Get([]byte("answer")) handle(err) var valNot, valCopy []byte err := item.Value(func(val []byte) error { // This func with val would only be called if item.Value encounters no error. // Accessing val here is valid. fmt.Printf("The answer is: %s\n", val) // Copying or parsing val is valid. valCopy = append([]byte{}, val...) // Assigning val slice to another variable is NOT OK. valNot = val // Do not do this. return nil }) handle(err) // DO NOT access val here. It is the most common cause of bugs. fmt.Printf("NEVER do this. %s\n", valNot) // You must copy it to use it outside item.Value(...). fmt.Printf("The answer is: %s\n", valCopy) // Alternatively, you could also use item.ValueCopy(). valCopy, err = item.ValueCopy(nil) handle(err) fmt.Printf("The answer is: %s\n", valCopy) return nil }) ``` -------------------------------- ### Open a Badger database Source: https://github.com/dgraph-io/badger/blob/main/docs/quickstart.md Initialize a database instance using default options. Ensure the directory is locked by only one process at a time. ```go package main import ( "log" badger "github.com/dgraph-io/badger/v4" ) func main() { // Open the Badger database located in the /tmp/badger directory. // It is created if it doesn't exist. db, err := badger.Open(badger.DefaultOptions("/tmp/badger")) if err != nil { log.Fatal(err) } defer db.Close() // your code here } ``` -------------------------------- ### Backup Badger Database (CLI) Source: https://github.com/dgraph-io/badger/blob/main/docs/quickstart.md Use this command to create a version-agnostic backup of your Badger database to a specified file. Ensure the Badger CLI tool is in your PATH. ```bash badger backup --dir ``` -------------------------------- ### Configure and Run Badger Stream Source: https://github.com/dgraph-io/badger/blob/main/docs/quickstart.md Set up and execute a stream for iterating over Badger DB data. Customize iteration with options like NumGo, Prefix, ChooseKey, and KeyToList. The Send function processes batched key-values serially. ```go stream := db.NewStream() // db.NewStreamAt(readTs) for managed mode. // -- Optional settings stream.NumGo = 16 // Set number of goroutines to use for iteration. stream.Prefix = []byte("some-prefix") // Leave nil for iteration over the whole DB. stream.LogPrefix = "Badger.Streaming" // For identifying stream logs. Outputs to Logger. // ChooseKey is called concurrently for every key. If left nil, assumes true by default. stream.ChooseKey = func(item *badger.Item) bool { return bytes.HasSuffix(item.Key(), []byte("er")) } // KeyToList is called concurrently for chosen keys. This can be used to convert // Badger data into custom key-values. If nil, uses stream.ToList, a default // implementation, which picks all valid key-values. stream.KeyToList = nil // -- End of optional settings. // Send is called serially, while Stream.Orchestrate is running. stream.Send = func(list *pb.KVList) error { return proto.MarshalText(w, list) // Write to w. } // Run the stream if err := stream.Orchestrate(context.Background()); err != nil { return err } // Done. ``` -------------------------------- ### Backup Badger Database (v0.8.1 and below) Source: https://github.com/dgraph-io/badger/blob/main/docs/quickstart.md For databases created with Badger v0.8 or earlier, use the `badger_backup` tool to create a backup file. This backup can then be restored using the standard `badger restore` command to upgrade the database. ```bash badger_backup --dir --backup-file badger.bak ``` -------------------------------- ### Set Key/Value Pair using Entry and SetEntry Source: https://github.com/dgraph-io/badger/blob/main/docs/quickstart.md Create an Entry object with key and value, then use Txn.SetEntry() to save it. This method allows setting additional entry properties. ```go err := db.Update(func(txn *badger.Txn) error { e := badger.NewEntry([]byte("answer"), []byte("42")) err := txn.SetEntry(e) return err }) ``` -------------------------------- ### Configure in-memory mode Source: https://github.com/dgraph-io/badger/blob/main/docs/quickstart.md Set the InMemory option to true to run the database without disk persistence. ```go opt := badger.DefaultOptions("").WithInMemory(true) ``` -------------------------------- ### Benchmark Database Open Operation Source: https://github.com/dgraph-io/badger/blob/main/table/README.md Commands to prepare the environment and execute the BenchmarkDBOpen test for a large database. ```sh badger fill -m 2000 --dir="/tmp/data" --sorted ``` ```sh free -mh && sync && echo 3 | sudo tee /proc/sys/vm/drop_caches && sudo swapoff -a && sudo swapon -a && free -mh ``` ```sh blockdev --flushbufs /dev/nvme0n1p4 ``` ```sh go test -run=^$ github.com/dgraph-io/badger -bench ^BenchmarkDBOpen$ -benchdir="/tmp/data" -v badger 2019/06/04 17:15:56 INFO: 126 tables out of 1028 opened in 3.017s badger 2019/06/04 17:15:59 INFO: 257 tables out of 1028 opened in 6.014s badger 2019/06/04 17:16:02 INFO: 387 tables out of 1028 opened in 9.017s badger 2019/06/04 17:16:05 INFO: 516 tables out of 1028 opened in 12.025s badger 2019/06/04 17:16:08 INFO: 645 tables out of 1028 opened in 15.013s badger 2019/06/04 17:16:11 INFO: 775 tables out of 1028 opened in 18.008s badger 2019/06/04 17:16:14 INFO: 906 tables out of 1028 opened in 21.003s badger 2019/06/04 17:16:17 INFO: All 1028 tables opened in 23.851s badger 2019/06/04 17:16:17 INFO: Replaying file id: 1998 at offset: 332000 badger 2019/06/04 17:16:17 INFO: Replay took: 9.81µs goos: linux goarch: amd64 pkg: github.com/dgraph-io/badger BenchmarkDBOpen-16 1 23930082140 ns/op PASS ok github.com/dgraph-io/badger 24.076s ``` -------------------------------- ### Clone Badger Project and Set Up Remotes Source: https://github.com/dgraph-io/badger/blob/main/CONTRIBUTING.md Clone the Badger repository from GitHub and configure remote repositories for upstream and push restrictions. This sets up your local environment for development. ```sh git clone https://github.com/$GITHUB_USER/badger cd badger git remote add upstream git@github.com:dgraph-io/badger.git # Never push to the upstream master git remote set-url --push upstream no_push ``` -------------------------------- ### Export and Restore Badger Database for Full Encryption Source: https://github.com/dgraph-io/badger/blob/main/docs/encryption-at-rest.md To encrypt all existing data immediately, first export the database using the 'backup' command. Then, create a new encrypted database instance and restore the data from the backup file. This ensures all data is encrypted upon restoration. ```shell badger backup --dir=badger_dir -f backup.bak ``` ```shell badger restore --dir=new_badger_dir -f backup.bak ``` -------------------------------- ### Run Population Benchmark Source: https://github.com/dgraph-io/badger/blob/main/skl/README.md Command used to populate the database for performance testing. ```sh rm -Rf tmp && /usr/bin/time -l ./populate -keys_mil 10 ``` -------------------------------- ### Build Badger Without Cgo Source: https://github.com/dgraph-io/badger/blob/main/docs/troubleshooting.md Build the Badger binary without Cgo support, which disables ZSTD compression. ```bash CGO_ENABLED=0 go build ``` -------------------------------- ### Implement pagination with prefix scans Source: https://github.com/dgraph-io/badger/blob/main/docs/quickstart.md Use a cursor to resume iteration from a specific point within a prefix scan. Ensure the prefix is reset after the initial seek to continue sequential iteration. ```go // startCursor may look like 'feed:tQpnEDVRoCxTFQDvyQEzdo:1733127486'. // A prefix scan with this cursor locates the specific key where // the previous iteration stopped. err = db.badger.View(func(txn *badger.Txn) error { it := txn.NewIterator(opts) defer it.Close() // Prefix example 'feed:tQpnEDVRoCxTFQDvyQEzdo' // if no cursor provided prefix scan starts from the beginning p := prefix if startCursor != nil { p = startCursor } iterNum := 0 // Tracks the number of iterations to enforce the limit. for it.Seek(p); it.ValidForPrefix(p); it.Next() { // The method it.ValidForPrefix ensures that iteration continues // as long as keys match the prefix. // For example, if p = 'feed:tQpnEDVRoCxTFQDvyQEzdo:1733127486', // it matches keys like // 'feed:tQpnEDVRoCxTFQDvyQEzdo:1733127889:pprRrNL2WP4yfVXsSNBSx6'. // Once the starting point for iteration is found, revert the prefix // back to 'feed:tQpnEDVRoCxTFQDvyQEzdo' to continue iterating sequentially. // Otherwise, iteration would stop after a single prefix-key match. p = prefix item := it.Item() key := string(item.Key()) if iterNum > limit { // Limit reached. nextCursor = key // Save the next cursor for future iterations. return nil } iterNum++ // Increment iteration count. err := item.Value(func(v []byte) error { fmt.Printf("key=%s, value=%s\n", k, v) return nil }) if err != nil { return err } } // If the number of iterations is less than the limit, // it means there are no more items for the prefix. if iterNum < limit { nextCursor = "" } return nil }) return nextCursor, err ``` -------------------------------- ### Set TTL and Metadata Together Source: https://github.com/dgraph-io/badger/blob/main/docs/quickstart.md Combine both TTL and user metadata on a single entry before committing. ```go err := db.Update(func(txn *badger.Txn) error { e := badger.NewEntry([]byte("answer"), []byte("42")).WithMeta(byte(1)).WithTTL(time.Hour) err := txn.SetEntry(e) return err }) ``` -------------------------------- ### Use a Merge Operator Source: https://github.com/dgraph-io/badger/blob/main/docs/quickstart.md Initialize a merge operator with a key and duration, then add values and retrieve the result. ```go key := []byte("merge") m := db.GetMergeOperator(key, add, 200*time.Millisecond) defer m.Stop() m.Add([]byte("A")) m.Add([]byte("B")) m.Add([]byte("C")) res, _ := m.Get() // res should have value ABC encoded ``` -------------------------------- ### Backup Badger Data Source: https://github.com/dgraph-io/badger/blob/main/docs/troubleshooting.md Create a backup of the existing Badger data directory. ```bash badger backup --dir path/to/badger/directory -f badger.backup ``` -------------------------------- ### Rsync-based Badger Database Backup Script Source: https://github.com/dgraph-io/badger/blob/main/docs/quickstart.md This bash script provides a rudimentary way to back up a Badger database using rsync. It repeatedly syncs the database directory and checks for updates to MANIFEST and SSTable files to ensure consistency. ```bash #!/bin/bash set -o history set -o histexpand # Makes a complete copy of a Badger database directory. # Repeat rsync if the MANIFEST and SSTables are updated. rsync -avz --delete db/ dst while !! | grep -q "(MANIFEST\|\.sst)$\"; do :; done ``` -------------------------------- ### Set TTL on an Entry Source: https://github.com/dgraph-io/badger/blob/main/docs/quickstart.md Apply a time-to-live duration to a key-value entry using WithTTL. ```go err := db.Update(func(txn *badger.Txn) error { e := badger.NewEntry([]byte("answer"), []byte("42")).WithTTL(time.Hour) err := txn.SetEntry(e) return err }) ``` -------------------------------- ### Set User Metadata on an Entry Source: https://github.com/dgraph-io/badger/blob/main/docs/quickstart.md Attach a single-byte user metadata value to an entry using WithMeta. ```go err := db.Update(func(txn *badger.Txn) error { e := badger.NewEntry([]byte("answer"), []byte("42")).WithMeta(byte(1)) err := txn.SetEntry(e) return err }) ``` -------------------------------- ### Run All Badger Tests Source: https://github.com/dgraph-io/badger/blob/main/CONTRIBUTING.md Execute all tests for the Badger project using the provided test script. This ensures your changes do not break existing functionality. ```sh ./test.sh ``` -------------------------------- ### Enable Encryption for New Badger Database Source: https://github.com/dgraph-io/badger/blob/main/docs/encryption-at-rest.md Configure Badger options to enable encryption when creating a new database. Specify the master encryption key and optionally set the data key rotation duration. The default rotation duration is 10 days. ```go opts := badger.DefaultOptions("/tmp/badger"). WithEncryptionKey(masterKey). WithEncryptionKeyRotationDuration(dataKeyRotationDuration) // defaults to 10 days ``` -------------------------------- ### Splash Go Index Cache Source: https://github.com/dgraph-io/badger/blob/main/contrib/RELEASE.md Splash the 'go index' cache to ensure the latest release is publicly available. Replace 'vX.X.X' with the actual version number. ```shell go list -m github.com/dgraph-io/badger/v4@vX.X.X ``` -------------------------------- ### Benchmark Read-Write Performance Source: https://github.com/dgraph-io/badger/blob/main/skl/README.md Performance metrics for read-write operations in Badger compared to other data structures. ```sh BenchmarkReadWrite/frac_0-8 3000000 537 ns/op BenchmarkReadWrite/frac_1-8 3000000 503 ns/op BenchmarkReadWrite/frac_2-8 3000000 492 ns/op BenchmarkReadWrite/frac_3-8 3000000 475 ns/op BenchmarkReadWrite/frac_4-8 3000000 440 ns/op BenchmarkReadWrite/frac_5-8 5000000 442 ns/op BenchmarkReadWrite/frac_6-8 5000000 380 ns/op BenchmarkReadWrite/frac_7-8 5000000 338 ns/op BenchmarkReadWrite/frac_8-8 5000000 294 ns/op BenchmarkReadWrite/frac_9-8 10000000 268 ns/op BenchmarkReadWrite/frac_10-8 100000000 26.3 ns/op ``` ```sh BenchmarkReadWriteMap/frac_0-8 2000000 774 ns/op BenchmarkReadWriteMap/frac_1-8 2000000 647 ns/op BenchmarkReadWriteMap/frac_2-8 3000000 605 ns/op BenchmarkReadWriteMap/frac_3-8 3000000 603 ns/op BenchmarkReadWriteMap/frac_4-8 3000000 556 ns/op BenchmarkReadWriteMap/frac_5-8 3000000 472 ns/op BenchmarkReadWriteMap/frac_6-8 3000000 476 ns/op BenchmarkReadWriteMap/frac_7-8 3000000 457 ns/op BenchmarkReadWriteMap/frac_8-8 5000000 444 ns/op BenchmarkReadWriteMap/frac_9-8 5000000 361 ns/op BenchmarkReadWriteMap/frac_10-8 10000000 212 ns/op ``` -------------------------------- ### Restore Badger Database (CLI) Source: https://github.com/dgraph-io/badger/blob/main/docs/quickstart.md Use this command to restore a Badger database from a backup file. Specify the target directory for the restored database. ```bash badger restore --dir ``` -------------------------------- ### Iterate Over Keys Source: https://github.com/dgraph-io/badger/blob/main/docs/quickstart.md Use an iterator to traverse keys in lexicographical order. ```go err := db.View(func(txn *badger.Txn) error { opts := badger.DefaultIteratorOptions opts.PrefetchSize = 10 it := txn.NewIterator(opts) defer it.Close() for it.Rewind(); it.Valid(); it.Next() { item := it.Item() k := item.Key() err := item.Value(func(v []byte) error { fmt.Printf("key=%s, value=%s\n", k, v) return nil }) if err != nil { return err } } return nil }) ``` -------------------------------- ### Restore Badger Data Source: https://github.com/dgraph-io/badger/blob/main/docs/troubleshooting.md Restore data from a backup file into a new directory using the new Badger binary. ```bash badger restore --dir path/to/new/badger/directory -f badger.backup ``` -------------------------------- ### Benchmark Read and Build Operation Source: https://github.com/dgraph-io/badger/blob/main/table/README.md Executes the BenchmarkReadAndBuild test to measure the performance of reading and building a table in memory. ```sh $ go test -bench BenchmarkReadAndBuild -run ^$ -count 3 goos: linux goarch: amd64 pkg: github.com/dgraph-io/badger/table BenchmarkReadAndBuild-16 1 1026755231 ns/op BenchmarkReadAndBuild-16 1 1009543316 ns/op BenchmarkReadAndBuild-16 1 1039920546 ns/op PASS ok github.com/dgraph-io/badger/table 12.081s ``` -------------------------------- ### Execute read-write transaction Source: https://github.com/dgraph-io/badger/blob/main/docs/quickstart.md Use DB.Update to perform write or delete operations. Always handle returned errors. ```go err := db.Update(func(txn *badger.Txn) error { // Your code here… return nil }) ``` -------------------------------- ### Perform Prefix Scans Source: https://github.com/dgraph-io/badger/blob/main/docs/quickstart.md Combine Seek and ValidForPrefix to iterate over keys matching a specific prefix. ```go db.View(func(txn *badger.Txn) error { it := txn.NewIterator(badger.DefaultIteratorOptions) defer it.Close() prefix := []byte("1234") for it.Seek(prefix); it.ValidForPrefix(prefix); it.Next() { item := it.Item() k := item.Key() err := item.Value(func(v []byte) error { fmt.Printf("key=%s, value=%s\n", k, v) return nil }) if err != nil { return err } } return nil }) ``` -------------------------------- ### Create Release Branch Source: https://github.com/dgraph-io/badger/blob/main/contrib/RELEASE.md Create a release branch from the main branch after tagging a new version. This facilitates backporting fixes. ```shell git checkout main git pull origin main git checkout -b release/v4.9 git push origin release/v4.9 ``` -------------------------------- ### Perform key-only iteration Source: https://github.com/dgraph-io/badger/blob/main/docs/quickstart.md Set PrefetchValues to false to enable key-only iteration, which improves performance by avoiding value lookups in the LSM-tree. ```go err := db.View(func(txn *badger.Txn) error { opts := badger.DefaultIteratorOptions opts.PrefetchValues = false it := txn.NewIterator(opts) defer it.Close() for it.Rewind(); it.Valid(); it.Next() { item := it.Item() k := item.Key() fmt.Printf("key=%s\n", k) } return nil }) ``` -------------------------------- ### Analyze Node Pooling Memory Usage Source: https://github.com/dgraph-io/badger/blob/main/skl/README.md pprof memory profiles showing resource allocation before and after node pooling implementation. ```sh 1311.53MB of 1338.69MB total (97.97%) Dropped 30 nodes (cum <= 6.69MB) Showing top 10 nodes out of 37 (cum >= 12.50MB) flat flat% sum% cum cum% 523.04MB 39.07% 39.07% 523.04MB 39.07% github.com/dgraph-io/badger/skl.(*Skiplist).Put 184.51MB 13.78% 52.85% 184.51MB 13.78% runtime.stringtoslicebyte 166.01MB 12.40% 65.25% 689.04MB 51.47% github.com/dgraph-io/badger/mem.(*Table).Put 165MB 12.33% 77.58% 165MB 12.33% runtime.convT2E 116.92MB 8.73% 86.31% 116.92MB 8.73% bytes.makeSlice 62.50MB 4.67% 90.98% 62.50MB 4.67% main.newValue 34.50MB 2.58% 93.56% 34.50MB 2.58% github.com/dgraph-io/badger/table.(*BlockIterator).parseKV 25.50MB 1.90% 95.46% 100.06MB 7.47% github.com/dgraph-io/badger/y.(*MergeIterator).Next 21.06MB 1.57% 97.04% 21.06MB 1.57% github.com/dgraph-io/badger/table.(*Table).read 12.50MB 0.93% 97.97% 12.50MB 0.93% github.com/dgraph-io/badger/table.header.Encode 128.31 real 329.37 user 17.11 sys 3355660288 maximum resident set size 0 average shared memory size 0 average unshared data size 0 average unshared stack size 2203080 page reclaims 764 page faults 0 swaps 275 block input operations 76 block output operations 0 messages sent 0 messages received 0 signals received 49173 voluntary context switches 599922 involuntary context switches ``` ```sh 1963.13MB of 2026.09MB total (96.89%) Dropped 29 nodes (cum <= 10.13MB) Showing top 10 nodes out of 41 (cum >= 185.62MB) flat flat% sum% cum cum% 658.05MB 32.48% 32.48% 658.05MB 32.48% github.com/dgraph-io/badger/skl.glob..func1 297.51MB 14.68% 47.16% 297.51MB 14.68% runtime.convT2E 257.51MB 12.71% 59.87% 257.51MB 12.71% runtime.stringtoslicebyte 249.01MB 12.29% 72.16% 1007.06MB 49.70% github.com/dgraph-io/badger/mem.(*Table).Put 142.43MB 7.03% 79.19% 142.43MB 7.03% bytes.makeSlice 100MB 4.94% 84.13% 758.05MB 37.41% github.com/dgraph-io/badger/skl.newNode 99.50MB 4.91% 89.04% 99.50MB 4.91% main.newValue 75MB 3.70% 92.74% 75MB 3.70% github.com/dgraph-io/badger/table.(*BlockIterator).parseKV 44.62MB 2.20% 94.94% 44.62MB 2.20% github.com/dgraph-io/badger/table.(*Table).read 39.50MB 1.95% 96.89% 185.62MB 9.16% github.com/dgraph-io/badger/y.(*MergeIterator).Next 135.58 real 374.29 user 17.65 sys 3740614656 maximum resident set size 0 average shared memory size 0 average unshared data size 0 average unshared stack size 2276566 page reclaims 770 page faults 0 swaps 128 block input operations 90 block output operations 0 messages sent 0 messages received 0 signals received 46434 voluntary context switches 597049 involuntary context switches ``` -------------------------------- ### BadgerDB v2.2007.2 API Additions Source: https://github.com/dgraph-io/badger/blob/main/CHANGELOG.md This section details the new APIs introduced in BadgerDB version 2.2007.2, focusing on cache metrics and configuration options. ```APIDOC ## New APIs in BadgerDB v2.2007.2 ### Badger.DB - **BlockCacheMetrics** - Description: Provides metrics for the block cache. - Usage: Access block cache performance statistics. - **IndexCacheMetrics** - Description: Provides metrics for the index cache. - Usage: Access index cache performance statistics. ### Badger.Option - **WithBlockCacheSize** - Description: Configures the size of the block cache. - Usage: `badger.WithBlockCacheSize(size int64)` - **WithIndexCacheSize** - Description: Configures the size of the index cache. - Usage: `badger.WithIndexCacheSize(size int64)` ``` -------------------------------- ### Enable Encryption on Existing Unencrypted Badger Database Source: https://github.com/dgraph-io/badger/blob/main/docs/encryption-at-rest.md Enable encryption on an existing unencrypted database using the 'rotate' command. This command enables encryption for new files only; existing data will be encrypted during subsequent compactions. Badger will operate in a hybrid mode, tracking encryption status per file. ```shell badger rotate --dir=badger_dir --new-key-path=new/path ``` -------------------------------- ### Batching Writes with WriteBatch API Source: https://github.com/dgraph-io/badger/blob/main/docs/troubleshooting.md Use the WriteBatch API for efficient bulk writes by batching multiple updates into a single transaction. This amortizes transaction costs and avoids blocking. Note that WriteBatch does not support reads; use the Transaction API for read-modify-write workloads. ```go wb := db.NewWriteBatch() deferr wb.Cancel() for i := 0; i < N; i++ { err := wb.Set(key(i), value(i), 0) // Will create txns as needed. handle(err) } handle(wb.Flush()) // Wait for all txns to finish. ``` -------------------------------- ### Implement a Counter Merge Operator Source: https://github.com/dgraph-io/badger/blob/main/docs/quickstart.md Define helper functions and a merge function to perform atomic uint64 additions. ```go func uint64ToBytes(i uint64) []byte { var buf [8]byte binary.BigEndian.PutUint64(buf[:], i) return buf[:] } func bytesToUint64(b []byte) uint64 { return binary.BigEndian.Uint64(b) } // Merge function to add two uint64 numbers func add(existing, new []byte) []byte { return uint64ToBytes(bytesToUint64(existing) + bytesToUint64(new)) } ``` ```go key := []byte("merge") m := db.GetMergeOperator(key, add, 200*time.Millisecond) defer m.Stop() m.Add(uint64ToBytes(1)) m.Add(uint64ToBytes(2)) m.Add(uint64ToBytes(3)) res, _ := m.Get() // res should have value 6 encoded ``` -------------------------------- ### Set index cache size for encryption Source: https://github.com/dgraph-io/badger/blob/main/docs/quickstart.md Configure the index cache size when encryption is enabled to maintain performance. ```go opts.IndexCache = 100 << 20 // 100 mb or some other size based on the amount of data ``` -------------------------------- ### BadgerDB v2.0.1 API Additions Source: https://github.com/dgraph-io/badger/blob/main/CHANGELOG.md This section details the new APIs introduced in BadgerDB version 2.0.1, including in-memory mode and ZSTD compression level options. ```APIDOC ## New APIs in BadgerDB v2.0.1 ### badger.Options - **WithInMemory** - Description: Enables in-memory mode for the database. - Usage: `badger.WithInMemory(enabled bool)` - **WithZSTDCompressionLevel** - Description: Sets the compression level for ZSTD compression. - Usage: `badger.WithZSTDCompressionLevel(level int)` ### Badger.TableInfo - **EstimatedSz** - Description: Returns the estimated size of the table. - Usage: `tableInfo.EstimatedSz() uint64` ``` -------------------------------- ### Commit Changes Source: https://github.com/dgraph-io/badger/blob/main/CONTRIBUTING.md Stage and commit your changes. This command will open your configured Git editor to write a commit message. ```sh git commit ``` -------------------------------- ### BadgerDB v2.2007.0 API Additions Source: https://github.com/dgraph-io/badger/blob/main/CHANGELOG.md This section details the new APIs introduced in BadgerDB version 2.2007.0, including batch operations, prefix dropping, and conflict detection. ```APIDOC ## New APIs in BadgerDB v2.2007.0 ### Badger.DB - **NewManagedWriteBatch** - Description: Creates a new managed write batch. - Usage: `db.NewManagedWriteBatch()` - **DropPrefix** - Description: Removes all keys with a given prefix. - Usage: `db.DropPrefix(prefix []byte) error` ### Badger.Option - **WithDetectConflicts** - Description: Enables or disables conflict detection. - Usage: `badger.WithDetectConflicts(detect bool)` - **WithKeepBlockIndicesInCache** - Description: Configures whether to keep block indices in the cache. - Usage: `badger.WithKeepBlockIndicesInCache(keep bool)` - **WithKeepBlocksInCache** - Description: Configures whether to keep blocks in the cache. - Usage: `badger.WithKeepBlocksInCache(keep bool)` ### Badger.WriteBatch - **DeleteAt** - Description: Marks a key for deletion at a specific timestamp. - Usage: `batch.DeleteAt(key []byte, timestamp uint64) error` - **SetEntryAt** - Description: Sets a key-value pair at a specific timestamp. - Usage: `batch.SetEntryAt(key, value []byte, timestamp uint64) error` - **Write** - Description: Writes the batch to the database. - Usage: `batch.Write() error` ``` -------------------------------- ### Handle ErrTxnTooBig in transactions Source: https://github.com/dgraph-io/badger/blob/main/docs/quickstart.md Commit and restart a transaction if the pending write limit is exceeded. ```go updates := make(map[string]string) txn := db.NewTransaction(true) for k,v := range updates { if err := txn.Set([]byte(k),[]byte(v)); err == badger.ErrTxnTooBig { _ = txn.Commit() txn = db.NewTransaction(true) _ = txn.Set([]byte(k),[]byte(v)) } } _ = txn.Commit() ``` -------------------------------- ### Set Key/Value Pair using DB.Update Source: https://github.com/dgraph-io/badger/blob/main/docs/quickstart.md The DB.Update method provides a convenient wrapper for setting a key-value pair within a transaction. It handles transaction creation and commit/discard automatically. ```go err := db.Update(func(txn *badger.Txn) error { err := txn.Set([]byte("answer"), []byte("42")) return err }) ``` -------------------------------- ### BadgerDB v2.0.3 API Additions Source: https://github.com/dgraph-io/badger/blob/main/CHANGELOG.md This section details the new APIs introduced in BadgerDB version 2.0.3, focusing on cache metrics and bypass lock guard options. ```APIDOC ## New APIs in BadgerDB v2.0.3 ### badger.DB - **BfCacheMetrics** - Description: Provides metrics for the bloom filter cache. - Usage: Access bloom filter cache performance statistics. - **DataCacheMetrics** - Description: Provides metrics for the data cache. - Usage: Access data cache performance statistics. ### badger.Options - **WithBypassLockGuard** - Description: Option to bypass the directory lock guard. - Usage: `badger.WithBypassLockGuard(bypass bool)` - **WithLoadBloomsOnOpen** - Description: Option to load bloom filters on database open. - Usage: `badger.WithLoadBloomsOnOpen(load bool)` - **WithMaxBfCacheSize** - Description: Sets the maximum size for the bloom filter cache. - Usage: `badger.WithMaxBfCacheSize(size int64)` ``` -------------------------------- ### Benchmark Random Read Operation Source: https://github.com/dgraph-io/badger/blob/main/table/README.md Executes the BenchmarkRandomRead test to measure performance of random key-value lookups. ```sh go test -bench BenchmarkRandomRead$ -run ^$ -count 3 goos: linux goarch: amd64 pkg: github.com/dgraph-io/badger/table BenchmarkRandomRead-16 500000 2645 ns/op BenchmarkRandomRead-16 500000 2648 ns/op BenchmarkRandomRead-16 500000 2614 ns/op PASS ok github.com/dgraph-io/badger/table 50.850s ``` -------------------------------- ### Benchmark Read Operation Source: https://github.com/dgraph-io/badger/blob/main/table/README.md Executes the BenchmarkRead test to measure sequential read performance. ```sh $ go test -bench ^BenchmarkRead$ -run ^$ -count 3 goos: linux goarch: amd64 pkg: github.com/dgraph-io/badger/table BenchmarkRead-16 10 154074944 ns/op BenchmarkRead-16 10 154340411 ns/op BenchmarkRead-16 10 151914489 ns/op PASS ok github.com/dgraph-io/badger/table 22.467s ``` -------------------------------- ### Execute read-only transaction Source: https://github.com/dgraph-io/badger/blob/main/docs/quickstart.md Use DB.View to perform read-only operations with a consistent view of the database. ```go err := db.View(func(txn *badger.Txn) error { // your code here return nil }) ``` -------------------------------- ### Push Feature Branch to Origin Source: https://github.com/dgraph-io/badger/blob/main/CONTRIBUTING.md Push your committed changes from your local feature branch to your fork on GitHub. This makes your changes available for review. ```sh git push origin my_new_feature ``` -------------------------------- ### Rotate Master Key in Badger Source: https://github.com/dgraph-io/badger/blob/main/docs/encryption-at-rest.md Use the 'rotate' command to manually rotate master keys for an existing Badger database. Ensure the database is offline during this process. This command re-encrypts only the data keys. ```shell badger rotate --dir=badger_dir --old-key-path=old/path --new-key-path=new/path ``` -------------------------------- ### BadgerDB API Changes (v3.2103.0) Source: https://github.com/dgraph-io/badger/blob/main/CHANGELOG.md Overview of the new, modified, and removed API methods introduced in BadgerDB v3.2103.0. ```APIDOC ## New APIs ### Badger.DB - BanNamespace - BannedNamespaces - Ranges ### Badger.Options - FromSuperFlag - WithNumGoRoutines - WithNamespaceOffset - WithVLogPercentile ### Badger.Trie - AddMatch - DeleteMatch ### Badger.Table - StaleDataSize ### Badger.Table.Builder - AddStaleKey ### Badger.InitDiscardStats ## Changed APIs ### Badger.DB - Subscribe (Added option to subscribe with holes in prefixes) ### Badger.Options - WithValueThreshold ## Removed APIs ### Badger.DB - KeySplits ### Badger.Options - SkipVlog ``` -------------------------------- ### Create New Feature Branch Source: https://github.com/dgraph-io/badger/blob/main/CONTRIBUTING.md Create a new branch for your feature or bug fix. This isolates your changes and makes it easier to manage contributions. ```sh git checkout -b my_new_feature ``` -------------------------------- ### Calculate reversed timestamp for descending order Source: https://github.com/dgraph-io/badger/blob/main/docs/quickstart.md Use this calculation to store timestamps in a way that results in descending order during lexicographical scans. ```go reversedTimestamp := math.MaxInt64-time.Now().Unix() ``` -------------------------------- ### Benchmark Merged Read Operation Source: https://github.com/dgraph-io/badger/blob/main/table/README.md Executes the ReadMerged benchmark to measure performance when merging multiple tables. ```sh $ go test -bench ReadMerged -run ^$ -count 3 goos: linux goarch: amd64 pkg: github.com/dgraph-io/badger/table BenchmarkReadMerged-16 2 977588975 ns/op BenchmarkReadMerged-16 2 982140738 ns/op BenchmarkReadMerged-16 2 962046017 ns/op PASS ok github.com/dgraph-io/badger/table 27.433s ``` -------------------------------- ### Define a Merge Function Source: https://github.com/dgraph-io/badger/blob/main/docs/quickstart.md Create a function of type MergeFunc to combine existing and new byte slice values. ```go // Merge function to append one byte slice to another func add(originalValue, newValue []byte) []byte { return append(originalValue, newValue...) } ``` -------------------------------- ### Update Local Master Branch Source: https://github.com/dgraph-io/badger/blob/main/CONTRIBUTING.md Fetch the latest changes from the upstream repository and rebase your local master branch to ensure it's up-to-date before creating a new feature branch. ```sh git fetch upstream git checkout master git rebase upstream/master ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.