### RocksDB Reads and Writes Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/docs/_docs/getting-started.md Shows how to perform basic database operations: `Get`, `Put`, and `Delete`. This example demonstrates moving a value from one key to another. ```C++ std::string value; rocksdb::Status s = db->Get(rocksdb::ReadOptions(), key1, &value); if (s.ok()) s = db->Put(rocksdb::WriteOptions(), key2, value); if (s.ok()) s = db->Delete(rocksdb::WriteOptions(), key1); ``` -------------------------------- ### Opening a RocksDB Database Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/docs/_docs/getting-started.md Demonstrates how to open a RocksDB database, creating it if it doesn't exist. It also shows how to set options like `create_if_missing` and `error_if_exists`. ```C++ #include #include "rocksdb/db.h" rocksdb::DB* db; rocksdb::Options options; options.create_if_missing = true; rocksdb::Status status = rocksdb::DB::Open(options, "/tmp/testdb", &db); assert(status.ok()); ... ``` ```C++ options.error_if_exists = true; ``` -------------------------------- ### Start and Stop Tendis Service Source: https://github.com/tencent/tendis/blob/unstable/pack/README.md Shell scripts to start and stop the Tendis service. These are the primary commands for managing the Tendis instance. ```shell ./start.sh ./stop.sh ``` -------------------------------- ### Compile Tendis Examples Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/examples/README.md This snippet demonstrates how to navigate to the examples directory and compile all provided examples for the Tendis project after RocksDB has been compiled. ```bash cd examples/ make all ``` -------------------------------- ### Closing a RocksDB Database Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/docs/_docs/getting-started.md Illustrates the process of closing a RocksDB database by deleting the database object when it's no longer needed. ```C++ /* open the db as described above */ /* do something with db */ delete db; ``` -------------------------------- ### Handling RocksDB Status Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/docs/_docs/getting-started.md Explains the use of `rocksdb::Status` to check for errors in RocksDB operations. It shows how to verify if an operation was successful and retrieve error messages. ```C++ rocksdb::Status s = ...; if (!s.ok()) cerr << s.ToString() << endl; ``` -------------------------------- ### Install Project Dependencies Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/docs/README.md Installs all necessary project dependencies defined in the Gemfile for the documentation site. This command should be run from the 'docs' directory. ```shell bundle install ``` -------------------------------- ### Lua Table Constructor Example Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/lua/doc/manual.html An example demonstrating how a complex table constructor is equivalent to a sequence of table assignments. ```Lua a = { [f(1)] = g; "x", "y"; x = 1, f(x), [30] = 23; 45 } is equivalent to do local t = {} t[f(1)] = g t[1] = "x" t[2] = "y" t.x = 1 t[3] = f(x) t[30] = 23 t[4] = 45 a = t end ``` -------------------------------- ### Run Tendis Source: https://github.com/tencent/tendis/blob/unstable/README.md Instructions to start the Tendis server with a configuration file and connect to it using redis-cli. ```bash $ ./build/bin/tendisplus tendisplus.conf $ redis-cli -p 51002 ``` -------------------------------- ### CMake Installation Configuration Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/CMakeLists.txt Configures CMake for installing Tendis, setting default installation paths on Linux and managing package configuration files. It handles the installation of include directories, static and shared libraries, and export files for package management. ```cmake if(WIN32) option(ROCKSDB_INSTALL_ON_WINDOWS "Enable install target on Windows" OFF) endif() if(NOT WIN32 OR ROCKSDB_INSTALL_ON_WINDOWS) if(CMAKE_INSTALL_PREFIX_INITIALIZED_TO_DEFAULT) if(${CMAKE_SYSTEM_NAME} STREQUAL "Linux") # Change default installation prefix on Linux to /usr set(CMAKE_INSTALL_PREFIX /usr CACHE PATH "Install path prefix, prepended onto install directories." FORCE) endif() endif() include(GNUInstallDirs) include(CMakePackageConfigHelpers) set(package_config_destination ${CMAKE_INSTALL_LIBDIR}/cmake/rocksdb) configure_package_config_file( ${CMAKE_CURRENT_LIST_DIR}/cmake/RocksDBConfig.cmake.in RocksDBConfig.cmake INSTALL_DESTINATION ${package_config_destination} ) write_basic_package_version_file( RocksDBConfigVersion.cmake VERSION ${ROCKSDB_VERSION} COMPATIBILITY SameMajorVersion ) install(DIRECTORY include/rocksdb COMPONENT devel DESTINATION "${CMAKE_INSTALL_INCLUDEDIR}") install( TARGETS ${ROCKSDB_STATIC_LIB} EXPORT RocksDBTargets COMPONENT devel ARCHIVE DESTINATION "${CMAKE_INSTALL_LIBDIR}" INCLUDES DESTINATION "${CMAKE_INSTALL_INCLUDEDIR}" ) install( TARGETS ${ROCKSDB_SHARED_LIB} EXPORT RocksDBTargets COMPONENT runtime RUNTIME DESTINATION "${CMAKE_INSTALL_BINDIR}" LIBRARY DESTINATION "${CMAKE_INSTALL_LIBDIR}" INCLUDES DESTINATION "${CMAKE_INSTALL_INCLUDEDIR}" ) install( EXPORT RocksDBTargets COMPONENT devel DESTINATION ${package_config_destination} NAMESPACE RocksDB:: ) install( FILES ${CMAKE_CURRENT_BINARY_DIR}/RocksDBConfig.cmake ${CMAKE_CURRENT_BINARY_DIR}/RocksDBConfigVersion.cmake COMPONENT devel DESTINATION ${package_config_destination} ) endif() ``` -------------------------------- ### Run Jekyll Serve (Standard) Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/docs/README.md Starts the Jekyll development server to preview the documentation site locally. This command performs a full build, suitable for initial runs or structural changes. ```shell bundle exec jekyll serve ``` -------------------------------- ### Install RocksDB on Windows via vcpkg Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/INSTALL.md Installs RocksDB on Windows using the vcpkg package manager. ```bash vcpkg install rocksdb:x64-windows ``` -------------------------------- ### Run Jekyll Serve (Incremental) Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/docs/README.md Starts the Jekyll development server with incremental build support for faster updates when only content changes are made. ```shell bundle exec jekyll serve --incremental ``` -------------------------------- ### Lua Coroutine Execution Flow Example Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/lua/doc/manual.html Provides a concrete example demonstrating the step-by-step execution of Lua coroutines, including creation, multiple resumes, yielding, and handling of return values and errors. The output clearly illustrates the control flow between the main thread and the coroutine. ```Lua function foo (a) print("foo", a) return coroutine.yield(2*a) end co = coroutine.create(function (a,b) print("co-body", a, b) local r = foo(a+1) print("co-body", r) local r, s = coroutine.yield(a+b, a-b) print("co-body", r, s) return b, "end" end) print("main", coroutine.resume(co, 1, 10)) print("main", coroutine.resume(co, "r")) print("main", coroutine.resume(co, "x", "y")) print("main", coroutine.resume(co, "x", "y")) ``` -------------------------------- ### Install Bundler Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/docs/README.md Installs the Bundler tool, which manages project dependencies for Ruby projects. This command may require administrator privileges. ```shell gem install bundler ``` -------------------------------- ### Install Dependencies on Ubuntu Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/INSTALL.md Installs necessary development libraries for Tendis on Ubuntu systems using apt-get. ```bash sudo apt-get install libgflags-dev sudo apt-get install libsnappy-dev sudo apt-get install zlib1g-dev sudo apt-get install libbz2-dev sudo apt-get install liblz4-dev sudo apt-get install libzstd-dev ``` -------------------------------- ### Build RocksDB from Source on FreeBSD Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/INSTALL.md Compiles and installs RocksDB static library from source on FreeBSD. ```bash cd ~ git clone https://github.com/facebook/rocksdb.git cd rocksdb gmake static_lib ``` -------------------------------- ### Build RocksJava from Source on FreeBSD Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/INSTALL.md Compiles and installs RocksJava from source on FreeBSD. ```bash cd rocksdb export JAVA_HOME=/usr/local/openjdk7 gmake rocksdbjava ``` -------------------------------- ### Build RocksDB from Source on OpenBSD Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/INSTALL.md Compiles and installs RocksDB static library from source on OpenBSD. ```bash cd ~ git clone https://github.com/facebook/rocksdb.git cd rocksdb gmake static_lib ``` -------------------------------- ### Configure and Start MySQL with RocksDB Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/utilities/env_librados.md Steps to configure MySQL for RocksDB usage, including creating directories, setting up configuration files, initializing the data directory, and starting the MySQL server with RocksDB as the default storage engine. ```bash mkdir -p /etc/mysql mkdir -p /var/lib/mysql mkdir -p /etc/mysql/conf.d echo -e '[mysqld_safe]\nsyslog' > /etc/mysql/conf.d/mysqld_safe_syslog.cnf cp /usr/share/mysql/my-medium.cnf /etc/mysql/my.cnf sed -i 's#.*datadir.*#datadir = /var/lib/mysql#g' /etc/mysql/my.cnf chown mysql:mysql -R /var/lib/mysql mysql_install_db --user=mysql --ldata=/var/lib/mysql/ export CEPH_CONFIG_PATH="path/of/ceph/config/file" mysqld_safe -user=mysql --skip-innodb --rocksdb --default-storage-engine=rocksdb --default-tmp-storage-engine=MyISAM & mysqladmin -u root password mysql -u root -p ``` -------------------------------- ### Compile RocksDB Static Library Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/examples/README.md This snippet shows the command to compile the RocksDB static library, which is a prerequisite for compiling the Tendis examples. ```bash make static_lib ``` -------------------------------- ### Example: Getting Function Line Defined Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/lua/doc/manual.html Demonstrates how to use lua_getinfo with the '>S' option to retrieve the source file and the line number where a global function 'f' was defined. ```C lua_Debug ar; lua_getfield(L, LUA_GLOBALSINDEX, "f"); /* get global 'f' */ lua_getinfo(L, ">S", &ar); printf("%d\n", ar.linedefined); ``` -------------------------------- ### Tendis Benchmarking with memtier_benchmark (SET, GET, INCR, LPUSH, SADD, ZADD, HSET) Source: https://github.com/tencent/tendis/blob/unstable/README.md These commands demonstrate how to use memtier_benchmark to test various Tendis operations. They specify the number of threads, connections, server address, port, and command details including key prefixes, data sizes, and test duration. The examples cover SET, GET, INCR, LPUSH, SADD, ZADD, and HSET operations. ```bash ./memtier_benchmark -t 20 -c 50 -s 127.0.0.1 -p 51002 --distinct-client-seed --command="set __key__ __data__" --key-prefix="kv_" --key-minimum=1 --key-maximum=500000000 --random-data --data-size=128 --test-time=1800 ``` ```bash ./memtier_benchmark -t 20 -c 50 -s 127.0.0.1 -p 51002 --distinct-client-seed --command="get __key__" --key-prefix="kv_" --key-minimum=1 --key-maximum=500000000 --test-time=1800 ``` ```bash ./memtier_benchmark -t 20 -c 50 -s 127.0.0.1 -p 51002 --distinct-client-seed --command="incr __key__" --key-prefix="int_" --key-minimum=1 --key-maximum=1000000 --test-time=1800 ``` ```bash ./memtier_benchmark -t 20 -c 50 -s 127.0.0.1 -p 51002 --distinct-client-seed --command="lpush __key__ __data__" --key-prefix="list_" --key-minimum=1 --key-maximum=1000000 --random-data --data-size=128 --test-time=1800 ``` ```bash ./memtier_benchmark -t 20 -c 50 -s 127.0.0.1 -p 51002 --distinct-client-seed --command="sadd __key__ __data__" --key-prefix="set_" --key-minimum=1 --key-maximum=1000000 --random-data --data-size=128 --test-time=1800 ``` ```bash ./memtier_benchmark -t 20 -c 50 -s 127.0.0.1 -p 51002 --distinct-client-seed --command="zadd __key__ __key__ __data__" --key-prefix="" --key-minimum=1 --key-maximum=1000000 --random-data --data-size=128 --test-time=1800 ``` ```bash ./memtier_benchmark -t 20 -c 50 -s 127.0.0.1 -p 51002 --distinct-client-seed --command="hset __key__ __data__ __data__" --key-prefix="hash_" --key-minimum=1 --key-maximum=1000000 --random-data --data-size=128 --test-time=1800 ``` -------------------------------- ### Lua Standard Libraries Initialization Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/lua/doc/manual.html Explains how to open all standard Lua libraries or individual libraries through the C API. ```C // To have access to these libraries, the C host program should call the luaL_openlibs function, which opens all standard libraries. // Alternatively, it can open them individually by calling luaopen_base (for the basic library), luaopen_package (for the package library), luaopen_string (for the string library), luaopen_table (for the table library), luaopen_math (for the mathematical library), luaopen_io (for the I/O library), luaopen_os (for the Operating System library), and luaopen_debug (for the debug library). // These functions are declared in lualib.h and should not be called directly: you must call them like any other Lua C function, e.g., by using lua_call. ``` -------------------------------- ### Jemalloc Initialization Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/WINDOWS_PORT.md Describes the method for initializing Jemalloc before global/static object initialization by injecting an initialization routine into the `".CRT$XCT"` data segment. It also mentions queuing `je-uninit` to `atexit()`. ```APIDOC Jemalloc Initialization: - Initialization routine injected into ".CRT$XCT" for pre-global/static initialization. - je-uninit queued to atexit(). - Jemalloc redirecting new/delete global operators used by the linker. ``` -------------------------------- ### RDB Compilation and Setup Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/tools/rdb/README.md Instructions for compiling RDB, a NodeJS shell for RocksDB. It details the necessary requirements like static RocksDB library, libsnappy, Node.js, node-gyp, and Python 2. The installation process involves configuring and building with node-gyp. ```shell make static_lib node-gyp configure node-gyp build ``` -------------------------------- ### Start RADOS Cluster Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/utilities/env_librados.md Commands to start a RADOS cluster for testing, including stopping existing instances, clearing development data, and starting a new cluster with specified configurations. ```bash cd ceph-path/src ( ( ./stop.sh; rm -rf dev/*; CEPH_NUM_OSD=3 ./vstart.sh --short --localhost -n -x -d ; ) ) 2>&1 ``` -------------------------------- ### Documentation Page Creation Steps Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/docs/CONTRIBUTING.md Details the steps for adding new documentation pages, including file creation in the `./_docs/` directory, updating navigation, local testing, and pushing changes. ```markdown 1. Add your markdown file to the `./_docs/` folder. See `./doc-type-examples/docs-hello-world.md` for an example of the YAML header format. **If the `./_docs/` directory does not exist, create it**. - You can use folders in the `./_docs/` directory to organize your content if you want. 1. Update `_data/nav_docs.yml` to add your new document to the navigation bar. Use the `docid` you put in your doc markdown in as the `id` in the `_data/nav_docs.yml` file. 1. [Run the site locally](./README.md) to test your changes. It will be at `http://127.0.0.1/docs/your-new-doc-permalink.html` 1. Push your changes to GitHub. ``` -------------------------------- ### RocksDB Get() with Mutex Lock Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/docs/_posts/2014-06-27-avoid-expensive-locks-in-get.markdown Original implementation of RocksDB's Get() operation that uses a mutex lock to safely access the SuperVersion. This approach caused performance bottlenecks on multi-core systems. ```C++ mutex_.Lock(); auto* s = super_version_->Ref(); mutex_.Unlock(); ``` -------------------------------- ### LevelDB Migration to RocksDB: RocksDB Code from Scratch Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/docs/_posts/2015-01-16-migrating-from-leveldb-to-rocksdb-2.markdown This C++ code snippet demonstrates how to initialize and open a RocksDB database directly, without using the LevelDB options conversion utility. ```c++ #include #include "rocksdb/db.h" #include "rocksdb/table.h" using namespace rocksdb; int main(int argc, char** argv) { DB *db; Options opt; opt.create_if_missing = true; opt.max_open_files = 1000; BlockBasedTableOptions topt; topt.block_size = 4096; opt.table_factory.reset(NewBlockBasedTableFactory(topt)); Status s = DB::Open(opt, "/tmp/mydb_rocks", &db); delete db; } ``` -------------------------------- ### RocksDB Get() with Thread-Local Storage Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/docs/_posts/2014-06-27-avoid-expensive-locks-in-get.markdown Optimized implementation of RocksDB's Get() operation using thread-local storage and atomic versioning. This avoids mutex locks for most read operations, improving performance and scalability. ```C++ SuperVersion* s = thread_local_->Get(); if (s->version_number != super_version_number_.load()) { // slow path, cleanup of current super version is omitted mutex_.Lock(); s = super_version_->Ref(); mutex_.Unlock(); } ``` -------------------------------- ### Install gflags on CentOS/RHEL (Source) Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/INSTALL.md Builds and installs gflags from source on CentOS/RHEL systems. ```bash git clone https://github.com/gflags/gflags.git cd gflags git checkout v2.0 ./configure && make && sudo make install ``` -------------------------------- ### Install RocksDB Dependencies for RocksJava on FreeBSD Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/INSTALL.md Installs OpenJDK 7 for RocksJava dependency on FreeBSD. ```bash export BATCH=yes cd /usr/ports/java/openjdk7 && make install ``` -------------------------------- ### Go Memory Usage Example Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/govendor/src/github.com/shirou/gopsutil/README.rst Demonstrates how to retrieve virtual memory statistics using gopsutil's mem package. It shows how to access total memory, free memory, and used percentage, and how to print the memory information as a string or JSON. ```go import ( "fmt" "github.com/shirou/gopsutil/mem" ) func main() { v, _ := mem.VirtualMemory() // almost every return value is a struct fmt.Printf("Total: %v, Free:%v, UsedPercent:%f%%\n", v.Total, v.Free, v.UsedPercent) // convert to JSON. String() is also implemented fmt.Println(v) } ``` -------------------------------- ### Install RocksDB Dependencies on OpenBSD Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/INSTALL.md Installs necessary dependencies for RocksDB on OpenBSD using pkg_add. ```bash pkg_add gmake gflags snappy bzip2 lz4 zstd git jdk bash findutils gnuwatch ``` -------------------------------- ### Lua Module Loading Search Strategy Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/lua/doc/manual.html Details the multi-step search process for modules. It prioritizes `package.preload`, then Lua files specified by `package.path`, followed by C libraries specified by `package.cpath`, and finally an all-in-one loader. ```lua -- Search Order for require: -- 1. package.preload[modname] -- 2. Lua files using package.path (e.g., "./?.lua;./?.lc;/usr/local/?/init.lua") -- 3. C libraries using package.cpath (e.g., "./?.so;./?.dll;/usr/local/?/init.so") -- - Searches for luaopen_ function. -- 4. All-in-one loader (searches C path for root name, then looks for submodule open function). ``` -------------------------------- ### Install RocksDB Dependencies on FreeBSD Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/INSTALL.md Installs necessary dependencies for RocksDB on FreeBSD using the Ports system. ```bash export BATCH=YES cd /usr/ports/devel/gmake && make install cd /usr/ports/devel/gflags && make install cd /usr/ports/archivers/snappy && make install cd /usr/ports/archivers/bzip2 && make install cd /usr/ports/archivers/liblz4 && make install cd /usr/ports/archivesrs/zstd && make install cd /usr/ports/devel/git && make install ``` -------------------------------- ### Lua Package Loading and Management Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/lua/doc/manual.html Dynamically links the host program with a C library and manages module loading paths and preloaded modules. ```APIDOC package.loadlib (libname, funcname) Dynamically links the host program with the C library `libname`. Inside this library, looks for a function `funcname` and returns this function as a C function. Parameters: - libname: The complete file name of the C library, including path and extension. - funcname: The exact name exported by the C library. Notes: - This is a low-level function that bypasses the package and module system. - Not supported by ANSI C; available on platforms like Windows, Linux, Mac OS X, Solaris, BSD, and other Unix systems supporting `dlfcn`. package.path The path used by `require` to search for a Lua loader. Initialization: - Initialized with the value of the environment variable `LUA_PATH` or a default path from `luaconf.h`. - Any "";" in `LUA_PATH` is replaced by the default path. package.preload A table to store loaders for specific modules. package.seeall (module) Sets a metatable for `module` with its `__index` field referring to the global environment, allowing the module to inherit values from the global environment. Usage: To be used as an option to function `module`. ``` -------------------------------- ### Tencent Tendis Executable Configuration Source: https://github.com/tencent/tendis/blob/unstable/src/tendisplus/replication/CMakeLists.txt Defines the executable 'binlog_tool' with its source file 'binlog_tool.cpp'. It links against 'glog', 'kvstore', 'rocks_kvstore', 'varint', 'utils_common', and 'commands'. Static linking flags are also set for GCC andstdc++ libraries. ```cmake add_executable(binlog_tool binlog_tool.cpp) target_link_libraries(binlog_tool glog kvstore rocks_kvstore varint utils_common commands) set_target_properties(binlog_tool PROPERTIES LINK_FLAGS "-static-libgcc -static-libstdc++") ``` -------------------------------- ### Build Tendis Source: https://github.com/tencent/tendis/blob/unstable/README.md Steps to clone the Tendis repository, initialize submodules, configure the build with CMake, and compile the project using make. ```bash $ git clone https://github.com/Tencent/tendis.git --recursive $ git submodule update --init --recursive $ mkdir build $ cd build & cmake .. $ make -j12 ``` -------------------------------- ### Blog Post Creation Steps Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/docs/CONTRIBUTING.md Outlines the four-step process for adding a new blog post, including file creation, author information, local testing, and pushing changes. ```markdown 1. Create your blog post in `./_posts/` in markdown (file extension `.md` or `.markdown`). See current posts in that folder or `./doc-type-examples/2016-04-07-blog-post-example.md` for an example of the YAML format. **If the `./_posts` directory does not exist, create it**. - You can add a `` tag in the middle of your post such that you show only the excerpt above that tag in the main `/blog` index on your page. 1. If you have not authored a blog post before, modify the `./_data/authors.yml` file with the `author` id you used in your blog post, along with your full name and Facebook ID to get your profile picture. 1. [Run the site locally](./README.md) to test your changes. It will be at `http://127.0.0.1/blog/your-new-blog-post-title.html` 1. Push your changes to GitHub. ``` -------------------------------- ### Install RocksDB on OS X via Homebrew Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/INSTALL.md Installs RocksDB on OS X using Homebrew, including GCC 4.8. ```bash xcode-select --install brew tap homebrew/versions brew install gcc48 --use-llvm brew install rocksdb ``` -------------------------------- ### Benchmark Executable Creation Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/CMakeLists.txt This section defines and builds benchmark executables. It iterates through a list of benchmark source files, creates an executable for each, links necessary libraries (gtest_rocks, LIBS, rt), and includes the 'testharness' object library. ```cmake set(BENCHMARKS cache/cache_bench.cc memtable/memtablerep_bench.cc tools/db_bench.cc table/table_reader_bench.cc utilities/column_aware_encoding_exp.cc utilities/persistent_cache/hash_table_bench.cc) add_library(testharness OBJECT util/testharness.cc) foreach(sourcefile ${BENCHMARKS}) get_filename_component(exename ${sourcefile} NAME_WE) add_executable(${exename}${ARTIFACT_SUFFIX} ${sourcefile} $) target_link_libraries(${exename}${ARTIFACT_SUFFIX} gtest_rocks ${LIBS} rt) endforeach(sourcefile ${BENCHMARKS}) ``` -------------------------------- ### Install Dependencies on CentOS/RHEL Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/INSTALL.md Installs necessary development libraries and tools for Tendis on CentOS/RHEL systems using yum and manual compilation. ```bash yum install gcc48-c++ sudo yum install snappy snappy-devel sudo yum install zlib zlib-devel sudo yum install bzip2 bzip2-devel sudo yum install lz4-devel sudo yum install libasan # For zstandard wget https://github.com/facebook/zstd/archive/v1.1.3.tar.gz mv v1.1.3.tar.gz zstd-1.1.3.tar.gz tar zxvf zstd-1.1.3.tar.gz cd zstd-1.1.3 make && sudo make install ``` -------------------------------- ### gopsutil HostInfo Documentation Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/govendor/src/github.com/shirou/gopsutil/README.rst Provides details on the HostInfo function, available on Linux systems. It returns a struct containing hostname, uptime, OS details, virtualization information, and more. ```APIDOC host/HostInfo() (linux) - Hostname - Uptime - Procs - OS (ex: "linux") - Platform (ex: "ubuntu", "arch") - PlatformFamily (ex: "debian") - PlatformVersion (ex: "Ubuntu 13.10") - VirtualizationSystem (ex: "LXC") - VirtualizationRole (ex: "guest"/"host") ``` -------------------------------- ### Tendis QPS on Different Payload (SET and GET) Source: https://github.com/tencent/tendis/blob/unstable/README.md This command set measures Tendis QPS for SET and GET operations with a focus on different payload sizes. The benchmark runs for half an hour for each operation. The results indicate that due to sufficient memory, GET QPS remains consistent across different payload sizes. ```bash ./memtier_benchmark -t 20 -c 50 -s 127.0.0.1 -p 51002 --distinct-client-seed --command="set __key__ __data__" --key-prefix="kv_" --command-key-pattern=R --random-data --data-size=128 --test-time=1800 ``` ```bash ./memtier_benchmark -t 20 -c 50 -s 127.0.0.1 -p 51002 --distinct-client-seed --command="get __key__" --key-prefix="kv_" --command-key-pattern=R --test-time=1800 ``` -------------------------------- ### Backup RocksDB using BackupEngine Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/docs/_posts/2014-03-27-how-to-backup-rocksdb.markdown Shows how to back up a RocksDB database using the BackupEngine directly. This method involves creating a BackupEngine instance, performing database operations, and then creating a new backup. ```C++ #include "rocksdb/db.h" #include "utilities/backupable_db.h" using namespace rocksdb; DB* db; DB::Open(Options(), "/tmp/rocksdb", &db); db->Put(...); // do your thing BackupEngine* backup_engine = BackupEngine::NewBackupEngine(Env::Default(), BackupableDBOptions("/tmp/rocksdb_backup")); backup_engine->CreateNewBackup(db); delete db; delete backup_engine; ``` -------------------------------- ### HostInfo Metrics Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/govendor/src/github.com/shirou/gopsutil/README.rst Provides system information such as hostname, uptime, operating system details, and virtualization type. Support varies across different operating systems. ```APIDOC HostInfo: hostname: Hostname of the system (Linux, FreeBSD, MacOSX, Windows) uptime: System uptime (Linux, FreeBSD, MacOSX) proces: Process information (Linux, FreeBSD) os: Operating system name (Linux, FreeBSD, MacOSX, Windows) platform: Operating system platform (Linux, FreeBSD, MacOSX) platformfamiliy: Operating system platform family (Linux, FreeBSD, MacOSX) virtualization: Virtualization technology used (Linux) ``` -------------------------------- ### Lua Coroutine Management Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/lua/doc/manual.html Functions for starting and resuming coroutines, and checking their status. lua_resume starts or resumes a coroutine, returning its status. lua_status returns the current status of a thread. ```APIDOC lua_resume: int lua_resume (lua_State *L, int narg); Starts and resumes a coroutine in a given thread. Parameters: L: The lua_State pointer. narg: The number of arguments to pass to the coroutine. Returns: LUA_YIELD if the coroutine yields. 0 if the coroutine finishes execution without errors. An error code in case of errors. lua_status: int lua_status (lua_State *L); Returns the status of the thread L. Returns: 0 for a normal thread. An error code if the thread finished with an error. LUA_YIELD if the thread is suspended. ``` -------------------------------- ### AltiVec Optimization for ppc64le Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/CMakeLists.txt Enables AltiVec and Power8 optimizations for ppc64le processors by checking for the '-maltivec' compiler flag and setting appropriate C and C++ flags. ```cmake include(CheckCCompilerFlag) if(CMAKE_SYSTEM_PROCESSOR MATCHES "ppc64le") CHECK_C_COMPILER_FLAG("-maltivec" HAS_ALTIVEC) if(HAS_ALTIVEC) message(STATUS " HAS_ALTIVEC yes") set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -maltivec") set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -maltivec") set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -mcpu=power8") set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -mcpu=power8") endif(HAS_ALTIVEC) endif(CMAKE_SYSTEM_PROCESSOR MATCHES "ppc64le") ``` -------------------------------- ### RocksDB Get() Locking Optimization Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/HISTORY.md Optimizes the locking mechanism for the `Get()` operation in RocksDB, resulting in a 1.5x increase in QPS for certain workloads. This reduces contention and improves read throughput. ```C++ // Optimized locking for `Get()` -- [1fdb3f](https://github.com/facebook/rocksdb/commit/1fdb3f7dc60e96394e3e5b69a46ede5d67fb976c) -- 1.5x QPS increase for some workloads ``` -------------------------------- ### Basic MySQL Operations with RocksDB Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/utilities/env_librados.md Demonstrates basic SQL commands after connecting to a MySQL instance configured with RocksDB, including showing databases, creating a database and table, inserting data, and selecting data. ```sql show databases; create database testdb; use testdb; show tables; CREATE TABLE tbl (id INT AUTO_INCREMENT primary key, str VARCHAR(32)); insert into tbl values (1, "val2"); select * from tbl; ``` -------------------------------- ### MongoDB RocksDB Storage Engine Integration (Get) Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/docs/_posts/2015-02-27-write-batch-with-index.markdown A specific C++ code snippet from MongoDB's RocksDB storage engine demonstrating how WriteBatchWithIndex is used to implement the read-your-own-write functionality for Get operations. ```C++ auto it = _writeBatchWithIndex.NewIterator(new rocksdb::ReadOptions()); it->Seek(key); if (it->Valid() && it->key() == key) { value = it->value(); } else { // Fallback to RocksDB's Get status = _db->Get(readOptions, key, &value); } ``` -------------------------------- ### Windows Platform Source Files Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/CMakeLists.txt Includes Windows-specific source files for I/O, environment, porting, and threading when the WIN32 condition is met. Also includes jemalloc support if WITH_JEMALLOC is enabled. ```CMake if(WIN32) list(APPEND SOURCES port/win/io_win.cc port/win/env_win.cc port/win/env_default.cc port/win/port_win.cc port/win/win_logger.cc port/win/win_thread.cc port/win/xpress_win.cc) if(WITH_JEMALLOC) list(APPEND SOURCES port/win/win_jemalloc.cc) endif() else() list(APPEND SOURCES port/port_posix.cc env/env_posix.cc env/io_posix.cc) endif() ``` -------------------------------- ### RocksDB Level 5 Read Latency Histogram Example Source: https://github.com/tencent/tendis/blob/unstable/src/thirdparty/rocksdb-5.13.4/rocksdb/docs/_posts/2015-11-16-analysis-file-read-latency-by-level.markdown An example of a RocksDB read latency histogram output for Level 5, showing statistics like count, average, standard deviation, median, max, and percentiles. It also details the distribution of read latencies across different microsecond ranges. ```text Level 5 read latency histogram (micros): Count: 7133903059 Average: 480.4357 StdDev: 309.18 Min: 0.0000 Median: 551.1491 Max: 224142.0000 Percentiles: P50: 551.15 P75: 651.44 P99: 996.52 P99.9: 2073.07 P99.99: 3196.32 —————————————————— [ 0, 1 ) 28587385 0.401% 0.401% [ 1, 2 ) 686572516 9.624% 10.025% ## [ 2, 3 ) 567317522 7.952% 17.977% ## [ 3, 4 ) 44979472 0.631% 18.608% [ 4, 5 ) 50379685 0.706% 19.314% [ 5, 6 ) 64930061 0.910% 20.224% [ 6, 7 ) 22613561 0.317% 20.541% …………more………. ```