### Install Afero Go Package Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/go/src/mapd/vendor/github.com/spf13/afero/README.md Use 'go get' to install the Afero library. Include it in your application using the import statement. ```bash $ go get github.com/spf13/afero ``` -------------------------------- ### Install etcd/clientv3 Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/go/src/mapd/vendor/go.etcd.io/etcd/clientv3/README.md Install the official Go etcd client v3 using go get. ```bash go get go.etcd.io/etcd/clientv3 ``` -------------------------------- ### Install bbolt Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/go/src/mapd/vendor/go.etcd.io/bbolt/README.md Install the bbolt library and command-line utility using go get. ```sh $ go get go.etcd.io/bbolt/... ``` -------------------------------- ### Install go-codec Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/go/src/mapd/vendor/github.com/ugorji/go/codec/README.md Install the go-codec library using the go get command. Ensure you are using a compatible Go SDK version. ```bash go get github.com/ugorji/go/codec ``` -------------------------------- ### Install gojsonschema Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/go/src/mapd/vendor/github.com/xeipuuv/gojsonschema/README.md Use 'go get' to install the gojsonschema library. ```bash go get github.com/xeipuuv/gojsonschema ``` -------------------------------- ### Install Gabs Utility Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/go/src/mapd/vendor/github.com/Jeffail/gabs/README.md Use 'go get' to install the Gabs library. This command fetches and installs the specified package. ```bash go get github.com/Jeffail/gabs ``` -------------------------------- ### Install etcd/client Go Package Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/go/src/mapd/vendor/go.etcd.io/etcd/client/README.md Use `go get` to install the etcd/client Go package. It is recommended to vendor dependencies for full compatibility. ```bash go get go.etcd.io/etcd/client ``` -------------------------------- ### Install graceful Package Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/go/src/mapd/vendor/gopkg.in/tylerb/graceful.v1/README.md Use 'go get' to install the graceful package. This command fetches and installs the specified package and its dependencies. ```bash go get gopkg.in/tylerb/graceful.v1 ``` -------------------------------- ### Install Sample Data Source: https://github.com/heavyai/heavydb/blob/master/CMakeLists.txt Installs the sample data to the current directory. ```cmake install(PROGRAMS insert_sample_data DESTINATION ".") ``` -------------------------------- ### Install and Import YAML Library for Go Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/go/src/mapd/vendor/github.com/ghodss/yaml/README.md Instructions for installing the YAML library using go get and importing it into your Go project. ```bash go get github.com/ghodss/yaml ``` ```go import "github.com/ghodss/yaml" ``` -------------------------------- ### Install multierr Package Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/go/src/mapd/vendor/go.uber.org/multierr/README.md Use the go get command to install the multierr package. This command fetches and installs the latest stable version. ```go go get -u go.uber.org/multierr ``` -------------------------------- ### Install go-metrics Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/go/src/mapd/vendor/github.com/rcrowley/go-metrics/README.md Standard Go installation command for the go-metrics library. ```sh go get github.com/rcrowley/go-metrics ``` -------------------------------- ### Install Zap Logging Library Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/go/src/mapd/vendor/go.uber.org/zap/README.md Use go get to install the Zap logging library. Ensure your Go version is one of the two most recent minor versions. ```go go get -u go.uber.org/zap ``` -------------------------------- ### Install Documentation and Headers Source: https://github.com/heavyai/heavydb/blob/master/CMakeLists.txt Installs license files, UDF/UDTF headers, and logger headers to their respective destinations. ```cmake install(DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/ThirdParty/licenses" DESTINATION "ThirdParty" COMPONENT "doc") install(FILES Shared/funcannotations.h DESTINATION "Shared/" COMPONENT "include") install(FILES Shared/InlineNullValues.h DESTINATION "Shared/" COMPONENT "include") install(FILES Logger/Logger.h DESTINATION "Logger/" COMPONENT "include") ``` -------------------------------- ### Install HeavyDB from GitHub Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/go/src/mapd/vendor/gopkg.in/fsnotify.v1/CONTRIBUTING.md Install the HeavyDB package from GitHub using the 'go get' command. This ensures you are using the original import path, which is recommended for smooth development. ```bash go get -u github.com/fsnotify/fsnotify ``` -------------------------------- ### Install BurntSushi/toml Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/go/src/mapd/vendor/github.com/BurntSushi/toml/README.md Use 'go get' to install the TOML package. ```bash go get github.com/BurntSushi/toml ``` -------------------------------- ### Install mapstructure Go Library Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/go/src/mapd/vendor/github.com/mitchellh/mapstructure/README.md Use 'go get' to install the mapstructure library. This is the standard method for obtaining Go packages. ```bash $ go get github.com/mitchellh/mapstructure ``` -------------------------------- ### Multithreaded Benchmark Setup Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/googlebenchmark/docs/user_guide.md Use `state.thread_index() == 0` to perform setup and teardown only once in multithreaded benchmarks. This ensures operations are executed before all threads start and after all threads finish. ```c++ static void BM_MultiThreaded(benchmark::State& state) { if (state.thread_index() == 0) { // Setup code here. } for (auto _ : state) { // Run the test as normal. } if (state.thread_index() == 0) { // Teardown code here. } } BENCHMARK(BM_MultiThreaded)->Threads(2); ``` -------------------------------- ### Install Google Benchmark Globally Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/googlebenchmark/README.md Command to install the benchmark library globally after building. This requires administrator privileges. ```bash sudo cmake --build "build" --config Release --target install ``` -------------------------------- ### Install gRPC-Go Package Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/go/src/mapd/vendor/google.golang.org/grpc/README.md Use this command to install the gRPC-Go library. Ensure Go and your Go workspace are set up. ```bash $ go get -u google.golang.org/grpc ``` -------------------------------- ### Python Example for Kafka Clickthrough Source: https://github.com/heavyai/heavydb/wiki/Tutorials-&-Demos This example demonstrates a 'clickthrough' scenario involving Kafka and OmniSciDB, likely for real-time data ingestion and analysis. Specific implementation details depend on the Kafka and OmniSciDB integration setup. ```python # This is a placeholder for a Python example related to Kafka clickthrough. # Actual code would involve Kafka client libraries (e.g., kafka-python) # and OmniSciDB client library (e.g., pymapd) to: # 1. Consume messages from Kafka topics. # 2. Process or transform the messages as needed. # 3. Insert the processed data into OmniSciDB tables. # Example structure: # import pymapd # from kafka import KafkaConsumer # # Kafka Consumer Setup # consumer = KafkaConsumer( # 'your_topic_name', # bootstrap_servers=['your_kafka_broker:9092'], # auto_offset_reset='earliest', # enable_auto_commit=True, # group_id='omnisci-group', # value_deserializer=lambda x: x.decode('utf-8')) # # OmniSciDB Connection Setup # db = pymapd.connect(conn_str="mapd://localhost:9092/mapd", user="mapd", password="mapd") # # Processing Loop # for message in consumer: # data = message.value # # Process 'data' here (e.g., parse JSON, extract fields) # processed_data = process_kafka_message(data) # # # Insert into OmniSciDB # # Assuming a table 'kafka_data' with appropriate columns # db.execute("INSERT INTO kafka_data (col1, col2) VALUES (?, ?)", processed_data['field1'], processed_data['field2']) # print(f"Inserted data: {processed_data}") # Placeholder function def process_kafka_message(raw_data): # Replace with actual data processing logic print(f"Processing raw data: {raw_data}") # Example: Assuming raw_data is a JSON string # import json # data_dict = json.loads(raw_data) # return {'field1': data_dict['key1'], 'field2': data_dict['key2']} return {'field1': 'value1', 'field2': 'value2'} # Dummy return print("Kafka clickthrough example structure outlined. Implement with actual Kafka and OmniSciDB logic.") ``` -------------------------------- ### Basic CORS Handler Setup in Go Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/go/src/mapd/vendor/github.com/rs/cors/README.md Sets up a basic HTTP server with CORS enabled using default options. This handler accepts all origins with simple methods (GET, POST). ```go package main import ( "net/http" "github.com/rs/cors" ) func main() { mux := http.NewServeMux() mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) { w.Header().Set("Content-Type", "application/json") w.Write([]byte("{\"hello\": \"world\"}")) }) // cors.Default() setup the middleware with default options being // all origins accepted with simple methods (GET, POST). See // documentation below for more options. handler := cors.Default().Handler(mux) http.ListenAndServe(":8080", handler) } ``` -------------------------------- ### Build warpcore Tests, Benchmarks, and Examples Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/warpcore/README.md Builds the tests, benchmarks, and examples for warpcore. Ensure you have set the WARPCORE_ROOT environment variable. ```bash cd $WARPCORE_ROOT mkdir -p build cd build cmake .. -DWARPCORE_BUILD_TESTS=ON -DDWARPCORE_BUILD_BENCHMARKS=ON -DDWARPCORE_BUILD_EXAMPLES=ON make ``` -------------------------------- ### Install Docker README Source: https://github.com/heavyai/heavydb/blob/master/CMakeLists.txt Installs the README file for the Docker directory. ```cmake install(FILES docker/README.md DESTINATION "docker") ``` -------------------------------- ### Install Project Components Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/googlebenchmark/src/CMakeLists.txt Installs the built targets, headers, CMake configuration files, and pkg-config files to the appropriate locations based on CMAKE_INSTALL_PREFIX. ```cmake if (BENCHMARK_ENABLE_INSTALL) # Install target (will install the library to specified CMAKE_INSTALL_PREFIX variable) install( TARGETS ${targets_to_export} EXPORT ${targets_export_name} ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR} LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR} RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR} INCLUDES DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}) install( DIRECTORY "${PROJECT_SOURCE_DIR}/include/benchmark" "${PROJECT_BINARY_DIR}/include/benchmark" DESTINATION ${CMAKE_INSTALL_INCLUDEDIR} FILES_MATCHING PATTERN "*.*h") install( FILES "${project_config}" "${version_config}" DESTINATION "${CMAKE_INSTALL_LIBDIR}/cmake/${PROJECT_NAME}") install( FILES "${pkg_config}" DESTINATION "${CMAKE_INSTALL_LIBDIR}/pkgconfig") install( EXPORT "${targets_export_name}" NAMESPACE "${namespace}" DESTINATION "${CMAKE_INSTALL_LIBDIR}/cmake/${PROJECT_NAME}") endif() ``` -------------------------------- ### Install tomll CLI Tool Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/go/src/mapd/vendor/github.com/pelletier/go-toml/README.md Install the 'tomll' command-line tool for linting TOML files. Ensure your Go environment is configured for installing tools. ```bash go install github.com/pelletier/go-toml/cmd/tomll tomll --help ``` -------------------------------- ### Start HeavyDB Server Source: https://github.com/heavyai/heavydb/blob/master/README.md Command to start the HeavyDB server. Assumes execution from the 'build' directory. ```bash ./bin/heavydb ``` -------------------------------- ### Install Zap Dependencies Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/go/src/mapd/vendor/go.uber.org/zap/CONTRIBUTING.md Install all necessary dependencies for Zap using the provided make command. ```bash make dependencies ``` -------------------------------- ### Install Query Engine Files Source: https://github.com/heavyai/heavydb/blob/master/QueryEngine/CMakeLists.txt Installs generated header and runtime files for the Query Engine component. ```cmake install(FILES ${CMAKE_CURRENT_BINARY_DIR}/heavydbTypes.h ${CMAKE_CURRENT_BINARY_DIR}/Utils/FlatBuffer.h ${CMAKE_CURRENT_BINARY_DIR}/RuntimeFunctions.bc ${CMAKE_CURRENT_BINARY_DIR}/GeosRuntime.bc ${CMAKE_CURRENT_BINARY_DIR}/H3Runtime.bc ${CMAKE_CURRENT_BINARY_DIR}/ExtensionFunctions.ast DESTINATION QueryEngine COMPONENT "QE") ``` -------------------------------- ### Install StatHat Go Client Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/go/src/mapd/vendor/github.com/rcrowley/go-metrics/README.md Additional installation command for the StatHat Go client, required for StatHat support. ```sh go get github.com/stathat/go ``` -------------------------------- ### Basic Benchmark Example Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/googlebenchmark/README.md A simple C++ example demonstrating how to define and register a benchmark function. Ensure `benchmark/benchmark.h` is included and `BENCHMARK_MAIN()` is called. ```c++ #include static void BM_SomeFunction(benchmark::State& state) { // Perform setup here for (auto _ : state) { // This code gets timed SomeFunction(); } } // Register the function as a benchmark BENCHMARK(BM_SomeFunction); // Run the benchmark BENCHMARK_MAIN(); ``` -------------------------------- ### Install Dependencies Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/googlebenchmark/docs/tools.md Install the required Python packages for the benchmark tools using pip. ```bash pip3 install -r requirements.txt ``` -------------------------------- ### Example Log Entry Source: https://github.com/heavyai/heavydb/blob/master/docs/source/components/logger.md Provides a concrete example of a log entry, demonstrating the fields and their typical appearance. ```cpp 2019-09-18T16:25:25.659248 I 26481 5 DBHandler.cpp:181 HEAVY.AI Server 4.9.0dev-20190918-bd97353685 ``` -------------------------------- ### Install DBEngine Library and Headers Source: https://github.com/heavyai/heavydb/blob/master/Embedded/CMakeLists.txt Configures the installation of the DBEngine library and its header files to specific destinations and components. ```cmake install(TARGETS DBEngine DESTINATION "lib" COMPONENT "DBE") install(FILES DBEngine.h DBETypes.h DESTINATION "Embedded" COMPONENT "include") install(FILES Python/DBEngine.pxd DESTINATION "Embedded" COMPONENT "include") ``` -------------------------------- ### RESTORE TABLE Syntax Example Source: https://github.com/heavyai/heavydb/blob/master/docs/source/data_model/dump_restore.md This example shows the basic syntax for the RESTORE TABLE statement, restoring a table from a specified tar file. ```sql RESTORE TABLE t FROM '/tmp/Orz_.tgz'; ``` -------------------------------- ### Initializing OsFs Backend Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/go/src/mapd/vendor/github.com/spf13/afero/README.md Demonstrates how to initialize the OsFs backend, which wraps native OS calls. This is useful for production environments or when direct OS interaction is needed. ```go appfs := afero.NewOsFs() appfs.MkdirAll("src/a", 0755)) ``` -------------------------------- ### Serve Built Docs Locally Source: https://github.com/heavyai/heavydb/blob/master/docs/README.md Run a simple HTTP server from the build directory to preview the documentation locally. ```bash python -m http.server ``` -------------------------------- ### Start OmniSciDB with startomnisci Wrapper Source: https://github.com/heavyai/heavydb/wiki/OmniSciDB-Documentation Use the startomnisci wrapper script to initialize the data directory, start the server, and optionally download sample data. Assumes execution from the build directory. ```bash ../startomnisci ``` -------------------------------- ### RPC Client Codec Setup Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/go/src/mapd/vendor/github.com/ugorji/go/codec/README.md Demonstrates how to establish an RPC client connection and create a client codec using either Go RPC or MsgpackSpec RPC. ```go //RPC Communication (client side) conn, err = net.Dial("tcp", "localhost:5555") rpcCodec := codec.GoRpc.ClientCodec(conn, h) //OR rpcCodec := codec.MsgpackSpecRpc.ClientCodec(conn, h) client := rpc.NewClientWithCodec(rpcCodec) ``` -------------------------------- ### DUMP TABLE Syntax Example Source: https://github.com/heavyai/heavydb/blob/master/docs/source/data_model/dump_restore.md This example demonstrates the basic syntax for the DUMP TABLE statement, archiving a table to a specified file path. ```sql DUMP TABLE t TO '/tmp/Orz_.tgz'; ``` -------------------------------- ### Configure Codec Handles and Extensions Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/go/src/mapd/vendor/github.com/ugorji/go/codec/README.md Demonstrates how to create and configure different codec handles (Binc, Msgpack, Cbor) and set up custom extensions, such as for time.Time in Msgpack. ```go var ( bh codec.BincHandle mh codec.MsgpackHandle ch codec.CborHandle ) mh.MapType = reflect.TypeOf(map[string]interface{}(nil)) // configure extensions // e.g. for msgpack, define functions and enable Time support for tag 1 // mh.SetExt(reflect.TypeOf(time.Time{}), 1, myExt) ``` -------------------------------- ### RPC Server Codec Setup Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/go/src/mapd/vendor/github.com/ugorji/go/codec/README.md Illustrates how to set up an RPC server codec using either the standard Go RPC or MsgpackSpec RPC, integrating with net/rpc. ```go //RPC Server go func() { for { conn, err := listener.Accept() rpcCodec := codec.GoRpc.ServerCodec(conn, h) //OR rpcCodec := codec.MsgpackSpecRpc.ServerCodec(conn, h) rpc.ServeCodec(rpcCodec) } }() ``` -------------------------------- ### Simple C++ Example for Stream Inserts Source: https://github.com/heavyai/heavydb/wiki/Tutorials-&-Demos A simplified C++ program for stream inserts into OmniSciDB. This example focuses on the core logic, assuming necessary setup. ```cpp #include "omnisci/mapd.h" #include #include int main() { // Connection parameters const char* host = "localhost"; const int port = 9092; const char* user = "mapd"; const char* password = "mapd"; const char* dbname = "mapd"; std::shared_ptr conn; try { conn = std::make_shared(host, port, dbname, user, password); std::cout << "Connected to OmniSciDB." << std::endl; // Assume table 'simple_stream_table' exists with columns: id INT, data TEXT const char* table_name = "simple_stream_table"; // Prepare data for insertion std::vector> data_rows; data_rows.push_back({"1", "first data point"}); data_rows.push_back({"2", "second data point"}); // Construct the INSERT statement for stream insert // The exact syntax for stream inserts might vary. This is a conceptual example. // Often, a dedicated API or method is used for efficient batch/stream inserts. std::string insert_sql = "INSERT INTO " + std::string(table_name) + " (id, data) VALUES (?, ?)"; // Execute the insert (this part would typically use a prepared statement or batch API) // For simplicity, showing a conceptual execution. // conn->execute(insert_sql, data_rows); // Hypothetical execution std::cout << "Stream insert would be executed here." << std::endl; std::cout << "Refer to OmniSciDB C++ client documentation for precise implementation." << std::endl; } catch (const std::exception& e) { std::cerr << "Error: " << e.what() << std::endl; return 1; } std::cout << "Disconnected." << std::endl; return 0; } ``` -------------------------------- ### Implement Benchmark Setup and Teardown Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/googlebenchmark/docs/user_guide.md Use `Setup` and `Teardown` callbacks to perform actions before and after each benchmark run. These callbacks are invoked once for each benchmark, and if the benchmark is multi-threaded, they are invoked once before each run with a specific number of threads. They are also invoked for each size group of threads if the benchmark uses different thread counts. ```c++ static void DoSetup(const benchmark::State& state) { } static void DoTeardown(const benchmark::State& state) { } static void BM_func(benchmark::State& state) {...} BENCHMARK(BM_func)->Arg(1)->Arg(3)->Threads(16)->Threads(32)->Setup(DoSetup)->Teardown(DoTeardown); ``` -------------------------------- ### Install Google Benchmark using CMake Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/googlebenchmark/README.md Steps to clone the repository, create a build directory, and generate build system files using CMake. This process downloads dependencies if specified. ```bash # Check out the library. $ git clone https://github.com/google/benchmark.git # Go to the library root directory $ cd benchmark # Make a build directory to place the build output. $ cmake -E make_directory "build" # Generate build system files with cmake, and download any dependencies. $ cmake -E chdir "build" cmake -DBENCHMARK_DOWNLOAD_DEPENDENCIES=on -DCMAKE_BUILD_TYPE=Release ../ # or, starting with CMake 3.13, use a simpler form: # cmake -DCMAKE_BUILD_TYPE=Release -S . -B "build" # Build the library. $ cmake --build "build" --config Release ``` -------------------------------- ### Manual Sphinx Docs Build with Virtual Environment Source: https://github.com/heavyai/heavydb/blob/master/docs/README.md Set up a Python virtual environment, install requirements, build Sphinx docs, and then deactivate the environment. ```bash python3 -m venv sphinx-env . sphinx-env/bin/activate pip install -r requirements.txt make html SPHINXOPTS="-D version=$(../scripts/parse-version.sh)" deactivate ``` -------------------------------- ### Start Probing an Endpoint Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/go/src/mapd/vendor/github.com/xiang90/probing/README.md Add an HTTP endpoint to probe and retrieve its status. Includes an example of checking total probes, loss, RTT, and clock difference. ```go id := "example" probingInterval = 5 * time.Second url := "http://example.com:12345/health" p.AddHTTP(id, probingInterval, url) time.Sleep(13 * time.Second) status, err := p.Status(id) fmt.Printf("Total Probing: %d, Total Loss: %d, Estimated RTT: %v, Estimated Clock Difference: %v\n", status.Total(), status.Loss(), status.SRTT(), status.ClockDiff()) // Total Probing: 2, Total Loss: 0, Estimated RTT: 320.771µs, Estimated Clock Difference: -35.869µs ``` -------------------------------- ### Serve Multiple Protocols on One Listener with cmux Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/go/src/mapd/vendor/github.com/soheilhy/cmux/README.md Use this pattern to create a main listener, set up a cmux, and then match incoming connections for different protocols like gRPC, HTTP, and generic TCP/RPC. Ensure servers are started in separate goroutines before calling m.Serve(). ```go // Create the main listener. l, err := net.Listen("tcp", ":23456") if err != nil { log.Fatal(err) } // Create a cmux. m := cmux.New(l) // Match connections in order: // First grpc, then HTTP, and otherwise Go RPC/TCP. grpcL := m.Match(cmux.HTTP2HeaderField("content-type", "application/grpc")) httpL := m.Match(cmux.HTTP1Fast()) trpcL := m.Match(cmux.Any()) // Any means anything that is not yet matched. // Create your protocol servers. grpcS := grpc.NewServer() grpchello.RegisterGreeterServer(grpcS, &server{}) httpS := &http.Server{ Handler: &helloHTTP1Handler{}, } trpcS := rpc.NewServer() trpcS.Register(&ExampleRPCRcvr{}) // Use the muxed listeners for your servers. go grpcS.Serve(grpcL) go httpS.Serve(httpL) go trpcS.Accept(trpcL) // Start serving! m.Serve() ``` -------------------------------- ### Install EPEL Repository for DKMS Source: https://github.com/heavyai/heavydb/blob/master/README.md Installs the Extra Packages for Enterprise Linux (EPEL) repository, which is required for installing DKMS, a prerequisite for CUDA installation on CentOS/RHEL systems. ```bash sudo yum install epel-release ``` -------------------------------- ### Installing Sample Code and Data Source: https://github.com/heavyai/heavydb/blob/master/SampleCode/CMakeLists.txt Installs the compiled sample executables, the entire sample code directory, and specific source files (Thrift definitions and CMake modules) to the 'SampleCode' destination. ```cmake install(TARGETS StreamInsertSimple StreamInsert DataGen DESTINATION SampleCode COMPONENT "data") install(DIRECTORY . DESTINATION SampleCode COMPONENT "data") install( FILES ${CMAKE_SOURCE_DIR}/heavy.thrift ${CMAKE_SOURCE_DIR}/cmake/Modules/FindThrift.cmake DESTINATION SampleCode COMPONENT "data") ``` -------------------------------- ### Install CUDA and Dependencies on macOS Source: https://github.com/heavyai/heavydb/blob/master/README.md This script installs and updates Homebrew, then uses it to install CUDA and other dependencies. It also adds the correct environment variables to ~/.bash_profile. Ensure macOS is up to date and Xcode is installed. ```bash scripts/mapd-deps-osx.sh ``` -------------------------------- ### Basic HTTP Server with Graceful Shutdown Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/go/src/mapd/vendor/gopkg.in/tylerb/graceful.v1/README.md This example demonstrates how to use the `graceful.Run` function with a standard `http.ServeMux`. It sets up a simple HTTP server that responds to requests on port 3001 and allows a 10-second timeout for active requests during shutdown. ```go package main import ( "gopkg.in/tylerb/graceful.v1" "net/http" "fmt" "time" ) func main() { mux := http.NewServeMux() mux.HandleFunc("/", func(w http.ResponseWriter, req *http.Request) { fmt.Fprintf(w, "Welcome to the home page!") }) graceful.Run(":3001",10*time.Second,mux) } ``` -------------------------------- ### Install CUDA, NVIDIA Drivers, and Headers on Arch Source: https://github.com/heavyai/heavydb/blob/master/README.md Installs CUDA, NVIDIA drivers, and necessary kernel headers on Arch Linux using 'yay'. Ensure 'yay' is installed first. A reboot is required after installation to activate the NVIDIA drivers. ```bash yay -S \ linux-headers \ cuda \ nvidia ``` -------------------------------- ### Basic CMake Installation Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/jwt-cpp/docs/install.md This snippet shows the basic steps to configure, build, and install JWT-CPP using CMake. Ensure OpenSSL is installed beforehand. ```sh cmake . cmake --build . cmake --install . ``` -------------------------------- ### Install Java JARs Source: https://github.com/heavyai/heavydb/blob/master/CMakeLists.txt Installs the generated HeavyDB utility JAR, JDBC JAR, and Calcite JAR to the 'bin' directory of the installation. This makes them accessible for runtime use. ```cmake install(FILES ${CMAKE_SOURCE_DIR}/java/heavydb/target/${UTILITY_JAR} DESTINATION bin COMPONENT "jar") install(FILES ${CMAKE_SOURCE_DIR}/java/heavyaijdbc/target/${JDBC_JAR} DESTINATION bin COMPONENT "jar") install(FILES ${CMAKE_SOURCE_DIR}/java/calcite/target/calcite-1.0-SNAPSHOT-jar-with-dependencies.jar DESTINATION bin COMPONENT "jar") ``` -------------------------------- ### Install CUDA Toolkit on Ubuntu Source: https://github.com/heavyai/heavydb/blob/master/README.md Installs the NVIDIA CUDA Toolkit and drivers from Ubuntu's standard repositories. A reboot is required after installation to activate the NVIDIA drivers. ```bash sudo apt install -y \ nvidia-cuda-toolkit ``` -------------------------------- ### Configure Python Setup Script Source: https://github.com/heavyai/heavydb/blob/master/Embedded/CMakeLists.txt Sets up the path for the Python setup.py script and configures a template file for it. ```cmake set(SETUP_PY "${CMAKE_CURRENT_BINARY_DIR}/setup.py") # Currently dbe.cpp is generated near dbe.pyx and not in build dir. This could be revised later set(OUTPUT "${CMAKE_CURRENT_SOURCE_DIR}/dbe.cpp") configure_file("${CMAKE_CURRENT_SOURCE_DIR}/setup.in.py" "${CMAKE_CURRENT_BINARY_DIR}/setup.py.in" @ONLY) # cannot be done in one step, splitting in configure_file and file commands file(GENERATE OUTPUT "${SETUP_PY}" INPUT "${CMAKE_CURRENT_BINARY_DIR}/setup.py.in") ``` -------------------------------- ### Basic etcd Client Usage in Go Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/go/src/mapd/vendor/go.etcd.io/etcd/client/README.md Demonstrates setting and getting a key-value pair using the etcd Go client. Ensure etcd is running and accessible at the specified endpoint. ```go package main import ( "log" "time" "context" "go.etcd.io/etcd/client" ) func main() { cfg := client.Config{ Endpoints: []string{"http://127.0.0.1:2379"}, Transport: client.DefaultTransport, // set timeout per request to fail fast when the target endpoint is unavailable HeaderTimeoutPerRequest: time.Second, } c, err := client.New(cfg) if err != nil { log.Fatal(err) } kapi := client.NewKeysAPI(c) // set "/foo" key with "bar" value log.Print("Setting '/foo' key with 'bar' value") resp, err := kapi.Set(context.Background(), "/foo", "bar", nil) if err != nil { log.Fatal(err) } else { // print common key info log.Printf("Set is done. Metadata is %q\n", resp) } // get "/foo" key's value log.Print("Getting '/foo' key value") resp, err = kapi.Get(context.Background(), "/foo", nil) if err != nil { log.Fatal(err) } else { // print common key info log.Printf("Get is done. Metadata is %q\n", resp) // print value log.Printf("%q key has %q value\n", resp.Node.Key, resp.Node.Value) } } ``` -------------------------------- ### Install Arch Linux Dependencies with yay Source: https://github.com/heavyai/heavydb/blob/master/docs/source/quickstart/deps.md Use `yay` to install core dependencies including `linux-headers`, `cuda`, and `nvidia`. Ensure you reboot after installation to activate NVIDIA drivers. ```shell yay -S \ linux-headers cuda nvidia ``` -------------------------------- ### Run startheavy script Source: https://github.com/heavyai/heavydb/blob/master/README.md Execute the startheavy wrapper script to initialize the database and start the HeavyDB server. Assumes execution from the build directory. ```bash ../startheavy ``` -------------------------------- ### Install jwt-cpp Library and Headers Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/jwt-cpp/CMakeLists.txt This section defines the installation rules for the 'jwt-cpp' library. It installs the target, headers, and CMake configuration files to their designated locations based on CMAKE_INSTALL_INCLUDEDIR and JWT_CMAKE_FILES_INSTALL_DIR. ```cmake install(TARGETS jwt-cpp EXPORT jwt-cpp-targets PUBLIC_HEADER DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}) install(EXPORT jwt-cpp-targets NAMESPACE jwt-cpp:: FILE jwt-cpp-targets.cmake DESTINATION ${JWT_CMAKE_FILES_INSTALL_DIR}) install(DIRECTORY ${JWT_INCLUDE_PATH}/jwt-cpp DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}) if(NOT JWT_EXTERNAL_PICOJSON AND NOT JWT_DISABLE_PICOJSON) install(FILES ${JWT_INCLUDE_PATH}/picojson/picojson.h DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}/picojson) endif() install(FILES ${CMAKE_CURRENT_BINARY_DIR}/jwt-cpp-config.cmake ${CMAKE_CURRENT_BINARY_DIR}/jwt-cpp-config-version.cmake DESTINATION ${JWT_CMAKE_FILES_INSTALL_DIR}) ``` -------------------------------- ### Configure SQLHintTest Build Process Source: https://github.com/heavyai/heavydb/blob/master/Tests/CMakeLists.txt Sets up the SQLHintTest executable, links execute test libraries, and registers it as a test. ```cmake add_executable(SQLHintTest SQLHintTest.cpp) target_link_libraries(SQLHintTest ${EXECUTE_TEST_LIBS}) add_test(SQLHintTest SQLHintTest ${TEST_ARGS}) list(APPEND SANITY_TEST_PROGRAMS SQLHintTest) ``` -------------------------------- ### Install Dockerfile (CPU) Source: https://github.com/heavyai/heavydb/blob/master/CMakeLists.txt Installs the CPU-only Dockerfile if CUDA and Rendering are not enabled. ```cmake install(FILES docker/Dockerfile.cpu RENAME Dockerfile DESTINATION "docker") ``` -------------------------------- ### Install Dockerfile (CUDA) Source: https://github.com/heavyai/heavydb/blob/master/CMakeLists.txt Installs the CUDA-enabled Dockerfile if CUDA or Rendering is enabled. ```cmake install(FILES docker/Dockerfile.cuda RENAME Dockerfile DESTINATION "docker") ``` -------------------------------- ### Install nvidia-ml-py3 with Pip Source: https://github.com/heavyai/heavydb/blob/master/Benchmarks/README.md Installs the nvidia-ml-py3 Python package using pip. ```bash pip install nvidia-ml-py3 ``` -------------------------------- ### Install nvidia-ml-py3 with Conda Source: https://github.com/heavyai/heavydb/blob/master/Benchmarks/README.md Installs the nvidia-ml-py3 Python package using conda. ```bash conda install -c fastai nvidia-ml-py3 ``` -------------------------------- ### Initializing MemMapFs Backend Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/go/src/mapd/vendor/github.com/spf13/afero/README.md Shows how to initialize the MemMapFs backend, an in-memory filesystem ideal for mocking and testing. It is fully concurrent and safe for use in goroutines. ```go mm := afero.NewMemMapFs() mm.MkdirAll("src/a", 0755)) ``` -------------------------------- ### Initialize and Use Cookie Store Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/go/src/mapd/vendor/github.com/gorilla/sessions/README.md Demonstrates initializing a cookie store with a secret key and using it to get, set, and save session values within an HTTP handler. Ensure to use context.ClearHandler if not using gorilla/mux to prevent memory leaks. ```go import ( "net/http" "github.com/gorilla/sessions" ) var store = sessions.NewCookieStore([]byte("something-very-secret")) func MyHandler(w http.ResponseWriter, r *http.Request) { // Get a session. We're ignoring the error resulted from decoding an // existing session: Get() always returns a session, even if empty. session, _ := store.Get(r, "session-name") // Set some session values. session.Values["foo"] = "bar" session.Values[42] = 43 // Save it before we write to the response/return from the handler. session.Save(r, w) } ``` -------------------------------- ### Install numpy with Pip Source: https://github.com/heavyai/heavydb/blob/master/Benchmarks/README.md Installs the numpy Python package using pip. ```bash pip install numpy ``` -------------------------------- ### Set GOPATH and PATH, Install govendor Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/go/src/mapd/vendor/README.md Configure your environment variables to point to the correct Go workspace and add govendor to your PATH. This is a prerequisite for managing dependencies. ```bash export GOPATH=/path/to/map-d/mapd2/ThirdParty/go export PATH=$GOPATH/bin:$PATH go get github.com/kardianos/govendor ``` -------------------------------- ### Install numpy with Conda Source: https://github.com/heavyai/heavydb/blob/master/Benchmarks/README.md Installs the numpy Python package using conda. ```bash conda install -c anaconda numpy ``` -------------------------------- ### Install pandas with Pip Source: https://github.com/heavyai/heavydb/blob/master/Benchmarks/README.md Installs the pandas Python package using pip. ```bash pip install pandas ``` -------------------------------- ### Build and Run JWT Example Source: https://github.com/heavyai/heavydb/blob/master/ThirdParty/jwt-cpp/README.md This command sequence demonstrates how to build a JWT example application using CMake and then run it. This requires the jwt-cpp library to be built first. ```sh cmake . cmake --build . --target print-claims ./print-claims ``` -------------------------------- ### Install pandas with Conda Source: https://github.com/heavyai/heavydb/blob/master/Benchmarks/README.md Installs the pandas Python package using conda. ```bash conda install pandas ``` -------------------------------- ### STDLOG Example Entries Source: https://github.com/heavyai/heavydb/blob/master/docs/source/components/logger.md Illustrates example log entries for query execution. The first entry marks the beginning of a query with `stdlog_begin`, and the second shows the standard `stdlog` entry at the end, including execution time. ```cpp 2019-09-20T17:15:28.215590 1 13080 DBHandler.cpp:846 stdlog_begin sql_execute 2 0 omnisci testuser 528-dyM2 {"query_str"} {"SELECT * FROM omnisci_counties LIMIT 1;"} 2019-09-20T17:15:28.924512 I 13080 DBHandler.cpp:846 stdlog sql_execute 2 709 omnisci testuser 528-dyM2 {"query_str","execution_time_ms","total_time_ms"} {"SELECT * FROM omnisci_counties LIMIT 1;","708","709"} ``` -------------------------------- ### Install pymapd with Pip Source: https://github.com/heavyai/heavydb/blob/master/Benchmarks/README.md Installs the pymapd Python package using pip. ```bash pip install pymapd ``` -------------------------------- ### Install pymapd with Conda Source: https://github.com/heavyai/heavydb/blob/master/Benchmarks/README.md Installs the pymapd Python package using conda. ```bash conda install -c conda-forge pymapd ```