### Initiate Interactive Start Guide for VerticaPy Source: https://www.vertica.com/python/documentation/1.1.x/html/user_guide_introduction_installation This snippet demonstrates how to launch an interactive guide for getting started with VerticaPy. The `help_start()` function is designed to provide a guided introduction to the library's functionalities. Ensure the `verticapy` library is imported before use. ```python vp.help_start() ``` -------------------------------- ### Start VerticaPyLab Containers (All) Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/verticapylab_gs This command launches both the Vertica and JupyterLab containers for VerticaPyLab. It automatically sets up a demo database. This command is part of the Make build system. ```bash make all ``` -------------------------------- ### Start Interactive Guide with VerticaPy Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/user_guide_introduction_installation This code snippet initiates an interactive start guide for VerticaPy using the `vp.help_start()` function. This function is designed to provide users with a guided introduction to the library's features and functionalities. ```python vp.help_start() ``` -------------------------------- ### Start VerticaPyLab Container (JupyterLab Only) Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/verticapylab_gs This command starts only the JupyterLab container for VerticaPyLab, allowing connection to an existing Vertica database. This command is part of the Make build system. ```bash make verticapylab-start ``` -------------------------------- ### Install Verticapy Package Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/contribution_guidelines_code_auto_doc Installs the Verticapy package from the setup file. This command should be run from the root of the VerticaPy directory. If Verticapy is already installed, it should be uninstalled first. ```bash pip install . ``` ```bash pip uninstall verticapy ``` -------------------------------- ### Clone VerticaPyLab Repository Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/verticapylab_gs This command clones the VerticaPyLab repository from GitHub. It requires Git to be installed on the system. ```bash git clone https://github.com/vertica/VerticaPyLab.git ``` -------------------------------- ### Load QueryProfiler from Schema Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/user_guide_performance_qprof Provides an example of loading a previously saved QueryProfiler object from a database schema using its target_schema and key_id. This enables sharing and reusing profile data. ```python from verticapy.performance.vertica import QueryProfiler, QueryProfilerInterface # Someone else can now connect to my DB and use the object. qprof = QueryProfiler( target_schema = "sc_demo", key_id = "unique_xx1", ) ``` -------------------------------- ### Get Queries from QueryProfiler Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/user_guide_performance_qprof Demonstrates retrieving and viewing the queries contained within a QueryProfiler object using the get_queries() method. The output can be further processed or visualized. ```python qprof.get_queries() ``` -------------------------------- ### Install WSL Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/verticapylab_gs This command installs the Windows Subsystem for Linux (WSL), enabling a Linux environment on Windows. It automatically sets up required components, downloads the latest Linux kernel, configures WSL 2 as the default, and installs the default Ubuntu distribution. This is crucial for running Docker Desktop on Windows. ```bash $ wsl --install ``` -------------------------------- ### Launch VerticaPyLab Containers Source: https://www.vertica.com/python/documentation/1.1.x/html/verticapylab_gs These commands manage the VerticaPyLab Docker containers. 'make all' launches both Vertica and JupyterLab containers, creating a demo database, while 'make verticapylab-start' launches only the JupyterLab container. ```shell make all ``` ```shell make verticapylab-start ``` -------------------------------- ### Start VerticaPy Interactive Help Source: https://www.vertica.com/python/documentation/1.1.x/html/utilities Initiates the VERTICAPY interactive help system, providing access to FAQs and documentation directly within the interactive environment. This is a convenient way to get help without leaving your current session. ```python help_start() ``` -------------------------------- ### Import QueryProfiler Profile Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/user_guide_performance_qprof Shows how to import a QueryProfiler object from a .tar file using the import_profile method, requiring the target_schema and key_id for proper restoration. ```python qprof.import_profile(filename="test_export_1.tar") ``` -------------------------------- ### Install Make Package Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/verticapylab_gs This command installs the 'make' utility within a Linux environment, typically used for compiling software. It requires administrator privileges (sudo) and uses the apt package manager, common in Debian-based Linux distributions like Ubuntu. ```bash $ sudo apt install make ``` -------------------------------- ### Import VerticaPy and Load Datasets Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/user_guide_performance_qprof Imports the verticapy package and loads the titanic and amazon datasets. These datasets are commonly used for demonstrating VerticaPy functionalities. ```python import verticapy as vp from verticapy.datasets import load_titanic, load_amazon # load datasets titanic = load_titanic() amazon = load_amazon() ``` -------------------------------- ### Create QueryProfiler with Multiple Queries Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/user_guide_performance_qprof Illustrates initializing a QueryProfilerInterface object with a list of query identifiers (tuples of transaction_id and statement_id) to analyze multiple queries collectively. ```python from verticapy.performance.vertica import QueryProfiler, QueryProfilerInterface qprof = QueryProfilerInterface([(45035996273780927,74), (45035996273780075,6)]) ``` -------------------------------- ### Get Help for VerticaPy Functions Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/user_guide_introduction_installation This example illustrates how to obtain detailed help information for VerticaPy functions, such as `vp.new_connection`. It utilizes Python's `inspect` module to display the function signature and its documentation, providing insights into its parameters and usage. ```python import inspect import re def help(obj): signature = f"Help on function {obj.__name__} in module {obj.__module__}:\n\n{obj.__name__}{inspect.signature(obj)}" doc = inspect.getdoc(obj) if doc: short_doc = re.split(r"\n\s*Examples\s*[-=]*\s*\n", doc)[0] print(f"{signature}\n\n{short_doc}") help(vp.new_connection) ``` -------------------------------- ### Visualize Query Plan Profile (Python) Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/user_guide_performance_qprof Generates a pie chart representing the profile of a query plan. The 'kind="pie"' argument specifies the chart type. This visualization helps in identifying the proportion of time spent on different parts of the query plan. Requires plotting libraries like Plotly to be installed and configured. ```python vp.set_option("plotting_lib", "plotly") fig = qprof.get_qplan_profile(kind = "pie") fig.write_html("/project/data/VerticaPy/docs/figures/user_guides_performance_qprof_pie.html") ``` -------------------------------- ### Navigate to VerticaPyLab Directory Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/verticapylab_gs This command changes the current directory to the cloned VerticaPyLab repository. This is a standard shell command. ```bash cd VerticaPyLab ``` -------------------------------- ### Install Make Utility Source: https://www.vertica.com/python/documentation/1.1.x/html/contribution_guidelines_code_auto_doc Provides instructions on how to install the 'make' utility, which is a prerequisite for building documentation using the 'make html' command. ```shell apt install make ``` -------------------------------- ### Install VerticaPy Development Dependencies Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/contribution_guidelines_code_setting_up Installs the necessary Python packages for developing VerticaPy using pip. This typically includes testing frameworks and other development tools. ```shell pip3 install -r requirements-dev.txt ``` -------------------------------- ### Import VerticaPy and Load Datasets Source: https://www.vertica.com/python/documentation/1.1.x/html/user_guide_performance_qprof This snippet demonstrates how to import the verticapy package and load sample datasets, 'titanic' and 'amazon', which are commonly used for data analysis and performance profiling within VerticaPy. ```python import verticapy as vp from verticapy.datasets import load_titanic, load_amazon # load datasets titanic = load_titanic() amazon = load_amazon() ``` -------------------------------- ### Export QueryProfiler Profile Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/user_guide_performance_qprof Demonstrates exporting a QueryProfiler object's data to a .tar file using the export_profile method. This facilitates backup and transfer of profile information. ```python qprof.export_profile(filename="test_export_1.tar") ``` -------------------------------- ### Create QueryProfiler from Query IDs Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/user_guide_performance_qprof Shows how to initialize a QueryProfiler object using a tuple of transaction_id and statement_id. This method is suitable when you have specific query identifiers. ```python from verticapy.performance.vertica import QueryProfiler, QueryProfilerInterface qprof = QueryProfiler((45035996273780927,76)) ``` -------------------------------- ### Load SQL Extension Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/user_guide_performance_qprof Loads the verticapy.sql extension, which enables the execution of SQL queries directly within Jupyter notebook cells using the %%sql magic command. ```python %load_ext verticapy.sql ``` -------------------------------- ### Initialize VerticaPy and Create a vDataFrame Source: https://www.vertica.com/python/documentation/1.1.x/html/vdataframe This example shows the initial setup for using VerticaPy, including aliasing the library for clarity. It then demonstrates creating a simple vDataFrame with two columns, 'cats' and 'reps', and accessing one of these columns. ```python import verticapy as vp vdf = vp.vDataFrame({ "cats": ["A", "B", "C"], "reps": [2, 4, 8], }) vdf["cats"] ``` -------------------------------- ### Install Make Build Tool Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/contribution_guidelines_code_auto_doc_render Installs the 'make' utility, a command-line tool used for building and managing projects. The 'sudo' command is optional and depends on user permissions. ```shell sudo apt install make (sudo is optional) ``` -------------------------------- ### Install Project Requirements Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/contribution_guidelines_code_auto_doc_render Installs all necessary Python packages for the project from the requirements.txt file. It is recommended to do this within a virtual environment. ```shell pip install -r requirements.txt ``` -------------------------------- ### Create Schema and Import Query Profile (Python) Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/user_guide_performance_qprof Creates a new schema in Vertica and imports a query profile. Ensure the 'key_id' is unique and unused. The 'auto_initialize' parameter dictates whether the profile should be automatically set up upon import. Dependencies include the verticapy library. ```python vp.create_schema("sc_demo_1") qprof = QueryProfiler.import_profile( target_schema = "sc_demo_1", key_id = "unique_load_xx1", filename = "test_export_1.tar", auto_initialize = True, ) ``` -------------------------------- ### Python Example with Simple Input/Output (RST) Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/contribution_guidelines_code_auto_doc_example Demonstrates how to include a simple Python code example within an RST docstring, showing input and expected output. The `>>>` indicates interactive Python input, and the following line is the output. This is useful for quick demonstrations. ```rst .. code-block:: python >>> x = [1,2,3] >>> max(x) 3 ``` -------------------------------- ### Create QueryProfiler from Query Statement Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/user_guide_performance_qprof Demonstrates creating a QueryProfiler object directly from an SQL query string. This is useful for analyzing ad-hoc queries or specific SQL commands. ```python qprof = QueryProfiler(""" select transaction_id, statement_id, request, request_duration from query_requests where start_timestamp > (now() - interval'1 hour') order by request_duration desc limit 10; """) ``` -------------------------------- ### Import VerticaPy and Get Version Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/examples_business_battery Imports the VerticaPy library and displays its current version. This is a standard setup step for using VerticaPy functionalities. ```python import verticapy as vp vp.__version__ ``` -------------------------------- ### View Vertica 'query_consumption' Table with QueryProfiler Source: https://www.vertica.com/python/documentation/1.1.x/html/user_guide_performance_qprof This Python code snippet illustrates how to access and display data from the 'query_consumption' performance table via the `QueryProfiler` object. It requires an initialized `QueryProfiler` instance. The output presents information such as start time, and whether a query was a retry or successful. ```python qprof.get_table("query_consumption") ``` -------------------------------- ### Install Make Utility Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/verticapylab_gs_queryprofiler Installs the 'make' utility on Debian-based Linux systems (like Ubuntu) if it is not already present. 'make' is essential for running the provided build scripts. ```bash sudo apt install make ``` -------------------------------- ### Install Requirements and VerticaPy Source: https://www.vertica.com/python/documentation/1.1.x/html/contribution_guidelines_code_auto_doc Installs project dependencies using pip and then installs the VerticaPy package itself. It also includes instructions for uninstalling if VerticaPy is already installed. ```shell pip install -r requirements.txt ``` ```shell pip install . ``` ```shell pip uninstall verticapy ``` -------------------------------- ### Visualize Query Steps (Python) Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/user_guide_performance_qprof Generates a bar graph visualizing the steps involved in a query execution. The 'kind="bar"' argument specifies the visualization type. The output can be saved as an HTML file for sharing or further inspection. Requires the plotting library to be configured. ```python qprof.get_qsteps(kind="bar") ``` -------------------------------- ### Python Query Profiler Comparison Setup Source: https://www.vertica.com/python/documentation/1.1.x/html/whats_new_v1_1_0 Illustrates the setup for comparing query profiling results using VerticaPy's QueryProfilerInterface and QueryProfilerComparison. It involves creating instances of QueryProfilerInterface for different keys and then comparing them using QueryProfilerComparison. ```python from verticapy.performance.vertica import QueryProfilerInterface qprof_interface_1 = QueryProfilerInterface( key_id = "key_1", target_schema = "schema_1", ) qprof_interface_2 = QueryProfilerInterface( key_id = "key_2", target_schema = "schema_1", ) from verticapy.performance.vertica import QueryProfilerComparison qprof_compare = QueryProfilerComparison(qprof_interface_1, qprof_interface_2) qprof_compare.get_qplan_tree() ``` -------------------------------- ### Display Query Plan Tree (Python) Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/user_guide_performance_qprof Outputs the tree structure of a query plan. This method, 'get_qplan_tree()', is essential for understanding the hierarchical execution path of a query. The output is typically a text-based representation of the plan. ```python qprof.get_qplan_tree() ``` -------------------------------- ### Generate and Save Last Query Execution to HTML (Python) Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/user_guide_performance_qprof Generates the last query execution visualization and saves it as an HTML file. This is useful for quickly inspecting the most recent query's performance. It requires the `verticapy` library. ```python fig = qprof.get_qexecution() fig.write_html("/project/data/VerticaPy/docs/figures/user_guides_performance_qprof_last.html") ``` -------------------------------- ### View Vertica 'dc_explain_plans' Table with QueryProfiler Source: https://www.vertica.com/python/documentation/1.1.x/html/user_guide_performance_qprof This Python code snippet demonstrates how to retrieve and view data from the 'dc_explain_plans' performance table using the `QueryProfiler` object. It assumes the `QueryProfiler` object is set up and the table is accessible. The output shows table details including time and path line index. ```python qprof.get_table('dc_explain_plans') ``` -------------------------------- ### reStructuredText IPython Directive for Executable Python Examples Source: https://www.vertica.com/python/documentation/1.1.x/html/contribution_guidelines_code_auto_doc_example This directive allows embedding and executing Python code directly within documentation using the IPython kernel. It displays both the code and its output, making it ideal for demonstrating more complex examples or when hardcoding results is undesirable. ```rst .. ipython:: python x = 2 y = 3 x + y ``` -------------------------------- ### Run VerticaPy Tests with Tox Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/contribution_guidelines_code_setting_up Provides examples of using Tox to run the VerticaPy test suite across different Python environments. Tox automates testing in various configurations. ```bash # Run all tests using tox: tox # Run tests on specified python versions with `tox -e ENV,ENV` tox -e py36,py37 # Run specific tests by filename (e.g.) `test_vDF_combine_join_sort.py` tox -- verticapy/tests/vDataFrame/test_vDF_combine_join_sort.py # Run all tests on the python version 3.6: tox -e py36 -- verticapy/tests # Run all tests on the python version 3.7 with verbose result outputs: tox -e py37 -v -- verticapy/tests # Run an individual test on specified python versions. # e.g.: Run the test `test_vDF_append` under `test_vDF_combine_join_sort.py` on the python versions 3.7 and 3.8 tox -e py37,py38 -- verticapy/tests/vDataFrame/test_vDF_combine_join_sort.py::TestvDFCombineJoinSort::test_vDF_append ``` -------------------------------- ### Load Iris Dataset using VerticaPy Source: https://www.vertica.com/python/documentation/1.1.x/html/api/verticapy.machine_learning.vertica.neighbors.KNeighborsClassifier This snippet demonstrates how to load the Iris dataset using the verticapy.datasets module. It is a common starting point for examples and requires the verticapy library to be installed. ```python import verticapy.datasets as vpd data = vpd.load_iris() ``` -------------------------------- ### Clone VerticaPy Repository and Setup Upstream Remote Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/contribution_guidelines_code_setting_up Instructions for cloning the VerticaPy project from GitHub and setting up the 'upstream' remote repository. This is essential for tracking changes from the main project. ```shell git clone git@github.com:YOURUSERNAME/VerticaPy.git cd VerticaPy git remote add upstream git@github.com:vertica/VerticaPy.git git fetch upstream ``` -------------------------------- ### Python Example with Execution (IPython) Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/contribution_guidelines_code_auto_doc_example Illustrates how to include Python code in a docstring that will be both displayed and executed using the `ipython` directive. This is useful for examples where hardcoding the result is not desirable. The code block defines variables and performs an addition. ```rst .. ipython:: python x = 2 y = 3 x + y ``` -------------------------------- ### Deprecated Directive for Python Docstrings Source: https://www.vertica.com/python/documentation/1.1.x/html/contribution_guidelines_code_auto_doc_example This directive is used to mark a function as deprecated, indicating that it will be removed in future versions. It includes the version number from which the deprecation started, guiding users to update their code. ```python .. deprecated:: Y.0 ``` -------------------------------- ### Retrieve Query Plan in Python Source: https://www.vertica.com/python/documentation/1.1.x/html/user_guide_performance_qprof Fetches and displays the detailed execution plan for a query profile. This output provides a tree-like structure showing how the database intends to execute the query, including join operations, sorts, and storage accesses. ```python qprof.get_qplan() ``` -------------------------------- ### List Vertica Performance Tables with QueryProfiler Source: https://www.vertica.com/python/documentation/1.1.x/html/user_guide_performance_qprof This Python code snippet demonstrates how to use the `QueryProfiler` object to retrieve a list of all available performance tables. It's useful for understanding the data accessible for performance analysis. The output is a list of table names. ```python qprof.get_table() ``` -------------------------------- ### Load Market Dataset and Get Max Price Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/contribution_guidelines_code_auto_doc_example Demonstrates how to load the market dataset and find the maximum price. ```APIDOC ## Load Market Dataset and Get Max Price ### Description Loads the market dataset and calculates the maximum value in the 'price' column. ### Method Python execution ### Endpoint N/A ### Parameters None ### Request Example ```python from verticapy.datasets import load_market market = load_market() market["price"].max() ``` ### Response #### Success Response (200) ``` 10.16 ``` #### Response Example ``` 10.16 ``` ``` -------------------------------- ### Initialize QueryProfiler with SQL Query (Python) Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/user_guide_performance_qprof Initializes a QueryProfiler object by executing a given SQL query. This is useful for analyzing the performance of specific queries directly. The query should select relevant performance metrics like transaction_id, statement_id, request, and request_duration. The results are stored within the qprof object. ```python qprof = QueryProfiler(""" select transaction_id, statement_id, request, request_duration from query_requests where start_timestamp > (now() - interval'1 hour') order by request_duration desc limit 10; """) ``` -------------------------------- ### Pylint Analysis Output Example Source: https://www.vertica.com/python/documentation/1.1.x/html/cicd_pylint Example output from running Pylint on a Python script. It highlights common issues like missing docstrings, naming convention violations, and unnecessary code constructs, along with a quality score. ```text ************* Module example example.py:1:0: C0111: Missing module docstring (missing-docstring) example.py:1:0: C0103: Function name "add_numbers" doesn't conform to snake_case naming style (invalid-name) example.py:1:0: C0111: Missing function docstring (missing-docstring) example.py:5:4: W0104: Statement seems to have no effect (pointless-statement) example.py:5:4: R1705: Unnecessary "else" after "return" (no-else-return) ------------------------------------ Your code has been rated at 5.45/10 ``` -------------------------------- ### Referencing Functions using ..seealso:: directive Source: https://www.vertica.com/python/documentation/1.1.x/html/contribution_guidelines_code_auto_doc_example Shows how to use the '.. seealso::' directive to create a 'See Also' section in the documentation, referencing other functions or modules. This example references the 'bar' function. ```restructuredtext .. seealso::\n\n:py:func:`~bar` : Similar bar plots\n ``` -------------------------------- ### Configure Git User Information Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/contribution_guidelines_code_setting_up Configures the global Git settings for username and email address. This is crucial for commit authorship and is a standard first-time Git setup step. ```shell git config --global user.name "John Smith" git config --global user.email "email@example.com" ``` -------------------------------- ### Load Winequality Dataset with VerticaPy Source: https://www.vertica.com/python/documentation/1.1.x/html/api/verticapy.machine_learning.vertica.TensorFlowModel This snippet shows how to load the winequality dataset using the verticapy.datasets module. This dataset is commonly used for examples and analysis within VerticaPy. The loaded data is stored in the 'data' variable. ```python import verticapy.datasets as vpd data = vpd.load_winequality() ``` -------------------------------- ### Staging and Committing Changes with Git Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/contribution_guidelines_code_setting_up This command sequence stages all modified files in the current directory and then creates a new commit with a descriptive message. It's a fundamental step in tracking changes during development. ```shell git add . git commit -m 'Added two more tests for #166' ``` -------------------------------- ### Stop VerticaPyLab Service Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/verticapylab_gs This command stops the running VerticaPyLab services. It is part of the Make build system. ```bash make stop ``` -------------------------------- ### Python Docstring Parameter Section Example Source: https://www.vertica.com/python/documentation/1.1.x/html/contribution_guidelines_code_auto_doc_example This snippet demonstrates the standard format for documenting function parameters within a Python docstring. It includes the parameter name, its type, and a description of its purpose, separated by a blank line. ```python """ Parameters ----------- x: integer x is the input """ ``` -------------------------------- ### Save Query Execution Report to HTML (Python) Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/user_guide_performance_qprof Saves the query execution report generated by QueryProfiler to an HTML file. This allows for offline viewing and sharing of performance data. It uses the `_repr_html_()` method to get the HTML representation. ```python res = qprof.get_qexecution_report().sort({"exec_time_us": "desc"}) html_file = open("/project/data/VerticaPy/docs/figures/user_guides_performance_qprof_query_report.html", "w") html_file.write(res._repr_html_()) html_file.close() ``` -------------------------------- ### Referencing Modules using ..seealso:: directive Source: https://www.vertica.com/python/documentation/1.1.x/html/contribution_guidelines_code_auto_doc_example Demonstrates using the '.. seealso::' directive to reference an entire module. This example shows how to link to the 'verticapy.vDataFrame' module. ```restructuredtext .. seealso::\n\n :py:mod:`~verticapy.vDataFrame`\n ``` -------------------------------- ### VerticaPy Test Configuration - Override with .conf File Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/contribution_guidelines_code_setting_up Demonstrates how to override default VerticaPy test settings using a `verticapy/tests/vp_test.conf` file. This allows for environment-specific configurations. ```python # edit under [vp_test_config] section VP_TEST_HOST=10.0.0.2 VP_TEST_PORT=5000 VP_TEST_USER=dbadmin VP_TEST_DATABASE=vdb1 VP_TEST_PASSWORD=abcdef1234 VP_TEST_LOG_DIR=my_log/year/month/date VP_TEST_LOG_LEVEL=DEBUG ``` -------------------------------- ### Python Function Docstring Example: One-Hot Encoding Source: https://www.vertica.com/python/documentation/1.1.x/html/contribution_guidelines_code_auto_doc_example An example docstring for a `one_hot_encode` function, illustrating how to describe the function's purpose, its parameters (with types and defaults), and the return value. It also shows how to use inline code formatting for object names. ```python def one_hot_encode( self, prefix: Optional[str] = None, prefix_sep: str = "_", drop_first: bool = True, use_numbers_as_suffix: bool = False, ) -> "vDataFrame": """ Encodes the :py:class:`~vDataColumn` with the One-Hot Encoding algorithm. One hot encoding will be done on the select column. The result will be outputted in new columns thus resulting in additional columns added to the table. The first category/dummy will be dropped by default unless stated otherwise by the parameter ``drop_first``. """ ``` -------------------------------- ### View Vertica 'query_events' Table with QueryProfiler Source: https://www.vertica.com/python/documentation/1.1.x/html/user_guide_performance_qprof This Python code snippet shows how to fetch and display data from the 'query_events' performance table using the `QueryProfiler` object. It requires the `QueryProfiler` object to be initialized and assumes the 'query_events' table exists. The output displays the table's contents with event timestamps and severity. ```python qprof.get_table("query_events") ``` -------------------------------- ### Uninstall VerticaPyLab Environment Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/verticapylab_gs This command cleans up the VerticaPyLab environment by deleting all associated Docker images. It is part of the Make build system. ```bash make uninstall ``` -------------------------------- ### Python Function Docstring Example (One-Hot Encoding) Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/contribution_guidelines_code_auto_doc_example Provides an example of a Python docstring for a 'one_hot_encode' function. It includes a summary line, a detailed description of the encoding process, and highlights the use of specific syntax for VerticaPy objects. The output is the formatted description. ```python def one_hot_encode( self, prefix: Optional[str] = None, prefix_sep: str = "_", drop_first: bool = True, use_numbers_as_suffix: bool = False, ) -> "vDataFrame": """ Encodes the :py:class:`~vDataColumn` with the One-Hot Encoding algorithm. One hot encoding will be done on the select column. The result will be outputted in new columns thus resulting in additional columns added to the table. The first category/dummy will be dropped by default unless stated otherwise by the parameter ``drop_first``. """ ``` -------------------------------- ### Stop and Uninstall VerticaPyLab Source: https://www.vertica.com/python/documentation/1.1.x/html/verticapylab_gs These commands are used to stop the running VerticaPyLab services and to clean up the environment by deleting all associated Docker images. ```shell make stop ``` ```shell make uninstall ``` -------------------------------- ### Generate UDF Library Installation Code Source: https://www.vertica.com/python/documentation/1.1.x/html/sdk-vertica Generates the necessary Python code and SQL statements for installing a library of Python functions in Vertica. Takes a list of UDFs and the library name as input. ```python generate_lib_udf(udf_list, library_name) ``` -------------------------------- ### Run All Tests with Tox Source: https://www.vertica.com/python/documentation/1.1.x/html/contribution_guidelines_code_setting_up Execute the entire test suite for VerticaPy using the 'tox' command. Tox will iterate through the Python environments defined in 'tox.ini'. ```bash # Run all tests using tox: tox ``` -------------------------------- ### Load Winequality Dataset with VerticaPy Source: https://www.vertica.com/python/documentation/1.1.x/html/api/verticapy.machine_learning.vertica.PMMLModel Imports the datasets module from VerticaPy and loads the 'winequality' dataset. This dataset is commonly used for examples and demonstrations. ```python import verticapy.datasets as vpd data = vpd.load_winequality() ``` -------------------------------- ### Run All Tests on a Specific Python Version with Verbose Output Source: https://www.vertica.com/python/documentation/1.1.x/html/contribution_guidelines_code_setting_up Execute all tests for a specific Python version (e.g., 3.7) and display verbose output to get detailed information about test execution. ```bash # Run all tests on the python version 3.7 with verbose result outputs: tox -e py37 -v -- verticapy/tests ``` -------------------------------- ### VAR Class Documentation Source: https://www.vertica.com/python/documentation/1.1.x/html/api/verticapy.machine_learning.vertica.tsa Documentation for the VAR class, including parameters, attributes, and examples for creating and using Vector Autoregression models. ```APIDOC ## class verticapy.machine_learning.vertica.tsa.VAR Creates a inDB VectorAutoregressor model. ### Parameters - **name** (str, optional) - Name of the model. The model is stored in the database. - **overwrite_model** (bool, optional) - If set to `True`, training a model with the same name as an existing model overwrites the existing model. - **p** (int, optional) - Integer in the range [1, 1999], the number of lags to consider in the computation. Larger values for p weaken the correlation. - **method** (str, optional) - One of the following algorithms for training the model: 'ols' (Ordinary Least Squares), 'yule-walker' (Yule-Walker). - **penalty** (str, optional) - Method of regularization: 'none' (No regularization), 'l2' (L2 regularization). - **C** (int | float | Decimal, optional) - The regularization parameter value. The value must be zero or non-negative. - **missing** (str, optional) - Method for handling missing values: 'drop' (Missing values are ignored), 'error' (Missing values raise an error), 'zero' (Missing values are set to zero), 'linear_interpolation' (Missing values are replaced by a linearly interpolated value). - **subtract_mean** (bool, optional) - For Yule Walker, if `subtract_mean is True`, then the mean of the column(s) will be subtracted before calculating the coefficients. This parameter has no effect for OLS. ### Attributes - **phi_** (numpy.array) - The coefficient of the AutoRegressive process. - **intercept_** (float) - Represents the expected value of the time series when the lagged values are zero. - **features_importance_** (numpy.array) - The importance of features is computed through the AutoRegressive part coefficients. - **mse_** (float) - The mean squared error (MSE) of the model. - **n_** (int) - The number of rows used to fit the model. ### Examples The following examples provide a basic understanding of usage. For more detailed examples, please refer to the Machine Learning or the Examples section on the website. ``` -------------------------------- ### Configure VerticaPyLab for Remote Access Source: https://www.vertica.com/python/documentation/1.1.x/html/verticapylab_gs This snippet shows how to modify the VERTICAPYLAB_BIND_ADDRESS in the configuration file to allow remote access to VerticaPyLab. After modification, the containers need to be restarted. ```shell VERTICAPYLAB_BIND_ADDRESS=0.0.0.0 ``` ```shell make all ``` -------------------------------- ### Create a New Git Branch Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/contribution_guidelines_code_setting_up Command to create a new Git branch for feature development or bug fixes. Using descriptive branch names helps in managing contributions. ```shell git checkout -b my-fix-branch ``` -------------------------------- ### Create QueryProfiler from Multiple Queries Source: https://www.vertica.com/python/documentation/1.1.x/html/user_guide_performance_qprof Creates a QueryProfiler object to analyze multiple queries by providing a list of tuples, where each tuple contains the statement_id and transaction_id for a query. ```python qprof = QueryProfilerInterface([(45035996273780927,74), (45035996273780075,6)]) ``` -------------------------------- ### StandardScaler Initialization and Methods - Python Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/api/verticapy.machine_learning.vertica.preprocessing.StandardScaler Demonstrates the initialization of the StandardScaler and its various methods for data preprocessing, model management, and deployment in Vertica. This includes fitting, transforming, and exporting models. ```python from verticapy.machine_learning.vertica.preprocessing import StandardScaler # Initialize StandardScaler scaler = StandardScaler() # Fit the scaler to data (assuming 'data' is a VerticaDBTable object) # scaler.fit(data) # Transform data # transformed_data = scaler.transform(data) # Inverse transform data # original_data = scaler.inverse_transform(transformed_data) # Deploy model to SQL # deploy_sql = scaler.deploySQL() # Export model to different formats # scaler.to_binary() # scaler.to_pmml() # scaler.to_python() # scaler.to_sql() # scaler.to_tf() # Get model attributes and parameters # attributes = scaler.get_attributes() # params = scaler.get_params() # Drop the model # scaler.drop() # Check if model exists # scaler.does_model_exists() # Summarize the model # scaler.summarize() ``` -------------------------------- ### KMeans Initialization and Methods (Python) Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/api/verticapy.machine_learning.vertica.cluster.KMeans Demonstrates the initialization of the KMeans class and outlines its available methods. This includes fitting the model, making predictions, deploying it to Vertica, and managing model attributes and parameters. ```python from verticapy.machine_learning.vertica.cluster import KMeans # Initialize KMeans model kmeans_model = KMeans(n_clusters=5, ...) # Fit the model kmeans_model.fit(data) # Make predictions predictions = kmeans_model.predict(new_data) # Other methods like deploySQL, drop, export_models, plot, etc. are available. ``` -------------------------------- ### Get Specific Model Metrics Source: https://www.vertica.com/python/documentation/1.1.x/html/api/verticapy.machine_learning.vertica.ensemble Optimizes performance by retrieving only the necessary regression metrics using the `report()` function with the `metrics` parameter. This example demonstrates fetching Mean Squared Error (MSE) and R-squared (r2). ```python model.report(metrics = ["mse", "r2"]) ``` -------------------------------- ### View Detailed Query Execution Information with Vertica Python Source: https://www.vertica.com/python/documentation/1.1.x/html/user_guide_performance_qprof Retrieves detailed information about the query execution. This function is useful for a granular look at how each part of the query performed. ```python qprof.get_qexecution() ``` -------------------------------- ### Save QueryProfiler to Schema Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/user_guide_performance_qprof Explains how to save a QueryProfiler object to a specified schema in the database, allowing for persistence and retrieval. Includes options for schema, key_id, and overwriting existing data. ```python # Save it to your schema qprof = QueryProfiler( (45035996273780927, 76), target_schema = "sc_demo", key_id = "unique_xx1", overwrite = True, ) ``` -------------------------------- ### Load Titanic Dataset and Display Head Source: https://www.vertica.com/python/documentation/1.1.x/html/user_guide_data_preparation_features_engineering Loads the 'titanic' dataset using VerticaPy and displays the first 100 rows. This is a common starting point for feature engineering examples. ```python import verticapy as vp from verticapy.datasets import load_titanic titanic = load_titanic() titanic.head(100) ``` -------------------------------- ### Configure Remote Access for VerticaPyLab Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/verticapylab_gs This snippet shows how to modify the VerticaPyLab configuration file to allow remote access. It involves changing the VERTICAPYLAB_BIND_ADDRESS to '0.0.0.0' and then restarting the services. This is typically done within the etc/VerticaPyLab.conf.default file. ```bash VERTICAPYLAB_BIND_ADDRESS=0.0.0.0 ``` -------------------------------- ### SQL Query: Group and Aggregate Data Source: https://www.vertica.com/python/documentation/1.1.x/html/user_guide_performance_qprof This SQL query calculates the average of the 'number' column, grouped by 'date', and extracts the month from the date. It orders the results by the calculated average in descending order. ```sql %%sql SELECT date, MONTH(date) as month, AVG(number) as avg_number_test FROM public.amazon GROUP BY date ORDER BY avg_number_test DESC; ``` -------------------------------- ### XGBClassifier Initialization and Methods in VerticaPy Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/api/verticapy.machine_learning.vertica.ensemble.XGBClassifier Demonstrates the initialization of the XGBClassifier and outlines its various methods for machine learning tasks. These include fitting the model, making predictions, evaluating performance, and managing the model within Vertica. ```python from verticapy.machine_learning.vertica.ensemble import XGBClassifier # Initialize the XGBClassifier model = XGBClassifier() # Example methods (actual usage would involve data and parameters) model.fit(X, y) predictions = model.predict(X_new) report = model.classification_report() model.plot() model.to_json() ``` -------------------------------- ### Execute SQL Query with Join and Analyze Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/user_guide_performance_qprof Executes a more complex SQL query involving a join between the amazon table and a subquery to find the maximum number per date. The results are analyzed using VerticaPy. ```python %%sql SELECT a.date, MONTH(a.date) AS month, AVG(a.number) AS avg_number_test, b.max_number FROM public.amazon AS a JOIN ( SELECT date, MAX(number) AS max_number FROM public.amazon GROUP BY date ) AS b ON a.date = b.date GROUP BY a.date, b.max_number ORDER BY avg_number_test DESC; ``` -------------------------------- ### Execute SQL Query and Analyze Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/user_guide_performance_qprof Executes a SQL query to calculate the average number of items per date and orders the results. The output is displayed using VerticaPy's vDataFrame, allowing for further analysis. ```python %%sql SELECT date, MONTH(date) as month, AVG(number) as avg_number_test FROM public.amazon GROUP BY date ORDER BY avg_number_test DESC; ``` -------------------------------- ### Plot a Specific Tree in XGBRegressor Model Source: https://www.vertica.com/python/documentation/1.1.x/html/api/verticapy.machine_learning.memmodel.ensemble This example demonstrates how to visualize a specific decision tree within the XGBRegressor ensemble using the `plot_tree()` method. The `tree_id` parameter specifies which tree to plot. This functionality requires the Graphviz module to be installed. ```python model_xgbr.plot_tree(tree_id = 0) ``` -------------------------------- ### Scaler Class Initialization and Methods in VerticaPy Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/api/verticapy.machine_learning.vertica.preprocessing.Scaler This snippet documents the initialization (`__init__`) and various methods of the Scaler class. These methods allow for data scaling, model deployment (SQL, binary, PMML, etc.), model management (import/export, drop), and attribute retrieval. It is designed for use with VerticaPy and Vertica. ```python from verticapy.machine_learning.vertica.preprocessing import Scaler # Example usage (conceptual, actual parameters depend on data) # scaler = Scaler(name="my_scaler_model", ...) # scaler.fit(data=my_dataframe) # scaled_data = scaler.transform(data=my_dataframe) # inverse_scaled_data = scaler.inverse_transform(data=scaled_data) # Model deployment examples # scaler.deploySQL() # scaler.to_binary() # scaler.to_pmml() # Model management # scaler.drop() # scaler.import_models(...) # scaler.export_models(...) # Parameter and attribute retrieval # scaler.get_params() # scaler.get_attributes() ``` -------------------------------- ### Export QueryProfiler Report to HTML with Vertica Python Source: https://www.vertica.com/python/documentation/1.1.x/html/user_guide_performance_qprof Exports the complete QueryProfiler report to an HTML file. This allows for offline analysis and easy sharing without requiring a database connection or Jupyter environment. ```python qprof.to_html("my-report.html") # Where "my-report.html" is the path of the file to save. ``` -------------------------------- ### Run Specific Tests with Tox Source: https://www.vertica.com/python/documentation/1.1.x/html/contribution_guidelines_code_setting_up Execute specific test files or directories using tox by appending the path to the tox command. This is useful for focusing on a particular area of the test suite. ```bash # Run specific tests by filename (e.g.) `test_vDF_combine_join_sort.py` tox -- verticapy/tests/vDataFrame/test_vDF_combine_join_sort.py ``` -------------------------------- ### Display CPU Time as Bar Graph in Python Source: https://www.vertica.com/python/documentation/1.1.x/html/user_guide_performance_qprof Visualizes the CPU time consumed by different stages of a query execution as a bar graph. This is helpful for identifying performance bottlenecks related to CPU utilization. ```python qprof.get_cpu_time(kind="bar") ``` -------------------------------- ### Pushing Local Branch to Remote Repository Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/contribution_guidelines_code_setting_up This command pushes the current local branch to its corresponding remote repository, typically GitHub. It makes your local changes available on the remote for collaboration or backup. ```shell git push origin my-fix-branch ``` -------------------------------- ### Get Query Explain Plan Trees with VerticaPy Source: https://www.vertica.com/python/documentation/1.1.x/html/performance_vertica The get_qplan_explain() method returns the query explain plan as a list of titles and trees. The display_trees parameter can be used to control the output format. ```python QueryProfiler.get_qplan_explain([display_trees]) ``` -------------------------------- ### Build HTML Documentation with Sphinx Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/contribution_guidelines_code_auto_doc Builds the HTML documentation using Sphinx. This command is typically run from the 'docs' directory. 'make' may need to be installed separately. ```bash make html ``` ```bash apt install make ``` -------------------------------- ### Get Help for VerticaPy's new_connection Function Source: https://www.vertica.com/python/documentation/1.1.x/html/user_guide_introduction_installation This snippet illustrates how to access the documentation and parameter information for the `new_connection()` function within VerticaPy. It uses Python's built-in `help()` function, which is useful for understanding function arguments, return types, and detailed descriptions. This requires the `verticapy` library to be imported. ```python help(vp.new_connection) ``` -------------------------------- ### Retrieve Query Request Details (Python) Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/user_guide_performance_qprof Fetches and displays the details of a query request associated with the QueryProfiler object. The 'get_request()' method provides a formatted output of the query, aiding in understanding what was executed. This is useful for debugging and analysis. ```python qprof.get_request() ``` -------------------------------- ### Visualize Query Steps as Bar Graph in Python Source: https://www.vertica.com/python/documentation/1.1.x/html/user_guide_performance_qprof Retrieves and displays the number of query steps for a given query profile as a bar graph. This function is useful for visualizing the breakdown of operations within a query execution plan. ```python qprof.get_qsteps(kind="bar") ``` -------------------------------- ### Import QueryProfiler Object from .tar File Source: https://www.vertica.com/python/documentation/1.1.x/html/user_guide_performance_qprof Imports a QueryProfiler object from a .tar archive file. The object is loaded into the specified target_schema with a new unique key_id. auto_initialize=True ensures the object is ready for use immediately after import. ```python vp.create_schema("sc_demo_1") qprof = QueryProfiler.import_profile( target_schema = "sc_demo_1", key_id = "unique_load_xx1", filename = "test_export_1.tar", auto_initialize = True, ) ``` -------------------------------- ### Export Full QueryProfiler Report to HTML (Python) Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/user_guide_performance_qprof Exports the entire QueryProfiler report to a specified HTML file path. This method is convenient for sharing comprehensive performance analysis offline without database or Jupyter environment dependencies. ```python qprof.to_html("my-report.html") ``` -------------------------------- ### AutoDataPrep Class Initialization (Python) Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/api/verticapy.machine_learning.vertica.automl.AutoDataPrep Initializes the AutoDataPrep object. This is the entry point for utilizing the automatic data preparation functionalities. No specific dependencies are mentioned other than the verticapy library itself. ```python from verticapy.machine_learning.vertica.automl import AutoDataPrep # Example initialization (assuming necessary context/data) # autoprep = AutoDataPrep(parameters...) ``` -------------------------------- ### Build HTML Documentation Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/contribution_guidelines_code_auto_doc_render Compiles the documentation source files into HTML format using the Sphinx build system. This command should be executed from within the 'docs' directory. ```shell make html ``` -------------------------------- ### Initialize BisectingKMeans Model (Python) Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/api/verticapy.machine_learning.memmodel.cluster.BisectingKMeans Demonstrates how to initialize the BisectingKMeans model from the verticapy library. This is the first step before applying the clustering algorithm. ```python from verticapy.machine_learning.memmodel.cluster import BisectingKMeans # Initialize the BisectingKMeans model model = BisectingKMeans() ``` -------------------------------- ### View Query Plan Profile as Pie Chart in Python Source: https://www.vertica.com/python/documentation/1.1.x/html/user_guide_performance_qprof Generates and displays a pie chart representing the profile of the query execution plan. This visualization helps in understanding the proportional cost or time spent on different parts of the query plan. ```python qprof.get_qplan_profile(kind = "pie") ``` -------------------------------- ### SQL Query: Join with Subquery for Max Value Source: https://www.vertica.com/python/documentation/1.1.x/html/user_guide_performance_qprof This SQL query joins the 'amazon' table with a subquery that finds the maximum 'number' for each date. It then calculates the average 'number' per date and includes the maximum value from the subquery, ordered by the average. ```sql %%sql SELECT a.date, MONTH(a.date) AS month, AVG(a.number) AS avg_number_test, b.max_number FROM public.amazon AS a JOIN ( SELECT date, MAX(number) AS max_number FROM public.amazon GROUP BY date ) AS b ON a.date = b.date GROUP BY a.date, b.max_number ORDER BY avg_number_test DESC; ``` -------------------------------- ### Import QueryProfiler Source: https://www.vertica.com/python/documentation/1.1.x/html/api/verticapy.performance.vertica.qprof.QueryProfiler Imports the `QueryProfiler` class from the `verticapy.performance.vertica` module. This is the first step to using query profiling functionalities. ```python from verticapy.performance.vertica import QueryProfiler ``` -------------------------------- ### RandomForestClassifier Initialization and Methods - Python Source: https://www.vertica.com/python/documentation/1.1.x/html/_sources/api/verticapy.machine_learning.vertica.ensemble.RandomForestClassifier Demonstrates the initialization of the RandomForestClassifier and showcases common methods used for model training, evaluation, and manipulation. This includes fitting the model, predicting outcomes, generating reports, and model deployment utilities. ```python from verticapy.machine_learning.vertica.ensemble import RandomForestClassifier # Initialize the classifier model = RandomForestClassifier(name='my_random_forest', ...) # Fit the model to data model.fit(X, y) # Make predictions predictions = model.predict(X_new) # Get classification report report = model.classification_report(X_test, y_test) # Deploy the model to Vertica model.deploySQL() # Get model parameters params = model.get_params() # Summarize model details model.summarize() ```