### Run All Examples Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/examples.rst Use this nox session to run all provided PyExasol examples. ```bash poetry run -- nox -s run:examples ``` -------------------------------- ### UDF Script Output Capture Example Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/examples.rst Shows how to run a query with a UDF script and capture its output using PyEXASOL. Note: This may not work on all local setups. ```python import pyexasol # Example for running a query with a UDF script and capturing its output. # This requires a UDF script to be defined in Exasol. conn = None try: conn = pyexasol.connect( dsn='your_exasol_dsn', user='your_user', password='your_password' ) # Assume 'my_udf_script' is a UDF script in Exasol that prints something. # The exact syntax for calling UDFs might vary. # Example: conn.execute("SELECT my_udf_script(column_name) FROM my_table") # To capture output, you might need to configure the UDF script itself # or use specific query patterns if supported by Exasol for output redirection. # This example is conceptual as direct output capture from UDFs via standard SQL # might not be a direct feature of the client library without specific Exasol setup. # A more common approach is to have the UDF return a value that can be captured. # Example: result = conn.execute("SELECT my_udf_script('input_param')").fetchone() # print(f"UDF script returned: {result[0]}") print("Conceptual example for UDF script output. Actual implementation may vary.") except pyexasol.ExaError as e: print(f"An Exasol error occurred: {e}") except Exception as e: print(f"An unexpected error occurred: {e}") finally: if conn: conn.close() print("Connection closed.") ``` -------------------------------- ### Install Dependencies with Poetry Source: https://github.com/exasol/pyexasol/blob/master/doc/developer_guide.rst Installs all project dependencies, including extras, for development. ```shell poetry install --all-extras ``` -------------------------------- ### Complete SQL Formatting Example Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/formatting_sql.rst A comprehensive example demonstrating various placeholder types and conversions within a SQL query, along with the resulting formatted SQL. ```python # SQL with formatting params = { 'random_value': 'abc', 'null_value': None, 'table_name_1': 'users', 'table_name_2': (config.schema, 'PAYMENTS'), 'user_rating': '0.5', 'user_score': 1e1, 'is_female': 'TRUE', 'user_statuses': ['ACTIVE', 'PASSIVE', 'SUSPENDED'], 'exclude_user_score': [10, 20], 'limit': 10 } query = ''' SELECT {random_value} AS random_value, {null_value} AS null_value, u.user_id, sum(gross_amt) AS gross_amt FROM {table_name_1!i} u JOIN {table_name_2!q} p ON (u.user_id=p.user_id) WHERE u.user_rating >= {user_rating!d} AND u.user_score > {user_score!f} AND u.is_female IS {is_female!r} AND u.status IN ({user_statuses}) AND u.user_rating NOT IN ({exclude_user_score!d}) GROUP BY 1,2,3 ORDER BY 4 DESC LIMIT {limit!d} ''' stmt = C.execute(query, params) print(stmt.query) ``` ```sql SELECT 'abc' AS random_value, NULL AS null_value, u.user_id, sum(gross_amt) AS gross_amt FROM users u JOIN "PYEXASOL_TEST"."PAYMENTS" p ON (u.user_id=p.user_id) WHERE u.user_rating >= 0.5 AND u.user_score > 10.0 AND u.is_female IS TRUE AND u.status IN ('ACTIVE', 'PASSIVE', 'SUSPENDED') AND u.user_rating NOT IN (10, 20) GROUP BY 1,2,3 ORDER BY 4 DESC LIMIT 10 ``` -------------------------------- ### Start Test Database for Performance Tests Source: https://github.com/exasol/pyexasol/blob/master/doc/developer_guide.rst Starts a Docker container with a test database, which is required before running performance tests. ```shell poetry run -- nox -s db:start ``` -------------------------------- ### Connection via HTTP Proxy Example Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/examples.rst Demonstrates how to establish a connection to Exasol through an HTTP proxy using PyEXASOL. ```python import pyexasol # Example for connecting via an HTTP proxy. # This is useful in environments where direct access to the Exasol instance is restricted. proxy_address = "http://your_proxy_host:proxy_port" conn = None try: conn = pyexasol.connect( dsn='your_exasol_dsn', user='your_user', password='your_password', proxy=proxy_address ) print(f"Connected successfully via HTTP proxy: {proxy_address}") except pyexasol.ExaError as e: print(f"An Exasol error occurred: {e}") except Exception as e: print(f"An unexpected error occurred: {e}") finally: if conn: conn.close() print("Connection closed.") ``` -------------------------------- ### Connect Using Local Config File Example Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/examples.rst Demonstrates how to establish a connection to Exasol using a local configuration file. ```python import pyexasol import os # Example for connecting using a local config file. # The config file should be in a format like: # [EXASOL] # DSN = your_exasol_dsn # USER = your_user # PASSWORD = your_password # Create a dummy config file for demonstration config_content = """ [EXASOL] DSN = your_exasol_dsn USER = your_user PASSWORD = your_password """ config_file_path = "pyexasol_config.ini" with open(config_file_path, "w") as f: f.write(config_content) conn = None try: # The config_location parameter points to the INI file. conn = pyexasol.connect(config_location=config_file_path) print(f"Connected successfully using config file: {config_file_path}") except pyexasol.ExaError as e: print(f"An Exasol error occurred: {e}") except Exception as e: print(f"An unexpected error occurred: {e}") finally: if conn: conn.close() print("Connection closed.") # Clean up the dummy config file if os.path.exists(config_file_path): os.remove(config_file_path) ``` -------------------------------- ### DB-API 2.0 Compatibility Wrapper Example Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/examples.rst Demonstrates the DB-API 2.0 compatibility wrapper provided by PyEXASOL. ```python import pyexasol import sqlite3 # Example using a standard DB-API 2.0 library # This example shows how PyEXASOL's DB-API 2.0 compatibility can be used. # It's more about demonstrating the interface than a direct Exasol operation. # Assume 'conn' is an established pyexasol.connect() object # For demonstration, we'll simulate a DB-API 2.0 connection object class MockExasolDBAPIConnection: def cursor(self): return MockExasolDBCursor() def close(self): print("Mock connection closed.") class MockExasolDBCursor: def execute(self, query, params=None): print(f"Executing: {query} with params: {params}") # Simulate returning some results self.description = [('col1', None, None, None, None, None, None), ('col2', None, None, None, None, None, None)] self._results = [('value1', 100), ('value2', 200)] return self def fetchall(self): return self._results def close(self): print("Mock cursor closed.") # --- Usage Example --- # In a real scenario, you would use: # exasol_conn = pyexasol.connect(...) # dbapi_conn = exasol_conn.db_api() # For this example, we use the mock object: dbapi_conn = MockExasolDBAPIConnection() cursor = dbapi_conn.cursor() cursor.execute("SELECT col1, col2 FROM my_table WHERE id = ?", (123,)) results = cursor.fetchall() print("Fetched results:", results) cursor.close() dbapi_conn.close() ``` -------------------------------- ### Start Script Output Server in Debug Mode Source: https://github.com/exasol/pyexasol/blob/master/doc/changes/changes_0.10.0.md Use this command to start the script output server in debug mode. The old method will now raise a RuntimeException. ```python python -m pyexasol_utils.script_output ``` -------------------------------- ### Extension Functions Example Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/examples.rst Provides an example of using extension functions to assist with common tasks outside the direct scope of the database driver. ```python import pyexasol C = pyexasol.connect( dsn='your_exasol_host:8563', user='your_username', password='your_password' ) # Example: Using an extension function (assuming one is defined and available) # Replace 'my_extension_function' with an actual available extension function name # result = C.execute("SELECT my_extension_function(column1) FROM your_table") # print(result.fetchall()) # Placeholder for actual extension function usage print("Extension function examples require specific setup.") C.close() ``` -------------------------------- ### Connection Redundancy Example Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/examples.rst Illustrates how to implement connection redundancy and handle missing nodes in PyEXASOL. ```python import pyexasol # Example demonstrating connection redundancy # This requires a cluster setup with multiple nodes. # Replace with your actual connection details. conn = None try: conn = pyexasol.connect( dsn='your_exasol_dsn_with_multiple_nodes', user='your_user', password='your_password' ) print("Connected with redundancy enabled.") # Perform operations here. If a node fails, PyEXASOL might automatically # switch to another available node if configured correctly. except pyexasol.ExaError as e: print(f"An Exasol error occurred: {e}") except Exception as e: print(f"An unexpected error occurred: {e}") finally: if conn: conn.close() print("Connection closed.") ``` -------------------------------- ### Prepare Data for Testing Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/examples.rst Run this script to prepare the data set for testing PyExasol examples. ```bash python examples/a00_prepare.py ``` -------------------------------- ### Custom Session Parameters Example Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/examples.rst Shows how to pass custom session parameters like client_name and client_version to Exasol. ```python import pyexasol # Example for passing custom session parameters. # These parameters can be useful for identifying client applications in Exasol logs. conn = None try: conn = pyexasol.connect( dsn='your_exasol_dsn', user='your_user', password='your_password', client_name='MyCustomApp', client_version='1.2.3', config_location='optional/path/to/config' ) print("Connected with custom session parameters.") # You can verify these parameters in Exasol by querying session information # For example: SELECT * FROM EXA_SESSIONS WHERE ws_client_name = 'MyCustomApp' except pyexasol.ExaError as e: print(f"An Exasol error occurred: {e}") except Exception as e: print(f"An unexpected error occurred: {e}") finally: if conn: conn.close() print("Connection closed.") ``` -------------------------------- ### Install and Configure Pre-commit Hooks Source: https://github.com/exasol/pyexasol/blob/master/doc/developer_guide.rst Installs pre-commit hooks to enforce code quality checks before committing and pushing. This command should be run within the activated Poetry shell. ```shell poetry run -- pre-commit install --hook-type pre-commit --hook-type pre-push ``` -------------------------------- ### JSON Library RapidJSON Example Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/examples.rst Demonstrates using the RapidJSON library for handling JSON data fetched from Exasol. ```python import pyexasol import rapidjson # Example using the RapidJSON library for JSON processing. # Assumes you have a table with a JSON column. conn = None try: conn = pyexasol.connect( dsn='your_exasol_dsn', user='your_user', password='your_password' ) # Assume 'my_json_table' has a column 'json_data' of type VARCHAR or JSON # conn.execute("CREATE OR REPLACE TABLE my_json_table (id INT, json_data VARCHAR(1000))") # sample_json = '{"name": "Exasol", "version": 7}' # conn.execute("INSERT INTO my_json_table VALUES (1, ?)", (sample_json,)) result = conn.execute("SELECT json_data FROM my_json_table WHERE id = 1").fetchone() if result and result[0]: json_string = result[0] # Use rapidjson to parse the JSON string data = rapidjson.loads(json_string) print(f"Parsed JSON using RapidJSON: {data}") print(f"Name: {data.get('name')}") else: print("No JSON data found.") except pyexasol.ExaError as e: print(f"An Exasol error occurred: {e}") except Exception as e: print(f"An unexpected error occurred: {e}") finally: if conn: conn.close() print("Connection closed.") ``` -------------------------------- ### JSON Library UJSON Example Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/examples.rst Demonstrates using the UJSON library for handling JSON data fetched from Exasol. ```python import pyexasol import ujson # Example using the UJSON library for JSON processing. # Assumes you have a table with a JSON column. conn = None try: conn = pyexasol.connect( dsn='your_exasol_dsn', user='your_user', password='your_password' ) # Assume 'my_json_table' has a column 'json_data' of type VARCHAR or JSON # sample_json = '{"city": "Ljubljana", "population": 295500}' # conn.execute("INSERT INTO my_json_table VALUES (2, ?)", (sample_json,)) result = conn.execute("SELECT json_data FROM my_json_table WHERE id = 2").fetchone() if result and result[0]: json_string = result[0] # Use ujson to parse the JSON string data = ujson.loads(json_string) print(f"Parsed JSON using UJSON: {data}") print(f"City: {data.get('city')}") else: print("No JSON data found.") except pyexasol.ExaError as e: print(f"An Exasol error occurred: {e}") except Exception as e: print(f"An unexpected error occurred: {e}") finally: if conn: conn.close() print("Connection closed.") ``` -------------------------------- ### Basic PyExasol Connection and Query Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/examples.rst A minimal code example to establish a connection to the Exasol database and execute a basic query. ```python import pyexasol # Connect to the database C = pyexasol.connect( dsn='your_exasol_host:8563', user='your_username', password='your_password' ) # Run a query R = C.execute("SELECT 1") # Fetch the result as a tuple print(R.fetchone()) # Close the connection C.close() ``` -------------------------------- ### Install PyExasol via pip Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/getting_started.rst Install the PyExasol package using pip. For optional dependencies, use the bracket notation. ```bash pip install pyexasol ``` ```bash pip install pyexasol[] ``` -------------------------------- ### JSON Library ORJSON Example Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/examples.rst Demonstrates using the ORJSON library for handling JSON data fetched from Exasol. ```python import pyexasol import orjson # Example using the ORJSON library for JSON processing. # Assumes you have a table with a JSON column. conn = None try: conn = pyexasol.connect( dsn='your_exasol_dsn', user='your_user', password='your_password' ) # Assume 'my_json_table' has a column 'json_data' of type VARCHAR or JSON # sample_json = '{"product": "Widget", "price": 19.99, "in_stock": true}' # conn.execute("INSERT INTO my_json_table VALUES (3, ?)", (sample_json,)) result = conn.execute("SELECT json_data FROM my_json_table WHERE id = 3").fetchone() if result and result[0]: json_string = result[0] # Use orjson to parse the JSON string # orjson.loads returns bytes if input is bytes, or dict/list if input is str data = orjson.loads(json_string) print(f"Parsed JSON using ORJSON: {data}") print(f"Product: {data.get('product')}") else: print("No JSON data found.") except pyexasol.ExaError as e: print(f"An Exasol error occurred: {e}") except Exception as e: print(f"An unexpected error occurred: {e}") finally: if conn: conn.close() print("Connection closed.") ``` -------------------------------- ### Export Data to Parquet Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/getting_started.rst Connect to Exasol and export query results to a parquet format. Ensure pyarrow is installed with `pip install pyexasol[pyarrow]`. ```python # pip install pyexasol[pyarrow] import pyexasol C = pyexasol.connect(dsn='', user='sys', password='exasol', compression=True) df = C.export_to_parquet("SELECT * FROM EXA_ALL_USERS") print(df.head()) ``` -------------------------------- ### Install Performance Testing Dependencies Source: https://github.com/exasol/pyexasol/blob/master/doc/developer_guide.rst Installs the 'performance' extra dependencies, which includes 'pytest-benchmark', necessary for running performance tests. ```shell poetry install --with performance ``` -------------------------------- ### Prepare Dataset Script Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/advanced_topics/performance.rst Use this bash script to prepare the dataset for performance metric collection. Ensure PyODBC, TurbODBC, PyExasol, and pandas are installed, along with the Exasol ODBC driver. ```bash python 00_prepare.py ``` -------------------------------- ### Extend PyEXASOL Classes Example Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/examples.rst Demonstrates how to extend core PyEXASOL classes to add custom logic or behavior. ```python import pyexasol # Example of extending core PyEXASOL classes. # This allows you to add custom methods or override existing ones. class MyConnection(pyexasol.ExaConnection): def my_custom_method(self): print("This is a custom method added to the connection.") # You can access self.execute() or other connection methods here # --- Usage Example --- conn = None try: # When connecting, you can specify your custom class conn = MyConnection( dsn='your_exasol_dsn', user='your_user', password='your_password' ) print("Connected using a custom connection class.") # Call the custom method conn.my_custom_method() # You can still use standard PyEXASOL methods # conn.execute("SELECT 1") except pyexasol.ExaError as e: print(f"An Exasol error occurred: {e}") except Exception as e: print(f"An unexpected error occurred: {e}") finally: if conn: conn.close() print("Connection closed.") ``` -------------------------------- ### Snapshot Transactions Mode Example Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/examples.rst Demonstrates the usage of snapshot transactions mode in PyEXASOL, which can help with metadata locking problems. ```python import pyexasol # Example for using snapshot transactions mode. # This mode can be beneficial in scenarios with high concurrency and potential metadata locking. conn = None try: conn = pyexasol.connect( dsn='your_exasol_dsn', user='your_user', password='your_password', snapshot_transactions=True # Enable snapshot transactions mode ) print("Connected in snapshot transactions mode.") # Perform operations within the snapshot transaction context. # Example: conn.execute("UPDATE my_table SET value = value + 1 WHERE id = 1") except pyexasol.ExaError as e: print(f"An Exasol error occurred: {e}") except Exception as e: print(f"An unexpected error occurred: {e}") finally: if conn: conn.close() print("Connection closed.") ``` -------------------------------- ### Last Query Profiling Example Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/examples.rst Shows how to retrieve and display profiling information for the last executed query in PyEXASOL. ```python import pyexasol # Example for retrieving profiling information of the last query. conn = None try: conn = pyexasol.connect( dsn='your_exasol_dsn', user='your_user', password='your_password' ) # Execute a query conn.execute("SELECT COUNT(*) FROM your_large_table") # Fetch profiling data for the last executed query profile_data = conn.profile_last_query() if profile_data: print("Profiling data for the last query:") for row in profile_data: print(row) else: print("No profiling data available for the last query.") except pyexasol.ExaError as e: print(f"An Exasol error occurred: {e}") except Exception as e: print(f"An unexpected error occurred: {e}") finally: if conn: conn.close() print("Connection closed.") ``` -------------------------------- ### DSN Parsing and Exception Handling Example Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/examples.rst Illustrates parsing of complex connection strings (DSN) and catching relevant exceptions in PyEXASOL. ```python import pyexasol # Example for parsing DSN strings and handling exceptions. # PyEXASOL can parse various DSN formats. # Example of a complex DSN string dsn_string = "EXASOL://your_host:8563/your_schema?param1=value1¶m2=value2" conn = None try: # Attempt to connect using the DSN string conn = pyexasol.connect(dsn=dsn_string, user='your_user', password='your_password') print(f"Successfully connected using DSN: {dsn_string}") except pyexasol.ExaError as e: # Catch specific PyEXASOL errors related to connection or parsing print(f"An Exasol error occurred during connection: {e}") # You might want to inspect e.code or other attributes for specific error handling except Exception as e: # Catch any other unexpected errors print(f"An unexpected error occurred: {e}") finally: if conn: conn.close() print("Connection closed.") # Example of intentionally malformed DSN to trigger an error try: conn = pyexasol.connect(dsn="INVALID_DSN_FORMAT", user='user', password='password') except pyexasol.ExaError as e: print(f"Caught expected error for invalid DSN: {e}") ``` -------------------------------- ### Connect to Exasol using DB-API 2.0 Facade Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/advanced_topics/dbapi.rst Import and use the DBAPI2 compliant facade for integration with DB-Agnostic frameworks. Ensure you have the 'exasol.driver' package installed. ```python from exasol.driver.websocket.dbapi2 import connect connection = connect(dsn='', username='sys', password='exasol', schema='TEST') with connection.cursor() as cursor: cursor.execute("SELECT 1;") ``` -------------------------------- ### Example .pyexasol.ini Configuration Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/configuration/local_configuration.rst Define Exasol connection parameters such as DSN, user, password, and schema in a standard INI format. Supports multiple sections for different Exasol instances. ```ini [my_exasol] dsn = myexasol1..5 user = my_user password = my_password schema = my_schema compression = True fetch_dict = True ``` -------------------------------- ### Thread Safety Example Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/examples.rst Demonstrates the built-in protection in PyEXASOL for accessing connection objects from multiple threads simultaneously. ```python import pyexasol import threading # Example demonstrating thread safety. # PyEXASOL connections are generally not thread-safe by default. # This example shows how to manage access if you need to use a connection from multiple threads. conn = None def worker_task(connection, task_id): try: # In a real scenario, you would acquire a lock before accessing the connection # For demonstration, we just show the call. print(f"Thread {task_id}: Executing query.") connection.execute("SELECT %s", (task_id,)) print(f"Thread {task_id}: Query executed successfully.") except Exception as e: print(f"Thread {task_id}: Error - {e}") try: conn = pyexasol.connect( dsn='your_exasol_dsn', user='your_user', password='your_password' ) print("Connected to Exasol.") threads = [] num_threads = 5 for i in range(num_threads): # Pass the connection object to each thread. # IMPORTANT: In a real application, you MUST use a lock to ensure only one thread # accesses the connection at a time to prevent race conditions. thread = threading.Thread(target=worker_task, args=(conn, i)) threads.append(thread) thread.start() for thread in threads: thread.join() print("All threads finished.") except pyexasol.ExaError as e: print(f"An Exasol error occurred: {e}") except Exception as e: print(f"An unexpected error occurred: {e}") finally: if conn: conn.close() print("Connection closed.") ``` -------------------------------- ### Lock-Free Metadata Requests Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/examples.rst Example of performing lock-free metadata requests to the Exasol database using PyExasol. ```python import pyexasol C = pyexasol.connect( dsn='your_exasol_host:8563', user='your_username', password='your_password' ) # Fetch metadata without acquiring locks # Example: Get information about tables tables_info = C.get_tables(lock=False) print(tables_info) # Example: Get column information for a specific table columns_info = C.get_columns(table='your_table', lock=False) print(columns_info) C.close() ``` -------------------------------- ### HTTP Transport Errors Example Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/examples.rst Demonstrates various ways to break HTTP transport and handle associated errors. ```python import pyexasol import os # Example of how to use the HTTP transport with error handling # This example assumes you have a running Exasol instance and appropriate credentials. # Replace with your actual connection details. conn = None try: conn = pyexasol.connect( dsn='your_exasol_dsn', user='your_user', password='your_password', http_transport=True # Enable HTTP transport ) print("Connected successfully using HTTP transport.") # Example of a query that might cause an error or timeout # conn.execute("SELECT SLEEP(1000000)") except pyexasol.ExaError as e: print(f"An Exasol error occurred: {e}") except Exception as e: print(f"An unexpected error occurred: {e}") finally: if conn: conn.close() print("Connection closed.") ``` -------------------------------- ### SSL Encrypted WebSocket Connection Example Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/examples.rst Demonstrates establishing an SSL-encrypted WebSocket connection and using HTTP transport with PyEXASOL. ```python import pyexasol # Example for SSL-encrypted WebSocket connection and HTTP transport. # This requires your Exasol instance to be configured for SSL. # Replace with your actual connection details and SSL certificate paths if needed. conn = None try: conn = pyexasol.connect( dsn='your_exasol_dsn', user='your_user', password='your_password', websocket_sslopt={"cert_reqs": "REQUIRED", "ca_certs": "/path/to/your/ca.pem"}, http_transport=True, # Also enable HTTP transport http_websocket_sslopt={"cert_reqs": "REQUIRED", "ca_certs": "/path/to/your/ca.pem"} # SSL options for HTTP transport ) print("Connected successfully with SSL encrypted WebSocket and HTTP transport.") except pyexasol.ExaError as e: print(f"An Exasol error occurred: {e}") except Exception as e: print(f"An unexpected error occurred: {e}") finally: if conn: conn.close() print("Connection closed.") ``` -------------------------------- ### Edge Cases for Data Types Example Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/examples.rst Shows how to store and fetch the biggest and smallest values for various data types available in Exasol. ```python import pyexasol import decimal # Example for storing and fetching extreme values for different data types. # This requires a connection to an Exasol database. conn = None try: conn = pyexasol.connect( dsn='your_exasol_dsn', user='your_user', password='your_password' ) # Example: Storing and fetching the smallest and largest DECIMAL values min_decimal = decimal.Decimal('-9999999999999999999999999999999999999.9999999999') max_decimal = decimal.Decimal('9999999999999999999999999999999999999.9999999999') conn.execute("CREATE OR REPLACE TABLE edge_case_test (val DECIMAL(38, 38))") conn.execute("INSERT INTO edge_case_test VALUES (?)", (min_decimal,)) conn.execute("INSERT INTO edge_case_test VALUES (?)", (max_decimal,)) result = conn.execute("SELECT val FROM edge_case_test ORDER BY val").fetchall() print(f"Smallest value fetched: {result[0][0]}") print(f"Largest value fetched: {result[1][0]}") # Add similar examples for other data types like INT, DOUBLE, DATE, etc. except pyexasol.ExaError as e: print(f"An Exasol error occurred: {e}") except Exception as e: print(f"An unexpected error occurred: {e}") finally: if conn: conn.close() ``` -------------------------------- ### Parallel Export and Import Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/examples.rst Shows an example of performing both parallel export and import operations concurrently using PyExasol's multi-process HTTP transport. ```python import pyexasol C = pyexasol.connect( dsn='your_exasol_host:8563', user='your_username', password='your_password' ) # Example of combined parallel export and import # This is a conceptual example, actual implementation might vary based on needs # For instance, exporting from one table and importing into another in parallel # C.export_to_parquet(query='SELECT * FROM source_table', path='/path/to/export_part_{}.parquet', num_processes=4) # C.import_from_parquet(path='/path/to/import_part_{}.parquet', table='target_table', num_processes=4) print("Conceptual example for parallel export and import.") C.close() ``` -------------------------------- ### Custom Export to List Callback Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/import_and_export/index.rst Example of a custom callback function to export data from Exasol into a basic Python list using CSV format. ```python # Define callback function def export_to_list(pipe, dst, **kwargs): wrapped_pipe = io.TextIOWrapper(pipe, newline='\n') reader = csv.reader(wrapped_pipe, lineterminator='\n', **kwargs) return [row for row in reader] # Run EXPORT using the defined callback function C.export_to_callback(export_to_list, None, 'my_table') ``` -------------------------------- ### Run Basic Query with Context Managers Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/getting_started.rst Connect to an Exasol database and execute a basic query using context managers for proper resource management. Demonstrates fetching rows individually, in batches, and all at once. ```python import pyexasol # Usage of the context manager for a DB connection is helpful as it ensures proper # resource management -- like closing the connection after proper usage or an # exception is raised. with pyexasol.connect(dsn='', user='sys', password='exasol') as C: with C.execute("SELECT * FROM EXA_ALL_USERS") as stmt: # to fetch 1 row print(stmt.fetchone()) # to fetch n=3 rows print(stmt.fetchmany(3)) # to fetch all remaining rows print(stmt.fetchall()) # This is not needed for the code to run, but it shows the value of a context manager. print(stmt.is_closed) # This is not needed for the code to run, but it shows the value of a context manager. print(C.is_closed) ``` ```python with pyexasol.connect(dsn='', user='sys', password='exasol') as C: with C.execute("SELECT * FROM EXA_ALL_USERS") as stmt: # to iterate through all rows for row in stmt: print(row) ``` -------------------------------- ### Creation and Execution of Prepared Statements Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/examples.rst Illustrates the creation of prepared statements and their single or bulk execution in PyExasol. ```python import pyexasol C = pyexasol.connect( dsn='your_exasol_host:8563', user='your_username', password='your_password' ) # Prepare a statement stmt = C.prepare("INSERT INTO your_table (col1, col2) VALUES (?, ?)") # Single execution stmt.execute((10, 'single_value')) # Bulk execution data_batch = [ (11, 'batch_value_1'), (12, 'batch_value_2') ] stmt.execute_many(data_batch) print("Prepared statements executed.") C.close() ``` -------------------------------- ### Export Data to Polars DataFrame Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/getting_started.rst Connect to Exasol and export query results into a polars DataFrame. Ensure polars is installed with `pip install pyexasol[polars]`. ```python # pip install pyexasol[polars] import pyexasol C = pyexasol.connect(dsn='', user='sys', password='exasol', compression=True) df = C.export_to_polars("SELECT * FROM EXA_ALL_USERS") print(df.head()) ``` -------------------------------- ### Export Data to Pandas DataFrame Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/getting_started.rst Connect to Exasol and export query results directly into a pandas DataFrame. Ensure pandas is installed with `pip install pyexasol[pandas]`. ```python # pip install pyexasol[pandas] import pyexasol C = pyexasol.connect(dsn='', user='sys', password='exasol', compression=True) df = C.export_to_pandas("SELECT * FROM EXA_ALL_USERS") print(df.head()) ``` -------------------------------- ### Connect to Exasol Using Local Config Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/configuration/local_configuration.rst Establish a connection to Exasol by specifying the configuration section and path to the .pyexasol.ini file. Ensure pyexasol is imported. ```python import pyexasol C = pyexasol.connect_local_config( config_section='my_exasol', config_path="~/.pyexasol.ini" ) st = C.execute("SELECT CURRENT_TIMESTAMP") print(st.fetchone()) ``` -------------------------------- ### Run Performance Tests with Nox Source: https://github.com/exasol/pyexasol/blob/master/doc/developer_guide.rst Executes the performance tests using the nox task runner. Ensure the test database is started beforehand using 'nox -s db:start'. ```shell nox -s test:performance ``` -------------------------------- ### Import and Export to Parquet Files Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/examples.rst Demonstrates how to import data from Exasol to Parquet file(s) and export data from Parquet file(s) to Exasol. ```python import pyexasol C = pyexasol.connect( dsn='your_exasol_host:8563', user='your_username', password='your_password' ) # Export data from Exasol to Parquet file C.export_to_parquet(query='SELECT * FROM your_source_table', path='/path/to/output.parquet') # Import data from Parquet file to Exasol C.import_from_parquet(path='/path/to/input.parquet', table='your_target_table') print("Parquet import/export operations completed.") C.close() ``` -------------------------------- ### No-SQL Metadata Commands Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/examples.rst Demonstrates how to use no-SQL metadata commands, introduced in Exasol v7.0+, via PyExasol. ```python import pyexasol C = pyexasol.connect( dsn='your_exasol_host:8563', user='your_username', password='your_password' ) # Example: Fetching no-SQL metadata (e.g., about UDFs) # The exact commands depend on the Exasol version and available metadata # This is a conceptual example, replace with actual no-SQL commands if known # try: # result = C.execute("SELECT * FROM EXASOL_METADATA.UDFS") # print(result.fetchall()) # except pyexasol.ExaError as e: # print(f"Could not fetch no-SQL metadata: {e}") print("No-SQL metadata examples require specific Exasol version and commands.") C.close() ``` -------------------------------- ### Enable Quoted Identifiers Example Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/examples.rst Shows how to enable quoted identifiers for import_*, export_*, and other relevant functions in PyEXASOL. ```python import pyexasol # Example for enabling quoted identifiers. # This is useful when dealing with table or column names that are keywords or contain spaces/special characters. conn = None try: conn = pyexasol.connect( dsn='your_exasol_dsn', user='your_user', password='your_password', quote_identifiers=True # Enable quoted identifiers ) print("Connected with quoted identifiers enabled.") # Now, functions like import_*, export_* will handle identifiers with quotes. # Example: conn.export_to_file('"my table with spaces"', 'output.csv') # Example: conn.import_from_file('"another table"', 'input.csv') except pyexasol.ExaError as e: print(f"An Exasol error occurred: {e}") except Exception as e: print(f"An unexpected error occurred: {e}") finally: if conn: conn.close() print("Connection closed.") ``` -------------------------------- ### Other Import and Export Methods Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/examples.rst Illustrates alternative methods for importing data into and exporting data from Exasol using PyExasol. ```python import pyexasol C = pyexasol.connect( dsn='your_exasol_host:8563', user='your_username', password='your_password' ) # Example: Exporting data to a file (e.g., CSV) C.export_to_csv(query='SELECT * FROM your_source_table', path='/path/to/output.csv') # Example: Importing data from a file (e.g., CSV) C.import_from_csv(path='/path/to/input.csv', table='your_target_table') print("CSV import/export operations completed.") C.close() ``` -------------------------------- ### Fetch Result Set as Tuples Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/examples.rst Demonstrates all methods for fetching a result set, with results returned as tuples. ```python import pyexasol C = pyexasol.connect( dsn='your_exasol_host:8563', user='your_username', password='your_password' ) # Fetch all rows as tuples print(C.execute("SELECT * FROM your_table").fetchall()) # Fetch one row as a tuple print(C.execute("SELECT * FROM your_table").fetchone()) # Fetch rows as tuples using an iterator for row in C.execute("SELECT * FROM your_table"): print(row) C.close() ``` -------------------------------- ### Enable Compression for WiFi Connections Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/best_practices.rst Enable zlib compression for fetching and importing data to improve performance over wireless networks. This can yield significant speedups. ```python C = pyexasol.connect(... , compression=True) ``` -------------------------------- ### Apply Custom Data Type Mapper Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/examples.rst Example of applying a custom data type mapper for fetching results from the database. ```python import pyexasol def my_mapper(value, type_code, type_name): # Custom logic to map data types if type_name == 'DATE': return value.strftime('%Y-%m-%d') return value C = pyexasol.connect( dsn='your_exasol_host:8563', user='your_username', password='your_password', # Apply the custom mapper fetch_mapper=my_mapper ) # Execute a query and fetch results using the custom mapper result = C.execute("SELECT CURRENT_DATE").fetchone() print(result) C.close() ``` -------------------------------- ### Use WITH Clause for Connection and Statement Objects Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/examples.rst Shows how to use the WITH clause (context manager) for managing ExaConnection and ExaStatement objects, ensuring proper cleanup. ```python import pyexasol # Using WITH for ExaConnection with pyexasol.connect( dsn='your_exasol_host:8563', user='your_username', password='your_password' ) as C: # Using WITH for ExaStatement with C.execute("SELECT 1") as R: print(R.fetchone()) # Connection and statement are automatically closed outside the 'with' blocks ``` -------------------------------- ### Detect Garbage Collection Problems Example Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/examples.rst Shows how to detect potential garbage collection problems in PyEXASOL, often caused by cross-references. ```python import pyexasol import gc # Example for detecting potential garbage collection issues. # This often involves creating circular references that might prevent objects from being freed. conn = None def create_circular_reference(connection): # This is a simplified example of creating a potential circular reference. # In real applications, these can occur more subtly. obj1 = [] obj2 = {'ref': obj1} obj1.append(obj2) # If 'connection' holds a reference to 'obj1' or 'obj2' indirectly, # and they hold a reference back, it can cause issues. # For demonstration, let's assume 'connection' might store some state related to these objects. connection._internal_state = obj1 # Hypothetical storage try: conn = pyexasol.connect( dsn='your_exasol_dsn', user='your_user', password='your_password' ) print("Connected to Exasol.") # Create a circular reference scenario create_circular_reference(conn) # Manually trigger garbage collection and check for issues gc.collect() print("Garbage collection performed.") # In a real scenario, you might monitor memory usage or use debugging tools # to identify if objects are not being released as expected. # PyEXASOL aims to break such references upon connection close, but complex # application logic can sometimes interfere. except pyexasol.ExaError as e: print(f"An Exasol error occurred: {e}") except Exception as e: print(f"An unexpected error occurred: {e}") finally: if conn: conn.close() print("Connection closed. Attempting to break references.") # After closing, Python's GC should ideally clean up. gc.collect() ``` -------------------------------- ### Parallel Import using Multi-Process HTTP Transport Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/examples.rst Demonstrates how to perform parallel IMPORT operations into Exasol using multi-process HTTP transport with PyExasol. ```python import pyexasol C = pyexasol.connect( dsn='your_exasol_host:8563', user='your_username', password='your_password' ) # Perform parallel import from multiple files # Adjust num_processes and path pattern as needed C.import_from_parquet( path='/path/to/input_part_{}.parquet', table='your_target_table', num_processes=4 # Number of parallel processes ) print("Parallel import completed.") C.close() ``` -------------------------------- ### Fetch Result Set as Dictionaries Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/examples.rst Demonstrates all methods for fetching a result set, with results returned as dictionaries. ```python import pyexasol C = pyexasol.connect( dsn='your_exasol_host:8563', user='your_username', password='your_password' ) # Fetch all rows as dictionaries print(C.execute("SELECT * FROM your_table").fetchall(dict=True)) # Fetch one row as a dictionary print(C.execute("SELECT * FROM your_table").fetchone(dict=True)) # Fetch rows as dictionaries using an iterator for row in C.execute("SELECT * FROM your_table", dict=True): print(row) C.close() ``` -------------------------------- ### Identifier Placeholder Examples Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/formatting_sql.rst Shows how to use safe_ident and quote_ident for validating and quoting schema and table names, including multi-level identifiers. ```python safe_ident('my_schema', 'my_table') >>> my_schema.my_table ``` ```python quote_ident('my_schema', 'my_table', 'my_column') >>> "my_schema"."my_table"."my_column" ``` -------------------------------- ### Transaction Management and Autocommit Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/examples.rst Illustrates transaction management, including how to handle autocommit behavior in PyExasol. ```python import pyexasol C = pyexasol.connect( dsn='your_exasol_host:8563', user='your_username', password='your_password', autocommit=False # Disable autocommit for manual transaction control ) try: # Start a transaction C.execute("INSERT INTO your_table (col1) VALUES (1)") C.execute("INSERT INTO your_table (col1) VALUES (2)") # Commit the transaction C.commit() print("Transaction committed.") except pyexasol.ExaError as e: # Rollback the transaction in case of an error C.rollback() print(f"Transaction rolled back: {e}") finally: # Close the connection C.close() ``` -------------------------------- ### Custom Import from Pandas Callback Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/exploring_features/import_and_export/index.rst Example of a custom callback function to import data from a Pandas DataFrame into Exasol, using CSV format. ```python df = def import_from_pandas(pipe, src, **kwargs): wrapped_pipe = io.TextIOWrapper(pipe, newline='\n') return src.to_csv(wrapped_pipe, header=False, index=False, quoting=csv.QUOTE_NONNUMERIC, **kwargs) # Run IMPORT using the defined callback function C.import_from_callback(import_from_pandas, df, 'my_table') ``` -------------------------------- ### Connect to Exasol On-Premise or Docker Source: https://github.com/exasol/pyexasol/blob/master/doc/user_guide/configuration/security.rst Use this snippet to connect to an on-premise or Docker-based Exasol instance using username and password authentication. ```python pyexasol.connect(dsn='myexasol:8563' , user='user' , password='password') ```