### Install apt Dependencies Source: https://github.com/kxsystems/pykx/blob/main/README.md Example command to install necessary dependencies on Debian/Ubuntu-based systems using apt. ```bash apt-install python3 python3-venv build-essential python3-dev ``` -------------------------------- ### Basic PyKX Usage Example Source: https://github.com/kxsystems/pykx/blob/main/README.md A simple example demonstrating how to import PyKX and execute a basic q query. Ensure PyKX is installed and a valid license is configured. ```python import pykx pykx.q('1+1') ``` -------------------------------- ### Configure and Start Gateway with All Setup Source: https://github.com/kxsystems/pykx/blob/main/docs/user-guide/advanced/streaming/gateways.md Consolidate gateway configuration, including connection details, libraries, APIs, and validation functions, into a single initialization block. This simplifies system setup and restart. ```python def validation_function(username, password): if username == 'test_user': pattern = re.compile("password.*") if bool(pattern.match(password)): return True return False def gateway_function(table, symbol): rtp = gateway.call_port('rtp', table, symbol) try: hdb = gateway.call_port('hdb', table, symbol) except BaseException: print('Failed to retrieve data from HDB') hdb = 0 return rtp + hdb gateway = kx.tick.GATEWAY( port=5015, connections={'rtp': 'localhost:5014', 'hdb': 'localhost:5012'}, libraries={'re':'re'}, apis={'sum_of_symbols': gateway_function}, connection_validator=validation_function ) gateway.start() ``` -------------------------------- ### Install PyKX from Source Source: https://github.com/kxsystems/pykx/blob/main/docs/getting-started/installing.md Navigate into the cloned repository and install PyKX using pip. ```sh cd pykx pip install . ``` -------------------------------- ### Install yum Dependencies Source: https://github.com/kxsystems/pykx/blob/main/README.md Example command to install necessary dependencies on Red Hat/CentOS-based systems using yum. ```bash yum install python3 gcc gcc-c++ python3-devel.x86_64 ``` -------------------------------- ### Build and Install PyKX from Source Source: https://github.com/kxsystems/pykx/blob/main/README.md Steps to build and install PyKX from source within a Python virtual environment. This includes activating the environment and running the pip install command. ```bash python3 -m venv pykx-dev source pykx-dev/bin/activate cd pykx pip3 install -U '.[all]' ``` -------------------------------- ### Start Embedded Q IPC Server Source: https://github.com/kxsystems/pykx/blob/main/docs/help/issues.md This example demonstrates how to start a q IPC server within a Python process using PyKX. Note that the embedded q process cannot respond to IPC requests. ```python >>> import pykx as kx >>> kx.q('\p 5001') pykx.Identity(pykx.q('::')) ``` -------------------------------- ### Create Example Table Source: https://github.com/kxsystems/pykx/blob/main/docs/user-guide/advanced/Pandas_API.ipynb Initializes a sample kx.Table for use in subsequent examples. This table has three rows and two columns, 'A' and 'B'. ```python tab = kx.Table([[4, 9]] * 3, columns=['A', 'B']) ``` ```python tab ``` -------------------------------- ### Start q process with PyKX Source: https://github.com/kxsystems/pykx/blob/main/docs/user-guide/advanced/remote-functions.md Starts a q process on port 5050 with PyKX loaded, ready to accept remote connections. ```bash q pykx.q -p 5050 ``` -------------------------------- ### Create Example Table Source: https://github.com/kxsystems/pykx/blob/main/docs/user-guide/advanced/Pandas_API.ipynb Initializes a pykx Table with sample data for demonstration purposes. ```python tab = kx.Table(data={ 'Animal': ['Falcon', 'Falcon', 'Parrot', 'Parrot'], 'Speed': [380., 370., 24., 26.], 'Max_Altitude': [570., 555., 275., 300.] }) tab ``` -------------------------------- ### TOML Configuration File Example Source: https://github.com/kxsystems/pykx/blob/main/docs/user-guide/configuration.md Example of a .pykx-config file using TOML syntax to define default, test, and beta profiles for PyKX initialization. ```toml [default] PYKX_IGNORE_QHOME="true" PYKX_KEEP_LOCAL_TIMES="true" [test] PYKX_GC="true" PYKX_RELEASE_GIL="true" [beta] PYKX_BETA_FEATURES="true" ``` -------------------------------- ### Install pykx.q into QHOME Source: https://github.com/kxsystems/pykx/blob/main/docs/pykx-under-q/api.md Installs the pykx.q library into your QHOME directory. Use `overwrite_embedpy=True` if you previously had embedPy installed. The `to_local_folder` argument can be used to specify an alternative installation path. ```python import pykx as kx kx.install_into_QHOME() ``` ```python python -c "import pykx;pykx.install_into_QHOME()" ``` ```python python -c "import pykx;pykx.install_into_QHOME(overwrite_embedpy=True)" ``` ```bash python -c "import pykx;pykx.install_into_QHOME(to_local_folder=True)" ``` ```bash python -c "import pykx;pykx.install_into_QHOME(to_local_folder='some/dir/path')" ``` -------------------------------- ### kx.install_into_QHOME Source: https://github.com/kxsystems/pykx/blob/main/docs/pykx-under-q/api.md Installs the pykx.q library into your QHOME directory. Can optionally overwrite embedPy and specify an installation path. ```APIDOC ## kx.install_into_QHOME ### Description Installs the pykx.q library into your QHOME directory. This function can optionally overwrite existing embedPy installations and allows for specifying a custom installation path. ### Method `kx.install_into_QHOME(overwrite_embedpy=False, to_local_folder=False)` ### Parameters #### Keyword Arguments - **overwrite_embedpy** (bool) - Optional - If True, will overwrite an existing embedPy installation. - **to_local_folder** (bool or str) - Optional - If True, installs to the current working directory. If a string path is provided, installs to that directory. ### Request Example ```python import pykx as kx kx.install_into_QHOME() ``` ### Request Example with overwrite ```python import pykx as kx kx.install_into_QHOME(overwrite_embedpy=True) ``` ### Request Example with custom folder ```bash python -c "import pykx;pykx.install_into_QHOME(to_local_folder='some/dir/path')" ``` ``` -------------------------------- ### Import PyKX and use q.til() Source: https://github.com/kxsystems/pykx/blob/main/docs/getting-started/installing.md Verify successful license installation by importing PyKX and executing a simple q function. ```bash $ python >>> import pykx as kx >>> kx.q.til(5) pykx.LongVector(pykx.q('0 1 2 3 4')) ``` -------------------------------- ### Start q Processes for IPC Source: https://github.com/kxsystems/pykx/blob/main/docs/examples/ipc/README.md Commands to start the necessary q processes for the IPC example: a tick process, a mock feed, and a q process to receive data from PyKX. ```q // run tick.q $ q tick/tick.q sym ./log/ q) // run the mock feed $ q tick/feed.q q) // Start the q process to receive data from PyKX $ q -p 5130 q) ``` -------------------------------- ### Install pykx.q script Source: https://github.com/kxsystems/pykx/blob/main/docs/pykx-under-q/intro.md Installs the pykx.q script into your QHOME directory. Use overwrite_embedpy=True if you previously had embedPy installed. Use to_local_folder=True or a specific path to install in the current or specified directory. ```python python -c "import pykx;pykx.install_into_QHOME()" ``` ```python python -c "import pykx;pykx.install_into_QHOME(overwrite_embedpy=True)" ``` ```bash python -c "import pykx;pykx.install_into_QHOME(to_local_folder=True)" ``` ```bash python -c "import pykx;pykx.install_into_QHOME(to_local_folder='some/dir/path')" ``` -------------------------------- ### Install Windows Dependencies Source: https://github.com/kxsystems/pykx/blob/main/README.md Instructions for installing Python, Build Tools for Visual Studio, and dlfcn-win32 on Windows. The `w64_install.ps1` script can automate this. ```powershell cd pykx .\w64_install.ps1 ``` -------------------------------- ### Install PyKX with Beta Features (Previous) Source: https://github.com/kxsystems/pykx/blob/main/docs/upgrades/2030.md Use this command to install PyKX with beta features enabled in previous versions. ```bash pip install pykx[beta] ``` -------------------------------- ### Install PyKX using uv Source: https://github.com/kxsystems/pykx/blob/main/docs/getting-started/installing.md Install the latest version of PyKX using the uv package installer. ```sh uv pip install pykx ``` -------------------------------- ### Run Multithreaded Example Scripts Source: https://github.com/kxsystems/pykx/blob/main/docs/examples/threaded_execution/threading.md Execute either the asyncio-based or threading-based Python script to run the example. ```bash $ python asyncio_threading.py // or $ python threads.py ``` -------------------------------- ### Install q executable to a specified location Source: https://github.com/kxsystems/pykx/blob/main/docs/getting-started/installing.md Use this command to install the q executable to a custom directory. The specified path will be used for QHOME and PYKX_Q_EXECUTABLE in the PyKX configuration. ```python >>> kx.util.install_q('~/custom') ``` -------------------------------- ### Initialize Temporary Directory Source: https://github.com/kxsystems/pykx/blob/main/docs/examples/db-management.ipynb Create a temporary directory to store the database files for this example. ```python tempdir = tempfile.TemporaryDirectory() ``` -------------------------------- ### Install PyKX into QHOME Source: https://github.com/kxsystems/pykx/blob/main/docs/user-guide/advanced/remote-functions.md Installs PyKX into the QHOME directory, which is necessary for PyKX under q functionality. ```bash python -c "import pykx;pykx.install_into_QHOME()" ``` -------------------------------- ### On-Disk Compression Example Source: https://github.com/kxsystems/pykx/blob/main/docs/examples/compress_and_encrypt/readme.md Demonstrates writing a PyKX table to disk using the lz4hc compression algorithm. ```text Writing on-disk trades table with lz4hc: { "compressedLength": 15233016, "uncompressedLength": 36666552, "algorithm": 4, "logicalBlockSize": 17, "zipLevel": 12 } ``` -------------------------------- ### Install PyKX from PyPI Source: https://github.com/kxsystems/pykx/blob/main/docs/getting-started/installing.md Install the latest version of PyKX using pip. ```sh pip install pykx ``` -------------------------------- ### Create Example Table for applymap Source: https://github.com/kxsystems/pykx/blob/main/docs/user-guide/advanced/Pandas_API.ipynb Defines an example kx.Table with nested lists and null values for demonstrating applymap functionality. ```python tab = kx.Table(data={ 'x': [[1, 2, 3], 1, [1, 2]], 'y': [kx.LongAtom.null, 1, 2]}) ``` -------------------------------- ### PyKX Print Conversion Example Source: https://github.com/kxsystems/pykx/blob/main/docs/release-notes/underq-changelog.md Shows how to print the return of a conversion object using `.pykx.print`. ```q q).pykx.print .pykx.topd ([]5?1f;5?1f) x x1 0 0.613745 0.493183 1 0.529481 0.578520 2 0.691610 0.083889 3 0.229662 0.195991 4 0.691953 0.375638 ``` -------------------------------- ### Install q executable to default location Source: https://github.com/kxsystems/pykx/blob/main/docs/getting-started/installing.md Use this command to install the q executable to the default location, typically '~/q' or 'C:\q'. This will also update the PyKX configuration file. ```python >>> kx.util.install_q() ``` -------------------------------- ### Create Tables for Replacement Examples Source: https://github.com/kxsystems/pykx/blob/main/docs/user-guide/advanced/Pandas_API.ipynb Create an unkeyed `Table` and a `KeyedTable` with elements that will be used for replacement operations. ```python tab = kx.q('([] a:2 2 3; b:4 2 6; c:(1b;0b;1b); d:(`a;`b;`c); e:(1;2;`a))') ktab = kx.q('([a:2 2 3]b:4 2 6; c:(1b;0b;1b); d:(`a;`b;`c); e:(1;2;`a))') ktab ``` -------------------------------- ### Create Example Table for agg Source: https://github.com/kxsystems/pykx/blob/main/docs/user-guide/advanced/Pandas_API.ipynb Defines an example kx.Table with numerical data and null values for demonstrating aggregation functions. ```python tab = kx.Table([[1, 2, 3], [4, 5, 6], [7, 8, 9], [kx.FloatAtom.null, kx.FloatAtom.null, kx.FloatAtom.null]], columns=['A', 'B', 'C']) tab ``` -------------------------------- ### Get Help on q Functions Source: https://github.com/kxsystems/pykx/blob/main/docs/getting-started/quickstart.md Shows how to use the `help()` function in PyKX to get documentation for q keyword functions. This is useful for understanding the behavior and usage of built-in q functions. ```python >>> help(kx.q.max) Help on UnaryPrimitive in pykx: pykx.UnaryPrimitive = pykx.UnaryPrimitive(pykx.q('max')) • max Maximum. >>> pykx.q.max([0, 7, 2, 4 , 1, 3]) pykx.LongAtom(q('7')) ``` ```python >>> help(kx.q.abs) Help on UnaryPrimitive in pykx: pykx.UnaryPrimitive = pykx.UnaryPrimitive(pykx.q('abs')) • abs Where x is a numeric or temporal, returns the absolute value of x. Null is returned if x is null. >>> pykx.q.abs(-5) pykx.LongAtom(q('5')) ``` -------------------------------- ### Run Compression and Encryption Script Source: https://github.com/kxsystems/pykx/blob/main/docs/examples/compress_and_encrypt/readme.md Execute the provided Python script to generate the compression and encryption examples. ```bash $ python compress_and_encrypt.py ``` -------------------------------- ### Initialize and Start Gateway Process Source: https://github.com/kxsystems/pykx/blob/main/docs/examples/streaming/Evolving System.ipynb Initializes the gateway process, configuring its port, libraries, APIs, connections to other services (hdb, rte), and a connection validator. It then starts the gateway. ```python gateway = kx.tick.GATEWAY( port=5015, libraries = {'kx': 'pykx'}, apis = {'gateway_function': gateway_function}, connections={'hdb': 'localhost:5011', 'rte': 'localhost:5014'}, connection_validator = user_validation ) gateway.start() ``` -------------------------------- ### Install PyKX Wheels in Air-Gapped Environment Source: https://github.com/kxsystems/pykx/blob/main/docs/getting-started/installing.md Install PyKX and its dependencies from local wheel files in an air-gapped environment. Ensure all wheel files are copied to the specified directory. ```bash pip install --no-cache /opt/airgap/wheels/* ``` -------------------------------- ### Define and Start Tickerplant Process Source: https://github.com/kxsystems/pykx/blob/main/docs/user-guide/advanced/streaming/complex.md Define trade and aggregate tables using schema builders and start a Tickerplant process. Ensure log directory is specified. ```python import pykx as kx trade = kx.schema.builder({ 'time': kx.TimespanAtom , 'sym': kx.SymbolAtom, 'price': kx.FloatAtom, 'volume': kx.LongAtom}) agg = kx.schema.builder({ 'time': kx.TimespanAtom, 'sym': kx.SymbolAtom, 'max_price': kx.FloatAtom , 'median_volume': kx.FloatAtom}) tick = kx.tick.TICK( port=5010, tables = {'trade': trade, 'aggregate': agg}, log_directory = 'log' ) tick.start() ``` -------------------------------- ### Start a q process Source: https://github.com/kxsystems/pykx/blob/main/docs/examples/subscriber/readme.md Run a q process with an open port for IPC communication. Ensure the port is not in use. ```q // run q $ q -p 5001 q) ``` -------------------------------- ### Start a Chained Tickerplant Source: https://github.com/kxsystems/pykx/blob/main/docs/user-guide/advanced/streaming/subscribe.md Initialize and start a chained tickerplant. This setup is used to subscribe to messages from a zero-latency tickerplant and publish them at a slower rate to downstream subscribers. ```python >>> chained_tp = kx.tick.TICK(port=5013, chained=True) >>> chained_tp.start({'tickerplant': 'localhost:5010'}) ``` -------------------------------- ### Create and Load a Sample Table Source: https://github.com/kxsystems/pykx/blob/main/docs/user-guide/fundamentals/query/sql.md Demonstrates how to create a sample pykx.Table with random data and assign it to a q symbol 'trades'. This table can then be queried using SQL. ```python import pykx as kx trades = kx.Table(data={ 'sym': kx.random.random(100, ['AAPL', 'GOOG', 'MSFT']), 'date': kx.random.random(100, kx.q('2022.01.01') + [0,1,2]), 'price': kx.random.random(100, 1000.0) }) kx.q['trades'] = trades ``` -------------------------------- ### Bash Commands to Run Example Servers Source: https://github.com/kxsystems/pykx/blob/main/docs/examples/AsynchronousQueries/async_querying.md Open two q servers on ports 5050 and 5051 to serve as targets for asynchronous queries. ```bash q -p 5050 ``` ```bash q -p 5051 ``` -------------------------------- ### Create a sample table for kurt() example Source: https://github.com/kxsystems/pykx/blob/main/docs/user-guide/advanced/Pandas_API.ipynb Initializes a pykx Table with sample data to demonstrate the `kurt()` function for calculating kurtosis. ```python tab = kx.Table(data= { 'a': [1, 2, 2, 4], 'b': [1, 2, 6, 7], 'c': [7, 8, 9, 10], 'd': [7, 11, 14, 14] } ) tab ``` -------------------------------- ### Retrieve PyKX Version with .pykx.version Source: https://github.com/kxsystems/pykx/blob/main/docs/pykx-under-q/api.md Get the currently installed version of PyKX in your Q session. This is useful for compatibility checks and debugging. ```q q).pykx.version[] "2.0.0" ``` -------------------------------- ### Get PyKX Debug Information Source: https://github.com/kxsystems/pykx/blob/main/docs/pykx-under-q/api.md Retrieves library and environment information for debugging purposes. The output is a list of strings containing details about the PyKX setup. ```q .pykx.debugInfo[] ``` -------------------------------- ### Initialize Basic Streaming Infrastructure Source: https://github.com/kxsystems/pykx/blob/main/docs/user-guide/advanced/streaming/basic.md Use `pykx.tick.BASIC` to set up the Tickerplant, RDB, and HDB processes. Configure table schemas, logging, and database directories as needed. Call `.start()` to run the pipeline. ```python import pykx as kx trade = kx.schema.builder({ 'time': kx.TimespanAtom , 'sym': kx.SymbolAtom, 'price': kx.FloatAtom, 'volume': kx.LongAtom}) agg = kx.schema.builder({ 'time': kx.TimespanAtom, 'sym': kx.SymbolAtom, 'max_price': kx.FloatAtom , 'median_volume': kx.FloatAtom}) basic = kx.tick.BASIC( tables = {'trade': trade, 'aggregate': agg}, log_directory = 'log', database = 'db') basic.start() ``` -------------------------------- ### Initialize an example table with random data Source: https://github.com/kxsystems/pykx/blob/main/docs/user-guide/advanced/Pandas_API.ipynb Set up a sample table with various data types, including integers, strings, floats, and nulls, for demonstrating sampling functionalities. ```python N = 1000 tab = kx.Table(data = { 'x': kx.q.til(N), 'y': kx.random.random(N, ['AAPL', 'GOOG', 'MSFT']), 'z': kx.random.random(N, 500.0), 'w': kx.random.random(N, 1000), 'v': kx.random.random(N, [kx.LongAtom.null, 0, 50, 100, 200, 250])}) tab.head() ``` -------------------------------- ### PyKX Example: Python Class and Instance Operations Source: https://github.com/kxsystems/pykx/blob/main/docs/pykx-under-q/intro.md Demonstrates loading a Python class, creating an instance, accessing/setting attributes, and calling methods using PyKX. ```bash $ cat test.p # used for tests class obj: def __init__(self,x=0,y=0): self.x = x # attribute self.y = y # property (incrementing on get) @property def y(self): a=self.__y self.__y+=1 return a @y.setter def y(self, y): self.__y = y def total(self): return self.x + self.y ``` -------------------------------- ### Install PyKX with PyTorch support Source: https://github.com/kxsystems/pykx/blob/main/docs/beta-features/torch.md Install PyKX with the necessary dependencies for PyTorch integration. Ensure torch>2.1 is installed. ```bash pip install pykx[torch] ``` -------------------------------- ### Install PyKX with Extras Source: https://github.com/kxsystems/pykx/blob/main/README.md Install PyKX with optional dependencies for specific features like PyArrow, Dashboards, Streaming, Streamlit, or Torch. Ensure the correct extra is specified during installation. ```bash pip install pykx[pyarrow] ``` ```bash pip install pykx[dashboards] ``` ```bash pip install pykx[streaming] ``` ```bash pip install pykx[streamlit] ``` ```bash pip install pykx[torch] ``` -------------------------------- ### Fix for install_into_QHOME with overwrite_embedpy=True Source: https://github.com/kxsystems/pykx/blob/main/docs/release-notes/underq-changelog.md Demonstrates the behavior of loading PyKX through `p)` before and after a fix for `install_into_QHOME` when `overwrite_embedpy=True`. Previously, this would result in an OS error. ```q q)p)print(1+1) 'pykx.q. OS reports: No such file or directory [3] /home/user/q/p.k:1: \l pykx.q ``` ```q q)p)print(1+1) 2 ``` -------------------------------- ### Verify PyKX Installation Source: https://github.com/kxsystems/pykx/blob/main/docs/getting-started/installing.md Check if PyKX is installed correctly by printing its version number. ```bash python -c"import pykx;print(pykx.__version__)" ``` -------------------------------- ### Get Help on DB Create Method Source: https://github.com/kxsystems/pykx/blob/main/docs/examples/db-management.ipynb Use the help() function to view documentation for the db.create method, which details its parameters and usage. ```python help(db.create) ``` -------------------------------- ### Start RTP Process Source: https://github.com/kxsystems/pykx/blob/main/docs/user-guide/advanced/streaming/rta.md Start the Real-Time Processor (RTP) with the specified tickerplant configuration. ```python rtp.start({'tickerplant': 'localhost:5014'}) ``` -------------------------------- ### Connect to q Process with Default Host Source: https://github.com/kxsystems/pykx/blob/main/docs/examples/jupyter-integration.ipynb Connect to a q process on port 5000 using '%%q', defaulting the host to 'localhost'. Creates a sample table. ```python %%q --port 5000 tab:([]a:1000?1000; b:1000?500.0; c:1000?`AAPL`MSFT`GOOG); ``` -------------------------------- ### Create and Display Partitioned Table (Two Empty Partitions) Source: https://github.com/kxsystems/pykx/blob/main/tests/test_repr_html_.ipynb Demonstrates creating a partitioned table with two partitions, both of which are empty. This scenario shows how the system handles empty partitions. ```python q('(`$':2001.01.01/partitionedTab/') set 0#([] a:.z.d-til 2000;b:til 2000)') q('(`$':2001.01.02/partitionedTab/') set 0#([] a:.z.d-til 2000;b:til 2000)') q('system"l ."') display(q('partitionedTab')) ``` -------------------------------- ### Create a sample table for prod() example Source: https://github.com/kxsystems/pykx/blob/main/docs/user-guide/advanced/Pandas_API.ipynb Generates a sample pykx Table with random data for demonstrating the `prod()` function. Ensures that 'traded' and 'price' columns do not contain zero values to avoid issues with product calculations. ```python N = 10 tab = kx.Table(data={ 'sym': kx.random.random(N, ['AAPL', 'GOOG', 'MSFT']), 'price': 2.5 - kx.random.random(N, 5.0), 'traded': 10 - kx.random.random(N, 20), 'hold': kx.random.random(N, False) }) tab[tab['traded'] == 0, 'traded'] = 1 tab[tab['price'] == 0, 'price'] = 1.0 tab ``` -------------------------------- ### Validate PyKX License Installation Source: https://github.com/kxsystems/pykx/blob/main/docs/getting-started/installing.md Verifies the correct installation of the PyKX license by executing a simple q function. ```python >>> kx.q.til(10) pykx.LongVector(pykx.q('0 1 2 3 4 5 6 7 8 9')) ``` -------------------------------- ### License Activation Method Selection Source: https://github.com/kxsystems/pykx/blob/main/docs/getting-started/installing.md Choose how to activate your license: by file path, activation key, or unlicensed mode. Press Enter or type '1' to use a license file. ```bash Select the method you wish to use to activate your license: [1] Download the license file provided in your welcome email and input the file path (Default) [2] Input the activation key (base64 encoded string) provided in your welcome email [3] Proceed with unlicensed mode Enter your choice here [1/2/3]: ``` -------------------------------- ### Install PyKX using pipx Source: https://github.com/kxsystems/pykx/blob/main/docs/getting-started/installing.md Install PyKX with its dependencies using pipx. This creates a virtual environment for PyKX. ```sh pipx install pykx --include-deps ``` -------------------------------- ### Install Latest PyKX Source: https://github.com/kxsystems/pykx/blob/main/README.md Install the latest version of PyKX using pip. Ensure pip is up-to-date first. ```bash pip install --upgrade pip ``` ```bash pip install pykx ``` -------------------------------- ### Configure and Start pykx RTA Process Source: https://github.com/kxsystems/pykx/blob/main/docs/user-guide/advanced/streaming/rta.md This code configures and starts the pykx Real-time Analytics (RTA) process. It defines custom pre- and post-processing functions for handling data streams and includes starting the RTA with specified subscriptions and tickerplant connection. ```python def preprocessor(table, data): if table == 'trade': return data else: return None def postprocessor(table, data): agg = kx.q[table].select( columns = {'min_px':'min price', 'max_px': 'max price', 'spread_px': 'max[price] - min price'}, by = {'symbol': 'symbol'}) kx.q['agg'] = agg # Make the table accessible from q with kx.SyncQConnection(port=5010, wait=False, no_ctx=True) as q: q('.u.upd', 'aggregate', agg._values) return None rtp = kx.tick.RTP(port=5014, subscriptions = ['trade'], libraries={'kx': 'pykx'}, pre_processor=preprocessor, post_processor=postprocessor, vanilla=False) rtp.start({'tickerplant': 'localhost:5013'}) ``` -------------------------------- ### Initialize PyKX library in q Source: https://github.com/kxsystems/pykx/blob/main/docs/pykx-under-q/intro.md Loads the pykx.q script into your q session, making the .pykx namespace available for use. ```q q)\l pykx.q ``` -------------------------------- ### Install PyKX from Anaconda Source: https://github.com/kxsystems/pykx/blob/main/docs/getting-started/installing.md Install PyKX from the kx channel on Anaconda for Linux x86 and arm-based architectures. Type 'y' to confirm. ```sh conda install -c kx pykx ``` -------------------------------- ### Accessing Configuration in Python (Default Profile) Source: https://github.com/kxsystems/pykx/blob/main/docs/user-guide/configuration.md Demonstrates how to import PyKX and access configuration settings, specifically 'ignore_qhome', when using the default profile. ```python >>> import pykx as kx >>> kx.config.ignore_qhome True ``` -------------------------------- ### Install PyKX with Streamlit Support Source: https://github.com/kxsystems/pykx/blob/main/docs/user-guide/advanced/streamlit.md Install PyKX with the necessary dependencies for Streamlit integration. This command ensures all required packages are included. ```bash pip install pykx[streamlit] ``` -------------------------------- ### Set up kdb+ Table for Streamlit Demo Source: https://github.com/kxsystems/pykx/blob/main/docs/user-guide/advanced/streamlit.md Initializes a kdb+ process on port 5050 and creates a sample table named 'tab' with columns 'sym', 'price', and 'size'. This is a prerequisite for the Streamlit script. ```q \p 5050 N:1000 tab:([]sym:N?`AAPL`MSFT`GOOG`FDP;price:100+N?100f;size:10+N?100) ``` -------------------------------- ### Create and Display Partitioned Table (First Populated, Second Empty) Source: https://github.com/kxsystems/pykx/blob/main/tests/test_repr_html_.ipynb Creates a partitioned table with the first partition populated and the second empty. This demonstrates the display order when an empty partition follows a populated one. ```python q('(`$':2001.01.01/partitionedTab/') set 1#([] a:.z.d-til 2000;b:til 2000)') q('(`$':2001.01.02/partitionedTab/') set 0#([] a:.z.d-til 2000;b:til 2000)') q('system"l ."') display(q('partitionedTab')) ``` -------------------------------- ### Install Specific PyKX Version Source: https://github.com/kxsystems/pykx/blob/main/README.md Install a specific version of PyKX by specifying the semver version. Replace with the desired version number. ```bash pip install pykx== ``` -------------------------------- ### Calling q from Python (Standard) Source: https://github.com/kxsystems/pykx/blob/main/docs/user-guide/advanced/performance.md Demonstrates a standard call from Python to q, which works without special configuration. ```python >>> kx.q('{x each 1 2 3}', lambda x: range(x)) pykx.List(pykx.q(' ,0 0 1 0 1 2 ')) ``` -------------------------------- ### Writing a Q Script for Context Loading Source: https://github.com/kxsystems/pykx/blob/main/docs/user-guide/advanced/context_interface.md Create a q script file in a location accessible by PyKX (e.g., `kx.qhome`). This script defines a namespace with variables and functions that can be loaded as a context. ```python >>> demo_extension_source = ''' ... \d .demo_extension ... N:100 ... test_data:([]N?`a`b`c;N?1f;N?10;N?0b) ... test_function:{[data] ... analytic_keys :`max_x1`avg_x2`med_x3; ... analytic_calcs:( ... (max;`x1); ... (avg;`x2); ... (med;`x3)); ... ?[data; ... (); ... .. k!k:enlist `x; ... analytic_keys!analytic_calcs ... ] ... } .. ''' >>> >>> demo_extension_filename = kx.qhome/'demo_extension.q' >>> with open(demo_extension_filename, 'w') as f: ... f.write(demo_extension_source) ``` -------------------------------- ### Register and Start Real-Time Processor Source: https://github.com/kxsystems/pykx/blob/main/docs/examples/streaming/Evolving System.ipynb Registers the defined pre- and post-processing functions with the real-time event processor and starts it, enabling it to listen for messages from the chained tickerplant. ```python rte.libraries({'kx': 'pykx'}) rte.pre_processor(pre_processor) rte.post_processor(post_processor) rte.start({'tickerplant': 'localhost:5013'}) ``` -------------------------------- ### Create a sample table for sem() example Source: https://github.com/kxsystems/pykx/blob/main/docs/user-guide/advanced/Pandas_API.ipynb Initializes a pykx Table with sample data to demonstrate the `sem()` function for calculating the standard error of the mean. ```python tab = kx.Table(data= { 'a': [1, 2, 2, 4], 'b': [1, 2, 6, 7], 'c': [7, 8, 9, 10], 'd': [7, 11, 14, 14], } ) tab ``` -------------------------------- ### Install PyKX License Source: https://github.com/kxsystems/pykx/blob/main/docs/help/troubleshooting.md Use this function to install a downloaded PyKX license file. Ensure the license file is valid and contains the 'pykx' and 'embedq' feature flags. ```python >>> import pykx as kx >>> kx.license.install('/path/to/downloaded/kc.lic') ``` -------------------------------- ### Emulate a kdb+/q server for testing Source: https://github.com/kxsystems/pykx/blob/main/docs/user-guide/advanced/ipc.md This snippet demonstrates how to start a basic Python script that emulates a kdb+/q server for testing PyKX IPC connections. Ensure the 'server.py' script is available. ```python import pykx as kx import subprocess import time with kx.PyKXReimport(): server = subprocess.Popen( ('python', 'server.py'), stdin=subprocess.PIPE, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, ) time.sleep(2) ``` -------------------------------- ### Start Data Subscriber Source: https://github.com/kxsystems/pykx/blob/main/docs/examples/streaming/Evolving System.ipynb Starts a Python script as a subprocess to subscribe to the real-time data, printing the number of processed data points. This is used to validate data ingestion. ```python with kx.PyKXReimport(): subscriber = subprocess.Popen( ['python', 'subscriber.py'], stdin=subprocess.PIPE, stdout=None, stderr=None, ) ``` -------------------------------- ### Start PyKX Subscriber/Publisher Source: https://github.com/kxsystems/pykx/blob/main/docs/examples/ipc/README.md Bash commands to start the Python script that acts as a PyKX subscriber and publisher. It shows variations for running with or without a valid k4.lic file. ```bash // When running with a valid k4.lic in $QHOME $ python readwrite.py Running example in presence of licensed q // When running in the absence of a valid k4.lic in $QHOME '2021.04.02T11:32:41.006 license error: k4.lic Running example in absence of licensed q ``` -------------------------------- ### Import Libraries and Set Environment Variables Source: https://github.com/kxsystems/pykx/blob/main/docs/examples/db-management.ipynb Import necessary libraries and configure environment variables for PyKX. This includes ignoring QHOME symlinking and setting beta features. ```python import os os.environ['PYKX_IGNORE_QHOME'] = '1' # Ignore symlinking PyKX q libraries to QHOME os.environ['PYKX_Q_LOADED_MARKER'] = '' # Only used here for running Notebook under mkdocs-jupyter during document generation. ``` ```python import os os.environ['PYKX_BETA_FEATURES'] = 'true' import pykx as kx from datetime import date import tempfile ``` -------------------------------- ### Licensed Mode: Query API Example Source: https://github.com/kxsystems/pykx/blob/main/docs/user-guide/advanced/modes.md Licensed mode enables the updated query API for PyKX tables. This example selects rows from 'qtab' where 'col2' is greater than 40. ```python >>> qtab.select(where=kx.Column('col2')>40) pykx.Table(pykx.q(' col1 col2 --------- a 76 a 81 a 75 ')) ``` -------------------------------- ### Run PyKX Tests Source: https://github.com/kxsystems/pykx/blob/main/README.md Set up environment variables for q and then run pytest to execute the project's test suite. Ensure q is on the PATH and QHOME is set. ```bash export PATH="$PATH:/location/of/your/q/l64" # q must be on PATH for tests export QHOME=/location/of/your/q #q needs QHOME available python -m pytest -vvv -n 0 --no-cov --junitxml=report.xml ``` -------------------------------- ### Initialize Libraries at Session Creation (New) Source: https://github.com/kxsystems/pykx/blob/main/docs/upgrades/2030.md Libraries can be specified using the `libraries` keyword during session creation in new PyKX versions. ```python >>> import pykx as kx # Initialise libraries at session creation >>> session = kx.remote.session(port=5050, libraries = {'kx': 'pykx', 'np': 'numpy'}) ``` -------------------------------- ### Start Real-Time Database (RDB) Process Source: https://github.com/kxsystems/pykx/blob/main/docs/user-guide/advanced/streaming/complex.md Start an RDB process on a specified port and configure its connections to the Tickerplant and HDB. This allows it to ingest real-time data and access historical data. ```python rdb = kx.tick.RTP(port=5011) rdb.start({ 'tickerplant': 'localhost:5010', 'hdb': 'localhost:5012', 'database': 'db'}) ``` -------------------------------- ### Start Tick Ingestion Framework Source: https://github.com/kxsystems/pykx/blob/main/docs/examples/streaming/Evolving System.ipynb Initializes and starts the pykx tick framework (Tickerplant, RDB, HDB) with defined tables and ports. This sets up the core real-time data processing infrastructure. ```python simple = kx.tick.BASIC( tables = {'trade': trade, 'quote': quote, 'aggregate': aggregate}, ports={'tickerplant': 5010, 'rdb': 5012, 'hdb': 5011}, log_directory = 'log', database = 'db' ) simple.start() ``` -------------------------------- ### Import PyKX and Set Environment Variables Source: https://github.com/kxsystems/pykx/blob/main/docs/examples/interface-overview.ipynb Import the pykx library and set environment variables to manage QHOME symlinking and the loaded marker. It's recommended to use the alias 'kx' for pykx. ```python import os os.environ['PYKX_IGNORE_QHOME'] = '1' # Ignore symlinking PyKX q libraries to QHOME os.environ['PYKX_Q_LOADED_MARKER'] = '' # Only used here for running Notebook under mkdocs-jupyter during document generation. ``` ```python import pykx as kx kx.q.system.console_size = [10, 80] ``` -------------------------------- ### Limit CPU Cores with start /affinity (Windows) Source: https://github.com/kxsystems/pykx/blob/main/docs/help/faq.md On Windows, use the `start` command with the `/affinity` argument to restrict the number of CPU cores available to a Python process. The argument is a hexadecimal mask. ```bat > start /affinity f python ``` -------------------------------- ### Create and Display Partitioned Table (Initial) Source: https://github.com/kxsystems/pykx/blob/main/tests/test_repr_html_.ipynb Initial creation and display of a partitioned table. This sets up the basic structure for partitioned data. ```python q('(`$':2001.01.01/partitionedTab/') set 0#([] a:.z.d-til 2000;b:til 2000)') q('system"l ."') display(q('partitionedTab')) ``` -------------------------------- ### Get string representation of q objects Source: https://github.com/kxsystems/pykx/blob/main/docs/pykx-under-q/intro.md Use .pykx.repr to get the string representation of a Python object, whether it's a pykx object or a foreign Python object. This is useful for debugging or logging. ```q q).pykx.repr x ``` -------------------------------- ### Set Python Variables from q Source: https://github.com/kxsystems/pykx/blob/main/docs/pykx-under-q/intro.md Shows how to set variables in the Python `__main__` namespace from q using `.pykx.set`, and how to retrieve them using `.pykx.qeval` or `.pykx.get`. ```q q).pykx.set[`var1;42] ``` ```q q).pykx.qeval"var1" 42 ``` ```q q).pykx.set[`var2;{x*2}] ``` ```q q)qfunc:.pykx.get[`var2;<] {[f;x].pykx.util.pykx[f;x]}[foreign]enlist ``` ```q q)qfunc[3] 6 ``` -------------------------------- ### .pykx.version Source: https://github.com/kxsystems/pykx/blob/main/docs/pykx-under-q/api.md Retrieves the installed PyKX version. ```APIDOC ## .pykx.version ### Description Retrieves and returns the currently installed version of the PyKX library. ### Method `.pykx.version` ### Endpoint N/A (Function call within q) ### Usage ```q q).pykx.version ``` ``` -------------------------------- ### Table.tail() Source: https://github.com/kxsystems/pykx/blob/main/docs/user-guide/advanced/Pandas_API.ipynb Get the last n rows from a table. ```APIDOC ## Table.tail() ### Description Get the last n rows from a table. ### Method `tail(n=5)` ### Parameters #### Path Parameters None #### Query Parameters - **n** (int) - Optional - The number of rows to return. Default is 5. ### Request Example ```python tab.tail() tab.tail(10) ``` ### Response #### Success Response (200) - **Table** (Table) - The last `n` rows of the table. #### Response Example None provided in source. ``` -------------------------------- ### Provide Activation Key Source: https://github.com/kxsystems/pykx/blob/main/docs/getting-started/installing.md Enter the base64 encoded activation key provided in your welcome email. ```bash Provide your activation key (base64 encoded string) provided with your welcome email: ```