Shakudo

https://docs.shakudo.io/
Admin
Shakudo is a data and AI operating system that runs on your VPC, providing compatibility across...
Tokens:59,215
Snippets:219
Trust Score:9.4
Update:1 year ago
Show doc for...
Tokens:
Raw
### Bash Script to Start React App

Source: https://docs.shakudo.io/tutorials/buildareactapp

An example bash script designed to set up a Node.js environment, install dependencies, and start a React application. It includes steps for Node.js installation and npm package management.

```bash
PROJECT_DIR="$(cd -P "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
cd "$PROJECT_DIR"

yes Y | curl -sL https://deb.nodesource.com/setup_14.x | bash - 
apt update
apt install nodejs
npm install
npm start
```

--------------------------------

### Create a React App on Shakudo

Source: https://docs.shakudo.io/category/tutorials

Learn how to create a React application on the Shakudo Platform as a service. This tutorial guides you through the initial steps to get started.

--------------------------------

### Quickstart Ray Cluster

Source: https://docs.shakudo.io/Shakudo-stack/distributedComputing/ray

Quickly spin up a Ray cluster using predefined t-shirt sizes. This function simplifies the initial setup of a distributed Ray environment.

```Python
from hyperplane import ray_common as rc
ray_cluster = rc.quickstart_ray(
    num_workers = 4,
    size = 'hyperplane-med-high-mem'
)
```

--------------------------------

### Spin up Dask Cluster with Shakudo

Source: https://docs.shakudo.io/legacy/quickstart

Initializes a Dask cluster using the `notebook_common` package from Shakudo. Allows configuration of worker nodes, memory, cores, and processes per worker. Provides a Dask dashboard link for monitoring.

```Python
from hyperplane import notebook_common as nc
num_workers = 2   ## number of worker nodes
total_memory = 12   ## total memory size for the worker nodes in GB
cors_per_worker = 15  ## total number of cores for the worker nodes
nprocs = 3  ## number of processes for each worker node
ram_gb_per_proc = total_memory/nprocs  ## calculated memory size per processes in GB
nthreads = int(cors_per_worker/nprocs) ## calculated number of threads per processes

client, cluster = nc.initialize_cluster(
        num_workers = num_workers,
        nprocs = nprocs,
        nthreads = nthreads,
        ram_gb_per_proc = ram_gb_per_proc,
        cores_per_worker = cors_per_worker,
        node_selector = {}
    )
```

--------------------------------

### Application Setup and Run Script (Bash)

Source: https://docs.shakudo.io/tutorials/confluenceapp

A bash script to set up and run a Streamlit application on Shakudo. It installs dependencies, sets environment variables, and starts the Streamlit server.

```bash
PROJECT_DIR="$(cd -P "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
cd "$PROJECT_DIR"

pip install -r requirements.txt

export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python
export STREAMLIT_RUNONSSAVE=True

streamlit run app.py --server.port 8787 --browser.serverAddress localhost

```

--------------------------------

### Clone Repository and Create Branch

Source: https://docs.shakudo.io/legacy/public-logo-cdn-setup

This snippet demonstrates cloning the CDN repository, navigating into the directory, and creating a new feature branch for logo uploads.

```bash
git clone https://github.com/devsentient/cdn.git  
cd cdn  
git checkout -b add-my-logo  
```

--------------------------------

### Define Data Preparation Pipeline

Source: https://docs.shakudo.io/legacy/quickstart

Defines a YAML file for a data preparation pipeline. It specifies the pipeline name and tasks, including running a Jupyter notebook for Dask data processing.

```YAML
pipeline:
  name: "data prep pipeline"
  tasks:
  - name: "dask processing data"
    type: "jupyter notebook"
    notebook_path: "example_notebooks/doc_demo/quick_start/dask.ipynb"  # path to the script to run
    notebook_output_path: "dask_output.ipynb"
```

--------------------------------

### Shakudo Keycloak Redirect URI Example

Source: https://docs.shakudo.io/Getting%20started/Sign%20in%20with%20external%20provider/example-2-microsoft

This snippet shows an example of the redirect URI format required when configuring Shakudo Keycloak for Microsoft SSO. This URI is obtained from the Shakudo Keycloak admin console and used in the Azure AD application setup.

```text
https://<keycloak-domain>/auth/realms/<realm-name>/broker/Microsoft/endpoint
```

--------------------------------

### Get Raw CDN URL for Logo

Source: https://docs.shakudo.io/legacy/public-logo-cdn-setup

This example shows the final raw URL format for accessing an uploaded logo directly from the GitHub CDN, suitable for embedding.

```url
https://raw.githubusercontent.com/devsentient/cdn/main/stack-component-logos/my-logo.svg
```

--------------------------------

### Add Logo to Repository

Source: https://docs.shakudo.io/legacy/public-logo-cdn-setup

This command copies a local logo file into the designated 'stack-component-logos' directory within the cloned repository.

```bash
cp ~/Downloads/my-logo.svg stack-component-logos/  
```

--------------------------------

### Bash Script for Microservice Startup

Source: https://docs.shakudo.io/shakudo-platform-core/service

This bash script is used to start a microservice. It sets the '-e' option to exit immediately if a command exits with a non-zero status, finds the project directory, installs a package, and runs the application. The script restarts if the application exits with an error.

```bash
#!/bin/bash
set -e
PROJECT_DIR="$(cd -P "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

cd "$PROJECT_DIR"

pip install foo-package==bar-version
python ./app.py

```

--------------------------------

### Customize Helm Chart Installation with valuesOverride.yaml

Source: https://docs.shakudo.io/shakudo-platform-core/adminSettings/cloudshell

Demonstrates how to customize a Helm chart deployment by providing a values file. This example shows overriding resource limits for CPU and memory for the 'n8n' chart.

```YAML
# valuesOverride.yaml - you may want to name this different as there may be multiple values file for different stack components  
resources:  
  limits:  
    cpu: "5000m"  
    memory: "12Gi"  

```

--------------------------------

### Commit and Push Logo Changes

Source: https://docs.shakudo.io/legacy/public-logo-cdn-setup

This sequence stages the new logo file, commits it with a descriptive message, and pushes the changes to the remote repository's feature branch.

```bash
git add stack-component-logos/my-logo.svg  
git commit -m "Add my logo to stack-component-logos"  
git push -u origin add-my-logo  
```

--------------------------------

### Parameter Example JSON

Source: https://docs.shakudo.io/shakudo-platform-core/adminSettings/graphql

Provides an example JSON object representing a Parameter, demonstrating the expected format for key, value, id, and pipelineJobId.

```JSON
{
    "key": "key",
    "value": "value",
    "id": "b50e8ea9-1627-4a5d-b7c7-ebad6c801d0a",
    "pipelineJobId": "833bd3d1-bb63-4289-99d8-25c2856f2fba"
}
```

--------------------------------

### Basic Flask App Example

Source: https://docs.shakudo.io/tutorials/flaskapp

A simple Flask application that can be deployed on the Shakudo platform. This example demonstrates a basic API structure.

```Python
from flask import Flask

app = Flask(__name__)

@app.route('/')
def hello_world():
    return 'Hello, World!'

if __name__ == '__main__':
    app.run(debug=True, host='0.0.0.0', port=8000)
```

--------------------------------

### Configure package.json for React App Hosting

Source: https://docs.shakudo.io/tutorials/buildareactapp

An example of a package.json file configured for hosting a React app on Shakudo. It includes settings for the homepage, host, port, and script commands.

```json
{
  "name": "my-react-app",
  "version": "0.1.0",
  "private": true,
  "homepage": "https://[CLUSTER_NAME].hyperplane.dev/[my-app]/",
  ...
  "scripts": {
    "start": "HOST=0.0.0.0 PORT=8787 react-scripts start",
    "build": "react-scripts build",
    "test": "react-scripts test",
    "eject": "react-scripts eject"
  },
  ...
}
```

--------------------------------

### Verify Helm Release and Kubernetes Resources

Source: https://docs.shakudo.io/shakudo-platform-core/adminSettings/cloudshell

Commands to verify the successful installation of a Helm release and its associated Kubernetes resources. The first command lists Helm releases in a specific namespace, while the second lists all resources within that namespace.

```Bash
helm list --namespace hyperplane-n8n-test  
```

```Bash
kubectl get all --namespace hyperplane-n8n-test  
```

--------------------------------

### Install nbdev Package

Source: https://docs.shakudo.io/shakudo-platform-core/sessions/versionControl

Installs or updates the nbdev package using pip. It first checks the currently installed version and then proceeds with the installation if it's not met.

```bash
pip show nbdev
pip install nbdev
pip show nbdev
```

--------------------------------

### Inject Custom Prompt Example

Source: https://docs.shakudo.io/tutorials/confluenceapp

Shows how to inject a custom prompt into the QA chain for potentially better results.

```Python
qa.combine_documents_chain.llm_chain.prompt = CUSTOMPROMPT
```

--------------------------------

### Dry Run Helm Chart Installation

Source: https://docs.shakudo.io/shakudo-platform-core/adminSettings/cloudshell

Simulates the installation of a Helm chart to verify the command and its parameters without making actual changes to the cluster. This is a crucial step for ensuring a successful deployment.

```Bash
helm upgrade --install n8n oci://us-docker.pkg.dev/shakudo-417723/charts/n8n --namespace hyperplane-n8n-test --version 1.2.4 --create-namespace --dry-run=server --debug  
```

--------------------------------

### Process Flight Data with Pandas

Source: https://docs.shakudo.io/legacy/quickstart

This Python code snippet demonstrates how to process flight data from S3 using Pandas. It reads multiple CSV files, calculates the mean delay for each origin-destination pair, and identifies pairs with a high proportion of delayed flights. The code utilizes `tqdm` for progress tracking and `pandas` for data manipulation.

```Python
from tqdm.notebook import tqdm
import pandas as pd

# Assuming 'files' is a list of S3 paths to the CSV files
# Example: files = ["bucket-name/file1.csv", "bucket-name/file2.csv", ...]

results = []
for file in tqdm(files):
    df = pd.read_csv(
        f"s3://{file}",
        encoding = "ISO-8859-1",
        usecols = ['DepTime','FlightNum','DepDelay','Origin', 'Dest','Distance']
    )
    df['isDelayed'] = df.DepDelay.fillna(16) > 15
    df_delayed = df.groupby(['Origin','Dest']).isDelayed.mean()
    results.append(df_delayed)

df_results = pd.concat(results).reset_index()
df_results = df_results.groupby(['Origin','Dest']).mean()
df_results = df_results[df_results.isDelayed==1].reset_index()
print(f'found {len(df_results)} most delayed flights Origin and destination pairs')
```

--------------------------------

### Configure TCP Liveness and Readiness Probes

Source: https://docs.shakudo.io/shakudo-platform-core/service

Example YAML configuration for enabling basic TCP socket liveness and readiness probes for a Microservice. These probes check the service's availability on a specified port.

```YAML
livenessProbe:  
  tcpSocket:  
    port: 8787  
  initialDelaySeconds: 30  
  periodSeconds: 10  
  timeoutSeconds: 5  
  failureThreshold: 3  
readinessProbe:  
  tcpSocket:  
    port: 8787  
  initialDelaySeconds: 10  
  periodSeconds: 5  
  timeoutSeconds: 3  
  failureThreshold: 3  
```

--------------------------------

### HyperHubSession Example Object

Source: https://docs.shakudo.io/shakudo-platform-core/adminSettings/graphql

An example JSON object representing a HyperHubSession, demonstrating typical values for various metadata fields. This includes user details, session identifiers, and status information.

```json
{
"billingProject": null,
"collaborative": false,
"completionTime": "2023-05-17T21:26:02.310Z",
"currentPodEvents": null,
"department": null,
"duration": 901,
"estimatedCost": null,
"gpuRequest": null,
"group": "",
"hyperplaneUser": {
"id": "3661bd41-6ca9-4b20-a74b-c46ce6ff6951",
"email": "demo@shakudo.io"
},
"hyperplaneUserEmail": "demo@shakudo.io",
"hyperplanepodspecName": null,
"id": "65e3a289-1371-4009-9fb3-c03bfbcbebd8",
"imageHash": "gcr.io/imageHash",
"imageType": "test-custom-image",
"jLabUrl": "",
"notebookURI": "",
"owner": "demo",
"podEventsLog": "Stopping container hyperhub-user",
"podSpec": null,
"podSpecTemplate": null,
"premptableNode": false,
"resourceCPUlimit": null,
"resourceCPUrequest": null,
"resourceRAMlimit": null,
"resourceRAMrequest": null,
"runId": null
}
```

--------------------------------

### Neo4j Streamlit Microservice Configuration (Python)

Source: https://docs.shakudo.io/shakudo-platform-core/service

This example demonstrates a Python Streamlit application that connects to a Neo4j database. It includes frontend interactivity, Neo4j driver usage, and result visualization. Configuration is managed via environment variables.

```Python
import streamlit as st
from neo4j import GraphDatabase

# Neo4j connection details from environment variables
NEO4J_URI = os.environ.get('NEO4J_URI')
NEO4J_USER = os.environ.get('NEO4J_USER')
NEO4J_PASSWORD = os.environ.get('NEO4J_PASSWORD')

def get_neo4j_driver(uri, user, password):
    try:
        driver = GraphDatabase.driver(uri, auth=(user, password))
        return driver
    except Exception as e:
        st.error(f"Failed to connect to Neo4j: {e}")
        return None

def run_cypher_query(driver, query):
    if driver is None:
        return None
    try:
        with driver.session() as session:
            result = session.run(query)
            return [record for record in result]
    except Exception as e:
        st.error(f"Error running query: {e}")
        return None

st.title("Neo4j Streamlit Explorer")

# Initialize Neo4j driver
driver = get_neo4j_driver(NEO4J_URI, NEO4J_USER, NEO4J_PASSWORD)

query = st.text_area("Enter your Cypher query:", "MATCH (n) RETURN n LIMIT 10")

if st.button("Run Query"):
    if query:
        results = run_cypher_query(driver, query)
        if results:
            st.write("Query Results:")
            for row in results:
                st.write(row)
    else:
        st.warning("Please enter a Cypher query.")

if driver:
    driver.close()
```

--------------------------------

### Install nbdev Git Hooks

Source: https://docs.shakudo.io/shakudo-platform-core/sessions/versionControl

Installs nbdev's Git hooks within the current repository. These hooks are used to clean up commits and ensure compatibility with version control systems.

```bash
nbdev_install_hooks
```

--------------------------------

### Sample Pipeline Job Variables (JSON)

Source: https://docs.shakudo.io/shakudo-platform-core/adminSettings/graphql

Provides an example of the JSON structure for defining variables when creating a pipeline job. This includes settings like job name, YAML path, user email, and branch name.

```json
{
  "type": "basic",
  "maxRetries": 2,
  "jobName": "test",
  "yamlPath": "examples/pipeline.yaml",
  "hyperplaneUserEmail": "demo@shakudo.io",
  "branchName": "demo"
}
```

--------------------------------

### Quickstart Dask Cluster

Source: https://docs.shakudo.io/legacy/shakudoApi

Initializes a Dask cluster quickly using predefined t-shirt sizes for worker configurations. It returns a Dask Client and KubeCluster object, allowing for parallel computing with familiar APIs like NumPy and Pandas.

```Python
from hyperplane.notebook_common import quickstart_dask
client, cluster = quickstart_dask(
    num_workers = 4,
    size = 'hyperplane-med-high-mem'
)
```

--------------------------------

### Create Jupyter Session with jhub-basic

Source: https://docs.shakudo.io/shakudo-platform-core/sessions/shared_sessions

This guide outlines the steps to create a new Jupyter session using the 'jhub-basic' environment configuration. It involves navigating to the Sessions panel, selecting the environment, and launching the JupyterLab server.

```Bash
Navigate to the Sessions panel and click on the create session button.
Select the Basic environment config and click on the create session button.
Wait for the session to start and initialize the JupyterLab server.
Once the JupyterLab icon appears, click on it to open the session.
```

--------------------------------

### Initialize and Run ConfluenceQA

Source: https://docs.shakudo.io/tutorials/confluenceapp

Demonstrates the initialization and execution of the ConfluenceQA class to query Confluence content. It covers setting up configuration, embeddings, models, vector database, and the retrieval QA chain.

```Python
config = {"persist_directory":"./chroma_db/",
          "confluence_url":"https://templates.atlassian.net/wiki/",
          "username":None,
          "api_key":None,
          "space_key":"RD"}

confluenceQA = ConfluenceQA(config=config)
confluenceQA.init_embeddings()
confluenceQA.init_models()

confluenceQA.vector_db_confluence_docs()

confluenceQA.retreival_qa_chain()

question = "How to organize content in a space?"
confluenceQA.answer_confluence(question)
```

--------------------------------

### Process Data with Dask

Source: https://docs.shakudo.io/legacy/quickstart

Processes data using Dask DataFrames by reading CSV files from S3. It performs data cleaning, grouping, and computation, demonstrating lazy evaluation with the `.compute()` function. The example calculates the mean of a 'isDelayed' column grouped by origin and destination.

```Python
import dask.dataframe as dd
df = dd.read_csv([f"s3://{file}" for file in files],
                 encoding = "ISO-8859-1",
                 usecols = ['DepTime','FlightNum','DepDelay','Origin', 'Dest','Distance'],
                 dtype={'Distance': 'float64'})
df['isDelayed'] = df.DepDelay.fillna(16) > 15
df_delayed = df.groupby(['Origin','Dest']).isDelayed.mean()
df_results = df_delayed[df_delayed==1].compute().reset_index()
print(f'found {len(df_results)} most delayed flights Origin and destination pairs')
```

--------------------------------

### Build Confluence Q&A App with Langchain and ChatGPT on Shakudo

Source: https://docs.shakudo.io/category/tutorials

This tutorial demonstrates how to build a Q&A app for an internal knowledge base using Shakudo, Langchain, and ChatGPT, with Confluence as the knowledge source. It covers the process from conceptualization to deployment, leveraging AI and NLP advancements.

--------------------------------

### Shutdown Dask Client

Source: https://docs.shakudo.io/legacy/quickstart

Shuts down the Dask client after computation is complete. This is a good practice to release resources.

```Python
client.shutdown()
```

--------------------------------

### General Usage and Help with Shakudo CLI

Source: https://docs.shakudo.io/shakudo-platform-core/sessions/sctl

Provides general usage information for the Shakudo CLI and how to access help for specific commands or topics.

```bash
sctl [COMMAND]
```

```bash
sctl microservices --help
```

```bash
sctl immediate-jobs --help
```

--------------------------------

### Streamlit App Initialization

Source: https://docs.shakudo.io/tutorials/confluenceapp

Sets up the basic Streamlit application, including page configuration and initializing the ConfluenceQA instance within the session state.

```Python
import streamlit as st
from confluence_qa import ConfluenceQA

st.set_page_config(
    page_title='Q&A Bot for Confluence Page',
    page_icon='⚡',
    layout='wide',
    initial_sidebar_state='auto',
)

st.session_state["config"] = {}
confluence_qa = None
```

--------------------------------

### Close Dask Cluster and Client

Source: https://docs.shakudo.io/legacy/shakudoApi

Demonstrates how to shut down a Dask cluster and client to free up resources. Includes an example of starting a cluster and then closing it, and retrieving a forgotten cluster before closing.

```Python
from hyperplane import notebook_common as nc
client, cluster = nc.initialize_dask_cluster(num_workers=2)

cluster.close()
client.close()
```

```Python
from hyperplane import notebook_common as nc
client = nc.get_dask_cluster("dask-cluster-with-some-random-hash")

cluster.close()
client.close()
```

--------------------------------

### Create React App using npx

Source: https://docs.shakudo.io/tutorials/buildareactapp

This command initializes a new React project named 'my-react-app' using the create-react-app package via npx.

```bash
npx create-react-app my-react-app
```

--------------------------------

### Get Existing Ray Cluster

Source: https://docs.shakudo.io/Shakudo-stack/distributedComputing/ray

Reconnect to an existing Ray cluster. This function is useful if you need to connect to a previously started cluster, potentially adding more workers to it.

```Python
from hyperplane import ray_common as rc
rc.get_ray_cluster(extra_workers = 1)
```

--------------------------------

### Create New Git Repository

Source: https://docs.shakudo.io/shakudo-platform-core/sessions/versionControl

Initializes a new Git repository, adds a README file, makes the first commit, sets the main branch name, and pushes the initial commit to a remote origin.

```bash
mkdir <new directory>
cd <new directory>
echo "# New repository" >> README.md
git init
git add README.md
git commit -m "first commit"
git branch -M main
git remote add origin <remote repository url starting with git@github.com>
git push -u origin main
```

--------------------------------

### Build and Deploy Streamlit App on Shakudo

Source: https://docs.shakudo.io/category/tutorials

Create a data app using Shakudo and Streamlit, a popular Python library for building and sharing data applications quickly. This tutorial covers the process of building and deploying your Streamlit app.

--------------------------------

### Create a Flask App on Shakudo

Source: https://docs.shakudo.io/category/tutorials

Discover how to create a Flask application on the Shakudo Platform as a service. Follow the provided steps to begin building your Flask app.

--------------------------------

### Configure Azure AD for preferred_username Mapping

Source: https://docs.shakudo.io/Getting%20started/Sign%20in%20with%20external%20provider/idp-mapper-setup-doc

This example demonstrates how to configure Azure AD to pass the 'preferred_username' attribute, which will then be mapped to 'email' and 'username' fields in Shakudo Keycloak using IDP mappers.

```Keycloak
1. Log in to the Shakudo Keycloak Admin Console.
2. Navigate to Identity Providers and select Azure AD.
3. Go to the Mappers tab and click Add.
4. Set Name to "Preferred Username to Email".
5. Choose Mapper Type as Attribute Importer.
6. Set Attribute Name to preferred_username.
7. Map this attribute to the email field by setting User Attribute Name to email.
8. Create another mapper with the same settings but set User Attribute Name to username.
9. Click Save for both mappers.
```

```Azure AD
1. Log in to the Azure portal.
2. Navigate to Azure Active Directory and select App registrations.
```

--------------------------------

### Get Shakudo Sessions

Source: https://docs.shakudo.io/shakudo-platform-core/adminSettings/graphql

Retrieves a list of user sessions from the Shakudo Platform. This query allows filtering by user email, session status, and image type, and supports pagination with a limit parameter. It returns detailed information about each session, including IDs, resource requests, and start times.

```GraphQL
query hyperhubSessions($limit: Int!, $email: String, $status: String, $imageType: String) {
    hyperHubSessions(orderBy:{startTime: desc}, take: $limit, where: {
      hyperplaneUserEmail: {equals: $email},
            imageType: {equals: $imageType},
      status: {equals: $status},
    }) {
      id
      hyperplaneUserEmail
      status
      imageType
      jLabUrl
      notebookURI
      estimatedCost
      resourceCPUlimit
      resourceRAMlimit
      resourceCPUrequest
      resourceRAMrequest
      gpuRequest
      startTime
    }
    countHyperHubSessions
  }
```

--------------------------------

### Running a Query

Source: https://docs.shakudo.io/tutorials/confluenceapp

Executes a query against the initialized QA chain and prints the answer.

```Python
question = "How to organize content in a space?"

answer = qa.run(question)
print(answer)
```

--------------------------------

### Create Microservices with Shakudo CLI

Source: https://docs.shakudo.io/shakudo-platform-core/sessions/sctl

Creates new microservices with various configuration options, including Git server name, pipeline type, job name, subdomain, notifications, autoscaling, and more.

```bash
sctl microservices create --gitServerName shakudo-dev-repo --ai
```

```bash
sctl microservices create --scriptPath fastapi-openapi-example/run.sh \
   --pipelineType BASH --jobName my-custom-ms --subdomain custom-subdomain
```

```bash
sctl microservices create --gitServerName shakudo-dev-repo \
   --notificationsEnabled \
   --notificationTargetIds 8ebc86f3-c76e-4ce7-99f9-666151923b0c,14960534-15dd-4ffd-8d46-2e00b2f55f36 \
   --healthCheckThreshold 5 --serviceAlertCooldownPeriod 7000 \
   --scriptPath fastapi-openapi-example/run.sh --pipelineType BASH
```

```bash
sctl microservices create --gitServerName shakudo-dev-repo \
   --minReplicas 2 --maxHpaRange 5
```

```bash
sctl microservices create \
   --gitServerName shakudo-dev-repo \
   --jobName test-service-example-cli \
   --jobType basic-system \
   --scriptPath fastapi-openapi-example/run.sh \
   --pipelineType BASH \
   --subdomain test-subdomain \
   --notificationsEnabled \
   --notificationTargetIds 8ebc86f3-c76e-4ce7-99f9-666151923b0c,14960534-15dd-4ffd-8d46-2e00b2f55f36 \
   --healthCheckThreshold 2 --serviceAlertCooldownPeriod 2000 \
   --minReplicas 1 --maxHpaRange 3 \
   --port 8888 \
   --billingProjectName prod-bp \
   --serviceAccountName admin-sa \
   --parameter ENV1=VAL1 --parameter ENV2=VAL2 \
   --customSecretNames testing,pass-openai \
   --readme "> this is a testing readme\n\n### this is a header\n"
```

--------------------------------

### List Microservices with Shakudo CLI

Source: https://docs.shakudo.io/shakudo-platform-core/sessions/sctl

Lists all microservices managed by Shakudo. Supports limiting the number of items and setting an offset for pagination.

```bash
sctl microservices ls
```

```bash
sctl microservices ls --limit 50 --offset 25
```

--------------------------------

### Install Helm Chart with Custom Values

Source: https://docs.shakudo.io/shakudo-platform-core/adminSettings/cloudshell

Applies a Helm chart installation or upgrade using a custom values file to override default configurations. This command integrates the specified values file into the deployment process.

```Bash
helm upgrade --install n8n oci://us-docker.pkg.dev/shakudo-417723/charts/n8n --namespace hyperplane-n8n-test -f valuesOverride.yaml  
```

--------------------------------

### Install n8n Stack Component using Helm

Source: https://docs.shakudo.io/shakudo-platform-core/adminSettings/cloudshell

Installs or upgrades the 'n8n' Helm chart from the Shakudo registry into a specified Kubernetes namespace. This command ensures the correct version is deployed and creates the namespace if it doesn't exist.

```Bash
helm upgrade --install n8n oci://us-docker.pkg.dev/shakudo-417723/charts/n8n --namespace hyperplane-n8n-test --version 1.2.4 --create-namespace  
```

--------------------------------

### Sample Variables for Immediate Job Creation (JSON)

Source: https://docs.shakudo.io/shakudo-platform-core/adminSettings/graphql

Example JSON payload for creating an immediate job. This includes essential parameters like job type, timeouts, retries, YAML path, user email, and branch name.

```JSON
{
  "type": "basic",
  "timeout": 86400,
  "activeTimeout": 86400,
  "maxRetries": 2,
  "yamlPath": "examples/pipeline.yaml",
  "hyperplaneUserEmail": "demo@shakudo.io",
  "branchName": "main"
}
```

--------------------------------

### List Immediate Jobs with Shakudo CLI

Source: https://docs.shakudo.io/shakudo-platform-core/sessions/sctl

Lists all immediate jobs managed by Shakudo. Supports limiting the number of items and setting an offset for pagination.

```bash
sctl immediate-jobs ls
```

```bash
sctl immediate-jobs ls --limit 10 --offset 5
```

--------------------------------

### RetrievalQA Initialization

Source: https://docs.shakudo.io/tutorials/confluenceapp

Initializes the RetrievalQA chain with a specified LLM, chain type, and retriever.

```Python
retriever = vectordb.as_retriever(search_kwargs={"k":4}) #Top4-Snippets
qa = RetrievalQA.from_chain_type(llm=llm, chain_type="stuff",retriever=retriever)
```

--------------------------------

### Start Spark Session with RayDP

Source: https://docs.shakudo.io/Shakudo-stack/distributedComputing/overview

Starts a Spark session using the RayDP integration. This allows for Spark computations to be run on the Ray cluster, configured with a specified number of executors, cores, and memory.

```Python
import raydp

spark = raydp.init_spark(
    'example',
    num_executors=2,
    executor_cores=4,
    executor_memory='4G'
)
```

--------------------------------

### Check Installed Packages with pip and conda

Source: https://docs.shakudo.io/shakudo-platform-core/adminSettings/podspecs

This snippet shows how to check installed Python packages within a Jupyter notebook environment. It uses `pip freeze` for pip packages and `conda list` for conda packages.

```bash
pip freeze

```

```bash
conda list

```

--------------------------------

### Add Git Repository - Example SSH URL

Source: https://docs.shakudo.io/shakudo-platform-core/adminSettings/gitrepos

This snippet shows an example of a Git repository SSH URL format required by Shakudo. Shakudo only supports SSH URLs, not HTTPS. The format typically includes the username, server address, and repository name.

```text
git@github.com:devsentient/examples.git
```

--------------------------------

### Quickstart Dask Cluster

Source: https://docs.shakudo.io/Shakudo-stack/distributedComputing/dask

Quickly spin up a Dask cluster using predefined t-shirt sizes. This function returns a tuple containing the Dask client and the Kubernetes cluster object. It requires specifying the number of workers and the desired size for the worker pools.

```Python
from hyperplane.notebook_common import quickstart_dask
client, cluster = quickstart_dask(
    num_workers = 4,
    size = 'hyperplane-med-high-mem'
)
```

--------------------------------

### ConfluenceQA Class Structure

Source: https://docs.shakudo.io/tutorials/confluenceapp

Provides the basic structure of the ConfluenceQA class, highlighting its initialization method and attributes for managing embeddings, vector database, LLM, and the QA chain.

```Python
class ConfluenceQA:
    def __init__(self,config:dict = {}):
        self.config = config
        self.embedding = None
        self.vectordb = None
        self.llm = None
        self.qa = None
        self.retriever = None
```

--------------------------------

### Get Session Pod Spec (JSON)

Source: https://docs.shakudo.io/shakudo-platform-core/adminSettings/graphql

Retrieves the full PodSpec as a JSON string for Shakudo Sessions. This allows for granular customization within the `createHyperHubSession` function.

```JSON
{
  "data": {
    "getHyperhubSessionPodSpec": {
      "imageUrl": "",
      "userPvcName": "",
      "userEmail": "",
      "imageType": ""
    }
  }
}
```
Tokens:
Raw