### Server Setup and Run (Local) Source: https://github.com/datarobot-community/ai-accelerators/blob/main/generative_ai/compliance-agent-mvp-main/README.md Set up a Python virtual environment, install dependencies, and run the FastAPI server locally. Ensure the server is running before starting the frontend. ```bash cd server python -m venv .venv source .venv/bin/activate # or .venv\Scripts\activate on Windows pip install -r requirements.txt python main.py ``` -------------------------------- ### Example Project and Model Setup Source: https://github.com/datarobot-community/ai-accelerators/blob/main/advanced_ml_and_api_approaches/multi_model_analysis/Multi-Model Analysis.ipynb Sets up example DataRobot projects and models by reading data from a CSV, creating a project for each unique 'grade', and training a model. This is useful for testing the feature effects functions when you don't have existing projects. ```python # use this cell if you need example project(s) # this same example is used across all 3 functions, # so you only need to run this once! it may take awhile # skip this cell and go to the next one if you have already run this data = pd.read_csv( "https://s3.amazonaws.com/datarobot_public_datasets/10K_Lending_Club_Loans.csv", encoding="iso-8859-1", ) adv_opt = dr.AdvancedOptions(prepare_model_for_deployment=False) project_dict = {} for grade in data["grade"].unique(): p = dr.Project.create( data[data["grade"] == grade], "Multi-Model Accuracy Example, Grade {}".format(grade), ) p.analyze_and_model("is_bad", worker_count=-1, advanced_options=adv_opt) print("Project for Grade {} begun.".format(grade)) project_dict[grade] = p model_dict = {} for grade, p in project_dict.items(): p.wait_for_autopilot(verbosity=0) models = p.get_models() results = pd.DataFrame( [ { "model_type": m.model_type, "blueprint_id": m.blueprint_id, "cv_logloss": m.metrics["LogLoss"]["crossValidation"], "model_id": m.id, "model": m, } for m in models ] ) best_model = results["model"].iat[results["cv_logloss"].idxmin()] model_dict[grade] = best_model print("Project for Grade {} is finished.".format(grade)) ``` -------------------------------- ### Start Local Development Environment Source: https://github.com/datarobot-community/ai-accelerators/blob/main/generative_ai/adaptive-agent/README.md Run this command to initiate an interactive wizard that guides you through configuring your application, cloning the repository, and creating a `.env` file. ```sh dr start ``` -------------------------------- ### Navigate to Application Directory Source: https://github.com/datarobot-community/ai-accelerators/blob/main/generative_ai/adaptive-agent/README.md Change to the application directory created during the initial setup. Use this command before starting the agent. ```sh cd datarobot-agent-application ``` -------------------------------- ### Display Summary and Example Usage Source: https://github.com/datarobot-community/ai-accelerators/blob/main/use_cases_and_horizontal_approaches/datarobot-neo4j-knowledge-graph-for-fraud-detection/01_create_neo4j_db.ipynb Displays a summary of the Neo4j setup and provides an example of how to connect to the newly loaded database using Neo4j Browser. ```python msg = f""" **Summary**: - Neo4j 4.4.11 home: `{NEO4J_HOME}` - Database name: `{DATABASE_NAME}` - Dump file used: `{DUMP_FILE_PATH}` You have now loaded and started Neo4j. You can connect to this new DB (4.4.11) and confirm your data. **Example**: In Neo4j Browser: :use {DATABASE_NAME} MATCH (n) RETURN n LIMIT 10; **Done.** """ display(Markdown(msg)) ``` -------------------------------- ### Install DataRobot API Source: https://github.com/datarobot-community/ai-accelerators/blob/main/advanced_ml_and_api_approaches/GDL Featurizer/GDL Featurizer.ipynb Installs the DataRobot API client. Follow the API quickstart guide for more installation details. ```python # !pip install datarobot ``` -------------------------------- ### Frontend Setup and Build Source: https://github.com/datarobot-community/ai-accelerators/blob/main/generative_ai/compliance-agent-mvp-main/README.md Install Node.js dependencies and build the React frontend for local development or deployment. The build output is placed in a directory served by the FastAPI backend. ```bash cd frontend npm install npm run build ``` -------------------------------- ### Start MCP Server for Local Development Source: https://github.com/datarobot-community/ai-accelerators/blob/main/generative_ai/adaptive-agent/mcp_server/dev.md Runs the 'mcp:dev' task to set up the virtual environment, install dependencies, and start the MCP server locally. ```bash task mcp:dev ``` -------------------------------- ### Get Example Dataset Source: https://github.com/datarobot-community/ai-accelerators/blob/main/advanced_ml_and_api_approaches/LIME with DataRobot Models/LIME analysis with DataRobot.ipynb Fetches an example housing price dataset from a public S3 bucket for use in DataRobot projects. ```python #### Get example dataset from DataRobot public S3 #### raw_data = pd.read_csv( "https://s3.amazonaws.com/datarobot_public_datasets/ai_accelerators/house_train_dataset.csv" ) ``` -------------------------------- ### Check Docker Installation Source: https://github.com/datarobot-community/ai-accelerators/blob/main/advanced_ml_and_api_approaches/Using Feature Discovery SQL in Spark clusters/Using Feature Discovery SQL in other Spark clusters.ipynb Verifies if Docker is installed and running on the system. If not, follow the official Docker installation guide. ```bash # Check Docker installation !docker --version ``` -------------------------------- ### Install Frontend Dependencies Source: https://github.com/datarobot-community/ai-accelerators/blob/main/generative_ai/adaptive-agent/README.md Navigate to the frontend directory and run this command to install all necessary npm packages. ```sh cd frontend_web npm install ``` -------------------------------- ### Get Recommended Model Source: https://github.com/datarobot-community/ai-accelerators/blob/main/advanced_ml_and_api_approaches/prediction_intervals_via_conformal_inference/prediction_intervals_via_conformal_inference.ipynb Retrieves the recommended model for a given project. This is a common first step after project setup. ```python # Get recommended model best_model = dr.ModelRecommendation.get(project.id).get_model() best_model ``` -------------------------------- ### Install Requirements Source: https://github.com/datarobot-community/ai-accelerators/blob/main/generative_ai/LLM_custom_inference_model_template/storage/templates/anthropic-claude/README.md Install the necessary Python packages for the wrapper. This should be done in a virtual environment. ```bash python -m pip venv venv source venv/bin/activate pip install -r requirements.txt ``` -------------------------------- ### Install Streamlit Source: https://github.com/datarobot-community/ai-accelerators/blob/main/use_cases_and_horizontal_approaches/object_detection_on_video/README.md Install the Streamlit framework, used for building the interactive frontend application. ```bash pip install streamlit ``` -------------------------------- ### Install DataRobotX Source: https://github.com/datarobot-community/ai-accelerators/blob/main/advanced_ml_and_api_approaches/LIME with DataRobot Models/LIME analysis with DataRobot.ipynb Use this command to install the DataRobotX library. Ensure you have pip installed. ```bash !pip install datarobotx ``` -------------------------------- ### Install Prerequisite Libraries Source: https://github.com/datarobot-community/ai-accelerators/blob/main/generative_ai/external_monitoring/huggingface_observability_starter.ipynb Installs the necessary libraries for HuggingFace models and DataRobot integration. Ensure these are installed before proceeding. ```python !pip install transformers torch py-readability-metrics nltk ``` ```python !pip install datarobotx[llm] datarobot-mlops datarobot-mlops-connected-client "langchain==0.0.335" ``` -------------------------------- ### Install Go-Task on Linux Source: https://github.com/datarobot-community/ai-accelerators/blob/main/generative_ai/adaptive-agent/README.md Installs Go-Task, a task runner, on Linux systems. This command downloads and executes the installation script. ```shell sh -c "$(curl --location https://taskfile.dev/install.sh)" -- -d ``` -------------------------------- ### Install Project Dependencies Source: https://github.com/datarobot-community/ai-accelerators/blob/main/advanced_ml_and_api_approaches/faster-rcnn-custom-model/FasterR-CNN_training.ipynb Installs all required packages listed in the training requirements file. A kernel restart may be necessary after installation. ```python %pip install -q -r training/requirements.txt ``` -------------------------------- ### Install and Import Libraries Source: https://github.com/datarobot-community/ai-accelerators/blob/main/use_cases_and_horizontal_approaches/healthcare_appointment_no_show_prediction/no_show.ipynb Installs necessary libraries like shapely and datarobot, then imports them for use in the project. Ensure you have the datarobot library installed if you intend to use its functionalities. ```python # Install the shapely and datarobot libraries !pip install shapely --quiet #!pip install datarobot --quiet import datetime as datetime import json import os from IPython.display import HTML import datarobot as dr import matplotlib.pyplot as plt import numpy as np import pandas as pd import requests import shapely.geometry import shapely.wkt import yaml %matplotlib inline ``` -------------------------------- ### Check Node.js Installation and PATH Source: https://github.com/datarobot-community/ai-accelerators/blob/main/generative_ai/adaptive-agent/mcp_server/docs/mcp_client_setup.md Verify Node.js installation and ensure npm global binaries are in your system's PATH. If not, install Node.js or add the directory to your PATH. ```bash node --version ``` ```bash npx mcp-remote@latest --version ``` -------------------------------- ### Install Required Packages Source: https://github.com/datarobot-community/ai-accelerators/blob/main/generative_ai/rssfeed/rssfeed.ipynb Uncomment and run these lines in a DataRobot notebook to install necessary packages for the accelerator. ```python # These are packages used in this accelerator # The below format is used in the Datarobot notebooks to install packages. If running this in a DR notebook, uncomment the below entries # !pip install beautifulsoup4 # !pip install datarobot # !pip install datarobot-early-access # !pip install feedparser # !pip install requests ``` -------------------------------- ### Install D2 CLI on Linux Source: https://github.com/datarobot-community/ai-accelerators/blob/main/generative_ai/compliance-agent-mvp-main/Architecture/README.md Installs the D2 command-line interface using a script. This command downloads and executes the installation script. ```bash curl -fsSL https://d2lang.com/install.sh | sh -s -- --prefix ~/.local ``` -------------------------------- ### Install Project Dependencies Source: https://github.com/datarobot-community/ai-accelerators/blob/main/advanced_ml_and_api_approaches/graph_financial_fraud_classification/Graph_Financial_Fraud_Classification.ipynb Installs all required packages from the requirements.txt file, forcing a reinstallation to ensure compatibility. ```bash pip install --force-reinstall -r requirements.txt ``` -------------------------------- ### Install Dependencies Source: https://github.com/datarobot-community/ai-accelerators/blob/main/generative_ai/adaptive-agent/mcp_server/docs/mcp_client_setup.md Run this command to install any missing dependencies required for the MCP server or related tasks. ```bash task install ``` -------------------------------- ### Install Python Dependencies Source: https://github.com/datarobot-community/ai-accelerators/blob/main/ecosystem_integration_templates/teams_datarobot/README.md Install the required Python libraries for the bot. Ensure you have a requirements.txt file. ```bash pip install -r requirements.txt ``` -------------------------------- ### Example Output Log Source: https://github.com/datarobot-community/ai-accelerators/blob/main/use_cases_and_horizontal_approaches/data_annotator_app/data_annotator_app.ipynb This is an example log output showing which files were moved during the data labeling preparation process. ```text Moving Banana_025.jpg to scoring folder Moving Banana_030.jpg to scoring folder Moving Banana_020.jpg to scoring folder Moving Banana_035.jpg to scoring folder Moving Banana_045.jpg to scoring folder Moving Banana_040.jpg to scoring folder Moving Banana_010.jpg to scoring folder Moving Banana_005.jpg to scoring folder Moving Banana_015.jpg to scoring folder Moving Pomegranate_020.jpg to scoring folder Moving Pomegranate_025.jpg to scoring folder Moving Pomegranate_015.jpg to scoring folder Moving Pomegranate_010.jpg to scoring folder Moving Pomegranate_005.jpg to scoring folder Moving Mango_015.jpg to scoring folder Moving Mango_005.jpg to scoring folder Moving Mango_010.jpg to scoring folder Moving Mango_020.jpg to scoring folder Moving Mango_030.jpg to scoring folder Moving Mango_025.jpg to scoring folder Moving Kiwi_005.jpg to scoring folder Moving Kiwi_010.jpg to scoring folder Moving Kiwi_015.jpg to scoring folder Moving Kiwi_045.jpg to scoring folder Moving Kiwi_040.jpg to scoring folder Moving Kiwi_030.jpg to scoring folder Moving Kiwi_025.jpg to scoring folder Moving Kiwi_035.jpg to scoring folder Moving Kiwi_020.jpg to scoring folder ``` -------------------------------- ### Install AWS Bedrock SDK Source: https://github.com/datarobot-community/ai-accelerators/blob/main/generative_ai/external_monitoring/anthropic_claude_observability_starter.ipynb Installs the AWS Bedrock Python SDK by downloading and unpacking the recommended distribution files. Ensure to check AWS documentation for the latest installation methods. ```python !mkdir storage/bedrock_sdk/ import shutil import requests url = "https://d2eo22ngex1n9g.cloudfront.net/Documentation/SDK/bedrock-python-sdk.zip" r = requests.get(url, allow_redirects=True) open("storage/bedrock-python-sdk.zip", "wb").write(r.content) shutil.unpack_archive("storage/bedrock-python-sdk.zip", "storage/bedrock_sdk/") !pip install storage/bedrock_sdk/botocore-1.31.21-py3-none-any.whl storage/bedrock_sdk/boto3-1.28.21-py3-none-any.whl ``` -------------------------------- ### Verify ffmpeg Installation Source: https://github.com/datarobot-community/ai-accelerators/blob/main/use_cases_and_horizontal_approaches/Whisper Speech Recognition Deployment/Whisper Speech Recognition Deployment.ipynb Checks if the ffmpeg executable is available at the specified path. Raises an error if ffmpeg is not found, guiding the user to complete setup steps. ```python ffmpeg_file_path = "./storage/ffmpeg" try: import os assert os.path.isfile(ffmpeg_file_path) print("Found ffmpeg file") except Exception as e: raise RuntimeError( "Please follow the setup steps before running the notebook to upload ffmpeg." ) from e ``` -------------------------------- ### Initialize and Setup Paths Source: https://github.com/datarobot-community/ai-accelerators/blob/main/generative_ai/LLM Multimodal PDF RAG/LLM Multimodal PDF RAG.ipynb Initializes the DataRobot client, downloads demo data, and creates necessary directories for PDF, image, and markdown files. Also sets up parameters for vector database and playground. ```python # Datarobot client dr.Client() # Download demo data to current directory zip_path = "pdf_demo.zip" # Create a folder to save pdf uncompressed from zip file pdf_path = "pdf" os.makedirs(pdf_path, exist_ok=True) with zipfile.ZipFile(zip_path, "r") as zip_ref: zip_ref.extractall(pdf_path) # Create a folder to save images converted from PDF image_path = "image" os.makedirs(image_path, exist_ok=True) # Create a folder to save markdown files extracted from images markdown_path = "markdown" os.makedirs(markdown_path, exist_ok=True) # Generate a .zip file to create a vector database vectordb_zip_path = "vectordb.zip" # Image size 1000 is recommended image_size = 1000 # Chunk parameters chunking_method = VectorDatabaseChunkingMethod.RECURSIVE chunk_size = 384 chunk_overlap_percentage = 50 separators = ["\n\n"] # Playground parameters playground_name = "multimodal_rag" chat_name = "gpt4o" # Chat parameters max_completion_length = 256 temperature = 0.4 top_p = 0.9 max_documents_retrieved_per_prompt = 5 max_tokens = 384 # For previous chat prompts (history) to be included in each subsequent prompt, PromptType.ONE_TIME_PROMPT is an alternative if you don't wish prompting_strategy = PromptType.CHAT_HISTORY_AWARE # Max retry for creating vector database and playground max_retry = 5 ``` -------------------------------- ### Copy Environment Example Source: https://github.com/datarobot-community/ai-accelerators/blob/main/generative_ai/compliance-agent-mvp-main/README.md Copy the example environment file to create a new configuration file for the server. Edit this file to set LLM connection details. ```bash cp server/.env.example server/.env ``` -------------------------------- ### Placeholder for Example Execution Source: https://github.com/datarobot-community/ai-accelerators/blob/main/advanced_ml_and_api_approaches/multi_model_analysis/Multi-Model Analysis.ipynb This cell is intended to be run after the example project and model setup has been completed. It serves as a placeholder for further execution steps. ```python # run this cell if you already ran the template example above ``` -------------------------------- ### Install Libraries Source: https://github.com/datarobot-community/ai-accelerators/blob/main/generative_ai/LLM Multimodal PDF RAG/LLM Multimodal PDF RAG.ipynb Installs necessary Python libraries for PDF processing, LLM interaction, and DataRobot integration. Use this at the beginning of your environment setup. ```python !pip install google-genai pdf2image pymupdf openai anthropic -q ``` -------------------------------- ### Run the Demo Source: https://github.com/datarobot-community/ai-accelerators/blob/main/generative_ai/adaptive-agent/ADAPTIVE_DEMO_README.md Execute the development task and navigate to the provided localhost URL to interact with the adaptive agent demo. ```bash task dev ``` -------------------------------- ### Python requirements.txt example Source: https://github.com/datarobot-community/ai-accelerators/blob/main/generative_ai/playground/DataRobot GenAI Playground Accelerator.ipynb List Python or R packages to add to the base environment. This pre-installs packages not included in the base environment. ```python pandas scikit-learn ``` -------------------------------- ### Install and Configure DataRobot Python Client Source: https://github.com/datarobot-community/ai-accelerators/blob/main/use_cases_and_horizontal_approaches/anti-money-laundering/Anti-Money Laundering (AML) Alert Scoring.ipynb Install the DataRobot Python client and its dependencies. Authenticate with your DataRobot API endpoint and token. This setup is not required for Notebooks within DataRobot Workbench. ```python # NOT required for Notebooks in DataRobot Workbench # ************************************************* ! pip install datarobot --quiet # Upgrade DR to datarobot-3.2.0b0 # ! pip uninstall datarobot --yes # ! pip install datarobot --pre ! pip install pandas --quiet ! pip install matplotlib --quiet import getpass import datarobot as dr endpoint = "https://app.eu.datarobot.com/api/v2" token = getpass.getpass() dr.Client(endpoint=endpoint, token=token) # ************************************************* ``` -------------------------------- ### Install Project Dependencies Source: https://github.com/datarobot-community/ai-accelerators/blob/main/generative_ai/adaptive-agent/README.md Run this command to install all necessary project dependencies. This is often required after cloning a repository or when encountering service startup problems. ```shell dr task run install ``` -------------------------------- ### Environment Configuration Example Source: https://github.com/datarobot-community/ai-accelerators/blob/main/generative_ai/compliance-agent-mvp-main/Architecture/ARCHITECTURE.md Example of setting up required environment variables in a .env file for the AI Accelerators project. Ensure to replace placeholder values with your actual credentials and endpoints. ```env MODE=dr-gateway DATAROBOT_ENDPOINT=https://app.datarobot.com/api/v2 DATAROBOT_API_TOKEN=your-token-here CHAT_COMPLETIONS_MODEL=gpt-4o-mini GATEKEEPER_CONFIDENCE_THRESHOLD=70 SCRIPT_NAME=/custom_applications/your-app-id ``` -------------------------------- ### Print Python and Client Versions Source: https://github.com/datarobot-community/ai-accelerators/blob/main/use_cases_and_horizontal_approaches/Demand_forecasting1_end_to_end/End_to_end_demand_forecasting.ipynb Prints the current Python version and the installed DataRobot client version. Useful for verifying the environment setup. ```python from datetime import datetime as dt from platform import python_version import datarobot as dr import dr_utils as dru import pandas as pd print("Python version:", python_version()) print("Client version:", dr.__version__) ``` -------------------------------- ### Initiate Autopilot and Monitor Project Progress Source: https://github.com/datarobot-community/ai-accelerators/blob/main/advanced_ml_and_api_approaches/feature_reduction_with_fire/feature_reduction_with_fire.ipynb Configures the project's target variable and starts Autopilot in quick mode. It also opens the leaderboard in a browser for progress monitoring and waits for Autopilot to complete. ```python project_fire.set_target( target="SAR", mode="quick", worker_count=-1, advanced_options=dr.AdvancedOptions(seed=RANDOM_SEED), ) # Open the project's Leaderboard to monitor progress in the UI project_fire.open_leaderboard_browser() # Wait for Autopilot to finish # Set verbosity to 0 if you do not wish to see progress updates project_fire.wait_for_autopilot(verbosity=0) ``` -------------------------------- ### Python Requirements Example Source: https://github.com/datarobot-community/ai-accelerators/blob/main/generative_ai/playground/DataRobot GenAI Playground Accelerator.ipynb Specify Python packages required for your custom model in this file. This ensures necessary libraries are installed in the model's environment. ```python user_provided_model_id: user/my-awesome-model-id target_type: regression target_name: "target_column" model_environment_id: "664602284569e66a83a14196" --- # Prepare custom models for deployment Custom inference models allow you to bring your own pre-trained models to DataRobot. By uploading a model artifact to the Custom Model Workshop, you can create, test, and deploy custom inference models to a centralized deployment hub. DataRobot supports models built with a variety of coding languages, including Python, R, and Java. If you've created a model outside of DataRobot and you want to upload your model to DataRobot, you need to define two components: * **Model content**: The compiled artifact, source code, and additional supporting files related to the model. * **Model environment**: The Docker image where the model will run. Model environments can be either _drop-in_ or _custom_, containing a Docker file and any necessary supporting files. DataRobot provides a variety of built-in environments. Custom environments are only required to accommodate very specialized models and use cases. !!! note Custom inference models are _not_ custom DataRobot models. They are _user-defined_ models created outside of DataRobot and assembled in the Custom Model Workshop for deployment, monitoring, and governance. See the associated [feature considerations](#feature-considerations). ## Model content {: #model-content } To define a custom model, create a local folder containing the files listed in the table below (detailed descriptions follow the table). !!! tip To ensure your assembled custom model folder has the correct contents, you can find examples of these files in the [DataRobot model template repository](https://github.com/datarobot/datarobot-user-models/tree/master/model_templates){ target=_blank } on GitHub. File | Description | Required -----|-------------|--------- Model artifact file
_or_
`custom.py`/`custom.R` file | Provide a model artifact and/or a custom code file.