### Install LiteLLM for generic setup

Source: https://docs.pandas-ai.com/v3/large-language-models

Install the pandasai-litellm extension for unified LLM access.

```bash
# Using poetry
poetry add pandasai-litellm

# Using pip
pip install pandasai-litellm
```

--------------------------------

### Install pandasai-yfinance extension

Source: https://docs.pandas-ai.com/v3/semantic-layer/data-ingestion

Install the Yahoo Finance data connector.

```bash
poetry add pandasai-yfinance
```

```bash
pip install pandasai-yfinance
```

--------------------------------

### Install dependencies with Poetry

Source: https://docs.pandas-ai.com/v3/contributing

Use this command to install all project dependencies and development tools.

```bash
poetry install --all-extras --with dev
```

--------------------------------

### Install OpenAI extension

Source: https://docs.pandas-ai.com/v3/large-language-models

Install the pandasai-openai package using poetry or pip.

```bash
# Using poetry
poetry add pandasai-openai

# Using pip
pip install pandasai-openai
```

--------------------------------

### Install Oracle Extension

Source: https://docs.pandas-ai.com/v3/semantic-layer/data-ingestion

Install the Oracle connector for PandasAI using poetry or pip.

```bash
poetry add pandasai-oracle
# or
pip install pandasai-oracle
```

--------------------------------

### Install pandasai-sql extension

Source: https://docs.pandas-ai.com/v3/semantic-layer/data-ingestion

Install the SQL connector with support for specific database drivers.

```bash
poetry add pandasai-sql[postgres|mysql|cockroachdb|sqlserver]
```

```bash
pip install pandasai-sql[postgres|mysql|cockroachdb|sqlserver]
```

--------------------------------

### Install pandasai-bigquery extension

Source: https://docs.pandas-ai.com/v3/semantic-layer/data-ingestion

Install the BigQuery data connector. Requires an enterprise license.

```bash
poetry add pandasai-bigquery
```

```bash
pip install pandasai-bigquery
```

--------------------------------

### Install pre-commit hooks

Source: https://docs.pandas-ai.com/v3/contributing

Run this command to set up pre-commit hooks to ensure code standards.

```bash
pre-commit install
```

--------------------------------

### Install pandasai-oracle extension

Source: https://docs.pandas-ai.com/v3/semantic-layer/data-ingestion

Install the Oracle data connector. Requires an enterprise license.

```bash
poetry add pandasai-oracle
```

```bash
pip install pandasai-oracle
```

--------------------------------

### Initialize an Agent with data

Source: https://docs.pandas-ai.com/v3/skills

Setup an Agent with multiple DataFrames and a configured LLM. Skills defined globally will be accessible to the agent during execution.

```python
import pandas as pd
import pandasai as pai
from pandasai import Agent
from pandasai_litellm.litellm import LiteLLM

# Add your model
llm = LiteLLM(model="ollama/llama3", api_base="http://localhost:11434/api/generate")

pai.config.set({"llm": llm})

# Sample employee data
employees_data = {
    "EmployeeID": [1, 2, 3, 4, 5],
    "Name": ["John", "Emma", "Liam", "Olivia", "William"],
    "Department": ["HR", "Sales", "IT", "Marketing", "Finance"],
    "Salary": [50000, 60000, 70000, 55000, 65000],
    "Performance": [85, 92, 78, 88, 95]
}

salaries_data = {
    "EmployeeID": [1, 2, 3, 4, 5],
    "Bonus": [7500, 9000, 7000, 5500, 9750]
}

employees_df = pai.DataFrame(employees_data)
salaries_df = pai.DataFrame(salaries_data)

# Create an agent with the dataframes
agent = Agent([employees_df, salaries_df], memory_size=10)
```

--------------------------------

### Install BigQuery Extension

Source: https://docs.pandas-ai.com/v3/semantic-layer/data-ingestion

Install the BigQuery connector for PandasAI using pip.

```bash
pip install pandasai-bigquery
```

--------------------------------

### Install pandasai-docker

Source: https://docs.pandas-ai.com/v3/privacy-security

Install the pandasai-docker package to enable the sandbox environment. Ensure Docker is running on your system.

```bash
pip install pandasai-docker
```

--------------------------------

### Install Yahoo Finance Extension

Source: https://docs.pandas-ai.com/v3/semantic-layer/data-ingestion

Install the Yahoo Finance connector for PandasAI using poetry or pip.

```bash
poetry add pandasai-yfinance
# or
pip install pandasai-yfinance
```

--------------------------------

### Install Pandas-AI and LiteLLM

Source: https://docs.pandas-ai.com/v3/migration-guide

Update your installation to include pandasai and pandasai-litellm for v3. For SQL connectors, install the relevant package (e.g., pandasai-sql[postgres]).

```bash
# Using pip
pip install pandasai pandasai-litellm

# Using poetry
poetry add pandasai pandasai-litellm

# For SQL connectors
pip install pandasai-sql[postgres]  # or mysql, sqlite, etc.
```

--------------------------------

### Install SQL database extensions

Source: https://docs.pandas-ai.com/v3/semantic-layer/data-ingestion

Install the required SQL extension via pip before connecting to specific database types.

```bash
pip install pandasai-sql[postgres]
pip install pandasai-sql[mysql]
pip install pandasai-sql[cockroachdb]
pip install pandasai-sql[sqlserver]
```

--------------------------------

### Install PandasAI with Poetry or Pip

Source: https://docs.pandas-ai.com/v3/getting-started

Use this command to install the PandasAI library. Poetry is recommended for dependency management.

```bash
poetry add pandasai
```

```bash
pip install pandasai
```

--------------------------------

### Install pandasai-snowflake extension

Source: https://docs.pandas-ai.com/v3/semantic-layer/data-ingestion

Install the Snowflake data connector. Requires an enterprise license.

```bash
poetry add pandasai-snowflake
```

```bash
pip install pandasai-snowflake
```

--------------------------------

### Install pandasai-databricks extension

Source: https://docs.pandas-ai.com/v3/semantic-layer/data-ingestion

Install the Databricks data connector. Requires an enterprise license.

```bash
poetry add pandasai-databricks
```

```bash
pip install pandasai-databricks
```

--------------------------------

### Install BigQuery Extension

Source: https://docs.pandas-ai.com/v3/semantic-layer/data-ingestion

Install the BigQuery extension for PandasAI Enterprise Edition using poetry or pip.

```bash
poetry add pandasai-bigquery
```

--------------------------------

### Install LLM Extension

Source: https://docs.pandas-ai.com/v3/migration-troubleshooting

When encountering a `ModuleNotFoundError` for pandasai LLM modules, install the appropriate extension using pip. For example, to use LiteLLM, run: `pip install pandasai-litellm`.

```bash
pip install pandasai-litellm
```

--------------------------------

### Install Databricks Extension

Source: https://docs.pandas-ai.com/v3/semantic-layer/data-ingestion

Install the Databricks extension for PandasAI Enterprise Edition using poetry or pip.

```bash
poetry add pandasai-databricks
# or
pip install pandasai-databricks
```

--------------------------------

### Configure Pandas-AI for Deterministic Responses

Source: https://docs.pandas-ai.com/v3/large-language-models

Use `temperature=0` and `seed` to ensure consistent outputs. This example demonstrates setting up a DataFrame and configuring the LLM for deterministic behavior before querying.

```python
import pandasai as pai

# Sample DataFrame
df = pai.DataFrame({
    "country": ["United States", "United Kingdom", "France", "Germany", "Italy", "Spain", "Canada", "Australia", "Japan", "China"],
    "gdp": [19294482071552, 2891615567872, 2411255037952, 3435817336832, 1745433788416, 1181205135360, 1607402389504, 1490967855104, 4380756541440, 14631844184064],
    "happiness_index": [6.94, 7.16, 6.66, 7.07, 6.38, 6.4, 7.23, 7.22, 5.87, 5.12]
})

# Configure the LLM
pai.config.set({
   "temperature" : 0,
   "seed" : 26
})

df.chat('Which are the 5 happiest countries?') # answer should me (mostly) consistent across devices.
```

--------------------------------

### Configure LLM Globally in v3

Source: https://docs.pandas-ai.com/v3/migration-guide

v3 requires global LLM configuration using pai.config.set(). This example shows how to configure LiteLLM with model, API key, and other settings.

```python
from pandasai_litellm.litellm import LiteLLM
import pandasai as pai

llm = LiteLLM(model="gpt-4o-mini", api_key="your-api-key")
pai.config.set({
    "llm": llm,
    "verbose": False,
    "save_logs": True,
    "max_retries": 3
})
```

--------------------------------

### Install Snowflake Extension

Source: https://docs.pandas-ai.com/v3/semantic-layer/data-ingestion

Install the Snowflake extension for PandasAI Enterprise Edition using poetry or pip.

```bash
poetry add pandasai-snowflake
# or
pip install pandasai-snowflake
```

--------------------------------

### Basic Sandbox Usage with PandasAI Agent

Source: https://docs.pandas-ai.com/v3/agent

Initialize and start a DockerSandbox, then create a PandasAI agent with the sandbox enabled. Code executed via `agent.chat()` will run within the isolated sandbox environment. Remember to stop the sandbox when finished.

```python
from pandasai import Agent
from pandasai_docker import DockerSandbox

# Initialize the sandbox
sandbox = DockerSandbox()
sandbox.start()

# Create an agent with the sandbox
df = pai.read_csv("data.csv")
agent = Agent([df], sandbox=sandbox)

# Chat with the agent - code will run in the sandbox
response = agent.chat("Calculate the average sales")

# Don't forget to stop the sandbox when done
sandbox.stop()
```

--------------------------------

### Define Sample Sales Data

Source: https://docs.pandas-ai.com/v3/semantic-layer/transformations

Example CSV structure for a sales dataset.

```csv
date,store_id,product_name,category,quantity,unit_price,customer_email
2024-01-15, ST001,  iPhone 13 Pro,Electronics,2,999.99,john.doe@email.com
2024-01-15,ST002,macBook Pro ,Electronics,-1,1299.99,invalid.email
2024-01-16,ST001,AirPods Pro,Electronics,3,249.99,jane@example.com
2024-01-16,ST003,iMac 27" ,Electronics,1,1799.99,
```

--------------------------------

### Implement practical custom skills

Source: https://docs.pandas-ai.com/v3/skills

Examples of skills for data calculation, visualization, and formatting. Note that skills requiring external libraries like matplotlib must import them inside the function scope.

```python
import pandasai as pai

@pai.skill()
def calculate_bonus(salary: float, performance: float) -> float:
    """
    Calculates employee bonus based on salary and performance score.

    Args:
        salary (float): Employee's base salary
        performance (float): Performance score (0-100)

    Returns:
        float: Calculated bonus amount
    """
    if performance >= 90:
        return salary * 0.15  # 15% bonus for excellent performance
    elif performance >= 70:
        return salary * 0.10  # 10% bonus for good performance
    else:
        return salary * 0.05  # 5% bonus for average performance

@pai.skill()
def plot_salaries(names: list[str], salaries: list[float]):
    """
    Creates a bar chart showing employee salaries.

    Args:
        names (list[str]): List of employee names
        salaries (list[float]): List of corresponding salaries
    """
    import matplotlib.pyplot as plt

    plt.figure(figsize=(10, 6))
    plt.bar(names, salaries)
    plt.xlabel("Employee Name")
    plt.ylabel("Salary ($)")
    plt.title("Employee Salaries")
    plt.xticks(rotation=45)
    plt.tight_layout()
    plt.show()

@pai.skill()
def format_currency(amount: float) -> str:
    """
    Formats a number as currency.

    Args:
        amount (float): The amount to format

    Returns:
        str: Formatted currency string
    """
    return f"${amount:,.2f}"
```

--------------------------------

### Chat with Agent for Data Analysis

Source: https://docs.pandas-ai.com/v3/skills

Use the agent.chat() method to ask questions about your data. Skills are automatically available. This example shows calculating bonuses and displaying results.

```python
response1 = agent.chat("Calculate bonuses for all employees and show the results")
print("Response 1:", response1)
```

```python
response2 = agent.chat("Show me the total bonus amount formatted as currency")
print("Response 2:", response2)
```

```python
response3 = agent.chat("Calculate bonuses, format them as currency, and create a chart")
print("Response 3:", response3)
```

--------------------------------

### Configure LiteLLM with environment variables

Source: https://docs.pandas-ai.com/v3/large-language-models

Set API keys as environment variables and initialize LiteLLM for specific providers.

```python
import os
import pandasai as pai
from pandasai_litellm import LiteLLM

# Set your API keys as environment variables
os.environ["OPENAI_API_KEY"] = "your-openai-api-key"
os.environ["ANTHROPIC_API_KEY"] = "your-anthropic-api-key"

# Example with OpenAI
llm = LiteLLM(model="gpt-4.1-mini")

# Example with Anthropic
llm = LiteLLM(model="claude-2")

# Set your LLM configuration
pai.config.set({"llm": llm})
```

--------------------------------

### Using SmartDataframe in v3 (Legacy)

Source: https://docs.pandas-ai.com/v3/migration-backwards-compatibility

Demonstrates the v2-style usage of SmartDataframe, which requires global LLM configuration in v3.

```python
from pandasai import SmartDataframe
import pandasai as pai
import pandas as pd
from pandasai_litellm.litellm import LiteLLM

# Configure LLM globally (required)
llm = LiteLLM(model="gpt-4o-mini", api_key="your-api-key")
pai.config.set({"llm": llm})

# v2 style still works
df = pd.DataFrame({
    "country": ["US", "UK", "France"],
    "sales": [5000, 3200, 2900]
})

smart_df = SmartDataframe(df)
response = smart_df.chat("What are the top countries by sales?")
```

--------------------------------

### Instantiate Vector Store for Agent Training

Source: https://docs.pandas-ai.com/v3/agent

Instantiate a local vector store (ChromaDB, Qdrant, LanceDB, or Pinecone) for training a PandasAI agent with few-shot learning. An enterprise license is required for this feature.

```python
from pandasai import Agent
from pandasai.ee.vectorstores import ChromaDB
from pandasai.ee.vectorstores import Qdrant
from pandasai.ee.vectorstores import Pinecone
from pandasai.ee.vector_stores import LanceDB

# Instantiate the vector store
vector_store = ChromaDB()
# or with Qdrant
# vector_store = Qdrant()
# or with LanceDB
vector_store = LanceDB()
# or with Pinecone
# vector_store = Pinecone(
#     api_key="*****",
#     embedding_function=embedding_function,
```

--------------------------------

### Configure Azure OpenAI

Source: https://docs.pandas-ai.com/v3/large-language-models

Instantiate the AzureOpenAI class with endpoint, key, and deployment details.

```python
import pandasai as pai
from pandasai_openai import AzureOpenAI

llm = AzureOpenAI(api_base="https://<your-endpoint>.openai.azure.com/",
    api_key="my-azure-openai-api-key",
    deployment_name="text-davinci-003")  # The name of your deployed model

pai.config.set({"llm": llm})
```

--------------------------------

### Configure LLM Integration

Source: https://docs.pandas-ai.com/v3/migration-guide

Transition from built-in LLM classes to the extension-based LiteLLM integration.

```python
from pandasai.llm import OpenAI
from pandasai import SmartDataframe

llm = OpenAI(api_token="your-api-key")
df = SmartDataframe(data, config={"llm": llm})
```

```bash
pip install pandasai-litellm
```

```python
import pandasai as pai
from pandasai_litellm.litellm import LiteLLM

llm = LiteLLM(model="gpt-4o-mini", api_key="your-api-key")
pai.config.set({"llm": llm})
df = pai.DataFrame(data)
```

--------------------------------

### Define SQL Data Source with pai.create()

Source: https://docs.pandas-ai.com/v3/semantic-layer/new

Use the create method to define a dataset schema for a MySQL database. Requires the pandasai-sql extra dependency.

```python
sql_table = pai.create(
    # Format: "organization/dataset"
    path="company/health-data",

    # Optional description
    description="Heart disease dataset from MySQL database",

    # Define the source of the data, including connection details and
    # table name
    source={
        "type": "mysql",
        "connection": {
            "host": "${DB_HOST}",
            "port": 3306,
            "user": "${DB_USER}",
            "password": "${DB_PASSWORD}",
            "database": "${DB_NAME}"
        },
        "table": "heart_data"
    }
)
```

--------------------------------

### Configure OpenAI

Source: https://docs.pandas-ai.com/v3/large-language-models

Instantiate the OpenAI class with an API token and set it in the global configuration.

```python
import pandasai as pai
from pandasai_openai import OpenAI

llm = OpenAI(api_token="my-openai-api-key")

# Set your OpenAI API key
pai.config.set({"llm": llm})
```

--------------------------------

### Using Agent Follow-up Methods

Source: https://docs.pandas-ai.com/v3/migration-backwards-compatibility

Demonstrates maintaining conversation context using the follow_up method.

```python
agent = Agent([df1, df2])

# Start conversation
response = agent.chat('What is the total revenue?')

# Follow up without losing context
follow_up = agent.follow_up('What about last quarter?')
```

--------------------------------

### Configure Data Connectors

Source: https://docs.pandas-ai.com/v3/migration-guide

Migrate from specific connector classes to the unified pai.create() method using extensions.

```python
from pandasai.connectors import PostgreSQLConnector
from pandasai import SmartDataframe

connector = PostgreSQLConnector(config={
    "host": "localhost",
    "database": "mydb",
    "table": "sales"
})
df = SmartDataframe(connector)
```

```bash
pip install pandasai-sql[postgres]
```

```python
import pandasai as pai

df = pai.create(
    path="company/sales",
    description="Sales data from PostgreSQL",
    source={
        "type": "postgres",
        "connection": {
            "host": "localhost",
            "database": "mydb",
            "user": "${DB_USER}",
            "password": "${DB_PASSWORD}"
        },
        "table": "sales"
    }
)
```

--------------------------------

### Connect to a MySQL database

Source: https://docs.pandas-ai.com/v3/semantic-layer/data-ingestion

Configure a MySQL connection using the create method and define the schema for the semantic layer.

```python
# MySQL example
sql_table = pai.create(
    path="example/mysql-dataset",
    description="Heart disease dataset from MySQL database",
    source={
        "type": "mysql",
        "connection": {
            "host": "database.example.com",
            "port": 3306,
            "user": "${DB_USER}",
            "password": "${DB_PASSWORD}",
            "database": "medical_data"
        },
        "table": "heart_data",
        "columns": [
            {"name": "Age", "type": "integer", "description": "Age of the patient in years"},
            {"name": "Sex", "type": "string", "description": "Gender of the patient (M = male, F = female)"},
            {"name": "ChestPainType", "type": "string", "description": "Type of chest pain (ATA, NAP, ASY, TA)"},
            {"name": "RestingBP", "type": "integer", "description": "Resting blood pressure in mm Hg"},
            {"name": "Cholesterol", "type": "integer", "description": "Serum cholesterol in mg/dl"},
            {"name": "FastingBS", "type": "integer", "description": "Fasting blood sugar > 120 mg/dl (1 = true, 0 = false)"},
            {"name": "RestingECG", "type": "string", "description": "Resting electrocardiogram results (Normal, ST, LVH)"},
            {"name": "MaxHR", "type": "integer", "description": "Maximum heart rate achieved"},
            {"name": "ExerciseAngina", "type": "string", "description": "Exercise-induced angina (Y = yes, N = no)"},
            {"name": "Oldpeak", "type": "float", "description": "ST depression induced by exercise relative to rest"},
            {"name": "ST_Slope", "type": "string", "description": "Slope of the peak exercise ST segment (Up, Flat, Down)"},
            {"name": "HeartDisease", "type": "integer", "description": "Heart disease diagnosis (1 = present, 0 = absent)"}
        ]
    }
)
```

--------------------------------

### Configure LiteLLM

Source: https://docs.pandas-ai.com/v3/large-language-models

Initialize the LiteLLM class with a model name and API key, then apply it to the PandasAI configuration.

```python
import pandasai as pai
from pandasai_litellm.litellm import LiteLLM

# For OpenAI models
llm = LiteLLM(model="gpt-4.1-mini", api_key="YOUR_OPENAI_API_KEY")

# For other providers, change the model name and provide appropriate credentials
# llm = LiteLLM(model="anthropic/claude-3-opus-20240229", api_key="YOUR_ANTHROPIC_API_KEY")

pai.config.set({
    "llm": llm
})
```

--------------------------------

### Using Agent in v3

Source: https://docs.pandas-ai.com/v3/migration-backwards-compatibility

Standard Agent usage in v3, requiring global LLM configuration.

```python
from pandasai import Agent
import pandasai as pai
from pandasai_litellm.litellm import LiteLLM

# Configure LLM globally (required in v3)
llm = LiteLLM(model="gpt-4o-mini", api_key="your-api-key")
pai.config.set({"llm": llm})

# Agent works as before
df1 = pai.DataFrame(sales_data)
df2 = pai.DataFrame(costs_data)

agent = Agent([df1, df2])
response = agent.chat("Analyze the data and provide insights")
```

--------------------------------

### Configure PandasAI with LiteLLM and OpenAI

Source: https://docs.pandas-ai.com/v3/getting-started

Initialize LiteLLM with your chosen OpenAI model and API key, then configure PandasAI to use this LLM. Ensure you replace 'YOUR_OPENAI_API_KEY' with your actual key.

```python
import pandasai as pai
from pandasai_litellm.litellm import LiteLLM

# Initialize LiteLLM with your OpenAI model
llm = LiteLLM(model="gpt-4.1-mini", api_key="YOUR_OPENAI_API_KEY")

# Configure PandasAI to use this LLM
pai.config.set({
    "llm": llm
})
```

--------------------------------

### Initialize and Query Data with PandasAI

Source: https://docs.pandas-ai.com/v3/introduction

Configures the library with a LiteLLM provider and executes a natural language query on a CSV dataset.

```python
import pandasai as pai
from pandasai_litellm.litellm import LiteLLM

# Initialize LiteLLM with your OpenAI model
llm = LiteLLM(model="gpt-4.1-mini", api_key="YOUR_OPENAI_API_KEY")

# Configure PandasAI to use this LLM
pai.config.set({
    "llm": llm
})

# Load your data
df = pai.read_csv("data/companies.csv")

response = df.chat("What is the average revenue by region?")
print(response)
```

--------------------------------

### Migrate to pai.DataFrame (Recommended)

Source: https://docs.pandas-ai.com/v3/migration-guide

v3 recommends using pai.DataFrame for a semantic layer, offering better integration and features compared to the backward-compatible SmartDataframe.

```python
from pandasai import SmartDataframe

df = SmartDataframe(your_data)
response = df.chat("Your question")
```

```python
import pandasai as pai

# Simple approach
df = pai.DataFrame(your_data)
response = df.chat("Your question")

# With semantic layer (best for production)
df = pai.create(
    path="company/sales-data",
    df=your_data,
    description="Sales data by country and region",
    columns={
        "country": {"type": "string", "description": "Country name"},
        "sales": {"type": "float", "description": "Sales amount in USD"}
    }
)
response = df.chat("Your question")
```

--------------------------------

### Use New Agent Chat and Follow-up Methods

Source: https://docs.pandas-ai.com/v3/migration-troubleshooting

Methods like `clarification_questions`, `rephrase_query`, and `explain` have been removed in v3. Use `agent.chat()` for general queries and `agent.follow_up()` to maintain context in subsequent interactions.

```python
# v2 - These methods are removed
agent.clarification_questions('What is the GDP?')
agent.rephrase_query('What is the GDP?')
agent.explain()

# v3 - Use these instead
response = agent.chat('What is the GDP?')
follow_up = agent.follow_up('What about last year?')  # Maintains context
```

--------------------------------

### Agent Training with Vector Stores in v3

Source: https://docs.pandas-ai.com/v3/migration-guide

v3 requires a vector store for training, enabling few-shot learning. v2 used a simpler train() method without explicit vector store configuration.

```python
from pandasai import Agent

agent = Agent(df)
agent.train(queries=["query"], codes=["code"])
```

```python
from pandasai import Agent
from pandasai.ee.vectorstores import ChromaDB

# Instantiate with vector store
vector_store = ChromaDB()
agent = Agent(df, vectorstore=vector_store)

# Train with vector store
agent.train(queries=["query"], codes=["code"])
```

--------------------------------

### Configure PandasAI NL Layer

Source: https://docs.pandas-ai.com/v3/overview-nl

Initialize LiteLLM with an OpenAI model and configure PandasAI settings like the LLM, log saving, verbosity, and retry attempts.

```python
import pandasai as pai
from pandasai_litellm.litellm import LiteLLM

# Initialize LiteLLM with your OpenAI model
llm = LiteLLM(model="gpt-4.1-mini", api_key="YOUR_OPENAI_API_KEY")

pai.config.set({
   "llm": llm,
   "save_logs": True,
   "verbose": False,
   "max_retries": 3
})
```

--------------------------------

### Create SQL Server Data Source

Source: https://docs.pandas-ai.com/v3/semantic-layer/data-ingestion

Use this to create a data source from a SQL Server database. Ensure you have the necessary connection details and table schema defined.

```python
sql_server_table = pai.create(
    path="example/sqlserver-dataset",
    description="Sales data from SQL Server database",
    source={
        "type": "sqlserver",
        "connection": {
            "host": "sqlserver.example.com",
            "port": 1433,
            "user": "${SQLSERVER_USER}",
            "password": "${SQLSERVER_PASSWORD}",
            "database": "sales_data"
        },
        "table": "transactions",
        "columns": [
            {"name": "transaction_id", "type": "string", "description": "Unique identifier for each transaction"},
            {"name": "customer_id", "type": "string", "description": "Customer identifier"},
            {"name": "transaction_date", "type": "datetime", "description": "Date and time of transaction"},
            {"name": "product_category", "type": "string", "description": "Product category"},
            {"name": "quantity", "type": "integer", "description": "Number of items sold"},
            {"name": "unit_price", "type": "float", "description": "Price per unit"},
            {"name": "total_amount", "type": "float", "description": "Total transaction amount"}
        ]
    }
)
```

--------------------------------

### Run linting with Ruff

Source: https://docs.pandas-ai.com/v3/contributing

Execute this command to check for linting errors and warnings.

```bash
make format_diff
```

--------------------------------

### Load and Query Dataset

Source: https://docs.pandas-ai.com/v3/semantic-layer/new

Load a previously created dataset and perform a natural language query.

```python
# Load the dataset
heart_data = pai.load("organization/health-data")

# Query the data
response = heart_data.chat("What is the average age of patients with heart disease?")
```

--------------------------------

### Using SmartDatalake in v3 (Legacy)

Source: https://docs.pandas-ai.com/v3/migration-backwards-compatibility

Legacy implementation of SmartDatalake for querying multiple dataframes.

```python
from pandasai import SmartDatalake
import pandasai as pai
import pandas as pd
from pandasai_litellm.litellm import LiteLLM

# Configure LLM globally (required)
llm = LiteLLM(model="gpt-4o-mini", api_key="your-api-key")
pai.config.set({"llm": llm})

# v2 style still works
employees_df = pd.DataFrame({
    "name": ["John", "Jane", "Bob"],
    "department": ["Sales", "Engineering", "Sales"]
})

salaries_df = pd.DataFrame({
    "name": ["John", "Jane", "Bob"],
    "salary": [60000, 80000, 55000]
})

lake = SmartDatalake([
    employees_df,
    salaries_df
])

response = lake.chat("Who gets paid the most?")
```

--------------------------------

### Handle Follow-up Questions with Agent

Source: https://docs.pandas-ai.com/v3/agent

Use the `follow_up` method to ask context-aware questions without clearing the agent's conversation memory. This is useful for maintaining the flow of a multi-turn conversation.

```python
# Start a new conversation
response = agent.chat('What is the total sales?')
print("First response:", response)

# Continue the conversation without clearing memory
follow_up_response = agent.follow_up('What about last year?')
print("Follow-up response:", follow_up_response)
```

--------------------------------

### Using `pai.create()` for SQL Databases

Source: https://docs.pandas-ai.com/v3/semantic-layer/new

This snippet demonstrates how to use the `pai.create()` method to define a SQL database as a data source in PandasAI. It includes connection details and table specification.

```APIDOC
## POST /api/create_sql_datasource

### Description
Creates a PandasAI data source object from a SQL database.

### Method
POST

### Endpoint
`/api/create_sql_datasource`

### Parameters
#### Request Body
- **path** (string) - Required - The path where the dataset will be stored (e.g., "organization/dataset").
- **description** (string) - Optional - A description of the dataset.
- **source** (object) - Required - Defines the data source details.
  - **type** (string) - Required - The type of SQL database (e.g., "mysql", "postgresql").
  - **connection** (object) - Required - Database connection credentials.
    - **host** (string) - Required - Database host address.
    - **port** (integer) - Required - Database port number.
    - **user** (string) - Required - Database username.
    - **password** (string) - Required - Database password.
    - **database** (string) - Required - Database name.
  - **table** (string) - Required - The name of the table to query.

### Request Example
```json
{
  "path": "company/health-data",
  "description": "Heart disease dataset from MySQL database",
  "source": {
    "type": "mysql",
    "connection": {
      "host": "${DB_HOST}",
      "port": 3306,
      "user": "${DB_USER}",
      "password": "${DB_PASSWORD}",
      "database": "${DB_NAME}"
    },
    "table": "heart_data"
  }
}
```

### Response
#### Success Response (200)
- **data_source_object** (object) - The created PandasAI data source object.

#### Response Example
```json
{
  "data_source_object": "<pandasai.data_source.SQLDataSource object at 0x...>"
}
```
```

--------------------------------

### Update Configuration in v3

Source: https://docs.pandas-ai.com/v3/migration-guide

In v3, several configuration options have been removed. Only keep essential settings like `llm`, `save_logs`, `verbose`, and `max_retries`.

```python
# Remove: save_charts, enable_cache, security,
# custom_whitelisted_dependencies, save_charts_path

# v3 (keep only these)
pai.config.set({
    "llm": llm,
    "save_logs": True,
    "verbose": False,
    "max_retries": 3
})
```

--------------------------------

### Register Skills

Source: https://docs.pandas-ai.com/v3/migration-guide

Update skill registration to use the global @pai.skill decorator instead of manual agent registration.

```python
from pandasai.skills import skill
from pandasai import Agent

@skill
def calculate_bonus(salary: float, performance: float) -> float:
    """Calculate employee bonus."""
    if performance >= 90:
        return salary * 0.15
    return salary * 0.10

agent = Agent([df])
agent.add_skills(calculate_bonus)
```

```python
import pandasai as pai
from pandasai import Agent

@pai.skill
def calculate_bonus(salary: float, performance: float) -> float:
    """Calculate employee bonus."""
    if performance >= 90:
        return salary * 0.15
    return salary * 0.10

# Skills automatically available - no need to add them
agent = Agent([df])
```

--------------------------------

### Configure a Data View with YAML

Source: https://docs.pandas-ai.com/v3/semantic-layer/views

Define a view structure including columns and relationships using a YAML configuration file.

```yaml
name: table_heart
columns:
  - name: parents.id
  - name: parents.name
  - name: parents.age
  - name: children.name
  - name: children.age
relations:
  - name: parent_to_children
    description: Relation linking the parent to its children
    from: parents.id
    to: children.id
```

--------------------------------

### Configure SQL Source in YAML

Source: https://docs.pandas-ai.com/v3/semantic-layer/new

Specify connection details and table information for a SQL data source in the schema.yaml file.

```yaml
source:
  type: postgres
  connection:
    host: postgres-host
    port: 5432
    database: postgres
    user: postgres
    password: ******
  table: orders
  view: false
```

--------------------------------

### Format code with Ruff

Source: https://docs.pandas-ai.com/v3/contributing

Use this command to automatically reformat the codebase.

```bash
make format
```

--------------------------------

### Create Schema with CSV and Parquet Files

Source: https://docs.pandas-ai.com/v3/semantic-layer/new

Use the `pai.create` method to define a semantic layer schema. This method takes a path, an input DataFrame, an optional description, and a list of column definitions.

```python
import pandasai as pai

# Load your data: for example, in this case, a CSV
file = pai.read_csv("data.csv")

df = pai.create(
    # Format: "organization/dataset"
    path="company/sales-data",

    # Input dataframe
    df = file,

    # Optional description
    description="Sales data from our retail stores",

    # Define the structure and metadata of your dataset's columns.
    # If not provided, all columns from the input dataframe will be included.
    columns=[
        {
            "name": "transaction_id",
            "type": "string",
            "description": "Unique identifier for each sale"
        },
        {
            "name": "sale_date"
            "type": "datetime",
            "description": "Date and time of the sale"
        }
    ]
)
```

--------------------------------

### Recommended v3 Approach for Dataframes

Source: https://docs.pandas-ai.com/v3/migration-backwards-compatibility

The preferred method for handling dataframes in v3 using pai.DataFrame for better integration.

```python
import pandasai as pai
import pandas as pd

# Configure LLM globally
pai.config.set({"llm": llm})

# Simple approach
df = pd.DataFrame({
    "country": ["US", "UK", "France"],
    "sales": [5000, 3200, 2900]
})
df = pai.DataFrame(df)
response = df.chat("What are the top countries by sales?")
```

--------------------------------

### Migrate Data Connectors to v3

Source: https://docs.pandas-ai.com/v3/migration-guide

v3 uses a unified 'source' configuration for data connectors within the pai.create() function, replacing the v2 connector instantiation.

```python
# v2
from pandasai.connectors import PostgreSQLConnector
connector = PostgreSQLConnector(config={...})
df = SmartDataframe(connector)

# v3
import pandasai as pai
df = pai.create(
    path="company/database-table",
    description="Description of your data",
    source={
        "type": "postgres",
        "connection": {
            "host": "localhost",
            "database": "mydb",
            "user": "${DB_USER}",
            "password": "${DB_PASSWORD}"
        },
        "table": "your_table"
    }
)
```

--------------------------------

### Instantiate and Train Pandas-AI Agent

Source: https://docs.pandas-ai.com/v3/agent

Instantiate the agent with a custom vector store and train it with specific queries and corresponding Python code. The training code is provided as a string.

```python
from pandasai import Agent
from pandasai.vector_store import VectorStore

# Assuming vector_store is already instantiated and configured
# For example:
# vector_store = VectorStore(
#     embeddings=OpenAIEmbeddings(),
#     # dimension of your embedding model
#     dimensions=384,
# )

# Instantiate the agent with the custom vector store
agent = Agent("data.csv", vectorstore=vector_store)

# Train the model
query = "What is the total sales for the current fiscal year?"
# The following code is passed as a string to the response variable
response = '\n'.join([
    'import pandas as pd',
    '',
    'df = dfs[0]',
    '',
    '# Calculate the total sales for the current fiscal year',
    'total_sales = df[df[\'date\'] >= pd.to_datetime(\'today\').replace(month=4, day=1)][\'sales\'].sum()',
    'result = { "type": "number", "value": total_sales }'
])

agent.train(queries=[query], codes=[response])

response = agent.chat("What is the total sales for the last fiscal year?")
print(response)
# The model will use the information provided in the training to generate a response
```

--------------------------------

### Run tests with Pytest

Source: https://docs.pandas-ai.com/v3/contributing

Execute this command to run the full test suite.

```bash
make test_all
```

--------------------------------

### Configure Oracle Data Source

Source: https://docs.pandas-ai.com/v3/semantic-layer/data-ingestion

Define the Oracle source, destination, schema, and transformation pipeline in YAML format.

```yaml
name: sales_data

source:
  type: oracle
  connection:
    host: your-host
    port: 1521
    service_name: your-service
    user: ${ORACLE_USER}
    password: ${ORACLE_PASSWORD}
  table: sales_data

destination:
  type: local
  format: parquet
  path: company/oracle-sales

columns:
  - name: transaction_id
    type: string
    description: Unique identifier for each sale
  - name: sale_date
    type: datetime
    description: Date and time of the sale
  - name: product_id
    type: string
    description: Product identifier
  - name: quantity
    type: integer
    description: Number of units sold
  - name: price
    type: float
    description: Price per unit

transformations:
  - type: convert_timezone
    params:
      column: sale_date
      from: UTC
      to: Australia/Sydney
  - type: calculate
    params:
      column: total_amount
      formula: quantity * price
  - type: round
    params:
      column: total_amount
      decimals: 2
  - type: calculate
    params:
      column: discount
      formula: "CASE WHEN quantity > 10 THEN 0.1 WHEN quantity > 5 THEN 0.05 ELSE 0 END"

update_frequency: daily

order_by:
  - sale_date DESC

limit: 100000
```

--------------------------------

### Configure PandasAI Dataframes

Source: https://docs.pandas-ai.com/v3/migration-guide

Compare the per-dataframe configuration of v2 with the global configuration approach in v3.

```python
from pandasai import SmartDataframe

config = {
    "llm": llm,
    "save_charts": True,
    "enable_cache": True,
    "security": "standard"
}
df = SmartDataframe(data, config=config)
```

```python
import pandasai as pai

pai.config.set({
    "llm": llm,
    "save_logs": True,
    "verbose": False,
    "max_retries": 3
})
df = pai.DataFrame(data)
```

--------------------------------

### Enable and Use Docker Sandbox in PandasAI

Source: https://docs.pandas-ai.com/v3/privacy-security

Configure PandasAI to use the Docker sandbox for secure code execution. This involves initializing the LLM, setting up the sandbox, and passing it during the chat interaction. Remember to stop the sandbox when done.

```python
import pandasai as pai
from pandasai_docker import DockerSandbox
from pandasai_litellm.litellm import LiteLLM

# Initialize LiteLLM with your OpenAI model
llm = LiteLLM(model="gpt-4.1-mini", api_key="YOUR_OPENAI_API_KEY")

# Configure PandasAI to use this LLM
pai.config.set({
    "llm": llm
})

# initialize the sandbox
sandbox = DockerSandbox()
sandbox.start()

# read a csv as df
df = pai.read_csv("./data/heart.csv")

# pass the df and the sandbox
result = pai.chat("plot total heart patients by gender", df, sandbox=sandbox)

# display the chart
result.show()

# stop the sandbox (docker container)
sandbox.stop()
```

--------------------------------

### Recommended v3 Approach for Multiple Dataframes

Source: https://docs.pandas-ai.com/v3/migration-backwards-compatibility

Directly querying multiple dataframes using pai.chat() instead of SmartDatalake.

```python
import pandasai as pai

# Configure LLM globally
pai.config.set({"llm": llm})

# Create dataframes
employees = pai.DataFrame(employees_df)
salaries = pai.DataFrame(salaries_df)

# Query across multiple dataframes directly
response = pai.chat("Who gets paid the most?", employees, salaries)
```

--------------------------------

### Instantiate a PandasAI Agent

Source: https://docs.pandas-ai.com/v3/agent

Instantiate an agent with a DataFrame to enable multi-turn conversations. The agent maintains conversation state for context-aware responses.

```python
import os
from pandasai import Agent
import pandas as pd

# Sample DataFrames
sales_by_country = pd.DataFrame({
    "country": ["United States", "United Kingdom", "France", "Germany", "Italy", "Spain", "Canada", "Australia", "Japan", "China"],
    "sales": [5000, 3200, 2900, 4100, 2300, 2100, 2500, 2600, 4500, 7000],
    "deals_opened": [142, 80, 70, 90, 60, 50, 40, 30, 110, 120],
    "deals_closed": [120, 70, 60, 80, 50, 40, 30, 20, 100, 110]
})

agent = Agent(sales_by_country)
agent.chat('Which are the top 5 countries by sales?')
# Output: China, United States, Japan, Germany, Australia
```

--------------------------------

### Apply Global Configuration

Source: https://docs.pandas-ai.com/v3/migration-troubleshooting

In v3, configuration is applied globally using `pai.config.set(config)` before creating the DataFrame, unlike v2 where configuration was passed directly to the SmartDataframe constructor.

```python
# v2
df = SmartDataframe(data, config=config)

# v3
pai.config.set(config)
df = pai.DataFrame(data)
```

--------------------------------

### Basic Chat Test with Pandas-AI v3

Source: https://docs.pandas-ai.com/v3/migration-guide

Test your migration by performing a basic chat operation on a Pandas DataFrame. Ensure you import `pandasai` and `pandas`.

```python
import pandasai as pai
import pandas as pd

df = pd.DataFrame({"x": [1, 2, 3], "y": [4, 5, 6]})
df = pai.DataFrame(df)
response = df.chat("What is the sum of x?")
print(response)
```

--------------------------------

### Configure BigQuery Data Source

Source: https://docs.pandas-ai.com/v3/semantic-layer/data-ingestion

Define the BigQuery source, destination, schema, and transformation pipeline in YAML format.

```yaml
name: inventory_data

source:
  type: bigquery
  connection:
    project_id: your-project-id
    credentials: ${GOOGLE_APPLICATION_CREDENTIALS}
  table: inventory

destination:
  type: local
  format: parquet
  path: company/bigquery-inventory

columns:
  - name: product_id
    type: string
    description: Unique identifier for each product
  - name: product_name
    type: string
    description: Name of the product
  - name: category
    type: string
    description: Product category
  - name: stock_level
    type: integer
    description: Current quantity in stock
  - name: last_updated
    type: datetime
    description: Last inventory update timestamp

transformations:
  - type: categorize
    params:
      column: stock_level
      bins: [0, 20, 100, 500]
      labels: ["Low", "Medium", "High"]
  - type: extract
    params:
      column: product_name
      pattern: "(.*?)\\s*-\\s*(.*)"
      into: [brand, model]
  - type: convert_timezone
    params:
      column: last_updated
      from: UTC
      to: Asia/Tokyo

update_frequency: hourly

order_by:
  - last_updated DESC

limit: 50000
```

--------------------------------

### Add Description to Dataset Schema

Source: https://docs.pandas-ai.com/v3/semantic-layer/new

The `description` parameter provides a text explanation of the dataset's contents and purpose, aiding in understanding. It is optional but recommended.

```python
file = pai.read_csv("data.csv")

pai.create(
    path="company/sales-data",
    df = file,
    description="Daily sales transactions from all retail stores, including transaction IDs, dates, and amounts",
    ...
)
```

--------------------------------

### Customize Docker Sandbox

Source: https://docs.pandas-ai.com/v3/agent

Customize the Docker sandbox by providing a custom name and the path to a Dockerfile for advanced configuration. This allows for tailored sandbox environments.

```python
sandbox = DockerSandbox(
    "custom-sandbox-name",
    "/path/to/custom/Dockerfile"
)
```

--------------------------------

### Define a basic custom skill

Source: https://docs.pandas-ai.com/v3/skills

Use the @pai.skill() decorator to register a function globally. Ensure the function includes type hints and a descriptive docstring for AI interpretation.

```python
import pandasai as pai

@pai.skill()
def my_custom_function(param1: str, param2: int) -> str:
    """
    A custom function that demonstrates skill creation.

    Args:
        param1 (str): First parameter description
        param2 (int): Second parameter description

    Returns:
        str: Result description
    """
    return f"Processed {param1} with value {param2}"
```