PandasAI

https://docs.pandas-ai.com/v3
Admin
PandasAI is a Python library that enables users to ask questions to their data in natural language,...
Tokens:21,838
Snippets:186
Trust Score:8.1
Update:1 month ago
Show doc for...
Tokens:
Raw
### Install PandasAI using Poetry or Pip

Source: https://docs.pandas-ai.com/v3/getting-started

Install the core PandasAI package. Poetry is recommended for dependency management, but standard pip installation is also supported.

```bash
# Using poetry (recommended)
poetry add pandasai

# Alternative: using pip
pip install pandasai
```

--------------------------------

### Migrate LLM Setup from v2 to v3

Source: https://docs.pandas-ai.com/v3/migration-guide

LLMs in v3 are extension-based. Install pandasai-litellm separately and configure the LLM globally.

```python
from pandasai.llm import OpenAI
from pandasai import SmartDataframe

llm = OpenAI(api_token="your-api-key")
df = SmartDataframe(data, config={"llm": llm})
```

```bash
pip install pandasai-litellm
```

```python
import pandasai as pai
from pandasai_litellm.litellm import LiteLLM

llm = LiteLLM(model="gpt-4o-mini", api_key="your-api-key")
pai.config.set({"llm": llm})
df = pai.DataFrame(data)
```

--------------------------------

### Install and configure the BigQuery extension

Source: https://docs.pandas-ai.com/v3/semantic-layer/data-ingestion

Install the BigQuery extension via poetry or pip, and configure the connection, schema, and transformations using a YAML file.

```bash
poetry add pandasai-bigquery
# or
pip install pandasai-bigquery
```

```yaml
name: inventory_data

source:
  type: bigquery
  connection:
    project_id: your-project-id
    credentials: ${GOOGLE_APPLICATION_CREDENTIALS}
  table: inventory

destination:
  type: local
  format: parquet
  path: company/bigquery-inventory

columns:
  - name: product_id
    type: string
    description: Unique identifier for each product
  - name: product_name
    type: string
    description: Name of the product
  - name: category
    type: string
    description: Product category
  - name: stock_level
    type: integer
    description: Current quantity in stock
  - name: last_updated
    type: datetime
    description: Last inventory update timestamp

transformations:
  - type: categorize
    params:
      column: stock_level
      bins: [0, 20, 100, 500]
      labels: ["Low", "Medium", "High"]
  - type: extract
    params:
      column: product_name
      pattern: "(.*?)\\s*-\\s*(.*)"
      into: [brand, model]
  - type: convert_timezone
    params:
      column: last_updated
      from: UTC
      to: Asia/Tokyo

update_frequency: hourly

order_by:
  - last_updated DESC

limit: 50000
```

--------------------------------

### Install Oracle Extension

Source: https://docs.pandas-ai.com/v3/semantic-layer/data-ingestion

Install the Oracle extension for PandasAI using either Poetry or pip.

```bash
poetry add pandasai-oracle
# or
pip install pandasai-oracle
```

--------------------------------

### Implement Practical Example Skills in Python

Source: https://docs.pandas-ai.com/v3/skills

Examples of custom skills for calculating bonuses, plotting salaries with matplotlib, and formatting currency. These functions are registered globally once decorated.

```python
import pandasai as pai

@pai.skill()
def calculate_bonus(salary: float, performance: float) -> float:
    """
    Calculates employee bonus based on salary and performance score.

    Args:
        salary (float): Employee's base salary
        performance (float): Performance score (0-100)

    Returns:
        float: Calculated bonus amount
    """
    if performance >= 90:
        return salary * 0.15  # 15% bonus for excellent performance
    elif performance >= 70:
        return salary * 0.10  # 10% bonus for good performance
    else:
        return salary * 0.05  # 5% bonus for average performance

@pai.skill()
def plot_salaries(names: list[str], salaries: list[float]):
    """
    Creates a bar chart showing employee salaries.

    Args:
        names (list[str]): List of employee names
        salaries (list[float]): List of corresponding salaries
    """
    import matplotlib.pyplot as plt

    plt.figure(figsize=(10, 6))
    plt.bar(names, salaries)
    plt.xlabel("Employee Name")
    plt.ylabel("Salary ($)")
    plt.title("Employee Salaries")
    plt.xticks(rotation=45)
    plt.tight_layout()
    plt.show()

@pai.skill()
def format_currency(amount: float) -> str:
    """
    Formats a number as currency.

    Args:
        amount (float): The amount to format

    Returns:
        str: Formatted currency string
    """
    return f"${amount:,.2f}"
```

--------------------------------

### Install and configure the Databricks extension

Source: https://docs.pandas-ai.com/v3/semantic-layer/data-ingestion

Install the Databricks extension via poetry or pip, and configure the connection, schema, and transformations using a YAML file.

```bash
poetry add pandasai-databricks
# or
pip install pandasai-databricks
```

```yaml
name: customer_data

source:
  type: databricks
  connection:
    host: your-workspace-url
    token: ${DATABRICKS_TOKEN}
  table: customers

destination:
  type: local
  format: parquet
  path: company/databricks-customers

columns:
  - name: customer_id
    type: string
    description: Unique identifier for each customer
  - name: name
    type: string
    description: Customer's full name
  - name: email
    type: string
    description: Customer's email address
  - name: join_date
    type: datetime
    description: Date when customer joined
  - name: total_purchases
    type: integer
    description: Total number of purchases made

transformations:
  - type: anonymize
    params:
      columns: [email, name]
  - type: convert_timezone
    params:
      column: join_date
      from: UTC
      to: Europe/London
  - type: calculate
    params:
      column: customer_tier
      formula: "CASE WHEN total_purchases > 100 THEN 'Gold' WHEN total_purchases > 50 THEN 'Silver' ELSE 'Bronze' END"

update_frequency: daily

order_by:
  - join_date DESC

limit: 100000
```

--------------------------------

### Install pandasai-litellm extension

Source: https://docs.pandas-ai.com/v3/large-language-models

Install the pandasai-litellm package using either Poetry or pip to enable multi-provider LLM support.

```bash
# Using poetry
poetry add pandasai-litellm

# Using pip
pip install pandasai-litellm
```

--------------------------------

### Install dependencies using Poetry

Source: https://docs.pandas-ai.com/v3/contributing

Use Poetry to install all dependencies, including extra and development packages. Do not use pip or conda.

```bash
poetry install --all-extras --with dev
```

--------------------------------

### Install and configure the Snowflake extension

Source: https://docs.pandas-ai.com/v3/semantic-layer/data-ingestion

Install the Snowflake extension via poetry or pip, and configure the connection, schema, and transformations using a YAML file.

```bash
poetry add pandasai-snowflake
# or
pip install pandasai-snowflake
```

```yaml
name: sales_data

source:
  type: snowflake
  connection:
    account: your-account
    warehouse: your-warehouse
    database: your-database
    schema: your-schema
    user: ${SNOWFLAKE_USER}
    password: ${SNOWFLAKE_PASSWORD}
  table: sales_data

destination:
  type: local
  format: parquet
  path: company/snowflake-sales

columns:
  - name: transaction_id
    type: string
    description: Unique identifier for each sale
  - name: sale_date
    type: datetime
    description: Date and time of the sale
  - name: product_id
    type: string
    description: Product identifier
  - name: quantity
    type: integer
    description: Number of units sold
  - name: price
    type: float
    description: Price per unit

transformations:
  - type: convert_timezone
    params:
      column: sale_date
      from: UTC
      to: America/Chicago
  - type: calculate
    params:
      column: revenue
      formula: quantity * price
  - type: round
    params:
      column: revenue
      decimals: 2

update_frequency: daily

order_by:
  - sale_date DESC

limit: 100000
```

--------------------------------

### Install pre-commit hooks

Source: https://docs.pandas-ai.com/v3/contributing

Set up pre-commit hooks to ensure code standards are met before committing changes.

```bash
pre-commit install
```

--------------------------------

### Install pandasai-docker package

Source: https://docs.pandas-ai.com/v3/privacy-security

Install the required package to enable the secure Docker sandbox environment for PandasAI.

```bash
pip install pandasai-docker
```

--------------------------------

### Initialize and Run Agent in Docker Sandbox

Source: https://docs.pandas-ai.com/v3/agent

Start the Docker sandbox, pass it to the Agent, and stop it when execution is complete. Ensure Docker is running on your machine.

```python
from pandasai import Agent
from pandasai_docker import DockerSandbox

# Initialize the sandbox
sandbox = DockerSandbox()
sandbox.start()

# Create an agent with the sandbox
df = pai.read_csv("data.csv")
agent = Agent([df], sandbox=sandbox)

# Chat with the agent - code will run in the sandbox
response = agent.chat("Calculate the average sales")

# Don't forget to stop the sandbox when done
sandbox.stop()
```

--------------------------------

### Install Yahoo Finance Extension

Source: https://docs.pandas-ai.com/v3/semantic-layer/data-ingestion

Install the Yahoo Finance extension for PandasAI using either Poetry or pip.

```bash
poetry add pandasai-yfinance
# or
pip install pandasai-yfinance
```

--------------------------------

### Install SQL Database Extensions

Source: https://docs.pandas-ai.com/v3/semantic-layer/data-ingestion

Install the required database driver extension using pip before connecting PandasAI to your SQL database.

```bash
pip install pandasai-sql[postgres]
pip install pandasai-sql[mysql]
pip install pandasai-sql[cockroachdb]
pip install pandasai-sql[sqlserver]
```

--------------------------------

### Install LiteLLM extension for PandasAI

Source: https://docs.pandas-ai.com/v3/migration-troubleshooting

Use this command to resolve ModuleNotFoundError for pandasai.llm by installing the litellm extension.

```bash
pip install pandasai-litellm
```

--------------------------------

### Update Installation for PandasAI v3

Source: https://docs.pandas-ai.com/v3/migration-guide

Install the updated pandasai package along with the new pandasai-litellm package and optional SQL connectors.

```bash
# Using pip
pip install pandasai pandasai-litellm

# Using poetry
poetry add pandasai pandasai-litellm

# For SQL connectors
pip install pandasai-sql[postgres]  # or mysql, sqlite, etc.
```

--------------------------------

### Migrate Data Connectors from v2 to v3

Source: https://docs.pandas-ai.com/v3/migration-guide

Connectors are now separate extensions. Install the specific database extension and use pai.create() with a semantic layer.

```python
from pandasai.connectors import PostgreSQLConnector
from pandasai import SmartDataframe

connector = PostgreSQLConnector(config={
    "host": "localhost",
    "database": "mydb",
    "table": "sales"
})
df = SmartDataframe(connector)
```

```bash
pip install pandasai-sql[postgres]
```

```python
import pandasai as pai

df = pai.create(
    path="company/sales",
    description="Sales data from PostgreSQL",
    source={
        "type": "postgres",
        "connection": {
            "host": "localhost",
            "database": "mydb",
            "user": "${DB_USER}",
            "password": "${DB_PASSWORD}"
        },
        "table": "sales"
    }
)
```

--------------------------------

### Configure LiteLLM for PandasAI

Source: https://docs.pandas-ai.com/v3/large-language-models

Install the LiteLLM extension and configure it to use OpenAI or other supported providers like Anthropic.

```bash
pip install pandasai-litellm
```

```python
import pandasai as pai
from pandasai_litellm.litellm import LiteLLM

# For OpenAI models
llm = LiteLLM(model="gpt-4.1-mini", api_key="YOUR_OPENAI_API_KEY")

# For other providers, change the model name and provide appropriate credentials
# llm = LiteLLM(model="anthropic/claude-3-opus-20240229", api_key="YOUR_ANTHROPIC_API_KEY")

pai.config.set({
    "llm": llm
})
```

--------------------------------

### Configure OpenAI for PandasAI

Source: https://docs.pandas-ai.com/v3/large-language-models

Install the OpenAI extension and configure it using your OpenAI API key.

```bash
# Using poetry
poetry add pandasai-openai

# Using pip
pip install pandasai-openai
```

```python
import pandasai as pai
from pandasai_openai import OpenAI

llm = OpenAI(api_token="my-openai-api-key")

# Set your OpenAI API key
pai.config.set({"llm": llm})
```

--------------------------------

### Instantiating a PandasAI Agent

Source: https://docs.pandas-ai.com/v3/agent

Initialize a PandasAI Agent with a pandas DataFrame to start a conversational session.

```python
import os
from pandasai import Agent
import pandas as pd

# Sample DataFrames
sales_by_country = pd.DataFrame({
    "country": ["United States", "United Kingdom", "France", "Germany", "Italy", "Spain", "Canada", "Australia", "Japan", "China"],
    "sales": [5000, 3200, 2900, 4100, 2300, 2100, 2500, 2600, 4500, 7000],
    "deals_opened": [142, 80, 70, 90, 60, 50, 40, 30, 110, 120],
    "deals_closed": [120, 70, 60, 80, 50, 40, 30, 20, 100, 110]
})

agent = Agent(sales_by_country)
agent.chat('Which are the top 5 countries by sales?')
# Output: China, United States, Japan, Germany, Australia
```

--------------------------------

### Configure Azure OpenAI for PandasAI

Source: https://docs.pandas-ai.com/v3/large-language-models

Install the OpenAI extension and configure Azure OpenAI with your endpoint, API key, and deployment name.

```bash
# Using poetry
poetry add pandasai-openai

# Using pip
pip install pandasai-openai
```

```python
import pandasai as pai
from pandasai_openai import AzureOpenAI

llm = AzureOpenAI(api_base="https://<your-endpoint>.openai.azure.com/",
    api_key="my-azure-openai-api-key",
    deployment_name="text-davinci-003")  # The name of your deployed model

pai.config.set({"llm": llm})
```

--------------------------------

### Configure LiteLLM with API keys and models

Source: https://docs.pandas-ai.com/v3/large-language-models

Set up environment variables for API keys and initialize LiteLLM with specific models like GPT-4 or Claude.

```python
import os
import pandasai as pai
from pandasai_litellm import LiteLLM

# Set your API keys as environment variables
os.environ["OPENAI_API_KEY"] = "your-openai-api-key"
os.environ["ANTHROPIC_API_KEY"] = "your-anthropic-api-key"

# Example with OpenAI
llm = LiteLLM(model="gpt-4.1-mini")

# Example with Anthropic
llm = LiteLLM(model="claude-2")

# Set your LLM configuration
pai.config.set({"llm": llm})
```

--------------------------------

### Migrating Agent Training to Vector Stores

Source: https://docs.pandas-ai.com/v3/migration-guide

Training in v3 requires a vector store (such as ChromaDB, Qdrant, Pinecone, or LanceDB) to enable few-shot learning.

```python
from pandasai import Agent

agent = Agent(df)
agent.train(queries=["query"], codes=["code"])
```

```python
from pandasai import Agent
from pandasai.ee.vectorstores import ChromaDB

# Instantiate with vector store
vector_store = ChromaDB()
agent = Agent(df, vectorstore=vector_store)

# Train with vector store
agent.train(queries=["query"], codes=["code"])
```

--------------------------------

### Customize Docker Sandbox with Custom Dockerfile

Source: https://docs.pandas-ai.com/v3/agent

Instantiate the DockerSandbox with a custom container name and a path to a custom Dockerfile.

```python
sandbox = DockerSandbox(
    "custom-sandbox-name",
    "/path/to/custom/Dockerfile"
)
```

--------------------------------

### Define a SQL Data Source using pai.create

Source: https://docs.pandas-ai.com/v3/semantic-layer/new

Requires the `pandasai-sql` extra dependency. Use environment variables for database credentials to ensure security.

```python
sql_table = pai.create(
    # Format: "organization/dataset"
    path="company/health-data",

    # Optional description
    description="Heart disease dataset from MySQL database",

    # Define the source of the data, including connection details and
    # table name
    source={
        "type": "mysql",
        "connection": {
            "host": "${DB_HOST}",
            "port": 3306,
            "user": "${DB_USER}",
            "password": "${DB_PASSWORD}",
            "database": "${DB_NAME}"
        },
        "table": "heart_data"
    }
)
```

--------------------------------

### Configure PandasAI with LiteLLM

Source: https://docs.pandas-ai.com/v3/getting-started

Initialize the LiteLLM client with an OpenAI model and set it as the default LLM for PandasAI.

```python
import pandasai as pai
from pandasai_litellm.litellm import LiteLLM

# Initialize LiteLLM with your OpenAI model
llm = LiteLLM(model="gpt-4.1-mini", api_key="YOUR_OPENAI_API_KEY")

# Configure PandasAI to use this LLM
pai.config.set({
    "llm": llm
})
```

--------------------------------

### Basic Chat Query in Python

Source: https://docs.pandas-ai.com/v3/chat-and-output

Query a single DataFrame using plain English to get quick insights.

```python
import pandasai as pai

df_customers = pai.read_csv("customers.csv")

response = df_customers.chat("Which are our top 5 customers?")
```

--------------------------------

### Apply global configuration in v3

Source: https://docs.pandas-ai.com/v3/migration-troubleshooting

Set configuration globally using pai.config.set instead of passing it to SmartDataframe.

```python
# v2
df = SmartDataframe(data, config=config)

# v3
pai.config.set(config)
df = pai.DataFrame(data)
```

--------------------------------

### Migrating Agent Methods from v2 to v3

Source: https://docs.pandas-ai.com/v3/migration-guide

In v3, clarification_questions(), rephrase_query(), and explain() are removed. Use chat() and follow_up() to maintain conversation context.

```python
from pandasai import Agent

agent = Agent(df)
clarifications = agent.clarification_questions('What is the GDP?')
rephrased = agent.rephrase_query('What is the GDP?')
explanation = agent.explain()
```

```python
from pandasai import Agent

agent = Agent(df)
# ❌ These methods are removed in v3
# Use chat() and follow_up() instead
response = agent.chat('What is the GDP?')
follow_up = agent.follow_up('What about last year?')  # New: maintains context
```

--------------------------------

### Configure the NL Layer in PandasAI

Source: https://docs.pandas-ai.com/v3/overview-nl

Initialize the LiteLLM client and configure global PandasAI settings such as the LLM instance, logging, console verbosity, and maximum retries.

```python
import pandasai as pai
from pandasai_litellm.litellm import LiteLLM

# Initialize LiteLLM with your OpenAI model
llm = LiteLLM(model="gpt-4.1-mini", api_key="YOUR_OPENAI_API_KEY")

pai.config.set({
   "llm": llm,
   "save_logs": True,
   "verbose": False,
   "max_retries": 3
})
```

--------------------------------

### Train PandasAI Agent with Local Vector Store

Source: https://docs.pandas-ai.com/v3/agent

Instantiate a local vector store (ChromaDB, Qdrant, Pinecone, or LanceDB) and train the agent with a custom query and code response.

```python
from pandasai import Agent
from pandasai.ee.vectorstores import ChromaDB
from pandasai.ee.vectorstores import Qdrant
from pandasai.ee.vectorstores import Pinecone
from pandasai.ee.vector_stores import LanceDB

# Instantiate the vector store
vector_store = ChromaDB()
# or with Qdrant
# vector_store = Qdrant()
# or with LanceDB
vector_store = LanceDB()
# or with Pinecone
# vector_store = Pinecone(
#     api_key="*****",
#     embedding_function=embedding_function,
#     dimensions=384, # dimension of your embedding model
# )

# Instantiate the agent with the custom vector store
agent = Agent("data.csv", vectorstore=vector_store)

# Train the model
query = "What is the total sales for the current fiscal year?"
# The following code is passed as a string to the response variable
response = '\n'.join([
    'import pandas as pd',
    '',
    'df = dfs[0]',
    '',
    '# Calculate the total sales for the current fiscal year',
    'total_sales = df[df[\'date\'] >= pd.to_datetime(\'today\').replace(month=4, day=1)][\'sales\'].sum()',
    'result = { "type": "number", "value": total_sales }'
])

agent.train(queries=[query], codes=[response])

response = agent.chat("What is the total sales for the last fiscal year?")
print(response)
# The model will use the information provided in the training to generate a response
```

--------------------------------

### Migrate Data Connectors to PandasAI v3

Source: https://docs.pandas-ai.com/v3/migration-guide

Transition from v2 PostgreSQLConnector to the new v3 pai.create source configuration.

```python
# v2
from pandasai.connectors import PostgreSQLConnector
connector = PostgreSQLConnector(config={...})
df = SmartDataframe(connector)

# v3
import pandasai as pai
df = pai.create(
    path="company/database-table",
    description="Description of your data",
    source={
        "type": "postgres",
        "connection": {
            "host": "localhost",
            "database": "mydb",
            "user": "${DB_USER}",
            "password": "${DB_PASSWORD}"
        },
        "table": "your_table"
    }
)
```

--------------------------------

### Format code with Ruff

Source: https://docs.pandas-ai.com/v3/contributing

Automatically reformat the codebase using Ruff to match the project's style guidelines.

```bash
make format
```

--------------------------------

### Load and Query a Dataset using pai.load

Source: https://docs.pandas-ai.com/v3/semantic-layer/new

Load a previously defined dataset and query it using natural language.

```python
# Load the dataset
heart_data = pai.load("organization/health-data")

# Query the data
response = heart_data.chat("What is the average age of patients with heart disease?")
```

--------------------------------

### Migrating from SmartDataframe to pai.DataFrame

Source: https://docs.pandas-ai.com/v3/migration-backwards-compatibility

Configure the LLM globally using pai.config.set(). While SmartDataframe is supported for backward compatibility, migrating to pai.DataFrame offers better performance and integration.

```python
from pandasai import SmartDataframe
import pandasai as pai
import pandas as pd
from pandasai_litellm.litellm import LiteLLM

# Configure LLM globally (required)
llm = LiteLLM(model="gpt-4o-mini", api_key="your-api-key")
pai.config.set({"llm": llm})

# v2 style still works
df = pd.DataFrame({
    "country": ["US", "UK", "France"],
    "sales": [5000, 3200, 2900]
})

smart_df = SmartDataframe(df)
response = smart_df.chat("What are the top countries by sales?")
```

```python
import pandasai as pai
import pandas as pd

# Configure LLM globally
pai.config.set({"llm": llm})

# Simple approach
df = pd.DataFrame({
    "country": ["US", "UK", "France"],
    "sales": [5000, 3200, 2900]
})
df = pai.DataFrame(df)
response = df.chat("What are the top countries by sales?")
```

--------------------------------

### Querying data with PandasAI and LiteLLM

Source: https://docs.pandas-ai.com/v3/introduction

Set up PandasAI with LiteLLM to query a CSV dataset using natural language. Ensure you replace 'YOUR_OPENAI_API_KEY' with your actual API key.

```python
import pandasai as pai
from pandasai_litellm.litellm import LiteLLM

# Initialize LiteLLM with your OpenAI model
llm = LiteLLM(model="gpt-4.1-mini", api_key="YOUR_OPENAI_API_KEY")

# Configure PandasAI to use this LLM
pai.config.set({
    "llm": llm
})

# Load your data
df = pai.read_csv("data/companies.csv")

response = df.chat("What is the average revenue by region?")
print(response)
```

--------------------------------

### Define a Data View using YAML Configuration

Source: https://docs.pandas-ai.com/v3/semantic-layer/views

Configure a view schema in YAML by specifying columns and relationships using the [table].[column] format.

```yaml
name: table_heart
columns:
  - name: parents.id
  - name: parents.name
  - name: parents.age
  - name: children.name
  - name: children.age
relations:
  - name: parent_to_children
    description: Relation linking the parent to its children
    from: parents.id
    to: children.id
```

--------------------------------

### Configure Dataset Description in YAML

Source: https://docs.pandas-ai.com/v3/semantic-layer/new

Define a clear text description for the dataset to help LLMs and users understand its purpose.

```yaml
description: Daily sales transactions from all retail stores, including transaction IDs, dates, and amounts
```

--------------------------------

### Migrate Skills from v2 to v3

Source: https://docs.pandas-ai.com/v3/migration-guide

Skills now use the @pai.skill decorator and are automatically registered globally, removing the need for agent.add_skills().

```python
from pandasai.skills import skill
from pandasai import Agent

@skill
def calculate_bonus(salary: float, performance: float) -> float:
    """Calculate employee bonus."""
    if performance >= 90:
        return salary * 0.15
    return salary * 0.10

agent = Agent([df])
agent.add_skills(calculate_bonus)
```

```python
import pandasai as pai
from pandasai import Agent

@pai.skill
def calculate_bonus(salary: float, performance: float) -> float:
    """Calculate employee bonus."""
    if performance >= 90:
        return salary * 0.15
    return salary * 0.10

# Skills automatically available - no need to add them
agent = Agent([df])
```

--------------------------------

### Handling Follow-up Questions

Source: https://docs.pandas-ai.com/v3/agent

Use the follow_up method to continue a conversation without clearing the agent's memory.

```python
# Start a new conversation
response = agent.chat('What is the total sales?')
print("First response:", response)

# Continue the conversation without clearing memory
follow_up_response = agent.follow_up('What about last year?')
print("Follow-up response:", follow_up_response)
```

--------------------------------

### Create Semantic Layers for MySQL and SQL Server

Source: https://docs.pandas-ai.com/v3/semantic-layer/data-ingestion

Define semantic tables using the `pai.create` method, specifying connection parameters and column schemas for MySQL and SQL Server.

```python
# MySQL example
sql_table = pai.create(
    path="example/mysql-dataset",
    description="Heart disease dataset from MySQL database",
    source={
        "type": "mysql",
        "connection": {
            "host": "database.example.com",
            "port": 3306,
            "user": "${DB_USER}",
            "password": "${DB_PASSWORD}",
            "database": "medical_data"
        },
        "table": "heart_data",
        "columns": [
            {"name": "Age", "type": "integer", "description": "Age of the patient in years"},
            {"name": "Sex", "type": "string", "description": "Gender of the patient (M = male, F = female)"},
            {"name": "ChestPainType", "type": "string", "description": "Type of chest pain (ATA, NAP, ASY, TA)"},
            {"name": "RestingBP", "type": "integer", "description": "Resting blood pressure in mm Hg"},
            {"name": "Cholesterol", "type": "integer", "description": "Serum cholesterol in mg/dl"},
            {"name": "FastingBS", "type": "integer", "description": "Fasting blood sugar > 120 mg/dl (1 = true, 0 = false)"},
            {"name": "RestingECG", "type": "string", "description": "Resting electrocardiogram results (Normal, ST, LVH)"},
            {"name": "MaxHR", "type": "integer", "description": "Maximum heart rate achieved"},
            {"name": "ExerciseAngina", "type": "string", "description": "Exercise-induced angina (Y = yes, N = no)"},
            {"name": "Oldpeak", "type": "float", "description": "ST depression induced by exercise relative to rest"},
            {"name": "ST_Slope", "type": "string", "description": "Slope of the peak exercise ST segment (Up, Flat, Down)"},
            {"name": "HeartDisease", "type": "integer", "description": "Heart disease diagnosis (1 = present, 0 = absent)"}
        ]
    }
)

# SQL Server example
sql_server_table = pai.create(
    path="example/sqlserver-dataset",
    description="Sales data from SQL Server database",
    source={
        "type": "sqlserver",
        "connection": {
            "host": "sqlserver.example.com",
            "port": 1433,
            "user": "${SQLSERVER_USER}",
            "password": "${SQLSERVER_PASSWORD}",
            "database": "sales_data"
        },
        "table": "transactions",
        "columns": [
            {"name": "transaction_id", "type": "string", "description": "Unique identifier for each transaction"},
            {"name": "customer_id", "type": "string", "description": "Customer identifier"},
            {"name": "transaction_date", "type": "datetime", "description": "Date and time of transaction"},
            {"name": "product_category", "type": "string", "description": "Product category"},
            {"name": "quantity", "type": "integer", "description": "Number of items sold"},
            {"name": "unit_price", "type": "float", "description": "Price per unit"},
            {"name": "total_amount", "type": "float", "description": "Total transaction amount"}
        ]
    }
)
```

--------------------------------

### Configuring Agent in PandasAI v3

Source: https://docs.pandas-ai.com/v3/migration-backwards-compatibility

In v3, configure the LLM globally using pai.config.set() instead of passing it per-agent.

```python
from pandasai import Agent
import pandasai as pai
from pandasai_litellm.litellm import LiteLLM

# Configure LLM globally (required in v3)
llm = LiteLLM(model="gpt-4o-mini", api_key="your-api-key")
pai.config.set({"llm": llm})

# Agent works as before
df1 = pai.DataFrame(sales_data)
df2 = pai.DataFrame(costs_data)

agent = Agent([df1, df2])
response = agent.chat("Analyze the data and provide insights")
```

--------------------------------

### Configure LLM Globally in PandasAI v3

Source: https://docs.pandas-ai.com/v3/migration-guide

Set up the global LLM configuration using pandasai_litellm and the pai.config.set method.

```python
from pandasai_litellm.litellm import LiteLLM
import pandasai as pai

llm = LiteLLM(model="gpt-4o-mini", api_key="your-api-key")
pai.config.set({
    "llm": llm,
    "verbose": False,
    "save_logs": True,
    "max_retries": 3
})
```

--------------------------------

### Define a Basic Custom Skill in Python

Source: https://docs.pandas-ai.com/v3/skills

Create a basic custom skill using the @pai.skill() decorator. Ensure you include clear type hints and a descriptive docstring so the AI knows when and how to use it.

```python
import pandasai as pai

@pai.skill()
def my_custom_function(param1: str, param2: int) -> str:
    """
    A custom function that demonstrates skill creation.

    Args:
        param1 (str): First parameter description
        param2 (int): Second parameter description

    Returns:
        str: Result description
    """
    return f"Processed {param1} with value {param2}"
```

--------------------------------

### Create a Data View Programmatically in Python

Source: https://docs.pandas-ai.com/v3/semantic-layer/views

Define source datasets, establish relationships, select columns, and create a unified view using the PandasAI Python API.

```python
import pandasai as pai

# Create source datasets for an e-commerce analytics system
# Orders dataset
orders_df = pai.read_csv("orders.csv")
orders_dataset = pai.create(
    "myorg/orders",
    orders_df,
    description="Customer orders and transaction data"
)

# Products dataset
products_df = pai.read_csv("products.csv")
products_dataset = pai.create(
    "myorg/products",
    products_df,
    description="Product catalog with categories and pricing"
)

# Customer dataset
customers_df = pai.read_csv("customers.csv")
customers_dataset = pai.create(
    "myorg/customers",
    customers_df,
    description="Customer demographics and preferences"
)

# Define relationships between datasets
view_relations = [
    {
        "name": "order_to_product",
        "description": "Links orders to their products",
        "from": "orders.product_id",
        "to": "products.id"
    },
    {
        "name": "order_to_customer",
        "description": "Links orders to customer profiles",
        "from": "orders.customer_id",
        "to": "customers.id"
    }
]

# Select relevant columns for the sales analytics view
view_columns = [
    # Order details
    {"name": "orders.id", "type": "integer"},
    {"name": "orders.order_date", "type": "date"},
    {"name": "orders.total_amount", "type": "float"},
    {"name": "orders.status", "type": "string"},

    # Product information
    {"name": "products.name", "type": "string"},
    {"name": "products.category", "type": "string"},
    {"name": "products.unit_price", "type": "float"},
    {"name": "products.stock_level", "type": "integer"},

    # Customer information
    {"name": "customers.segment", "type": "string"},
    {"name": "customers.country", "type": "string"},
    {"name": "customers.join_date", "type": "date"},
]

# Create a comprehensive sales analytics view
sales_view = pai.create(
    "myorg/sales-analytics",
    description="Unified view of sales data combining orders, products, and customer information",
    relations=view_relations,
    columns=view_columns,
    view=True
)

# This view enables powerful analytics queries like:
# - Sales trends by customer segment and product category
# - Customer purchase history and preferences
# - Inventory management based on order patterns
# - Geographic sales distribution
```

--------------------------------

### Configure SQL Data Source in YAML

Source: https://docs.pandas-ai.com/v3/semantic-layer/new

Specify connection details and table/view information for SQL-based datasets.

```yaml
source:
  type: postgres
  connection:
    host: postgres-host
    port: 5432
    database: postgres
    user: postgres
    password: ******
  table: orders
  view: false
```

--------------------------------

### Migrating from SmartDatalake to pai.chat

Source: https://docs.pandas-ai.com/v3/migration-backwards-compatibility

Query multiple dataframes directly using pai.chat() instead of instantiating SmartDatalake. Ensure the LLM is configured globally.

```python
from pandasai import SmartDatalake
import pandasai as pai
import pandas as pd
from pandasai_litellm.litellm import LiteLLM

# Configure LLM globally (required)
llm = LiteLLM(model="gpt-4o-mini", api_key="your-api-key")
pai.config.set({"llm": llm})

# v2 style still works
employees_df = pd.DataFrame({
    "name": ["John", "Jane", "Bob"],
    "department": ["Sales", "Engineering", "Sales"]
})

salaries_df = pd.DataFrame({
    "name": ["John", "Jane", "Bob"],
    "salary": [60000, 80000, 55000]
})

lake = SmartDatalake([
    employees_df,
    salaries_df
])

response = lake.chat("Who gets paid the most?")
```

```python
import pandasai as pai

# Configure LLM globally
pai.config.set({"llm": llm})

# Create dataframes
employees = pai.DataFrame(employees_df)
salaries = pai.DataFrame(salaries_df)

# Query across multiple dataframes directly
response = pai.chat("Who gets paid the most?", employees, salaries)
```

--------------------------------

### Check code linting with Ruff

Source: https://docs.pandas-ai.com/v3/contributing

Run the linter to check for formatting differences and potential errors before submitting a pull request.

```bash
make format_diff
```

--------------------------------

### Migrate Configuration from v2 to v3

Source: https://docs.pandas-ai.com/v3/migration-guide

In v3, configuration is set globally using pai.config.set() instead of per-dataframe. Several options like save_charts and enable_cache have been removed.

```python
from pandasai import SmartDataframe

config = {
    "llm": llm,
    "save_charts": True,
    "enable_cache": True,
    "security": "standard"
}
df = SmartDataframe(data, config=config)
```

```python
import pandasai as pai

pai.config.set({
    "llm": llm,
    "save_logs": True,
    "verbose": False,
    "max_retries": 3
})
df = pai.DataFrame(data)
```

--------------------------------

### Configure determinism using temperature and seed

Source: https://docs.pandas-ai.com/v3/large-language-models

Set temperature to 0 and specify a seed to ensure reproducible and consistent outputs across runs. Note that seed is currently not supported for AzureOpenAI.

```python
import pandasai as pai

# Sample DataFrame
df = pai.DataFrame({
    "country": ["United States", "United Kingdom", "France", "Germany", "Italy", "Spain", "Canada", "Australia", "Japan", "China"],
    "gdp": [19294482071552, 2891615567872, 2411255037952, 3435817336832, 1745433788416, 1181205135360, 1607402389504, 1490967855104, 4380756541440, 14631844184064],
    "happiness_index": [6.94, 7.16, 6.66, 7.07, 6.38, 6.4, 7.23, 7.22, 5.87, 5.12]
})

# Configure the LLM
pai.config.set({
   "temperature" : 0,
   "seed" : 26
})

df.chat('Which are the 5 happiest countries?') # answer should me (mostly) consistent across devices.
```

--------------------------------

### Configure Oracle Data Ingestion

Source: https://docs.pandas-ai.com/v3/semantic-layer/data-ingestion

YAML configuration for ingesting data from an Oracle database, applying timezone conversions and calculated columns, and saving to local Parquet format.

```yaml
name: sales_data

source:
  type: oracle
  connection:
    host: your-host
    port: 1521
    service_name: your-service
    user: ${ORACLE_USER}
    password: ${ORACLE_PASSWORD}
  table: sales_data

destination:
  type: local
  format: parquet
  path: company/oracle-sales

columns:
  - name: transaction_id
    type: string
    description: Unique identifier for each sale
  - name: sale_date
    type: datetime
    description: Date and time of the sale
  - name: product_id
    type: string
    description: Product identifier
  - name: quantity
    type: integer
    description: Number of units sold
  - name: price
    type: float
    description: Price per unit

transformations:
  - type: convert_timezone
    params:
      column: sale_date
      from: UTC
      to: Australia/Sydney
  - type: calculate
    params:
      column: total_amount
      formula: quantity * price
  - type: round
    params:
      column: total_amount
      decimals: 2
  - type: calculate
    params:
      column: discount
      formula: "CASE WHEN quantity > 10 THEN 0.1 WHEN quantity > 5 THEN 0.05 ELSE 0 END"

update_frequency: daily

order_by:
  - sale_date DESC

limit: 100000
```

--------------------------------

### Configure Yahoo Finance Data Ingestion

Source: https://docs.pandas-ai.com/v3/semantic-layer/data-ingestion

YAML configuration for ingesting stock market data from Yahoo Finance, calculating daily returns and price ranges, and saving to local Parquet format.

```yaml
name: stock_data

source:
  type: yahoo_finance
  symbols: 
    - GOOG
    - MSFT
    - AAPL
  start_date: 2023-01-01
  end_date: 2023-12-31

destination:
  type: local
  format: parquet
  path: company/market-data

columns:
  - name: date
    type: datetime
    description: Date of the trading day
  - name: open
    type: float
    description: Opening price of the stock
  - name: high
    type: float
    description: Highest price of the stock during the day
  - name: low
    type: float
    description: Lowest price of the stock during the day
  - name: close
    type: float
    description: Closing price of the stock
  - name: volume
    type: integer
    description: Number of shares traded during the day

transformations:
  - type: calculate
    params:
      column: daily_return
      formula: (close - open) / open * 100
  - type: calculate
    params:
      column: price_range
      formula: high - low
  - type: round
    params:
      columns: [daily_return, price_range]
      decimals: 2
  - type: convert_timezone
    params:
      column: date
      from: UTC
      to: America/New_York

update_frequency: daily

order_by:
  - date DESC

limit: 100000
```

--------------------------------

### Use Skills with PandasAI Agent in Python

Source: https://docs.pandas-ai.com/v3/skills

Configure an Agent with custom DataFrames and LLM settings. Registered skills are automatically available during multi-turn agent conversations.

```python
import pandas as pd
import pandasai as pai
from pandasai import Agent
from pandasai_litellm.litellm import LiteLLM

# Add your model
llm = LiteLLM(model="ollama/llama3", api_base="http://localhost:11434/api/generate")

pai.config.set({"llm": llm})

# Sample employee data
employees_data = {
    "EmployeeID": [1, 2, 3, 4, 5],
    "Name": ["John", "Emma", "Liam", "Olivia", "William"],
    "Department": ["HR", "Sales", "IT", "Marketing", "Finance"],
    "Salary": [50000, 60000, 70000, 55000, 65000],
    "Performance": [85, 92, 78, 88, 95]
}

salaries_data = {
    "EmployeeID": [1, 2, 3, 4, 5],
    "Bonus": [7500, 9000, 7000, 5500, 9750]
}

employees_df = pai.DataFrame(employees_data)
salaries_df = pai.DataFrame(salaries_data)

# Create an agent with the dataframes
agent = Agent([employees_df, salaries_df], memory_size=10)

# Chat with the agent - skills are automatically available
response1 = agent.chat("Calculate bonuses for all employees and show the results")
print("Response 1:", response1)

response2 = agent.chat("Show me the total bonus amount formatted as currency")
print("Response 2:", response2)

# The agent can use multiple skills in one conversation
response3 = agent.chat("Calculate bonuses, format them as currency, and create a chart")
print("Response 3:", response3)
```

--------------------------------

### Using Agent follow-up in PandasAI v3

Source: https://docs.pandas-ai.com/v3/migration-backwards-compatibility

Use the follow_up method to continue conversations without losing context. Note that clarification_questions, explain, and rephrase_query have been removed.

```python
agent = Agent([df1, df2])

# Start conversation
response = agent.chat('What is the total revenue?')

# Follow up without losing context
follow_up = agent.follow_up('What about last quarter?')
```

--------------------------------

### Sample Sales Dataset CSV

Source: https://docs.pandas-ai.com/v3/semantic-layer/transformations

Raw sales data containing inconsistent formatting, negative quantities, and invalid email addresses.

```csv
date,store_id,product_name,category,quantity,unit_price,customer_email
2024-01-15, ST001,  iPhone 13 Pro,Electronics,2,999.99,john.doe@email.com
2024-01-15,ST002,macBook Pro ,Electronics,-1,1299.99,invalid.email
2024-01-16,ST001,AirPods Pro,Electronics,3,249.99,jane@example.com
2024-01-16,ST003,iMac 27" ,Electronics,1,1799.99,
```

--------------------------------

### Chain Transformations in YAML

Source: https://docs.pandas-ai.com/v3/semantic-layer/transformations

Apply multiple transformations sequentially in the order they are defined in the YAML configuration.

```yaml
transformations:
  - type: to_lowercase
    params:
      column: product_name
  - type: strip
    params:
      column: product_name
  - type: truncate
    params:
      column: product_name
      length: 50
```

--------------------------------

### Continuing a Multi-turn Conversation

Source: https://docs.pandas-ai.com/v3/agent

Ask follow-up questions in the same session. The agent automatically tracks the state of the conversation.

```python
agent.chat('And which one has the most deals?')
# Output: United States has the most deals
```

--------------------------------

### Enable Docker Sandbox in PandasAI

Source: https://docs.pandas-ai.com/v3/privacy-security

Configure and run PandasAI queries inside a secure, isolated Docker sandbox container to prevent malicious code execution. Ensure Docker is running on your system before executing.

```python
import pandasai as pai
from pandasai_docker import DockerSandbox
from pandasai_litellm.litellm import LiteLLM

# Initialize LiteLLM with your OpenAI model
llm = LiteLLM(model="gpt-4.1-mini", api_key="YOUR_OPENAI_API_KEY")

# Configure PandasAI to use this LLM
pai.config.set({
    "llm": llm
})

# initialize the sandbox
sandbox = DockerSandbox()
sandbox.start()

# read a csv as df
df = pai.read_csv("./data/heart.csv")

# pass the df and the sandbox
result = pai.chat("plot total heart patients by gender", df, sandbox=sandbox)

# display the chart
result.show()

# stop the sandbox (docker container)
sandbox.stop()
```
Tokens:
Raw