### Development Setup and Installation

Source: https://github.com/lotus-data/lotus/blob/main/CONTRIBUTING.md

Instructions for setting up the Lotus development environment, including creating a conda environment, cloning the repository, and installing the project in development mode with its dependencies.

```bash
conda create -n lotus python=3.10 -y
conda activate lotus

git clone git@github.com:lotus-data/lotus.git
cd lotus

pip install -e .

pip install -r requirements-dev.txt

pre-commit install
```

--------------------------------

### Install LOTUS with Pip

Source: https://github.com/lotus-data/lotus/blob/main/docs/installation.rst

Installs the LOTUS Python library using pip after setting up a Conda environment. It also includes instructions for installing the Faiss library, a dependency for LOTUS, which is crucial for performance.

```console
$ conda create -n lotus python=3.10 -y
$ conda activate lotus
$ pip install lotus-ai
```

```console
# CPU-only version
$ conda install -c pytorch faiss-cpu=1.8.0

# GPU(+CPU) version
$ conda install -c pytorch -c nvidia faiss-gpu=1.8.0
```

--------------------------------

### Install LOTUS You.com Submodule

Source: https://github.com/lotus-data/lotus/blob/main/docs/web_search.rst

Installs the LOTUS library with support for You.com searches.

```shell
pip install lotus[you]
```

--------------------------------

### Install Qdrant and Lotus

Source: https://github.com/lotus-data/lotus/blob/main/docs/vector_store.rst

Installs the necessary Qdrant client and the Lotus library with Qdrant support. This is the first step to using Qdrant with Lotus.

```bash
pip install qdrant-client lotus[qdrant]
```

--------------------------------

### Install LOTUS Serpapi (Google) Submodule

Source: https://github.com/lotus-data/lotus/blob/main/docs/web_search.rst

Installs the LOTUS library with support for Google searches via SerpApi.

```shell
pip install lotus[serpapi]
```

--------------------------------

### Install LOTUS Arxiv Submodule

Source: https://github.com/lotus-data/lotus/blob/main/docs/web_search.rst

Installs the LOTUS library with support for Arxiv searches.

```shell
pip install lotus[arxiv]
```

--------------------------------

### Install LOTUS from GitHub (Latest Features)

Source: https://github.com/lotus-data/lotus/blob/main/README.md

Installs the latest features of LOTUS directly from its GitHub repository. This method also requires creating and activating a Conda environment before installing from the Git URL.

```bash
conda create -n lotus python=3.10 -y
conda activate lotus
pip install git+https://github.com/lotus-data/lotus.git@main
```

--------------------------------

### Install Faiss and Lotus

Source: https://github.com/lotus-data/lotus/blob/main/docs/vector_store.rst

Installs the Faiss CPU version and the Lotus library. This is required to use Faiss as a vector store backend.

```bash
pip install faiss-cpu lotus
```

--------------------------------

### LM Rate Limiting Examples

Source: https://github.com/lotus-data/lotus/blob/main/docs/llm.rst

Provides examples of configuring rate limits for the LM class. It shows basic rate limiting (requests per minute) and how to use `max_batch_size` for local serving or stricter limits.

```python
from lotus.models import LM

# Basic rate limiting - 30 requests per minute
lm = LM(
    model="gpt-4o",
    rate_limit=30  # 30 requests per minute
)

# For strict rate limits (e.g., free tier APIs)
lm = LM(
    model="gpt-4o",
    max_batch_size=10,  # Optional: for local serving
    rate_limit=10      # Caps at 10 requests per minute
)

# Traditional approach using only max_batch_size
lm = LM(
    model="gpt-4o",
    max_batch_size=64
)
```

--------------------------------

### Install LOTUS Tavily Submodule

Source: https://github.com/lotus-data/lotus/blob/main/docs/web_search.rst

Installs the LOTUS library with support for Tavily searches.

```shell
pip install lotus[tavily]
```

--------------------------------

### Pre-commit Hooks for Code Quality

Source: https://github.com/lotus-data/lotus/blob/main/CONTRIBUTING.md

Instructions on installing and running pre-commit hooks, which automate linting and formatting using tools like ruff and mypy to maintain code quality.

```bash
# Install pre-commit if you haven't already
pip install pre-commit

# Install the pre-commit hooks defined in .pre-commit-config.yaml
pre-commit install

# Run all pre-commit hooks on all files
pre-commit run --all-files

# To run a specific hook (e.g., ruff)
pre-commit run ruff --all-files

# To run pre-commit checks before every commit (recommended), just commit as usual:
git commit -m "Your commit message"
# The hooks will run automatically
```

--------------------------------

### Install LOTUS Bing Submodule

Source: https://github.com/lotus-data/lotus/blob/main/docs/web_search.rst

Installs the LOTUS library with support for Bing searches.

```shell
pip install lotus[bing]
```

--------------------------------

### Initialize LM with GPT-4o

Source: https://github.com/lotus-data/lotus/blob/main/CONTRIBUTING.md

Example of initializing the `LM` class from `lotus.models` to use the GPT-4o model.

```python
from lotus.models import LM

lm = LM(model="gpt-4o")
```

--------------------------------

### Initialize LM with vLLM

Source: https://github.com/lotus-data/lotus/blob/main/CONTRIBUTING.md

Example of initializing the `LM` class to use a model hosted via vLLM, including specifying the API base, context length, and max tokens.

```python
from lotus.models import LM

lm = LM(
    model='hosted_vllm/meta-llama/Meta-Llama-3-8B-Instruct',
    api_base='http://localhost:8000/v1',
    max_ctx_len=8000,
    max_tokens=1000
)
```

--------------------------------

### QdrantVS Example Usage

Source: https://github.com/lotus-data/lotus/blob/main/docs/vector_store.rst

Demonstrates how to initialize and use the QdrantVS with Lotus. It involves setting up the Qdrant client, a retrieval model, configuring Lotus settings, indexing data, and performing a semantic search.

```python
import pandas as pd
from qdrant_client import QdrantClient
import lotus
from lotus.models import LiteLLMRM  # or SentenceTransformersRM
from lotus.vector_store import QdrantVS

# Start Qdrant server before running this code
client = QdrantClient(url="http://localhost:6333")
rm = LiteLLMRM(model="text-embedding-3-small")
vs = QdrantVS(client)
lotus.settings.configure(rm=rm, vs=vs)

data = {"Course Name": ["Machine Learning 101", "Introduction to Cooking"]}
df = pd.DataFrame(data)
df = df.sem_index("Course Name", "my_qdrant_index")
result = df.sem_search("Course Name", "Find the course about machine learning", K=1)
print(result)
```

--------------------------------

### Install Weaviate and Lotus

Source: https://github.com/lotus-data/lotus/blob/main/docs/vector_store.rst

Installs the Weaviate client and the Lotus library with Weaviate support. This enables the use of Weaviate as a vector store.

```bash
pip install weaviate-client lotus[weaviate]
```

--------------------------------

### Initialize LM with Ollama

Source: https://github.com/lotus-data/lotus/blob/main/CONTRIBUTING.md

Example of initializing the `LM` class to use a model hosted via Ollama, specifically llama3.2.

```python
from lotus.models import LM

lm = LM(model="ollama/llama3.2")
```

--------------------------------

### Semantic Indexing and Search in LOTUS

Source: https://github.com/lotus-data/lotus/blob/main/docs/examples.rst

Shows how to create a semantic index on a DataFrame column and then perform a semantic search using that index. This example indexes course descriptions and searches for a relevant class for 'Convolutional Neural Network'.

```python
# Create a semantic index on the description column and save it to the index_dir directory
df = df.sem_index("Description", "index_dir")
top_conv_df = df.sem_search("Description", "Convolutional Neural Network", K=1)
```

--------------------------------

### Install LOTUS with Conda (Stable)

Source: https://github.com/lotus-data/lotus/blob/main/README.md

Installs the latest stable release of LOTUS using Conda. This involves creating a new Conda environment, activating it, and then installing the 'lotus-ai' package.

```bash
conda create -n lotus python=3.10 -y
conda activate lotus
pip install lotus-ai
```

--------------------------------

### Run Qdrant with Docker

Source: https://github.com/lotus-data/lotus/blob/main/docs/vector_store.rst

Starts a Qdrant instance locally using Docker, mapping the necessary ports and persisting data to a local directory. Ensure Qdrant is running before connecting.

```bash
docker run -p 6333:6333 -p 6334:6334 \
    -v "$(pwd)/qdrant_storage:/qdrant/storage:z" \
    qdrant/qdrant
```

--------------------------------

### FaissVS Example Usage

Source: https://github.com/lotus-data/lotus/blob/main/docs/vector_store.rst

Shows how to initialize and use the FaissVS with Lotus. It involves setting up a retrieval model, configuring Lotus settings, indexing data, and performing a semantic search.

```python
import pandas as pd
import lotus
from lotus.models import LiteLLMRM
from lotus.vector_store import FaissVS

rm = LiteLLMRM(model="text-embedding-3-small")
vs = FaissVS()
lotus.settings.configure(rm=rm, vs=vs)

data = {"Course Name": ["Machine Learning 101", "Introduction to Cooking"]}
df = pd.DataFrame(data)
df = df.sem_index("Course Name", "my_faiss_index")
result = df.sem_search("Course Name", "Find the course about machine learning", K=1)
print(result)
```

--------------------------------

### Run Weaviate with Docker

Source: https://github.com/lotus-data/lotus/blob/main/docs/vector_store.rst

Starts a Weaviate instance locally using Docker, mapping the necessary ports. Ensure Weaviate is running before connecting.

```bash
docker run -p 8080:8080 -p 50051:50051 cr.weaviate.io/semitechnologies/weaviate:1.29.1
```

--------------------------------

### Few-Shot Reasoning Strategy

Source: https://github.com/lotus-data/lotus/blob/main/docs/prompt_strategies.rst

This strategy involves providing the model with a limited number of examples. The model learns to imitate the behavior demonstrated in these examples to answer the final prompt, without explicit reasoning steps being shown.

```Python
class ReasoningStrategy:
    FEW_SHOT = "Few-Shot Learning"
```

--------------------------------

### Google Web Search Example

Source: https://github.com/lotus-data/lotus/blob/main/docs/web_search.rst

Shows how to perform a web search using Google through LOTUS. Requires the `SERPAPI_API_KEY` environment variable. It retrieves results, selects 'title' and 'snippet', and finds the most exciting snippets.

```python
import lotus
from lotus import WebSearchCorpus, web_search
from lotus.models import LM

lm = LM(model="gpt-4o-mini")

lotus.settings.configure(lm=lm)

df = web_search(WebSearchCorpus.GOOGLE, "deep learning research", 5)[["title", "snippet"]]
print(f"Results from Google\n{df}")
most_interesting_articles = df.sem_topk("Which {snippet} is the most exciting?", K=1)
print(f"Most interesting articles\n{most_interesting_articles}")
```

--------------------------------

### You.com Web Search Example

Source: https://github.com/lotus-data/lotus/blob/main/docs/web_search.rst

Demonstrates searching You.com using LOTUS. Requires the `YOU_API_KEY` environment variable. It fetches results, selects 'title' and 'snippet', and identifies the top 3 most groundbreaking snippets.

```python
import lotus
from lotus import WebSearchCorpus, web_search
from lotus.models import LM

lm = LM(model="gpt-4o-mini")

lotus.settings.configure(lm=lm)

df = web_search(WebSearchCorpus.YOU, "latest AI breakthroughs", 10)[["title", "snippet"]]
print(f"Results from You.com:\n{df}\n")
top_you_articles = df.sem_topk("Which {snippet} is the most groundbreaking?", K=3)
print(f"Top 3 most interesting articles from You.com:\n{top_you_articles}")
```

--------------------------------

### Create LM Object for GPT-4o

Source: https://github.com/lotus-data/lotus/blob/main/docs/llm.rst

Demonstrates how to instantiate the LM class to use the 'gpt-4o' model. This is a basic setup for interacting with OpenAI's GPT-4o model.

```python
from lotus.models import LM
lm = LM(model="gpt-4o")
```

--------------------------------

### Configure LOTUS with LLM and Embedding Models

Source: https://github.com/lotus-data/lotus/blob/main/docs/examples.rst

Configures LOTUS with specified Language Model (LM) and Retrieval Model (RM) for semantic operations. It also sets up a vector store. Dependencies include pandas, lotus, and specific models from lotus.models and lotus.vector_store.

```python
import pandas as pd

import lotus
from lotus.models import SentenceTransformersRM, LM
from lotus.vector_store import FaissVS

# Configure models for LOTUS
lm = LM(model="gpt-4o-mini")
rm = SentenceTransformersRM(model="intfloat/e5-base-v2")
vs = FaissVS()

lotus.settings.configure(lm=lm, rm=rm, vs=vs)
```

--------------------------------

### WeaviateVS Example Usage

Source: https://github.com/lotus-data/lotus/blob/main/docs/vector_store.rst

Demonstrates how to initialize and use the WeaviateVS with Lotus. It involves setting up the Weaviate client, a retrieval model, configuring Lotus settings, indexing data, and performing a semantic search.

```python
import pandas as pd
import weaviate
import lotus
from lotus.models import LiteLLMRM
from lotus.vector_store import WeaviateVS

client = weaviate.Client("http://localhost:8080")
rm = LiteLLMRM(model="text-embedding-3-small")
vs = WeaviateVS(client)
lotus.settings.configure(rm=rm, vs=vs)

data = {"Course Name": ["Machine Learning 101", "Introduction to Cooking"]}
df = pd.DataFrame(data)
df = df.sem_index("Course Name", "my_weaviate_index")
result = df.sem_search("Course Name", "Find the course about machine learning", K=1)
print(result)
```

--------------------------------

### LM Usage Limits Configuration

Source: https://github.com/lotus-data/lotus/blob/main/docs/llm.rst

Demonstrates how to set physical and virtual usage limits for the LM class to control costs and token consumption. It includes examples of `UsageLimit` objects and handling `LotusUsageLimitException`.

```python
from lotus.models import LM
from lotus.types import UsageLimit, LotusUsageLimitException
import pandas as pd

# Set physical limit (actual API calls)
physical_limit = UsageLimit(
    prompt_tokens_limit=4000,
    completion_tokens_limit=1000,
    total_tokens_limit=5000,
    total_cost_limit=1.00
)

# Set virtual limit (includes cached responses)
virtual_limit = UsageLimit(
    prompt_tokens_limit=10000,
    completion_tokens_limit=2000,
    total_tokens_limit=12000,
    total_cost_limit=5.00
)

# Apply both limits to the LM
lm = LM(
    model="gpt-4o",
    physical_usage_limit=physical_limit,
    virtual_usage_limit=virtual_limit
)

try:
    course_df = pd.read_csv("course_df.csv")
    course_df = course_df.sem_filter("What {Course Name} requires a lot of math")
except LotusUsageLimitException as e:
    print(f"Usage limit exceeded: {e}")
    # Handle the exception as needed
```

--------------------------------

### sem_topk Function Example

Source: https://github.com/lotus-data/lotus/blob/main/docs/sem_topk.rst

Demonstrates how to use the sem_topk function for semantic top-k ranking with a pandas DataFrame. It shows how to configure the language model and apply the sem_topk operation with different methods.

```python
import pandas as pd

import lotus
from lotus.models import LM

lm = LM(model="gpt-4o-mini")

lotus.settings.configure(lm=lm)
data = {
    "Course Name": [
        "Probability and Random Processes",
        "Optimization Methods in Engineering",
        "Digital Design and Integrated Circuits",
        "Computer Security",
    ]
}
df = pd.DataFrame(data)

for method in ["quick", "heap", "naive"]:
    sorted_df, stats = df.sem_topk(
        "Which {Course Name} requires the least math?",
        K=2,
        method=method,
        return_stats=True,
    )
    print(sorted_df)
    print(stats)
```

--------------------------------

### Configure LM, LiteLLMRM, and FaissVS

Source: https://github.com/lotus-data/lotus/blob/main/docs/retriever_models.rst

Configures the Lotus project with a Language Model (LM), a LiteLLM Retrieval Model (LiteLLMRM), and a Faiss Vector Store (FaissVS). This setup allows for both language generation and retrieval using specified models.

```python
import pandas as pd

import lotus
from lotus.models import LM, LiteLLMRM
from lotus.vector_store import FaissVS

lm = LM(model="gpt-4o-mini")
rm = LiteLLMRM(model="text-embedding-3-small")
vs = FaissVS()

lotus.settings.configure(lm=lm, rm=rm, vs=vs)
```

--------------------------------

### Bing Web Search Example

Source: https://github.com/lotus-data/lotus/blob/main/docs/web_search.rst

Illustrates how to search Bing using LOTUS. Requires the `BING_API_KEY` environment variable. It retrieves results, selects 'title' and 'snippet', and finds the top 3 articles offering the best insight into AI models.

```python
import lotus
from lotus import WebSearchCorpus, web_search
from lotus.models import LM

lm = LM(model="gpt-4o-mini")

lotus.settings.configure(lm=lm)

df = web_search(WebSearchCorpus.BING, "state-of-the-art AI models", 10)[["title", "snippet"]]
print(f"Results from Bing:\n{df}\n")
top_bing_articles = df.sem_topk("Which {snippet} provides the best insight into AI models?", K=3)
print(f"Top 3 most insightful articles from Bing:\n{top_bing_articles}")
```

--------------------------------

### Arxiv Web Search Example

Source: https://github.com/lotus-data/lotus/blob/main/docs/web_search.rst

Demonstrates how to use the `web_search` function with the Arxiv corpus to find deep learning papers. It fetches results, selects 'title' and 'abstract' columns, and identifies the most exciting article using semantic top-k.

```python
import lotus
from lotus import WebSearchCorpus, web_search
from lotus.models import LM

lm = LM(model="gpt-4o-mini")

lotus.settings.configure(lm=lm)

df = web_search(WebSearchCorpus.ARXIV, "deep learning", 5)[["title", "abstract"]]
print(f"Results from Arxiv\n{df}\n\n")

most_interesting_articles = df.sem_topk("Which {abstract} is most exciting?", K=1)
print(f"Most interesting article: \n{most_interesting_articles.iloc[0]}")
```

--------------------------------

### Using sem_partition_by with Clustering

Source: https://github.com/lotus-data/lotus/blob/main/docs/sem_partition.rst

Demonstrates how to use the sem_partition_by utility with a clustering function to partition DataFrame rows. This example sets up Language Model (LM) and Retrieval Model (RM) configurations, indexes a DataFrame column, partitions it, and then performs semantic aggregation.

```python
import pandas as pd

import lotus
from lotus.models import LM, SentenceTransformersRM

lm = LM(max_tokens=2048)
rm = SentenceTransformersRM(model="intfloat/e5-base-v2")

lotus.settings.configure(lm=lm, rm=rm)
data = {
    "Course Name": [
        "Probability and Random Processes",
        "Optimization Methods in Engineering",
        "Digital Design and Integrated Circuits",
        "Computer Security",
        "Cooking",
        "Food Sciences",
    ]
}
df = pd.DataFrame(data)
df = df.sem_index("Course Name", "course_name_index").sem_partition_by(lotus.utils.cluster("Course Name", 2))
out = df.sem_agg("Summarize all {Course Name}")._output[0]
print(out)
```

--------------------------------

### Semantic Filtering and Aggregation in LOTUS

Source: https://github.com/lotus-data/lotus/blob/main/docs/examples.rst

Demonstrates filtering a DataFrame for machine learning courses based on descriptions and then aggregating the results to provide study tips. It uses pandas for data manipulation and LOTUS's semantic filtering and aggregation operators.

```python
# Dataset containing courses and their descriptions/workloads
data = [
    (
        "Probability and Random Processes",
        "Focuses on markov chains and convergence of random processes. The workload is pretty high.",
    ),
    (
        "Deep Learning",
        "Fouces on theory and implementation of neural networks. Workload varies by professor but typically isn't terrible.",
    ),
    (
        "Digital Design and Integrated Circuits",
        "Focuses on building RISC-V CPUs in Verilog. Students have said that the workload is VERY high.",
    ),
    (
        "Databases",
        "Focuses on implementation of a RDBMS with NoSQL topics at the end. Most students say the workload is not too high.",
    ),
]
df = pd.DataFrame(data, columns=["Course Name", "Description"])

# Applies semantic filter followed by semantic aggregation
ml_df = df.sem_filter("{Description} indicates that the class is relevant for machine learning.")
tips = ml_df.sem_agg(
    "Given each {Course Name} and its {Description}, give me a study plan to succeed in my classes."
)._output[0]
```

--------------------------------

### Load Data from Google BigQuery

Source: https://github.com/lotus-data/lotus/blob/main/docs/data_connectors.rst

Provides an example of loading data from Google BigQuery using DataConnector.load_from_db. The connection string includes the GCP project and dataset, and the code executes a BigQuery SQL query before applying a semantic filter.

```python
import lotus
from lotus.data_connectors import DataConnector
from lotus.models import LM

lm = LM(model="gpt-4o-mini")
lotus.settings.configure(lm=lm)

query = "SELECT date, MAX(title) as title, AVG(rating) as rating FROM movies GROUPBY date ORDERBY rating desc"
df = DataConnector.load_from_db("bigquery://my-gcp-project/my_dataset", query=query)

user_instruction = "{title} that are science fiction"
df = df.sem_filter(user_instruction)
print(df)
```

--------------------------------

### Tavily Web Search Example

Source: https://github.com/lotus-data/lotus/blob/main/docs/web_search.rst

Demonstrates using the `web_search` function with the Tavily corpus. Requires the `TAVILY_API_KEY` environment variable. It fetches results, selects 'title' and 'summary', and identifies the top 3 summaries that best explain AI ethics.

```python
import lotus
from lotus import WebSearchCorpus, web_search
from lotus.models import LM

lm = LM(model="gpt-4o-mini")

lotus.settings.configure(lm=lm)

df = web_search(WebSearchCorpus.TAVILY, "AI ethics in 2025", 10)[["title", "summary"]]
print(f"Results from Tavily:\n{df}\n")
top_tavily_articles = df.sem_topk("Which {summary} best explains ethical concerns in AI?", K=3)
print(f"Top 3 articles from Tavily on AI ethics:\n{top_tavily_articles}")
```

--------------------------------

### Chain of Thought (CoT) with Semantic Filter

Source: https://github.com/lotus-data/lotus/blob/main/docs/prompt_strategies.rst

Demonstrates using the Chain of Thought (CoT) reasoning strategy with the `sem_filter` operator in Lotus. This strategy guides the model through a step-by-step process to arrive at a more accurate and logical output, especially when provided with relevant examples.

```python
import pandas as pd

import lotus
from lotus.models import LM
from lotus.types import ReasoningStrategy

lm = LM(model="gpt-4o-mini")

lotus.settings.configure(lm=lm)
data = {
    "Course Name": [
        "Probability and Random Processes",
        "Optimization Methods in Engineering",
        "Digital Design and Integrated Circuits",
        "Computer Security",
    ]
}
df = pd.DataFrame(data)
user_instruction = "{Course Name} requires a lot of math"

example_data = {
    "Course Name": ["Machine Learning", "Reaction Mechanisms", "Nordic History"], 
    "Answer": [True, True, False],
    "Reasoning": ["Machine Learning requires a solid understanding of linear alebra and calculus",
                  "Reaction Engineering requires Ordinary Differential Equations to solve reactor design problems",
                  "Nordic History has no math involved"]
}
examples = pd.DataFrame(example_data)

df = df.sem_filter(user_instruction, examples=examples, strategy=ReasoningStrategy.COT)
print(df)
```

--------------------------------

### Install FAISS for Mac (GPU+CPU)

Source: https://github.com/lotus-data/lotus/blob/main/README.md

Installs the GPU-accelerated version of the FAISS library on macOS using Conda. This installation requires both PyTorch and NVIDIA channels.

```conda
conda install -c pytorch -c nvidia faiss-gpu=1.8.0
```

--------------------------------

### Semantic Map Operator in LOTUS

Source: https://github.com/lotus-data/lotus/blob/main/docs/examples.rst

Explains and demonstrates the semantic map operator in LOTUS, which is used to apply a transformation to each row of a DataFrame, potentially with provided examples. This example finds next topics to explore for each class.

```python
examples_df = pd.DataFrame(
    [("Computer Graphics", "Computer Vision"), ("Real Analysis", "Complex Analysis")],
    columns=["Course Name", "Answer"]
)
next_topics = df.sem_map(
    "Given {Course Name}, list a topic that will be good to explore next. \n        Respond with just the topic name and nothing else.", examples=examples_df, suffix="Next Topics"
)
```

--------------------------------

### Configure Lotus with LM, RM, and ReRanker

Source: https://github.com/lotus-data/lotus/blob/main/docs/reranker_models.rst

Demonstrates how to initialize and configure the Lotus project's settings with a Language Model (LM), a SentenceTransformers Retrieval Model (RM), and a CrossEncoderReranker. It also shows the initialization of a Faiss Vector Store (VS).

```python
import pandas as pd

import lotus
from lotus.models import LM, CrossEncoderReranker, SentenceTransformersRM
from lotus.vector_store import FaissVS

lm = LM(model="gpt-4o-mini")
rm = SentenceTransformersRM(model="intfloat/e5-base-v2")
reranker = CrossEncoderReranker(model="mixedbread-ai/mxbai-rerank-large-v1")
vs = FaissVS()

lotus.settings.configure(lm=lm, rm=rm, reranker=reranker, vs=vs)
```

--------------------------------

### Install FAISS for Mac (CPU-only)

Source: https://github.com/lotus-data/lotus/blob/main/README.md

Installs the CPU-only version of the FAISS library on macOS using Conda. FAISS is a library for efficient similarity search and clustering of dense vectors.

```conda
conda install -c pytorch faiss-cpu=1.8.0
```

--------------------------------

### Semantic Join Operator in LOTUS

Source: https://github.com/lotus-data/lotus/blob/main/docs/examples.rst

Demonstrates the use of LOTUS's semantic join operator to join two DataFrames based on a semantic predicate. This example finds courses that help improve specific skills.

```python
skills_df = pd.DataFrame(
    [("SQL"), ("Chip Design")], columns=["Skill"]
)
classes_for_skills = skills_df.sem_join(
    df, "Taking {Course Name} will make me better at {Skill}"
)
```

--------------------------------

### Semantic Top-K Operator in LOTUS

Source: https://github.com/lotus-data/lotus/blob/main/docs/examples.rst

Illustrates how to use LOTUS's semantic top-k operator to retrieve the top K rows from a DataFrame based on a semantic query. This example finds the top 2 courses with the highest workload.

```python
top_2_hardest = df.sem_topk("What {Description} indicates the highest workload?", K=2)
```

--------------------------------

### Zero-Shot Chain of Thought (ZS_COT) with Semantic Filter

Source: https://github.com/lotus-data/lotus/blob/main/docs/prompt_strategies.rst

Illustrates the use of the Zero-Shot Chain of Thought (ZS_COT) strategy with Lotus's `sem_filter` operator. This approach prompts the model to generate its own reasoning steps without explicit examples, useful for tasks where providing examples might be difficult or unnecessary.

```python
import pandas as pd

import lotus
from lotus.models import LM
from lotus.types import ReasoningStrategy

lm = LM(model="ollama/deepseek-r1:7b", temperature=0.6)
lotus.settings.configure(lm=lm)

data = {
    "Review": [
        "This vacuum cleaner is the best I've ever owned. Highly recommend it!",
        "It's okay, not sure I would buy it again.",
        "Terrible experience, broke after a few uses.",
        "Amazing build quality and customer support. Would absolutely recommend.",
    ]
}
df = pd.DataFrame(data)

user_instruction = "{Review} suggests that the user would recommend the product to others"

df = df.sem_filter(
    user_instruction, 
    strategy=ReasoningStrategy.ZS_COT, 
    return_explanations=True,
    return_all=True
)

print(df)
```

--------------------------------

### Set LM Usage Limits and Handle Exceptions

Source: https://github.com/lotus-data/lotus/blob/main/docs/usage.rst

Demonstrates how to set various usage limits (prompt tokens, completion tokens, total tokens, total cost) for an LM instance and how to catch the LotusUsageLimitException when a limit is exceeded. It includes setting up an LM with specific limits and processing data, with error handling for exceeded limits.

```python
from lotus.models import LM
from lotus.types import UsageLimit, LotusUsageLimitException
import pandas as pd

# Set limits
usage_limit = UsageLimit(
    prompt_tokens_limit=4000,
    completion_tokens_limit=1000,
    total_tokens_limit=3000,
    total_cost_limit=1.00
)
lm = LM(model="gpt-4o", physical_usage_limit=usage_limit)

try:
    course_df = pd.read_csv("course_df.csv")
    course_df = course_df.sem_filter("What {Course Name} requires a lot of math")
except LotusUsageLimitException as e:
    print(f"Usage limit exceeded: {e}")
    # Handle the exception as needed
```

--------------------------------

### Python sem_map Example

Source: https://github.com/lotus-data/lotus/blob/main/docs/sem_map.rst

Demonstrates how to use the sem_map operator to perform a semantic projection on a pandas DataFrame. It requires the 'lotus' library and a configured language model.

```python
import pandas as pd

import lotus
from lotus.models import LM

lm = LM(model="gpt-4o-mini")

lotus.settings.configure(lm=lm)
data = {
    "Course Name": [
        "Probability and Random Processes",
        "Optimization Methods in Engineering",
        "Digital Design and Integrated Circuits",
        "Computer Security",
    ]
    }
df = pd.DataFrame(data)
user_instruction = "What is a similar course to {Course Name}. Be concise."
df = df.sem_map(user_instruction)
print(df)
```

--------------------------------

### LOTUS Data Loading and DB Connectors Overview

Source: https://github.com/lotus-data/lotus/blob/main/docs/index.rst

This section details how to load data into LOTUS and connect to various databases using the provided data connectors.

```APIDOC
Data Loading and DB Connectors:

- data_connectors: General information about the data connectors available in LOTUS.
- DirectoryReader: A specific utility for reading data from directories.
```

--------------------------------

### Example of sem_filter Operator

Source: https://github.com/lotus-data/lotus/blob/main/docs/core_concepts.rst

Demonstrates how to use the sem_filter operator to filter a DataFrame based on a natural language expression. This operator keeps records that match the specified predicate.

```python
langex = "The {abstract} suggests that LLMs efficeintly utilize long context"
filtered_df = papers_df.sem_filter(langex)
```

--------------------------------

### Testing and Code Quality Checks

Source: https://github.com/lotus-data/lotus/blob/main/CONTRIBUTING.md

Commands to run the test suite, perform linting, and check type hints to ensure code quality and adherence to project standards.

```bash
# Run the test suite
pytest

# Run linting
pre-commit run --all-files

# Run type checking
mypy lotus/
```

--------------------------------

### Semantic Extraction without Quotes

Source: https://github.com/lotus-data/lotus/blob/main/docs/sem_extract.rst

Illustrates the usage of the sem_extract operator to extract structured data without including supporting quotes. This example shows how to omit descriptions for output columns.

```python
import pandas as pd

import lotus
from lotus.models import LM

lm = LM(model="gpt-4o-mini")
lotus.settings.configure(lm=lm)

df = pd.DataFrame(
    {
        "description": [
            "Yoshi is 25 years old",
            "Bowser is 45 years old",
            "Luigi is 15 years old",
        ]
    }
)
input_cols = ["description"]

# A description can also be omitted for each output column
output_cols = {
    "name": None,
    "age": None,
}
new_df = df.sem_extract(input_cols, output_cols)
print(new_df)
```

--------------------------------

### Documentation Tools

Source: https://github.com/lotus-data/lotus/blob/main/docs/requirements-docs.txt

Packages used for generating project documentation.

```python
sphinx==7.3.7
sphinx-rtd-theme==2.0.0
```

--------------------------------

### Configure Primary and Helper LMs

Source: https://github.com/lotus-data/lotus/blob/main/docs/configurations.rst

Demonstrates configuring both the primary language model (LM) and a secondary helper LM. This is useful when you need to manage multiple language models within the application.

```python
gpt_4o_mini = LM("gpt-4o-mini")
gpt_4o = LM("gpt-4o")

lotus.settings.configure(lm=gpt_4o, helper_lm=gpt_4o_mini)
```