### Install Dependencies and Run Development Server Source: https://context7.com/jiguuur/research_project_university_chatbot/llms.txt Commands to install project dependencies using npm and start the Next.js development server. The application will be available at http://localhost:3000. ```bash # Start the development server npm install npm run dev # Application available at http://localhost:3000 ``` -------------------------------- ### Start Development Server Source: https://github.com/jiguuur/research_project_university_chatbot/blob/main/README.md Launch the local development server to access the application at http://localhost:3000. ```bash npm run dev ``` -------------------------------- ### Install Project Dependencies Source: https://github.com/jiguuur/research_project_university_chatbot/blob/main/README.md Run this command in the root directory to install all required project packages. ```bash npm install ``` -------------------------------- ### Configure Environment Variables Source: https://github.com/jiguuur/research_project_university_chatbot/blob/main/README.md Create a .env.local file with the necessary API keys for OpenAI, Pinecone, and Supabase. ```env # Optional: OpenAI (Required for embeddings and LLM) OPENAI_API_KEY=your_openai_api_key # Pinecone (Required for Vector Storage) PINECONE_API_KEY=your_pinecone_api_key PINECONE_INDEX_NAME=your_pinecone_index_name # Supabase (Required for Metadata & Auth) NEXT_PUBLIC_SUPABASE_URL=your_supabase_project_url NEXT_PUBLIC_SUPABASE_ANON_KEY=your_supabase_anon_key SUPABASE_SERVICE_ROLE_KEY=your_supabase_service_role_key ``` -------------------------------- ### Environment Configuration for .env.local Source: https://context7.com/jiguuur/research_project_university_chatbot/llms.txt Lists the required environment variables for OpenAI, Pinecone, and Supabase services. These should be configured in a `.env.local` file at the project root. ```bash # .env.local configuration # OpenAI - Required for embeddings and LLM chat OPENAI_API_KEY=sk-your-openai-api-key # Pinecone - Required for vector storage (must use 1536 dimensions) PINECONE_API_KEY=your-pinecone-api-key PINECONE_INDEX_NAME=university-rag-index # Supabase - Required for metadata and authentication NEXT_PUBLIC_SUPABASE_URL=https://your-project.supabase.co NEXT_PUBLIC_SUPABASE_ANON_KEY=your-supabase-anon-key SUPABASE_SERVICE_ROLE_KEY=your-supabase-service-role-key ``` -------------------------------- ### Configure Supabase Clients Source: https://context7.com/jiguuur/research_project_university_chatbot/llms.txt Initialize browser or server-side clients to interact with PostgreSQL tables. Server-side clients require proper cookie handling for secure data access. ```typescript // Browser-side client (for React components) import { createSupabaseBrowserClient } from "@/lib/supabase/client"; const supabase = createSupabaseBrowserClient(); // Fetch documents list const { data: documents, error } = await supabase .from("documents") .select("*") .order("upload_date", { ascending: false }); // Server-side client (for API routes and server components) import { createSupabaseServerClient } from "@/lib/supabase/server"; const supabase = await createSupabaseServerClient(); // Insert a new document record const { error: insertError } = await supabase .from("documents") .insert({ filename: "handbook.pdf", file_type: "pdf", file_size: 2048576, status: "ready", chunk_count: 45, vector_count: 45 }); ``` -------------------------------- ### Implement React Chat Interface Source: https://context7.com/jiguuur/research_project_university_chatbot/llms.txt Integrate the chat interface using the Vercel AI SDK's useChat hook. Supports streaming responses and markdown rendering. ```typescript "use client"; import { useChat } from "@ai-sdk/react"; // Basic usage in a page component export default function ChatPage() { return ; } // The ChatInterface component internally uses: const { messages, sendMessage, status, error } = useChat({ api: "/api/chat", }); // Check loading state const isLoading = status === "submitted" || status === "streaming"; // Send a message programmatically const handleSend = () => { if (input.trim() && !isLoading) { sendMessage({ role: "user", content: input.trim() }); setInput(""); } }; // Messages are rendered with markdown support // using ReactMarkdown with remark-gfm plugin {messageContent} ``` -------------------------------- ### Ingest Documents via API Source: https://context7.com/jiguuur/research_project_university_chatbot/llms.txt Uploads PDF or DOCX files to the /api/ingest endpoint for processing and vector storage. ```bash # Upload a PDF document for ingestion into the knowledge base curl -X POST http://localhost:3000/api/ingest \ -F "file=@university-handbook.pdf" ``` -------------------------------- ### Chat with the University Chatbot Source: https://context7.com/jiguuur/research_project_university_chatbot/llms.txt Sends a user query to the /api/chat endpoint to receive a streaming response based on retrieved document context. ```bash # Send a chat message and receive a streaming response curl -X POST http://localhost:3000/api/chat \ -H "Content-Type: application/json" \ -d '{ "messages": [ { "role": "user", "content": "What is the tuition fee policy?" } ] }' ``` -------------------------------- ### Manage Document Uploads Source: https://context7.com/jiguuur/research_project_university_chatbot/llms.txt Use the DocumentUpload component for drag-and-drop file ingestion. Files are processed via POST requests to the ingestion API. ```typescript "use client"; import { DocumentUpload } from "@/components/admin/document-upload"; // Usage in admin page export default function AdminPage() { return (
); } // The component accepts PDF and DOCX files up to 10MB // Internally handles: // 1. File validation and queue management // 2. Progress tracking during upload // 3. POST to /api/ingest with FormData // 4. Success/error toast notifications // Example upload flow: const formData = new FormData(); formData.append("file", file); const response = await fetch("/api/ingest", { method: "POST", body: formData, }); const result = await response.json(); // { success: true, document: { chunkCount: 45, vectorCount: 45, ... } } ``` -------------------------------- ### Supabase PostgreSQL Database Schema Source: https://context7.com/jiguuur/research_project_university_chatbot/llms.txt Defines the tables for users, documents, and chat history, including primary keys, foreign keys, and constraints. Row Level Security is enabled for all tables. ```sql -- Users table for admin roles create table if not exists public.users ( id uuid primary key default gen_random_uuid(), email text not null unique, role text not null check (role in ('admin', 'user')), created_at timestamptz not null default now() ); -- Documents table for tracking uploaded files create table if not exists public.documents ( id uuid primary key default gen_random_uuid(), filename text not null, upload_date timestamptz not null default now(), chunk_count integer not null default 0, vector_count integer not null default 0, created_by uuid references public.users(id) on delete set null, created_at timestamptz not null default now() ); -- Chat history table for conversation tracking create table if not exists public.chat_history ( id uuid primary key default gen_random_uuid(), user_id uuid references public.users(id) on delete cascade, session_id text not null, message text not null, role text not null check (role in ('user', 'assistant', 'system')), created_at timestamptz not null default now() ); -- Enable Row Level Security alter table public.users enable row level security; alter table public.documents enable row level security; alter table public.chat_history enable row level security; ``` -------------------------------- ### Manage Pinecone Vector Database Operations Source: https://context7.com/jiguuur/research_project_university_chatbot/llms.txt Use these methods to upsert document embeddings with metadata and perform similarity searches. Ensure the index is properly configured and embeddings match the expected dimensions. ```typescript import { getPineconeClient, getPineconeIndex, type ChunkMetadata } from "@/lib/pinecone"; import type { PineconeRecord } from "@pinecone-database/pinecone"; // Get the configured Pinecone index const index = getPineconeIndex(); // Upsert document vectors with metadata const records: PineconeRecord[] = [ { id: "policy_pdf_1704067200000_chunk_0", values: embeddings[0], // 1536-dimensional vector metadata: { source: "policy.pdf", chunkIndex: 0, text: "The tuition fee for undergraduate programs...", documentId: "policy_pdf_1704067200000" } } ]; await index.upsert({ records }); // Query for similar documents const queryEmbedding = await generateEmbeddings(["What are the fees?"]); const searchResponse = await index.query({ vector: queryEmbedding[0], topK: 8, includeMetadata: true }); // Extract relevant text chunks from results const relevantChunks = searchResponse.matches .map((match) => match.metadata?.text) .filter((text): text is string => typeof text === "string"); ``` -------------------------------- ### Document Ingestion API Source: https://context7.com/jiguuur/research_project_university_chatbot/llms.txt Accepts document uploads (PDF, DOCX), processes them, and stores their vector representations in Pinecone. ```APIDOC ## POST /api/ingest ### Description This endpoint allows administrators to upload documents (PDF, DOCX) to be processed, chunked, embedded, and stored in the knowledge base. ### Method POST ### Endpoint /api/ingest ### Parameters #### Request Body - **file** (file) - Required - The document file to upload (PDF or DOCX). ### Request Example ```bash curl -X POST http://localhost:3000/api/ingest \ -F "file=@university-handbook.pdf" ``` ### Response #### Success Response (200) - **success** (boolean) - Indicates if the ingestion was successful. - **document** (object) - Contains details about the ingested document. - **id** (string) - Unique identifier for the document. - **filename** (string) - The original filename. - **fileType** (string) - The type of the file (e.g., 'pdf'). - **fileSize** (integer) - The size of the file in bytes. - **chunkCount** (integer) - The number of text chunks created. - **vectorCount** (integer) - The number of vectors generated. - **supabaseStatus** (string) - Status of metadata storage in Supabase. #### Error Response (400) - **error** (string) - Description of the error, e.g., 'Unsupported file type'. #### Response Example (Success) ```json { "success": true, "document": { "id": "university_handbook_pdf_1704067200000", "filename": "university-handbook.pdf", "fileType": "pdf", "fileSize": 2048576, "chunkCount": 45, "vectorCount": 45, "supabaseStatus": "saved" } } ``` #### Response Example (Error) ```json { "error": "Unsupported file type: .txt. Allowed: pdf, docx" } ``` ``` -------------------------------- ### Chat API Endpoint Source: https://context7.com/jiguuur/research_project_university_chatbot/llms.txt Handles user queries by retrieving relevant document chunks and streaming AI-generated responses. ```APIDOC ## POST /api/chat ### Description This endpoint processes user messages, retrieves relevant information from the knowledge base, and returns a streaming AI-generated response. ### Method POST ### Endpoint /api/chat ### Parameters #### Request Body - **messages** (array) - Required - An array of message objects, where each object has a 'role' (e.g., 'user') and 'content' (the message text). ### Request Example ```json { "messages": [ { "role": "user", "content": "What is the tuition fee policy?" } ] } ``` ### Response #### Success Response (200) - The response is a Server-Sent Events (SSE) stream containing the AI-generated answer. #### Response Example (SSE Stream) ``` data: {"text": "The tuition fee policy..."} data: {"text": "...further details can be found..."} ``` ``` -------------------------------- ### Document Parser Functions Source: https://context7.com/jiguuur/research_project_university_chatbot/llms.txt Utility functions for extracting text from PDF and DOCX files and chunking the text for RAG. ```APIDOC ## Document Parser Module ### Description Provides functions to extract text from various document formats and segment the text into manageable chunks suitable for retrieval. ### Functions - **extractText(buffer: Buffer, filename: string): Promise** - Extracts raw text content from a document buffer. - **parsePdf(buffer: Buffer): Promise** - Specifically parses PDF files. - **parseDocx(buffer: Buffer): Promise** - Specifically parses DOCX files. - **chunkText(text: string, options?: { chunkSize: number, chunkOverlap: number }): Promise>** - Chunks the provided text into smaller segments with optional overlap. ### Usage Example ```typescript import { extractText, chunkText, parsePdf, parseDocx } from "@/lib/document-parser"; import fs from 'fs'; // Extract text from any supported document type const buffer = fs.readFileSync("policy.pdf"); const rawText = await extractText(buffer, "policy.pdf"); // Returns: "University Policy Document\n\nSection 1: Admissions..." // Parse PDF directly const pdfBuffer = fs.readFileSync("handbook.pdf"); const pdfText = await parsePdf(pdfBuffer); // Parse DOCX directly const docxBuffer = fs.readFileSync("guidelines.docx"); const docxText = await parseDocx(docxBuffer); // Chunk text with custom settings for RAG pipeline const chunks = await chunkText(rawText, { chunkSize: 500, // Characters per chunk chunkOverlap: 100 // Overlap between chunks }); // Returns: [ // { text: "University Policy Document...", chunkIndex: 0 }, // { text: "...Section 1: Admissions...", chunkIndex: 1 }, // ... // ] ``` ``` -------------------------------- ### Parse and Chunk Documents Source: https://context7.com/jiguuur/research_project_university_chatbot/llms.txt Extracts text from documents and splits them into overlapping segments using the document-parser utility. ```typescript import { extractText, chunkText, parsePdf, parseDocx } from "@/lib/document-parser"; // Extract text from any supported document type const buffer = fs.readFileSync("policy.pdf"); const rawText = await extractText(buffer, "policy.pdf"); // Returns: "University Policy Document\n\nSection 1: Admissions..." // Parse PDF directly const pdfBuffer = fs.readFileSync("handbook.pdf"); const pdfText = await parsePdf(pdfBuffer); // Parse DOCX directly const docxBuffer = fs.readFileSync("guidelines.docx"); const docxText = await parseDocx(docxBuffer); // Chunk text with custom settings for RAG pipeline const chunks = await chunkText(rawText, { chunkSize: 500, // Characters per chunk chunkOverlap: 100 // Overlap between chunks }); // Returns: [ // { text: "University Policy Document...", chunkIndex: 0 }, // { text: "...Section 1: Admissions...", chunkIndex: 1 }, // ... // ] ``` -------------------------------- ### Embeddings Generation Source: https://context7.com/jiguuur/research_project_university_chatbot/llms.txt Generates vector embeddings for text using OpenAI's models, with automatic batching. ```APIDOC ## Embeddings Generation Module ### Description This module provides functionality to generate vector embeddings for text data using OpenAI's embedding models. It supports automatic batching to efficiently process large volumes of text. ### Function - **generateEmbeddings(texts: string[]): Promise** - Takes an array of text strings and returns a promise that resolves to an array of corresponding vector embeddings. ### Usage Example ```typescript import { generateEmbeddings } from "@/lib/embeddings"; // Generate embeddings for document chunks const texts = [ "The tuition fee for undergraduate programs is $5,000 per semester.", "Students must maintain a GPA of 2.0 to remain in good standing.", "Registration opens two weeks before the semester begins." ]; const embeddings = await generateEmbeddings(texts); // Returns: [ // [0.0123, -0.0456, 0.0789, ...], // 1536-dimensional vector // [0.0234, -0.0567, 0.0890, ...], // [0.0345, -0.0678, 0.0901, ...] // ] // Embeddings are automatically batched (512 texts per batch) // to stay within OpenAI API limits ``` ``` -------------------------------- ### Generate Text Embeddings Source: https://context7.com/jiguuur/research_project_university_chatbot/llms.txt Converts text strings into vector representations using the embeddings module, which handles batching automatically. ```typescript import { generateEmbeddings } from "@/lib/embeddings"; // Generate embeddings for document chunks const texts = [ "The tuition fee for undergraduate programs is $5,000 per semester.", "Students must maintain a GPA of 2.0 to remain in good standing.", "Registration opens two weeks before the semester begins." ]; const embeddings = await generateEmbeddings(texts); // Returns: [ // [0.0123, -0.0456, 0.0789, ...], // 1536-dimensional vector // [0.0234, -0.0567, 0.0890, ...], // [0.0345, -0.0678, 0.0901, ...] // ] // Embeddings are automatically batched (512 texts per batch) // to stay within OpenAI API limits ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.