### Install PySQLit Source: https://github.com/python51888/pysqlit/blob/master/docs/index.md Installs the PySQLit package using pip. This is the first step to get started with the library. ```bash pip install py-sqlit ``` -------------------------------- ### Build Real-time Data Pipeline with Async pysqlit Source: https://github.com/python51888/pysqlit/blob/master/docs/advanced-topics.md Provides an example of building a real-time data pipeline using pysqlit's asynchronous support and aiohttp. It demonstrates ingesting data from a URL, buffering it, and processing it in batches for insertion into the database. ```python import asyncio import aiohttp from pysqlit.async_support import AsyncDatabase from datetime import datetime class RealtimeDataPipeline: """实时数据管道""" def __init__(self, db_path: str): self.db = AsyncDatabase(db_path) self.buffer = asyncio.Queue(maxsize=1000) async def ingest_data(self, source_url: str): """数据摄取""" async with aiohttp.ClientSession() as session: async with session.get(source_url) as response: async for line in response.content: await self.buffer.put(line) async def process_data(self): """数据处理""" while True: batch = [] for _ in range(100): # 批处理100条 try: item = await asyncio.wait_for( self.buffer.get(), timeout=1.0 ) batch.append(item) except asyncio.TimeoutError: break if batch: await self.db.executemany( "INSERT INTO events (data, timestamp) VALUES (?, ?)", [(item, datetime.now()) for item in batch] ) ``` -------------------------------- ### Running Python SQLite Examples Source: https://github.com/python51888/pysqlit/blob/master/examples/汇总.md Provides instructions on how to run individual or all Python SQLite example scripts. Includes commands for executing single files and running them as modules. ```bash python 01_basic_connection.py python -m 02_insert_and_query.py ``` -------------------------------- ### Command-Line Interface Tools Source: https://github.com/python51888/pysqlit/blob/master/docs/index.md Provides examples of using PySQLit's command-line tools for interactive sessions, running SQL scripts, and checking database integrity. ```bash # Start interactive REPL python -m pysqlit.repl mydb.db # Run SQL script python -m pysqlit.cli --file script.sql # Database check python -m pysqlit.check --db mydb.db ``` -------------------------------- ### Install PySQLit Dependencies Source: https://github.com/python51888/pysqlit/blob/master/README.md Installs project dependencies using pip. This involves cloning the repository, creating a virtual environment, and then installing the required packages. ```bash git clone https://gitee.com/Python51888/PySqlit.git cd py-sqlit python -m venv venv source venv/bin/activate # Linux/Mac # 或 venv\Scripts\activate # Windows pip install -r requirements.txt ``` -------------------------------- ### Logging Setup for Debugging Source: https://github.com/python51888/pysqlit/blob/master/docs/development.md Illustrates how to configure and use Python's `logging` module with PySqlit's `setup_debug_logging` function. It shows how to set the logging level and use a logger instance to output debug messages for executed SQL queries. ```python import logging from pysqlit.debug import setup_debug_logging # 设置调试日志 setup_debug_logging(level=logging.DEBUG) # 使用日志 logger = logging.getLogger(__name__) class DatabaseManager: def execute(self, sql: str, params: tuple = (): logger.debug(f"Executing SQL: {sql} with params: {params}") # ... 执行逻辑 ... ``` -------------------------------- ### PySqlit Benchmark Suite Implementation Source: https://github.com/python51888/pysqlit/blob/master/docs/development.md Demonstrates the `BenchmarkSuite` class for conducting performance tests. It includes methods for setting up tests, running them over multiple iterations, and collecting statistics like mean, median, and standard deviation. The example shows how to benchmark insert operations. ```python import time import statistics from typing import List, Callable from pysqlit.database import EnhancedDatabase class BenchmarkSuite: """性能基准测试套件""" def __init__(self, iterations: int = 5): self.iterations = iterations self.results = {} def benchmark( self, name: str, setup: Callable, test: Callable, teardown: Callable = None ) -> float: """运行基准测试""" times = [] for _ in range(self.iterations): if setup: setup() start = time.perf_counter() test() elapsed = time.perf_counter() - start times.append(elapsed) if teardown: teardown() avg_time = statistics.mean(times) self.results[name] = { 'mean': avg_time, 'median': statistics.median(times), 'stdev': statistics.stdev(times) if len(times) > 1 else 0, 'min': min(times), 'max': max(times) } return avg_time def run_all(self): """运行所有基准测试""" # 插入性能测试 self.benchmark_insert_performance() self.benchmark_query_performance() self.benchmark_transaction_performance() return self.results def benchmark_insert_performance(self): """测试插入性能""" def setup(): self.db = EnhancedDatabase(":memory:") self.db.execute("CREATE TABLE test (id INTEGER, name TEXT, age INTEGER)") def test(): data = [(i, f"User{i}", i % 100) for i in range(1000)] self.db.executemany("INSERT INTO test VALUES (?, ?, ?)", data) def teardown(): self.db.close() self.benchmark("insert_1000_rows", setup, test, teardown) ``` -------------------------------- ### Makefile for Build and Release Source: https://github.com/python51888/pysqlit/blob/master/docs/development.md A Makefile defining common development tasks for the Python project. It includes targets for installation, setting up development environments, running tests, linting, formatting, cleaning build artifacts, building distributions, and uploading packages to PyPI. ```makefile .PHONY: install dev test lint format clean build upload # 安装 install: pip install -e . # 开发环境 dev: pip install -e ".[dev]" pre-commit install # 运行测试 test: pytest tests/ -v --cov=pysqlit --cov-report=html # 代码检查 lint: flake8 pysqlit/ tests/ mypy pysqlit/ pylint pysqlit/ # 格式化 format: black pysqlit/ tests/ isort pysqlit/ tests/ # 清理 clean: find . -type f -name "*.pyc" -delete find . -type d -name "__pycache__" -delete rm -rf build/ dist/ *.egg-info/ # 构建 build: python -m build # 上传到测试PyPI upload-test: python -m twine upload --repository testpypi dist/* # 上传到PyPI upload: python -m twine upload dist/* ``` -------------------------------- ### Profile Database Operations with cProfile Source: https://github.com/python51888/pysqlit/blob/master/docs/advanced-topics.md Illustrates how to use Python's built-in cProfile module to analyze the performance of database operations. It profiles the execution time of various database tasks like table creation, insertions, and selections. ```python # 使用cProfile进行性能分析 import cProfile import pstats def profile_database_operations(): """分析数据库操作性能""" profiler = cProfile.Profile() profiler.enable() # 执行数据库操作 db = EnhancedDatabase("profile.db") db.execute("CREATE TABLE test (id INTEGER, data TEXT)") for i in range(1000): db.execute("INSERT INTO test VALUES (?, ?)", (i, f"data{i}")) results = db.execute("SELECT * FROM test WHERE id > 500") profiler.disable() # 输出分析结果 stats = pstats.Stats(profiler) stats.sort_stats('cumulative') stats.print_stats(20) ``` -------------------------------- ### Analyze Memory Usage with tracemalloc Source: https://github.com/python51888/pysqlit/blob/master/docs/advanced-topics.md Demonstrates how to start memory tracing using Python's `tracemalloc` module. It captures a snapshot of memory allocations and prints the top 10 memory-consuming lines of code, aiding in identifying memory leaks or high usage patterns within PySQLit operations. ```python import tracemalloc from pysqlit.database import EnhancedDatabase # 启动内存跟踪 tracemalloc.start() # 创建数据库并执行操作 db = EnhancedDatabase("large_dataset.db") # 分析内存使用 snapshot = tracemalloc.take_snapshot() top_stats = snapshot.statistics('lineno') print("内存使用TOP 10:") for stat in top_stats[:10]: print(stat) ``` -------------------------------- ### pysqlit: Composite Conditions Example Source: https://github.com/python51888/pysqlit/blob/master/docs/api-reference.md Demonstrates how to create complex query conditions using AndCondition and OrCondition from the pysqlit library. This allows for building sophisticated WHERE clauses programmatically. ```python from pysqlit.models import AndCondition, OrCondition complex_condition = AndCondition([ WhereCondition("age", ">", 18), OrCondition([ WhereCondition("status", "=", "active"), WhereCondition("role", "=", "admin") ]) ]) ``` -------------------------------- ### pysqlit: Exception Hierarchy and Handling Source: https://github.com/python51888/pysqlit/blob/master/docs/api-reference.md Illustrates the exception hierarchy in pysqlit, which categorizes errors related to database operations, storage, parsing, and backups. Includes examples of catching specific exceptions like ParseError and TransactionError. ```python from pysqlit.exceptions import ( DatabaseError, StorageError, ParseError, TransactionError ) try: db.execute("INVALID SQL") except ParseError as e: print(f"SQL语法错误: {e}") try: db.execute("INSERT INTO users VALUES (1, NULL)") except DatabaseError as e: print(f"数据库错误: {e}") try: with db.transaction(): db.execute("UPDATE nonexistent SET x = 1") except TransactionError as e: print(f"事务错误: {e}") ``` -------------------------------- ### tracemalloc for Memory Debugging Source: https://github.com/python51888/pysqlit/blob/master/docs/development.md Explains how to use Python's `tracemalloc` module to track memory allocations. It covers starting the tracer, performing operations, taking a snapshot of memory usage, and printing the top memory consumers by line number. ```python import tracemalloc # 开始内存跟踪 tracemalloc.start() # 执行操作 db = EnhancedDatabase("test.db") # ... 大量操作 ... # 获取内存快照 snapshot = tracemalloc.take_snapshot() top_stats = snapshot.statistics('lineno') print("[ Top 10 memory consumers ]") for stat in top_stats[:10]: print(stat) ``` -------------------------------- ### Database Transaction Example Source: https://github.com/python51888/pysqlit/blob/master/docs/index.md Illustrates how to perform database operations within a transaction context. The 'with db.transaction() as tx:' block ensures that operations are either committed or rolled back automatically. ```python # Transaction example with db.transaction() as tx: # Transfer operation db.execute("UPDATE accounts SET balance = balance - ? WHERE id = ?", (100, 1)) db.execute("UPDATE accounts SET balance = balance + ? WHERE id = ?", (100, 2)) # Auto commit or rollback ``` -------------------------------- ### Analyze Query Patterns for Indexing Source: https://github.com/python51888/pysqlit/blob/master/docs/advanced-topics.md Introduces an `IndexOptimizer` class with a method to analyze query patterns. It parses SQL queries to extract `WHERE` conditions, `JOIN` clauses, and `ORDER BY` columns, providing insights into query behavior for effective indexing. ```python class IndexOptimizer: """索引优化器""" def analyze_query_patterns(self, queries): """分析查询模式""" patterns = {} for query in queries: # 提取WHERE条件 where_conditions = self.extract_where_conditions(query) # 提取JOIN条件 join_conditions = self.extract_join_conditions(query) # 提取ORDER BY order_columns = self.extract_order_columns(query) patterns[query] = { 'where': where_conditions, 'join': join_conditions, 'order': order_columns } return patterns def recommend_indexes(self, patterns): """推荐索引""" recommendations = [] for query, pattern in patterns.items(): # 基于查询频率和选择性推荐索引 if pattern['where']: for col, op in pattern['where']: if self.is_selective(col): recommendations.append({ 'table': self.get_table_from_query(query), 'columns': [col], 'type': 'btree', 'reason': f'WHERE {col} {op}' }) return recommendations ``` -------------------------------- ### Optimize Pagination with Keyset Pagination Source: https://github.com/python51888/pysqlit/blob/master/docs/advanced-topics.md Demonstrates an `optimize_pagination` method within `QueryRewriter`. It suggests switching from traditional `OFFSET` based pagination to keyset pagination for large datasets when the offset exceeds a certain threshold, improving performance for deep pagination. ```python class QueryRewriter: """查询重写器""" def optimize_pagination(self, query, offset, limit): """优化分页查询""" # 使用键集分页替代OFFSET if offset > 1000: return self.keyset_pagination(query, offset, limit) return query ``` -------------------------------- ### Python SQLite: Basic Connection and Table Creation Source: https://github.com/python51888/pysqlit/blob/master/examples/汇总.md Demonstrates establishing a connection to an SQLite database, creating user and product tables, and inspecting the database structure. Essential for setting up the database environment. ```python 01_basic_connection.py ``` -------------------------------- ### pyproject.toml Build Configuration Source: https://github.com/python51888/pysqlit/blob/master/docs/development.md This TOML file specifies the build system requirements, project metadata such as name, version, description, author, and license. It also lists optional development dependencies and tool-specific configurations for Black, isort, mypy, and pytest. ```toml [build-system] requires = ["setuptools>=61.0", "wheel"] build-backend = "setuptools.build_meta" [project] name = "py-sqlit" version = "1.0.0" description = "Enterprise-grade SQLite database engine in Python" readme = "README.md" requires-python = ">=3.8" license = {text = "MIT"} authors = [ {name = "Your Name", email = "your.email@example.com"}, ] keywords = ["database", "sqlite", "sql", "storage"] classifiers = [ "Development Status :: 4 - Beta", "Intended Audience :: Developers", "License :: OSI Approved :: MIT License", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.8", "Programming Language :: Python :: 3.9", "Programming Language :: Python :: 3.10", "Programming Language :: Python :: 3.11", "Topic :: Database", "Topic :: Database :: Database Engines/Servers", ] [project.optional-dependencies] dev = [ "pytest>=7.0.0", "pytest-cov>=4.0.0", "pytest-mock>=3.10.0", "black>=23.0.0", "isort>=5.12.0", "mypy>=1.0.0", "flake8>=6.0.0", "pylint>=2.17.0", "pre-commit>=3.0.0", "tox>=4.0.0", "sphinx>=6.0.0", "sphinx-rtd-theme>=1.2.0", ] [project.urls] Homepage = "https://gitee.com/Python51888/PySqlit" Documentation = "https://py-sqlit.readthedocs.io/" Repository = "https://gitee.com/Python51888/PySqlit.git" Issues = "https://gitee.com/Python51888/PySqlit/issues" [tool.setuptools.packages.find] where = ["."] include = ["pysqlit*"] [tool.black] line-length = 88 target-version = ['py38'] include = '\.pyi?$' [tool.isort] profile = "black" multi_line_output = 3 line_length = 88 [tool.mypy] python_version = "3.8" warn_return_any = true warn_unused_configs = true disallow_untyped_defs = true disallow_incomplete_defs = true check_untyped_defs = true no_implicit_optional = true warn_redundant_casts = true warn_unused_ignores = true warn_no_return = true warn_unreachable = true strict_equality = true [tool.pytest.ini_options] testpaths = ["tests"] python_files = ["test_*.py"] python_classes = ["Test*"] python_functions = ["test_*"] addopts = "-v --tb=short --strict-markers" markers = [ "slow: marks tests as slow", "integration: marks tests as integration tests", "performance: marks tests as performance tests", ] [tool.coverage.run] source = ["pysqlit"] omit = ["*/tests/*", "*/test_*"] [tool.coverage.report] exclude_lines = [ "pragma: no cover", "def __repr__", "raise AssertionError", "raise NotImplementedError", ] ``` -------------------------------- ### pysqlit: Connection Pool Management Source: https://github.com/python51888/pysqlit/blob/master/docs/api-reference.md Demonstrates the usage of ConnectionPool for managing database connections efficiently. It shows how to create a pool, acquire connections using context managers, and execute queries. ```python from pysqlit.pool import ConnectionPool # 创建连接池 pool = ConnectionPool( max_connections=10, database_path="myapp.db", timeout=5.0 ) # 获取连接 with pool.get_connection() as db: users = db.execute("SELECT * FROM users LIMIT 10") # 或使用上下文管理器 with pool.context() as db: db.execute("INSERT INTO users VALUES (?, ?)", (1, "Alice")) ``` -------------------------------- ### Rewrite Subqueries to Joins Source: https://github.com/python51888/pysqlit/blob/master/docs/advanced-topics.md Shows a `QueryRewriter` class with a method `rewrite_subquery_to_join`. This function aims to convert inefficient subqueries (like those using `IN`) into more performant JOIN operations, improving query execution speed. ```python class QueryRewriter: """查询重写器""" def rewrite_subquery_to_join(self, query): """将子查询重写为JOIN""" # 示例:将IN子查询重写为JOIN # 原查询: SELECT * FROM users WHERE id IN (SELECT user_id FROM orders) # 重写后: SELECT DISTINCT u.* FROM users u JOIN orders o ON u.id = o.user_id pass ``` -------------------------------- ### Project Release Process Commands Source: https://github.com/python51888/pysqlit/blob/master/docs/development.md A sequence of bash commands outlining the steps for releasing a new version of the project. This includes updating version numbers, running tests and linters, building distribution packages, uploading to PyPI (test and production), and creating Git tags. ```bash # 1. 更新版本号 # 编辑 pyproject.toml 中的 version # 2. 更新CHANGELOG.md # 添加新版本说明 # 3. 运行完整测试 make test make lint # 4. 构建发布包 make build # 5. 上传到测试PyPI make upload-test # 6. 测试安装 pip install --index-url https://test.pypi.org/simple/ py-sqlit # 7. 上传到正式PyPI make upload # 8. 创建Git标签 git tag v1.0.0 git push origin v1.0.0 ``` -------------------------------- ### Enable Debugging Mode in pysqlit Source: https://github.com/python51888/pysqlit/blob/master/docs/advanced-topics.md Shows how to enable and utilize the debugging features of pysqlit, including detailed query logging and setting breakpoints for specific SQL statements. This aids in troubleshooting and understanding database interactions. ```python # 启用调试模式 import logging from pysqlit.debug import DebugDatabase # 配置调试日志 logging.basicConfig( level=logging.DEBUG, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s' ) # 创建调试数据库 db = DebugDatabase("app.db", debug=True) # 启用查询日志 db.enable_query_logging() # 设置断点 db.set_breakpoint("SELECT * FROM users WHERE id = 42") # 执行查询 result = db.execute("SELECT * FROM users") ``` -------------------------------- ### Create Database and Table Source: https://github.com/python51888/pysqlit/blob/master/docs/index.md Demonstrates how to create a new SQLite database file, define a 'users' table with various constraints (primary key, unique, not null, check, default timestamp), and insert initial data. ```python from pysqlit.database import EnhancedDatabase # Create database db = EnhancedDatabase("myapp.db") # Create users table db.execute(""" CREATE TABLE users ( id INTEGER PRIMARY KEY AUTOINCREMENT, username TEXT UNIQUE NOT NULL, email TEXT UNIQUE NOT NULL, age INTEGER CHECK(age > 0), created_at DATETIME DEFAULT CURRENT_TIMESTAMP ) """) # Insert test data db.execute("INSERT INTO users (username, email, age) VALUES (?, ?, ?)", ("alice", "alice@example.com", 25)) # Query data users = db.execute("SELECT * FROM users WHERE age > ?", (20,)) for user in users: print(f"User: {user['username']}, Email: {user['email']}") db.close() ``` -------------------------------- ### Check Memory Leaks with Python Source: https://github.com/python51888/pysqlit/blob/master/docs/advanced-topics.md Demonstrates how to check for potential memory leaks in Python applications using the psutil library and garbage collection. It measures memory usage before and after database operations to detect significant increases. ```python import gc import psutil def check_memory_leaks(): """检查内存泄漏""" process = psutil.Process() # 获取初始内存使用 initial_memory = process.memory_info().rss # 执行大量操作 db = EnhancedDatabase("test.db") for i in range(10000): db.execute("INSERT INTO test VALUES (?, ?)", (i, f"data{i}")) # 关闭数据库 db.close() # 强制垃圾回收 gc.collect() # 检查内存是否释放 final_memory = process.memory_info().rss if final_memory > initial_memory * 1.1: print("Potential memory leak detected") return False return True ``` -------------------------------- ### Database Backup and Restore Source: https://github.com/python51888/pysqlit/blob/master/docs/index.md Shows how to create a backup of the current database and restore it later. The `create_backup` method generates a backup file, and `restore_backup` reverts the database to a previous state. ```python # Create backup backup_name = db.create_backup("daily_backup") print(f"Backup created: {backup_name}") # Restore from backup db.restore_backup("daily_backup") ``` -------------------------------- ### Configure Advanced Connection Pooling Source: https://github.com/python51888/pysqlit/blob/master/docs/advanced-topics.md Details advanced configuration options for PySQLit's `ConnectionPool`. It shows how to set parameters like `max_connections`, `min_connections`, `connection_timeout`, `idle_timeout`, `max_lifetime`, and `health_check_interval` for robust connection management. ```python from pysqlit.pool import ConnectionPool, PoolConfig # 高级连接池配置 config = PoolConfig( max_connections=50, min_connections=5, connection_timeout=30, idle_timeout=300, max_lifetime=3600, health_check_interval=60 ) pool = ConnectionPool( database_path="production.db", config=config ) # 监控连接池状态 def monitor_pool(pool): stats = pool.get_stats() print(f""" 活跃连接: {stats['active_connections']} 空闲连接: {stats['idle_connections']} 等待连接: {stats['waiting_connections']} 总连接数: {stats['total_connections']} """) ``` -------------------------------- ### cProfile for Performance Analysis Source: https://github.com/python51888/pysqlit/blob/master/docs/development.md Details how to use Python's built-in `cProfile` module for profiling code execution. It shows commands to run a Python module with `cProfile`, save the stats, and then analyze them to identify performance bottlenecks. ```bash # 性能分析 python -m cProfile -o profile.stats -m pysqlit.repl test.db # 查看分析结果 python -c " import pstats p = pstats.Stats('profile.stats') p.sort_stats('cumulative').print_stats(20) " ``` -------------------------------- ### Implement Distributed Cache with Redis and pysqlit Source: https://github.com/python51888/pysqlit/blob/master/docs/advanced-topics.md Details how to create a cached database layer using pysqlit and Redis. It implements a caching mechanism for frequently accessed data, such as user information, to improve performance by reducing database load. ```python import redis import json from functools import lru_cache # Assuming EnhancedDatabase is defined elsewhere and works with pysqlit # class EnhancedDatabase: # def __init__(self, db_path: str): # pass # def execute(self, query, params=None): # # Dummy implementation for example # print(f"Executing: {query} with {params}") # if 'users' in query and params and params[0] == 123: # return [{'id': 123, 'name': 'Test User'}] # return [] class CachedDatabase: """带缓存的数据库""" def __init__(self, db_path: str, redis_url: str): # Assuming EnhancedDatabase is a valid class for database operations self.db = EnhancedDatabase(db_path) self.cache = redis.from_url(redis_url) self.cache_ttl = 300 # 5分钟 @lru_cache(maxsize=1000) def get_user(self, user_id: int): """获取用户信息(带缓存)""" cache_key = f"user:{user_id}" # 尝试从缓存获取 cached = self.cache.get(cache_key) if cached: return json.loads(cached) # 从数据库获取 user = self.db.execute( "SELECT * FROM users WHERE id = ?", (user_id,) )[0] # 写入缓存 self.cache.setex( cache_key, self.cache_ttl, json.dumps(user) ) return user ``` -------------------------------- ### PySQLit EnhancedDatabase Transaction Management Source: https://github.com/python51888/pysqlit/blob/master/docs/api-reference.md Illustrates transaction handling within EnhancedDatabase, showing manual transaction control with begin, commit, and rollback, as well as the usage of context managers for simplified transaction management. ```python # Manual transaction tx_id = db.begin_transaction() try: db.execute("UPDATE accounts SET balance = balance - 100 WHERE id = 1") db.execute("UPDATE accounts SET balance = balance + 100 WHERE id = 2") db.commit_transaction(tx_id) except Exception as e: db.rollback_transaction(tx_id) raise # Context manager with db.transaction() as tx: db.execute("INSERT INTO users VALUES (?, ?)", (1, "Alice")) db.execute("INSERT INTO users VALUES (?, ?)", (2, "Bob")) ``` -------------------------------- ### Create Blog Posts Table Source: https://github.com/python51888/pysqlit/blob/master/docs/index.md Defines a 'posts' table for a blog system, including fields for title, content, author, status, and creation timestamp. It also sets up foreign key constraints and indexes for efficient querying. ```python # Create blog database db.execute(""" CREATE TABLE posts ( id INTEGER PRIMARY KEY AUTOINCREMENT, title TEXT NOT NULL, content TEXT, author_id INTEGER, status TEXT DEFAULT 'draft', created_at DATETIME DEFAULT CURRENT_TIMESTAMP, FOREIGN KEY (author_id) REFERENCES users(id) ) """) # Create indexes db.execute("CREATE INDEX idx_posts_author ON posts(author_id)") db.execute("CREATE INDEX idx_posts_status ON posts(status)") ``` -------------------------------- ### pysqlit: Migration Manager for Schema Changes Source: https://github.com/python51888/pysqlit/blob/master/docs/api-reference.md Details how to manage database schema migrations using the MigrationManager. It shows how to add migration scripts and apply them to the database. ```python from pysqlit.migration import MigrationManager # 创建迁移 migration = MigrationManager(db) # 添加迁移 migration.add_migration("001_add_users_table", """ CREATE TABLE users ( id INTEGER PRIMARY KEY AUTOINCREMENT, username TEXT UNIQUE NOT NULL, email TEXT UNIQUE NOT NULL ) """) # 执行迁移 migration.migrate() ``` -------------------------------- ### pysqlit: TestDatabase for Isolated Testing Source: https://github.com/python51888/pysqlit/blob/master/docs/api-reference.md Demonstrates the use of TestDatabase for creating isolated and automatically cleaned-up database environments for testing. It simplifies setting up test data and running tests. ```python from pysqlit.testing import TestDatabase # 创建测试数据库 test_db = TestDatabase() # 自动清理 with test_db as db: db.execute("CREATE TABLE test (id INTEGER, name TEXT)") db.execute("INSERT INTO test VALUES (1, 'Alice')") # 测试结束后自动清理 ``` -------------------------------- ### Python API Imports Source: https://github.com/python51888/pysqlit/blob/master/docs/index.md Demonstrates importing key classes from the PySQLit library for advanced usage, including database management, connection pooling, and ORM-style models. ```python # Advanced API from pysqlit.database import EnhancedDatabase # Connection pool from pysqlit.pool import ConnectionPool # ORM style from pysqlit.models import Table, Row ``` -------------------------------- ### Implement Read-Write Separation Strategy Source: https://github.com/python51888/pysqlit/blob/master/docs/advanced-topics.md Presents a Python class `ReadWriteRouter` for implementing read-write separation. It manages a master database for writes and multiple replica databases for reads, distributing read requests across replicas to improve concurrency and availability. ```python class ReadWriteRouter: """读写分离路由器""" def __init__(self, master_db, replica_dbs): self.master = master_db self.replicas = replica_dbs self.current_replica = 0 def get_read_db(self): """获取读数据库""" replica = self.replicas[self.current_replica] self.current_replica = (self.current_replica + 1) % len(self.replicas) return replica def get_write_db(self): """获取写数据库""" return self.master def execute_read(self, sql, params=()): return self.get_read_db().execute(sql, params) def execute_write(self, sql, params=()): return self.master.execute(sql, params) ``` -------------------------------- ### Tune PySQLit Caching and Page Size Source: https://github.com/python51888/pysqlit/blob/master/docs/advanced-topics.md Illustrates how to configure PySQLit's database settings for performance. It shows adjusting `cache_size` for larger datasets, `page_size` to reduce I/O operations, and `max_memory_usage` to limit the database's memory footprint. ```python from pysqlit.config import DatabaseConfig # 根据数据量调整缓存 config = DatabaseConfig( cache_size=1000, # 大数据集使用更大缓存 page_size=8192, # 大页减少I/O次数 max_memory_usage="512MB" # 限制内存使用 ) db = EnhancedDatabase("bigdata.db", config=config) ``` -------------------------------- ### Performance Testing Execution Commands Source: https://github.com/python51888/pysqlit/blob/master/docs/development.md Provides commands to execute performance tests. This includes running the benchmark script directly, using pytest-benchmark for more advanced testing, and generating performance reports in JSON format. ```bash # 运行性能测试 python benchmarks/benchmark.py # 使用pytest-benchmark pytest tests/test_performance.py --benchmark-only # 生成性能报告 pytest tests/test_performance.py --benchmark-json=benchmark.json ``` -------------------------------- ### pysqlit: MockDataGenerator for Test Data Source: https://github.com/python51888/pysqlit/blob/master/docs/api-reference.md Explains how to use MockDataGenerator to populate a database with realistic test data. It allows generating large volumes of mock records for tables like users and posts. ```python from pysqlit.testing import MockDataGenerator # 生成测试数据 generator = MockDataGenerator(db) generator.generate_users(count=1000) generator.generate_posts(count=5000) ``` -------------------------------- ### Handle Database Errors with pysqlit Exceptions Source: https://github.com/python51888/pysqlit/blob/master/docs/usage-guide.md Provides an example of robust error handling for database operations using pysqlit's custom exceptions. It shows how to catch specific errors like ParseError, DatabaseError (e.g., unique constraint violations), and general exceptions. ```python from pysqlit.exceptions import ( DatabaseError, StorageError, ParseError, TransactionError ) # Assuming 'db' is an initialized EnhancedDatabase instance and 'logger' is configured # def safe_insert_user(db, username, email): # try: # user_id = db.execute( # "INSERT INTO users (username, email) VALUES (?, ?)", # (username, email), # return_id=True # ) # return user_id # # except ParseError as e: # logger.error(f"SQL语法错误: {e}") # raise # # except DatabaseError as e: # if "UNIQUE constraint failed" in str(e): # logger.error(f"用户名或邮箱已存在: {username}, {email}") # raise ValueError("用户名或邮箱已存在") # else: # logger.error(f"数据库错误: {e}") # raise # # except Exception as e: # logger.error(f"未知错误: {e}") # raise ``` -------------------------------- ### Run PySQLit Tests with Pytest Source: https://github.com/python51888/pysqlit/blob/master/tests/TEST_SUMMARY.md This snippet shows how to execute all tests for the PySQLit project using the pytest framework. It covers running all tests, targeting specific files, and enabling verbose output for detailed logging. ```bash # Run all tests python -m pytest tests/ ``` ```bash # Run specific module tests python -m pytest tests/test_database.py ``` ```bash # Verbose output python -m pytest tests/ -v ``` -------------------------------- ### Optimize Batch Operations in PySQLit Source: https://github.com/python51888/pysqlit/blob/master/docs/advanced-topics.md Provides a function to find the optimal batch size for bulk insert operations using `executemany`. It benchmarks different batch sizes to determine which one yields the fastest execution time, crucial for large data imports. ```python # 最优批量大小测试 def find_optimal_batch_size(db, data_size=10000): batch_sizes = [100, 500, 1000, 2000, 5000] results = {} for batch_size in batch_sizes: start = time.time() for i in range(0, data_size, batch_size): batch = data[i:i+batch_size] db.executemany( "INSERT INTO test VALUES (?, ?, ?)", batch ) elapsed = time.time() - start results[batch_size] = elapsed return min(results, key=results.get) ``` -------------------------------- ### Basic PySQLit Database Operations Source: https://github.com/python51888/pysqlit/blob/master/README.md Demonstrates core functionalities including establishing a database connection, executing SQL for table creation and data insertion, managing transactions, querying data, and creating backups. ```python from pysqlit.database import EnhancedDatabase # 创建数据库连接 db = EnhancedDatabase("myapp.db") # 创建表 db.execute(""" CREATE TABLE users ( id INTEGER PRIMARY KEY AUTOINCREMENT, username TEXT UNIQUE NOT NULL, email TEXT UNIQUE NOT NULL, created_at DATETIME DEFAULT CURRENT_TIMESTAMP ) """) # 插入数据 user_id = db.execute( "INSERT INTO users (username, email) VALUES (?, ?)", ("alice", "alice@example.com") ) # 事务操作 with db.transaction(): db.execute("UPDATE users SET email = ? WHERE id = ?", ("new@example.com", user_id)) # 查询数据 users = db.execute("SELECT * FROM users WHERE username LIKE ?", ("ali%",)) for user in users: print(f"User: {user['username']}, Email: {user['email']}") # 创建备份 backup_path = db.create_backup("daily_backup") print(f"Backup created: {backup_path}") db.close() ``` -------------------------------- ### LRUCache Implementation in Python Source: https://github.com/python51888/pysqlit/blob/master/docs/architecture.md Implements a Least Recently Used (LRU) cache strategy using Python's OrderedDict. It allows for setting a capacity and provides methods to get and put key-value pairs, automatically evicting the least recently used item when the capacity is reached. ```python from collections import OrderedDict from typing import Any class LRUCache: """LRU缓存实现""" def __init__(self, capacity: int): self.capacity = capacity self.cache = OrderedDict() def get(self, key: str) -> Any: if key in self.cache: self.cache.move_to_end(key) return self.cache[key] return None def put(self, key: str, value: Any): if key in self.cache: self.cache.move_to_end(key) else: if len(self.cache) >= self.capacity: self.cache.popitem(last=False) self.cache[key] = value ``` -------------------------------- ### line_profiler for Line-by-Line Analysis Source: https://github.com/python51888/pysqlit/blob/master/docs/development.md Explains the usage of `line_profiler` for detailed line-by-line performance analysis. It covers adding the `@profile` decorator to functions and running the analysis using the `kernprof` command-line tool. ```python # 在代码中添加装饰器 from line_profiler import LineProfiler @profile def complex_query(db): return db.execute(""" SELECT u.username, COUNT(o.id) as order_count FROM users u LEFT JOIN orders o ON u.id = o.user_id GROUP BY u.id """) # 运行分析 kernprof -l -v script.py ``` -------------------------------- ### Pytest Configuration Source: https://github.com/python51888/pysqlit/blob/master/docs/development.md Configuration file for pytest, specifying test discovery paths, naming conventions, verbosity, reporting options, and custom markers for test categorization (e.g., slow, integration, performance). ```ini [tool:pytest] testpaths = tests python_files = test_*.py python_classes = Test* python_functions = test_* addopts = -v --tb=short --strict-markers --cov=pysqlit --cov-report=html --cov-report=term-missing markers = slow: marks tests as slow (deselect with '-m "not slow"') integration: marks tests as integration tests performance: marks tests as performance tests ``` -------------------------------- ### Database Backup and Restore Test Source: https://github.com/python51888/pysqlit/blob/master/docs/development.md Tests the backup and restore functionality of the EnhancedDatabase class. It creates a temporary database, adds data, creates a backup, modifies data, and then restores from the backup to verify data consistency. ```python # tests/test_integration.py import tempfile import os from pysqlit.database import EnhancedDatabase class TestIntegration: def test_backup_restore(self): """测试备份恢复功能""" with tempfile.TemporaryDirectory() as tmpdir: # 创建测试数据库 db_path = os.path.join(tmpdir, "test.db") db = EnhancedDatabase(db_path) # 添加测试数据 db.execute("CREATE TABLE test (id INTEGER, name TEXT)") db.execute("INSERT INTO test VALUES (1, 'Alice')") # 创建备份 backup_path = db.create_backup("test_backup") assert os.path.exists(backup_path) # 修改数据 db.execute("INSERT INTO test VALUES (2, 'Bob')") # 恢复备份 db.restore_backup("test_backup") # 验证数据 rows = db.execute("SELECT * FROM test") assert len(rows) == 1 assert rows[0]['name'] == "Alice" db.close() ``` -------------------------------- ### Python EnhancedDatabase Class with Type Hinting Source: https://github.com/python51888/pysqlit/blob/master/docs/development.md Demonstrates a Python class with comprehensive type hinting for its constructor and methods. It showcases advanced typing for parameters and return values, enhancing code readability and maintainability. ```python from typing import List, Dict, Optional, Any, Tuple, Union class EnhancedDatabase: def __init__( self, filename: str, page_size: int = 4096, cache_size: int = 100, timeout: float = 5.0 ) -> None: self.filename: str = filename self.page_size: int = page_size self._cache: Dict[int, bytearray] = {} def execute( self, sql: str, params: Union[Tuple[Any, ...], List[Any], Dict[str, Any]] = () ) -> List[Dict[str, Any]]: """执行SQL语句。""" pass def create_backup( self, name: str, *, compress: bool = False, description: Optional[str] = None ) -> str: """创建数据库备份。""" pass ``` -------------------------------- ### PySQLit EnhancedDatabase Backup and Restore Operations Source: https://github.com/python51888/pysqlit/blob/master/docs/api-reference.md Details the backup and restore functionalities of EnhancedDatabase. This includes creating backups, listing existing backups, restoring from a specific backup, and deleting backups. ```python # Create backup backup_path = db.create_backup("daily_backup") print(f"Backup created: {backup_path}") # List backups backups = db.list_backups() for backup in backups: print(f"{backup['name']} - {backup['created']}") # Restore backup db.restore_backup("daily_backup") # Delete backup db.delete_backup("old_backup") ``` -------------------------------- ### PySqlit Debugging Mode Features Source: https://github.com/python51888/pysqlit/blob/master/docs/development.md Demonstrates the `DebugDatabase` class for enabling PySqlit's debugging mode. It shows how to instantiate it and use its methods to retrieve execution plans (`explain`), lock information (`get_lock_info`), and cache statistics (`get_cache_stats`). ```python from pysqlit.debug import DebugDatabase # 启用调试模式 db = DebugDatabase("test.db", debug=True) # 查看执行计划 plan = db.explain("SELECT * FROM users WHERE age > 25") print(plan) # 查看锁信息 locks = db.get_lock_info() print(locks) # 查看缓存状态 cache_stats = db.get_cache_stats() print(cache_stats) ``` -------------------------------- ### pysqlit: Performance Monitor for Query Stats Source: https://github.com/python51888/pysqlit/blob/master/docs/api-reference.md Illustrates how to use the PerformanceMonitor to track database query performance. It covers enabling monitoring, executing queries, and retrieving statistics like query count and average execution time. ```python from pysqlit.monitor import PerformanceMonitor # 创建监控器 monitor = PerformanceMonitor(db) # 启用监控 monitor.enable() # 执行查询 # results = db.execute("SELECT * FROM users") # 获取统计 # stats = monitor.get_stats() # print(f"查询次数: {stats['query_count']}") # print(f"平均查询时间: {stats['avg_query_time']}ms") # print(f"缓存命中率: {stats['cache_hit_rate']}%") ``` -------------------------------- ### PySQLit BackupManager Functionality Source: https://github.com/python51888/pysqlit/blob/master/docs/api-reference.md Describes the BackupManager class for advanced backup features. It covers creating manual backups, setting up automatic backups at specified intervals, and validating the integrity of backup files. ```python from pysqlit.backup import BackupManager # Create backup manager backup_mgr = BackupManager("myapp.db") # Create backup backup_name = backup_mgr.create_backup("manual_backup") # Automatic backup backup_thread = backup_mgr.auto_backup(interval_hours=24) # Validate backup is_valid = backup_mgr.validate_backup("manual_backup") ``` -------------------------------- ### Database Instance Creation Factory Source: https://github.com/python51888/pysqlit/blob/master/docs/architecture.md A factory class for creating database instances. It abstracts the creation logic, allowing for different database types like in-memory or file-based databases. ```python class DatabaseFactory: """数据库工厂类""" @staticmethod def create_database(filename: str) -> EnhancedDatabase: """创建数据库实例""" if filename == ":memory:": return InMemoryDatabase() else: return FileDatabase(filename) ```