### Start Main Application

Source: https://github.com/666ghj/bettafish/blob/main/README-EN.md

Initiates the main BettaFish application. Ensure all dependencies are met and refer to notes for troubleshooting port conflicts and data scraping.

```bash
# Start main application
python app.py
```

--------------------------------

### Launch Individual Agents (Streamlit)

Source: https://github.com/666ghj/bettafish/blob/main/README-EN.md

Starts individual agents using Streamlit, each on a specified port. These agents include QueryEngine, MediaEngine, and InsightEngine.

```bash
# Start QueryEngine
streamlit run SingleEngineApp/query_engine_streamlit_app.py --server.port 8503

# Start MediaEngine  
streamlit run SingleEngineApp/media_engine_streamlit_app.py --server.port 8502

# Start InsightEngine
streamlit run SingleEngineApp/insight_engine_streamlit_app.py --server.port 8501
```

--------------------------------

### Install Browser Driver with Playwright CLI

Source: https://github.com/666ghj/bettafish/blob/main/MindSpider/DeepSentimentCrawling/MediaCrawler/README_en.md

Installs the necessary browser drivers for Playwright using the 'playwright install' command. This is a prerequisite for browser automation tasks.

```shell
playwright install
```

--------------------------------

### Install Python Packages with pip

Source: https://github.com/666ghj/bettafish/blob/main/MindSpider/DeepSentimentCrawling/MediaCrawler/README_en.md

Installs Python dependency libraries listed in the 'requirements.txt' file using pip. This command should be run after activating a Python virtual environment.

```shell
pip install -r requirements.txt
```

--------------------------------

### Install Python Packages with uv

Source: https://github.com/666ghj/bettafish/blob/main/MindSpider/DeepSentimentCrawling/MediaCrawler/README_en.md

Installs Python dependencies and ensures consistency of the Python version and related packages using the 'uv sync' command. This is the recommended method for managing project dependencies.

```shell
# Enter project directory
cd MediaCrawler

# Use uv sync command to ensure consistency of python version and related dependency packages
uv sync
```

--------------------------------

### Clone Repository - Bash

Source: https://github.com/666ghj/bettafish/blob/main/CONTRIBUTING-EN.md

Clones the project repository to your local machine and navigates into the repository directory. Ensure you replace '<your-username>' and '<repository-name>' with your actual GitHub username and the repository name.

```bash
git clone https://github.com/<your-username>/<repository-name>.git
cd <repository-name>
```

--------------------------------

### Install Browser Driver with uv

Source: https://github.com/666ghj/bettafish/blob/main/MindSpider/DeepSentimentCrawling/MediaCrawler/README_en.md

Installs the necessary browser drivers for the Playwright library using the 'uv run playwright install' command. This is required for browser automation tasks within the project.

```shell
# Install browser driver
uv run playwright install
```

--------------------------------

### Docker Compose Deployment for Bettafish

Source: https://context7.com/666ghj/bettafish/llms.txt

Instructions for deploying the Bettafish system using Docker Compose. This involves configuring environment variables, starting services in the background, checking status, viewing logs, stopping services, and accessing the deployed applications via specified URLs. The deployment includes a Flask app, PostgreSQL, and Streamlit UIs.

```bash
# 1. Configure environment variables
cp .env.example .env
# Edit .env with your API keys and database credentials

# 2. Start all services in background
docker compose up -d

# 3. Check service status
docker compose ps

# 4. View logs
docker compose logs -f app

# 5. Stop services
docker compose down

# 6. Access the application
# Main interface: http://localhost:5000
# InsightEngine: http://localhost:8501
# MediaEngine: http://localhost:8502
# QueryEngine: http://localhost:8503

# Database configuration in .env:
# DB_HOST=db  (service name from docker-compose.yml)
# DB_PORT=5432
# DB_USER=bettafish
# DB_PASSWORD=bettafish
# DB_NAME=bettafish
```

--------------------------------

### Run Crawler: Get Post Details with uv

Source: https://github.com/666ghj/bettafish/blob/main/MindSpider/DeepSentimentCrawling/MediaCrawler/README_en.md

Executes the main crawler script using 'uv run' to retrieve detailed information and comments for specified posts using a list of post IDs from a configuration file. It requires specifying the platform and login type.

```shell
# Read specified post ID list from configuration file to get information and comment information of specified posts
uv run main.py --platform xhs --lt qrcode --type detail
```

--------------------------------

### Commit Changes - Bash

Source: https://github.com/666ghj/bettafish/blob/main/CONTRIBUTING-EN.md

Stages all changes in the current directory and commits them with a message following the Conventional Commits specification. This helps maintain a clean and readable commit history.

```bash
git add .
git commit -m "type: short description"
```

--------------------------------

### Run Crawler: Get Post Details (venv)

Source: https://github.com/666ghj/bettafish/blob/main/MindSpider/DeepSentimentCrawling/MediaCrawler/README_en.md

Executes the main crawler script using the Python interpreter to retrieve detailed information and comments for specified posts using a list of post IDs from a configuration file. It requires specifying the platform and login type.

```shell
# Read specified post ID list from configuration file to get information and comment information of specified posts
python main.py --platform xhs --lt qrcode --type detail
```

--------------------------------

### Project Initialization and Configuration

Source: https://github.com/666ghj/bettafish/blob/main/MindSpider/DeepSentimentCrawling/MediaCrawler/media_platform/tieba/test_data/note_detail.html

Initializes the project's global namespace and configures parameters for various modules. It supports different page types and flags, initiating specific handlers based on the configuration. Dependencies include `N`, `q`, `E`, `A`, `o`, and `j` modules.

```javascript
var ecomNsPcGlobal = this.ecomNsPcGlobal || {};
(function (t) {
    t.PARAMS = {},
    t.init = function (e) {
        t.PARAMS = e;
        var n = "." + t.PARAMS.resultClass;
        E.getInstance({}).support(function () {
            t.PARAMS.flags.tiebaCkFix && -1 !== ["pc_tieba_list", "pc_tieba_detail"].indexOf(t.PARAMS.pageid) ? A.init(n, t.PARAMS.imTimeSign) : o.init(n, t.PARAMS.imTimeSign),
            q.init(),
            t.PARAMS.flags.tiebaCkFix && -1 !== ["pc_tieba_list", "pc_tieba_detail"].indexOf(t.PARAMS.pageid) && j.init(n, t.PARAMS)
        })
    },
    t.request = e
}(this.ecomNsPcGlobal = this.ecomNsPcGlobal || {}));

(function (variable) {
    window.ecomNsPcGlobal.init(variable);
})({
    "searchid": "000000003236df83",
    "eid": "120201_120702_7869012_300003",
    "bwsid": 0,
    "osid": 0,
    "pageid": "pc_tieba_detail",
    "baiduid": "00E51832EC6DAA0D10E1C0699C5E6670",
    "ovlid": "129424-dz#129771-dz#122828-4#129937-dz#129551-1#129665-5#126544-4#121730-5#109505-1#129380-dz#119725-1#47135-dz#123nj-0",
    "wpt": 0,
    "netType": 0,
    "cuid": "",
    "feedCuid": "",
    "query": "%E4%BB%A5%E5%A4%AA%E6%AF%94%E7%89%B9",
    "imTimeSign": 73,
    "asynMode": 0,
    "asynUrl": "",
    "isWiseDropDown": false,
    "asynsid": "",
    "aspTime": 1722942843174,
    "sourceAdNum": {"ads_2327": 1},
    "asynQuery": "",
    "jFieldLinkMap": 

```

--------------------------------

### Initialize and Use MySQL Database for Data Storage

Source: https://github.com/666ghj/bettafish/blob/main/MindSpider/DeepSentimentCrawling/MediaCrawler/README_en.md

Shows the process of initializing a MySQL database and then configuring the project to save data to it. Note that the MySQL database must be created beforehand. The `--init_db mysql` command initializes the database connection, and `--save_data_option db` is used for saving data, with 'db' retained for backward compatibility.

```shell
# Initialize MySQL database
uv run main.py --init_db mysql
# Use MySQL to store data (the db parameter is retained for compatibility with historical updates)
uv run main.py --platform xhs --lt qrcode --type search --save_data_option db
```

--------------------------------

### Module Loading and Initialization in JavaScript

Source: https://github.com/666ghj/bettafish/blob/main/MindSpider/DeepSentimentCrawling/MediaCrawler/media_platform/tieba/test_data/note_comments.html

Demonstrates the use of a module system ('_.Module.use') to load and initialize various widgets and functionalities. It includes calls for creativeplatform, pcommon, common, ucenter, frs, and pb widgets, often with specific configurations or callback functions.

```javascript
_.Module.use('creativeplatform/widget/aopApp', [[ ]]); 
_.Module.use('pcommon/widget/preview', [], function(){}); 
_.Module.use('common/widget/httpTransform', [], function(){}); 
_.Module.use('common/widget/suggestion', [], function(){});
_.Module.use('common/widget/searchBright', [$('#head'), { style:'bright', theme:'bright_pb', forumName:'网球风云', searchFixed:'', sugOn:'1'}]);
_.Module.use('ucenter/widget/sign_mod', [$('#sign_mod'),{'hasClass': 1, 'page': ''}]);
_.Module.use('frs/widget/forum_card');
_.Module.use('frs/widget/forum_card/focus_btn', [{"islike":"0", "isCategoryOfGame": true, "forum_name":"网球风云" , "fr":"", "userForumList": []}]);
_.Module.use('common/widget/tbnav_bright', [$('#tb_nav'),{jq_search:$('#tb_nav').find('.j_search_internal_forum'),forumName:'网球风云'},{promotion_setting: [[ ]]}]);
_.Module.use('pb/widget/ForumTitle', [{'is_pic_act_underway': false}], function(forumTitle){ window.forumTitle = forumTitle; });
_.Module.use('pb/widget/saveFace', [{"isLogin": "1", "props": {"all_level":{"2":{"end_time":"1421113470","level":2,"score_limit":8000}},"level":{"end_time":"1421113470","expired_notify":1,"expiring_notify":1,"left_num":0,"max_free_score":8000,"open_status":null,"pic_url":null,"props_category":105,"props_id":2,"props_type":0,"update_time":null,"used_status":1}}, "forumId": "4513750", "threadId": "9119688421" }], function(){});
_.Module.use("encourage/widget/pb_marry", {});
_.Module.use('user/widget/SingleIcons');
_.Module.use('tbmall/widget/pb_post_foot_send_gift', { container: '.post-foot-send-gift-container', box: '.post-foot-send-gift', authorId: 1635505954, postId: 150726491368, presentNum: 0});
```

--------------------------------

### Object Property Check and Manipulation

Source: https://github.com/666ghj/bettafish/blob/main/MindSpider/DeepSentimentCrawling/MediaCrawler/media_platform/tieba/test_data/note_detail.html

This snippet defines functions related to object property checking and manipulation. `q` likely represents a way to get the internal `__proto__` property, `W` is a list of common object methods, and `Y` is a function to get the keys of an object, including potentially inherited ones.

```javascript
var q=G[T="IE_PROTO"]||(G[T]=f(T)),W="constructor,hasOwnProperty,isPrototypeOf,propertyIsEnumerable,toLocaleString,toString,valueOf".split(","),Y=Object.keys||function(n){return function(n,t){var r,e=A(n),o=0,i=[];for(r in e)r!=q&&c(e,r)&&i.push(r);for(;t.length>o;)c(e,r=t[o++])&&(~U(i,r)||i.push(r));return
```

--------------------------------

### Initialize and Use SQLite Database for Data Storage

Source: https://github.com/666ghj/bettafish/blob/main/MindSpider/DeepSentimentCrawling/MediaCrawler/README_en.md

Demonstrates how to initialize an SQLite database and subsequently use it to store data. SQLite is recommended for personal users due to its lightweight nature and lack of server requirement. The `--init_db` flag is used for initialization, and `--save_data_option sqlite` is used for saving data.

```shell
# Initialize SQLite database (when using '--init_db', no other optional arguments are needed)
uv run main.py --init_db sqlite
# Use SQLite to store data (recommended for personal users)
uv run main.py --platform xhs --lt qrcode --type search --save_data_option sqlite
```

--------------------------------

### JavaScript Module Loading and Initialization

Source: https://github.com/666ghj/bettafish/blob/main/MindSpider/DeepSentimentCrawling/MediaCrawler/media_platform/tieba/test_data/note_comments.html

This JavaScript code snippet demonstrates asynchronous module loading using a custom `_.Module.use` function. It loads various widgets like `data_handler_async`, `grab_treasure_dialog_success`, `NoRefresh`, `fixed_bar`, `normalApp`, `AdStats`, `tbskin`, and `Stat`. It also initializes user-related data and configurations.

```javascript
\_.Module.use('adsense/widget/data_handler_async', [], function (instance){ instance.addData({ 'forum_vdir': null}); });
\_.Module.use('tbmall/widget/grab_treasure_dialog_success',[{ diamondData: []}]);
/\*加载无刷新组件\*\
\_.Module.use('pb/widget/NoRefresh', ["\\/p\\/9119688421?pn=", false]);
\_.Module.use('spage/widget/fixed_bar', [], function(){});
if ('' === 'showBar'){
  $.stats.track('底部', '新用户红包', 'spage', 'show');
}
\_.Module.use('creativeplatform/widget/normalApp', [[ ]]);
PageData.lcs = {};
PageData.lcs.sbduss = '6OC0dZS7NfIjUYDTwVI3zOZjAL7yD8MY8lDs9FZMXTlb\/XpIjjRep98GDwuufTb2q1TK5Ku+yLHr3U5kdcjZYzfQdjoUHU+p0P2QQKbNitPfpLXCmS84lGwv2QA6WN+UeYdtI6fZ7nMWEbewI2Qq50F6sDg8FyuOQXqwODwXK45BerA4PBcrjkF6sDg8FyuOQXqwODwXK45BerA4PBcrjkF6sDg8FyuOQXqwODwXK46F2JPjzJJF2Ryz1avRGHOI';
PageData.user.joinEmoji = "initial";
PageData.user.is_uc_account = null;
PageData.user.business_account = null;
PageData.userBarExtraConfig = {"switch":"0","word":"hao123","url":"https:\/\/www.hao123.com\/?tn=50000076_hao_pg"};
\_.Module.use("common/widget/Userbar", [PageData.user, PageData.product, { bluePushUnit: {"url":["http:\/\/push.dui1dui.com\/tieba\/awaken"],"button":["\u9a6c\u4e0a\u62a2"],"text":["\u4eb2\u7231\u7684\uff0c\u4f60\u4e0d\u77e5\u9053\u5417\uff1f\u4f60\u8d26\u6237\u4e2d\u7684\u84dd\u94bb\u53ef\u4ee5\u7528\u6765\u5151\u6362\u8d34\u5427\u4f1a\u5458\uff01\u540d\u989d\u6709\u9650\uff0c\u901f\u6765\u62a2\uff01"]}, bluePushFlag: "all", userDiamond: null}]);
\_.Module.use('common/widget/AdStats', [{}]);
\_.Module.use('common/widget/tbskin', ["normal"]);
\_.Module.use("pb/widget/Stat");
var topic_thread = PageData.thread.topic || {},
    thread_topic_type = topic_thread.is_topic ? "1" : "0",
    thread_topic_subtype = (thread_topic_type == "1" && topic_thread.topic_type == "2"
```

--------------------------------

### Push to Remote - Bash

Source: https://github.com/666ghj/bettafish/blob/main/CONTRIBUTING-EN.md

Pushes the local feature branch to the remote repository on GitHub. This makes your changes available for creating a pull request.

```bash
git push origin feature/your-feature-name
```

--------------------------------

### JavaScript Utility for Getting Prototype

Source: https://github.com/666ghj/bettafish/blob/main/MindSpider/DeepSentimentCrawling/MediaCrawler/media_platform/tieba/test_data/note_detail.html

This utility function in JavaScript is designed to retrieve the prototype of an object. It handles different JavaScript environments and ensures compatibility. It does not have external dependencies.

```javascript
function _getPrototypeOf(t) {
  if (typeof Symbol === "function" && t[Symbol.iterator] === "function") {
    var e = t[Symbol.iterator]();
    if (e) return e;
  }
  return Object.getPrototypeOf(t);
}
```

--------------------------------

### Agent Configuration Parameters (Python)

Source: https://github.com/666ghj/bettafish/blob/main/README-EN.md

Illustrates default configuration parameters for QueryEngine, MediaEngine, and InsightEngine agents. These parameters control aspects like reflection rounds, search limits, and content length.

```python
# QueryEngine/utils/config.py
class Config:
    max_reflections = 2           # Reflection rounds
    max_search_results = 15       # Maximum search results
    max_content_length = 8000     # Maximum content length
    
# MediaEngine/utils/config.py  
class Config:
    comprehensive_search_limit = 10  # Comprehensive search limit
    web_search_limit = 15           # Web search limit
    
# InsightEngine/utils/config.py
class Config:
    default_search_topic_globally_limit = 200    # Global search limit
    default_get_comments_limit = 500             # Comment retrieval limit
    max_search_results_for_llm = 50              # Max results for LLM
```

--------------------------------

### Configure Business Database Connection

Source: https://github.com/666ghj/bettafish/blob/main/README-EN.md

Sets up the connection parameters for the business database in the `config.py` file. These constants are used to establish a connection to the database for data retrieval and manipulation.

```python
# Add your business database configuration in config.py
BUSINESS_DB_HOST = "your_business_db_host"
BUSINESS_DB_PORT = 3306
BUSINESS_DB_USER = "your_business_user"
BUSINESS_DB_PASSWORD = "your_business_password"
BUSINESS_DB_NAME = "your_business_database"
```

--------------------------------

### Create Feature Branch - Bash

Source: https://github.com/666ghj/bettafish/blob/main/CONTRIBUTING-EN.md

Creates a new Git branch for developing a new feature or fix. The branch name should follow the convention 'feature/xxx' or 'fix/xxx' for clarity.

```bash
git checkout -b feature/your-feature-name
```

--------------------------------

### Poster Context Initialization (JavaScript)

Source: https://github.com/666ghj/bettafish/blob/main/MindSpider/DeepSentimentCrawling/MediaCrawler/media_platform/tieba/test_data/tieba_note_list.html

Initializes the PosterContext module, loading user-specific data and authorization settings. It extends the existing PageData with user information and defines allowed domains for content embedding. Dependencies include the 'poster/widget/poster_context' module.

```javascript
_.Module.use('poster/widget/poster_context', [{"blockInfo": { "is_block" : 0, "block_reason":'', "opgroup" : '', "days_tofree":0, "block_errno": '0' }}], function() { PosterContext.loadContextFromPageData($.extend({},PageData,{ user:{ id: PageData.user.id, name: PageData.user.name, level: 1, isFavorForum: 0, isLogin: !!PageData.user.is_login, isBlocked: 0, isBlack: 0, noUsername: !!PageData.user.no_un, canPost: 0, canVote: 0, forbidFlag: 0, signNumber: 0, superboy: "", memberLevel: 0 }, is_zone_forum: "0", is_user_has_zone_right: "0", })); PosterContext.loadAuthorities({"img_num":50,"video_num":1,"smiley_num":50,"white_list":["http:\/\/www.tudou.com\/","http:\/\/v.blog.sohu.com\/","http:\/\/tv.sohu.com\/","http:\/\/share.vrs.sohu.com\/","http:\/\/my.tv.sohu.com\/","http:\/\/player.56.com\/","http:\/\/www.56.com\/","http:\/\/kankanews.com\/","http:\/\/video6.smgbb.cn\/","http:\/\/www.youku.com\/","http:\/\/player.youku.com\/","http:\/\/static.youku.com\/","http:\/\/www.ku6.com\/","http:\/\/player.ku6.com\/","http:\/\/video.sina.com.cn\/","http:\/\/vhead.blog.sina.com.cn\/","http:\/\/you.video.sina.com.cn\/","http:\/\/video.qq.com\/","http:\/\/www.baidu.com\/","http:\/\/box.baidu.com\/","http:\/\/hi.baidu.com\/","http:\/\/mv.baidu.com\/","http:\/\/mvimg.baidu.com\/","http:\/\/mvideo.baidu.com\/","http:\/\/player.cntv.cn\/","http:\/\/player.xiyou.cntv.cn\/","http:\/\/www.yinyuetai.com\/","http:\/\/player.yinyuetai.com\/","http:\/\/www.aipai.com\/","http:\/\/www.cutv.com\/","http:\/\/player.cutv.com\/","http:\/\/www.pptv.com\/","http:\/\/v.pptv.com\/","http:\/\/www.letv.com\/","http:\/\/www.iqiyi.com\/","http:\/\/yule.i"]}); });
```

--------------------------------

### Integrating OpenAI-compatible LLMs (Python)

Source: https://github.com/666ghj/bettafish/blob/main/README-EN.md

Demonstrates how to integrate custom LLM providers by configuring API key, base URL, and model name in `config.py`, adhering to the OpenAI request format.

```python
from openai import OpenAI

client = OpenAI(api_key="your_api_key",
                base_url="https://api.siliconflow.cn/v1")

response = client.chat.completions.create(
    model="Qwen/Qwen2.5-72B-Instruct",
    messages=[
        {
            'role': 'user',
            'content': "What new opportunities will reasoning models bring to the market?"
        }
    ],
)

complete_response = response.choices[0].message.content
print(complete_response)
```

--------------------------------

### Root Rectangle and Threshold Calculation

Source: https://github.com/666ghj/bettafish/blob/main/MindSpider/DeepSentimentCrawling/MediaCrawler/media_platform/tieba/test_data/note_detail.html

Calculates the bounding rectangle of the root element, expanding it by the root margin values. It also determines if an intersection threshold has been crossed between the previous and current intersection states. Dependencies include DOM utility functions for getting element dimensions.

```javascript
r.prototype._getRootRect=function(){var e;if(this.root)e=a(this.root);else{var n=t.documentElement,r=t.body;e={top:0,left:0,right:n.clientWidth||r.clientWidth,width:n.clientWidth||r.clientWidth,bottom:n.clientHeight||r.clientHeight,height:n.clientHeight||r.clientHeight}}return this._expandRectByRootMargin(e)},r.prototype._expandRectByRootMargin=function(t){var e=this._rootMarginValues.map(function(e,n){return"px"==e.unit?e.value:e.value*(n%2?t.width:t.height)/100}),n={top:t.top-e[0],right:t.right+e[1],bottom:t.bottom+e[2],left:t.left-e[3]};return n.width=n.right-n.left,n.height=n.bottom-n.top,n},r.prototype._hasCrossedThreshold=function(t,e){var n=t&&t.isIntersecting?t.intersectionRatio||0:-1,r=e.isIntersecting?e.intersectionRatio||0:-1;if(n!==r)for(var o=0;o<this.thresholds.length;o++){var i=this.thresholds[o];if(i==n||i==r||i<n!=i<r)return!0}},r.prototype._rootIsInDom=function(){return!this.root||c(t,this.root)},r.prototype._rootContainsTarget=function(e){return c(this.root||t,e)},r.prototype._registerInstance=function(){0>e.indexOf(this)&&e.push(this)},r.prototype._unregisterInstance=function(){var t=e.indexOf(this);-1!=t&&e.splice(t,1)},window.IntersectionObserver=r,window.IntersectionObserverEntry=n}function n(t){this.time=t.time,this.target=t.target,this.rootBounds=t.rootBounds,this.boundingClientRect=t.boundingClientRect,this.intersectionRect=t.intersectionRect
```

--------------------------------

### Weirwood Parameters Initialization

Source: https://github.com/666ghj/bettafish/blob/main/MindSpider/DeepSentimentCrawling/MediaCrawler/media_platform/tieba/test_data/tieba_note_list.html

Initializes Weirwood parameters, including error collection settings (sample rate, window errors, rejections, resource errors) and performance settings (sample rate, SPA, history).

```javascript
window.weirwoodParams = {
    "common": {
        "buildid": "pc_frs",
        "token": "e78f3741fe004f9a94290a022bdbaeba",
        "ignoreUrls": []
    },
    "error": {
        "sampleRate": 0.003,
        "collectWindowErrors": true,
        "collectUnhandledRejections": true,
        "collectResourceLoadErrors": true
    },
    "perf": {
        "sampleRate": 0.003,
        "spa": false,
        "history": true
    }
};

if (window.weirwoodParams) {
    !function() {
        "use strict";

        function i(e, t) {
            for (var n = 0; n < t.length; n++) {
                var i = t[n];
                i.enumerable = i.enumerable || !1,
                    i.configurable = !0,
                    "value" in i && (i.writable = !0),
                    Object.defineProperty(e, i.key, i)
            }
        }

        function a(t, e) {
            var n = Object.keys(t);
            return Object.getOwnPropertySymbols && n.push.apply(n, Object.getOwnPropertySymbols(t))
        }
    }()
}
```

--------------------------------

### Create Custom Business Database Query Tool

Source: https://github.com/666ghj/bettafish/blob/main/README-EN.md

Implements a `CustomBusinessDBTool` class for querying a business database. It initializes with connection details from `config.py` and provides methods for searching data and retrieving customer feedback. Dependencies include the `config` module.

```python
# InsightEngine/tools/custom_db_tool.py
import config

class CustomBusinessDBTool:
    """Custom business database query tool"""
    
    def __init__(self):
        self.connection_config = {
            'host': config.BUSINESS_DB_HOST,
            'port': config.BUSINESS_DB_PORT,
            'user': config.BUSINESS_DB_USER,
            'password': config.BUSINESS_DB_PASSWORD,
            'database': config.BUSINESS_DB_NAME,
        }
    
    def search_business_data(self, query: str, table: str):
        """Query business data"""
        # Implement your business logic
        pass
    
    def get_customer_feedback(self, product_id: str):
        """Get customer feedback data"""
        # Implement customer feedback query logic
        pass
```

--------------------------------

### Create and Activate Python venv Environment

Source: https://github.com/666ghj/bettafish/blob/main/MindSpider/DeepSentimentCrawling/MediaCrawler/README_en.md

Demonstrates how to create and activate a Python virtual environment named 'venv' using the 'venv' module. This is an alternative to using uv for environment management.

```shell
# Enter project root directory
cd MediaCrawler

# Create virtual environment
# My python version is: 3.9.6, the libraries in requirements.txt are based on this version
# If using other python versions, the libraries in requirements.txt may not be compatible, please resolve on your own
python -m venv venv

# macOS & Linux activate virtual environment
source venv/bin/activate

# Windows activate virtual environment
venv\Scripts\activate
```

--------------------------------

### Small Parameter Qwen3 Fine-tuning (Bash)

Source: https://github.com/666ghj/bettafish/blob/main/README-EN.md

Runs sentiment analysis using a small parameter Qwen3 fine-tuned model. This command is executed from the respective model directory.

```bash
cd SentimentAnalysisModel/WeiboSentiment_SmallQwen
python predict_universal.py --text "This event was very successful"
```

--------------------------------

### Run Crawler: Search Posts with uv

Source: https://github.com/666ghj/bettafish/blob/main/MindSpider/DeepSentimentCrawling/MediaCrawler/README_en.md

Executes the main crawler script using 'uv run' to search for posts based on keywords from a configuration file. It specifies the platform (e.g., 'xhs') and login type (e.g., 'qrcode'). This command crawls post information and comments.

```shell
# Read keywords from configuration file to search related posts and crawl post information and comments
uv run main.py --platform xhs --lt qrcode --type search
```

--------------------------------

### MindSpider Crawler System Commands

Source: https://github.com/666ghj/bettafish/blob/main/README-EN.md

Provides commands for using the MindSpider crawler system standalone. This includes project initialization, running topic extraction, complete workflows, and deep crawling on specified platforms.

```bash
# Enter crawler directory
cd MindSpider

# Project initialization
python main.py --setup

# Run topic extraction (get hot news and keywords)
python main.py --broad-topic

# Run complete crawler workflow
python main.py --complete --date 2024-01-20

# Run topic extraction only
python main.py --broad-topic --date 2024-01-20

# Run deep crawling only
python main.py --deep-sentiment --platforms xhs dy wb
```

--------------------------------

### Module Usage and Icon Widget Loading (JavaScript)

Source: https://github.com/666ghj/bettafish/blob/main/MindSpider/DeepSentimentCrawling/MediaCrawler/media_platform/tieba/test_data/note_comments.html

This snippet demonstrates the use of a module system to load a 'SingleIcons' widget from the 'user/widget' path. It also includes a conditional check to hide an element with the class 'd_pb_icons' if its HTML content is empty after trimming whitespace.

```javascript
\_.Module.use('user/widget/SingleIcons');
if ($.trim($('.d_pb_icons').html()).length === 0) {
  $('.d_pb_icons').hide();
}
```

--------------------------------

### Advertisement Wrapper and User Info Card Styling (CSS)

Source: https://github.com/666ghj/bettafish/blob/main/MindSpider/DeepSentimentCrawling/MediaCrawler/media_platform/tieba/test_data/note_detail.html

Styles for the "mediago-ad-pb-751-wrapper" and its internal components like user info, header, nickname, title, description, and check button. This section focuses on the presentation of an advertisement card with user-related details.

```CSS
.mediago-ad-pb-751-wrapper.svelte-1js710s.svelte-1js710s{text-align:center;width:738px;background:#ffffff;border-bottom:1px solid #e5e5e5;border-left:1px solid #e5e5e5;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;position:relative;font-size:0px}.mediago-ad-pb-751-wrapper.svelte-1js710s .mediago-userInfo.svelte-1js710s{display:inline-block;-webkit-flex-shrink:0;-ms-flex-negative:0;flex-shrink:0;width:130px;background:#fafbfc;vertical-align:top}.mediago-ad-pb-751-wrapper.svelte-1js710s .mediago-userInfo .mediago-header.svelte-1js710s{width:88px;height:88px;margin:auto;margin-top:20px;-webkit-box-sizing:border-box;box-sizing:border-box;border:1px solid #cccccc}.mediago-ad-pb-751-wrapper.svelte-1js710s .mediago-userInfo .mediago-header img.svelte-1js710s{width:80px;height:80px;margin:auto;margin-top:3px}.mediago-ad-pb-751-wrapper.svelte-1js710s .mediago-userInfo .mediago-nickName.svelte-1js710s{margin:auto;margin-top:24px;width:90%;text-align:center;font-size:12px;color:#2d64b3;letter-spacing:0;white-space:nowrap;text-overflow:ellipsis;overflow:hidden;word-break:break-all;line-height:12px}.mediago-ad-pb-751-wrapper.svelte-1js710s .mediago-content.svelte-1js710s{display:inline-block;width:608px;height:100%;padding-left:20px;position:relative;vertical-align:top;-webkit-box-sizing:border-box;box-sizing:border-box}.mediago-ad-pb-751-wrapper.svelte-1js710s .mediago-content .mediago-title.svelte-1js710s{width:568px;font-size:14px;color:#3066b4;letter-spacing:0;text-align:justify;line-height:18px;margin-top:30px}.mediago-ad-pb-751-wrapper.svelte-1js710s .mediago-content .mediago-desc.svelte-1js710s{width:568px;font-size:12px;color:#3066b4;letter-spacing:0;text-align:justify;line-height:18px;margin-top:6px}.mediago-ad-pb-751-wrapper.svelte-1js710s .mediago-content .mediago-checkBtn.svelte-1js710s{margin-top:12px;width:64px;height:28px;line-height}
```

--------------------------------

### Ad Integration Module (JavaScript)

Source: https://github.com/666ghj/bettafish/blob/main/MindSpider/DeepSentimentCrawling/MediaCrawler/media_platform/tieba/test_data/note_detail.html

This snippet demonstrates the use of a custom module system ('adsense/widget/tpl_ext') for ad integration. It configures an ad to be of type 'POST' and includes ad data. The provided JavaScript function `e` appears to be for logging image-related events, handling image loading and errors, and dynamically setting image sources.

```javascript
_.Module.use('adsense/widget/tpl_ext', [{
  'type': 'POST',
  'adDataList': {
    "PC_FC_FL": {
      "ad_num": 1,
      "js": "!function(t){\"use strict\";function e(t,e){if(t){var n=new Date().getTime(),r=new Image;t+=&quot;&amp;rand=&quot;+(n+Math.random()),window.ecomLogImage||(window.ecomLogImage={}),window.ecomLogImage[\"--IMAGE\"+n]=r,e=e||{};var o=!1;r.onload=r.onerror=r.onabort=function(i){if(!e.charge){if(i&amp;&amp;&quot;load&quot;!==i.type){if(o)&quot;error&quot;in e&amp;&amp;e.error();else{r.src=t,o=!0;return}}else&quot;load&quot;in e&amp;&amp;e.load()}}r.onload=r.onerror=r.onabort=null,window.ecomLogImage[\"--IMAGE\"+n]=null,r=null},r.src=t}}

var n,r=function(){var t,e,n,r=0,o=0,i=0,a=0,s=0,c=0,u=0,f=0,h=\/link\\?url\\=(\[^\\&amp;\]+)\/,l=\/\\?url\\=(\[^\\.\]+)\\.\\/,p=&quot;data-cklink&quot;,d=\/[?&amp;]tn=(\[^&amp;]*)\/;function v(t){if(!t)return!1;var e=h.exec(t)||l.exec(t);return!!e&amp;&amp;(h.exec(t)?e[1].length&lt;20?e[1]:e[1].substr(0,20):e[1])}function g(n){var a,s,h,l,d,g=(l=f.href,d=f.getAttribute(p),v(l)||v(d));!1!==g&amp;&amp;(a=&quot;&amp;ck=&quot;+[function(t,e){for(var n=0,r=0;r&lt;i*e%99+9;r++)n+=t.charCodeAt(u*r%t.length);return n}(g,n),i,u,Math.round(r),Math.round(o),Math.round(t),Math.round(e),c].join(&quot;.&quot;),s=f.href,h=f.getAttribute(p),s&amp;&amp;v(s)&amp;&amp;(f.href=y(s,a)+m(s)),h&amp;&amp;v(h)&amp;&amp;f.setAttribute(p,y(h,a)+m(h)))}function m(t){var e=&quot;&quot;;if(-1===t.indexOf(&quot;&amp;shh=&quot;)&amp;&amp;-1===t.indexOf(&quot;?shh=&quot;)&amp;&amp;(e=&quot;&amp;shh=&quot;+location.hostname),-1===t.indexOf(&quot;&amp;sht=&quot;)&amp;&amp;-1===t.indexOf(&quot;?sht=&quot;)){var n=location.href.match(d);n&amp;&amp;(e+=&quot;&amp;sht=&quot;+n[1])}return e}function y(t,e){return v(t)&amp;&amp;(-1===t.indexOf(&quot;&amp;ck=&quot;)?t+=e:t=t.replace(\/&amp;ck=[\\d.]*\/,e)),t}function b(n,h,l){void 0===n.length&amp;&amp;(n=[n]);for(var d=n.length,b=0,w=[function(n){var s;f=(s=(s=n)||window.event).target||s.srcElement,i++,void 0===t&amp;&amp;(t=s.clientX),void 0===e&amp;&amp;(e=s.clientY),r=0,o=0,u=0,c=0,a=new Date().getTime(),g(h)},function(t){!function(t,e){for(f=(t=t||window.event).target||t.srcElement;f&amp;&amp;&quot;A&quot;!==f.tagName;)f=f.parentNode;s=new Date().getTime(),u=9999,r=t.clientX,o=t.clientY,c=0===a?0:s-a,g(e)}(t,h)},function(t){u=new Date().getTime()-s,g(h)}];b&lt;d;b++)l||function(t){var e=t.href;if(v(e)){var n=&quot;&amp;ck=&quot;+[0,i,u,Math.round(r),Math.round(o),0,0,c].join(&quot;.&quot;),a=t.getAttribute(p);e&amp;&amp;(t.href=y(e,n)+m(e)),a&amp;&amp;t.setAttribute(p,y(a,n)+m(a))}}(n[b]),function(t,e,n){for(var r,o,i=e.length,a=0;a&lt;i;a++)r=e[a],o=n[a],window.attachEvent?t.attachEvent(&quot;on&quot;+r,o):t.addEventListener(r,o,!1)}(n[b],[&quot;mouseover&quot;,&quot;mousedown&quot;,&quot;mouseup&quot;],w)}return{get
```

--------------------------------

### AJAX Performance Monitoring in JavaScript

Source: https://github.com/666ghj/bettafish/blob/main/MindSpider/DeepSentimentCrawling/MediaCrawler/media_platform/tieba/test_data/note_comments.html

Implements a performance monitoring system named AJAX_MONITOR for tracking AJAX requests and page load times. It utilizes localStorage for caching data and sends performance metrics via image requests to a specified endpoint. Includes methods for initializing, creating instances, marking events, and sending data.

```javascript
(function(){
  function b(p,o,n){
    if(p.length===+p.length){
      for(var m=0,j=p.length;m<j;m++){
        if(o.call(n,m,p[m],p)===false){
          return
        }
      }
    } else {
      for(var k in p){
        if(p.hasOwnProperty(k)){
          if(o.call(n,k,p[k],p)===false){
            return
          }
        }
      }
    }
  }
  var g=[];
  var i={
    push:function(k){
      g.push(k);
      if(window.localStorage&&window.JSON){
        try{
          localStorage.setItem("WPO_NR",JSON.stringify(g))
        }catch(j){}
      }
    },
    get:function(j){
      var k=[];
      if(window.localStorage&&window.JSON){
        try{
          k=JSON.parse(localStorage.getItem("WPO_NR"))||[]
          j&&localStorage.removeItem("WPO_NR")
        }catch(l){}
      } else {
        k=g
      }
      if(j){
        g=[]
      }
      return k
    }
  };
  var c,f={},h={},a={"PDC":{"init":function(k){
    var j=navigator.connection||navigator.mozConnection||navigator.webkitConnection||{type:0};
    f={p:k.product_id,is_sample:Math.random()<=(k.sample||0.01),max:k.max||5,mnt:k.mnt||j.type};
    h={p:k.product_id,mnt:f.mnt,b:50};
    if(window.localStorage&&window.JSON&&window.addEventListener){
      c=i.get(true);
      window.addEventListener("load",function(){
        e.send(c)
      },false)
    }
  },"createInstance":function(j){
    return new d(j)
  }}};
  if((!window.localStorage||!window.JSON)&&document.attachEvent){
    window.attachEvent("onbeforeunload",function(){
      e.send()
    })
  }
  var e={"send":function(k){
    var m=[],l=[],o=k||i.get(true),n;
    if(o.length>0){
      b(o,function(p,r){
        var q=[];
        b(r.timing,function(s,t){
          q.push('"'+s+'":'+t)
        });
        m.push('{"t":{'+q.join(",")+"}","a":'+r.appId+"}")
        if(!n&&k&&r.start){n=r.start}
      });
      b(h,function(p,q){
        l.push(p+"="+q)
      });
      l.push("d=["+m.join(",")+"]")
      if(n){l.push("_st="+n)}
      else{
        l.push("_t="+(+
ew Date))
      }
      var j=new Image();
      j.src="https://gsp0.baidu.com/5aAHeD3nKhI2p27j8IqW0jdnxx1xbK/tb/pms/img/st.gif?"+l.join("&")
      window["___pms_img_"+new Date()*1]=j
    }
  }};
  var d=function(j){
    this.appId=j;
    this.timing={};
    this.start=+new Date
  };
  d.prototype={"mark":function(j,k){
    this.timing[j]=k||(new Date-this.start)
  },"start_event":function(){
    this.start=+new Date
  },"start_send":function(){
    this.mark("sts")
  },"transfer_time":function(){
    this.mark("tt")
  },"view_time":function(){
    this.mark("vt")
  },"ready":function(){
    if(f.is_sample){
      i.push(this);
      if(i.get().length>=f.max){
        e.send()
      }
    }
  },"error":function(j){}};
  window.AJAX_MONITOR=a}
)()
```