### MongoDB Query Examples

Source: https://context7.com/sniper970119/dianping_spider/llms.txt

Provides example MongoDB shell commands for querying data stored by the spider. Demonstrates finding shops by name pattern and reviews by shop ID.

```mongo
# MongoDB查询示例
# mongo shell:
# use dianping
# db.info.find({'店铺名': /自助餐/})
# db.review.find({'店铺id': 'k30YbaScPKFS0hfP'})
```

--------------------------------

### Install Dependencies with Pip

Source: https://github.com/sniper970119/dianping_spider/blob/master/README.md

Installs all necessary Python packages for the spider. Ensure you have Python 3 installed.

```bash
pip install -r requirements.txt
```

--------------------------------

### Configure IP Proxy Settings

Source: https://context7.com/sniper970119/dianping_spider/llms.txt

Example configuration for IP proxy settings in `config.ini`. Supports enabling proxies, setting repeat usage count, and configuring either HTTP extraction or key tunnel modes.

```ini
# config.ini 代理配置示例

[proxy]
# 启用代理
use_proxy = True

# IP重复使用次数（一个IP通常有效1分钟左右）
repeat_nub = 5

# HTTP提取模式
http_extract = True
key_extract = False
http_link = http://your-proxy-provider.com/api/get?num=1&format=json

# 秘钥隧道模式
# http_extract = False
# key_extract = True
# proxy_host = tunnel.proxy-provider.com
# proxy_port = 12345
# key_id = your_user_id
# key_key = your_secret_key
```

--------------------------------

### Alternative JSON Proxy Format (Not Supported)

Source: https://github.com/sniper970119/dianping_spider/blob/master/docs/proxy.md

This is an example of a JSON format that is not currently supported by the library. Manual modification in `requests_utils.py` is required if your provider uses this format.

```json
"state":"ok",
    "data":[
            {
                "ip":"xxx.xxx.xxx.xxx",
                "port":"xxxxx"
            },
            {
                "ip":"xxx.xxx.xxx.xxx",
                "port":"xxxxx"
            },
        ]
```

--------------------------------

### Get Basic Review Information

Source: https://context7.com/sniper970119/dianping_spider/llms.txt

Fetches basic review information, including review summaries and counts for different rating categories, without requiring login cookies. Uses the `get_basic_review` function.

```python
# 获取基础评论信息（不需要登录Cookie）
basic_review = get_basic_review(shop_id)
# 返回: {'店铺id': 'xxx', '评论摘要': [...], '评论总数': 1234, '好评个数': '1100', ...}
```

--------------------------------

### Fetch Specific Shop Details

Source: https://context7.com/sniper970119/dianping_spider/llms.txt

Allows for fetching details of a specific shop by its ID. Set `detail=True` to get full details (requires cookies), or `detail=False` to use the encrypted API.

```python
# 单独爬取指定店铺详情
# shop_id: 店铺ID
# detail: True=获取完整详情（需Cookie），False=使用加密接口
controller.get_detail(shop_id='k30YbaScPKFS0hfP', detail=False)
```

--------------------------------

### Get Basic Hidden Shop Info

Source: https://context7.com/sniper970119/dianping_spider/llms.txt

Fetches basic hidden shop information such as name, address, and phone number using the shop ID. Requires the `get_basic_hidden_info` function from `function.get_encryption_requests`.

```python
from function.get_encryption_requests import (
    get_basic_hidden_info,
    get_review_and_star,
    get_basic_review,
    get_lat_and_lng
)

shop_id = 'k30YbaScPKFS0hfP'

# 获取基础隐藏信息（名称、地址、电话）
hidden_info = get_basic_hidden_info(shop_id)
# 返回: {'店铺id': 'xxx', '店铺名': 'xxx', '店铺地址': 'xxx', '店铺电话': 'xxx'}
```

--------------------------------

### Get Shop Score and Average Price

Source: https://context7.com/sniper970119/dianping_spider/llms.txt

Retrieves the shop's overall score, detailed scores for aspects like taste and environment, average price, and total review count. Uses the `get_review_and_star` function.

```python
# 获取评分和人均价格
score_info = get_review_and_star(shop_id)
# 返回: {'店铺id': 'xxx', '店铺总分': 4.5, '店铺均分': {'口味': '8.5', '环境': '8.2'}, '人均价格': '128', '评论总数': '1234'}
```

--------------------------------

### Get Shop Latitude and Longitude

Source: https://context7.com/sniper970119/dianping_spider/llms.txt

Obtains the geographical coordinates (latitude and longitude) of a shop using its ID. This function is `get_lat_and_lng` from `function.get_encryption_requests`.

```python
# 获取店铺经纬度
location = get_lat_and_lng(shop_id)
# 返回: {'店铺id': 'xxx', '店铺名': 'xxx', '店铺纬度': 38.9123, '店铺经度': 121.6234}
```

--------------------------------

### Review Module Initialization and Usage

Source: https://context7.com/sniper970119/dianping_spider/llms.txt

Initializes the `Review` class and fetches user reviews for a given shop ID. The `request_type` parameter determines the request method (proxy/cookie). The function returns a dictionary containing review summaries and individual reviews.

```python
from function.review import Review

# 初始化评论类
review_spider = Review()

# 获取店铺评论
# shop_id: 16位店铺ID
# request_type: 请求类型
review_data = review_spider.get_review(
    shop_id='k30YbaScPKFS0hfP',
    request_type='proxy, cookie'
)

# 返回数据结构示例
# {
#     '店铺id': 'k30YbaScPKFS0hfP',
#     '评论摘要': [{'描述': '环境好', '个数': '256'}, {'描述': '服务好', '个数': '189'}],
#     '评论总数': 1234,
#     '好评个数': '1100',
#     '中评个数': '100',
#     '差评个数': '34',
#     '带图评论个数': '456',
#     '精选评论': [
#         {

```

--------------------------------

### Execute Main Script (main.py)

Source: https://context7.com/sniper970119/dianping_spider/llms.txt

Runs the main scraping process. Supports full workflow (search, detail, review) or custom flows by specifying parameters like `--normal`, `--detail`, `--review`, `--shop_id`, and `--need_more`. Using `--need_more True` requires login cookies and carries a higher risk.

```bash
# 完整流程：搜索 -> 详情 -> 评论
python main.py

# 只爬取指定店铺的详情信息
python main.py --normal 0 --detail 1 --review 0 --shop_id k30YbaScPKFS0hfP --need_more False

# 只爬取指定店铺的评论信息
python main.py --normal 0 --detail 0 --review 1 --shop_id k30YbaScPKFS0hfP --need_more False

# 爬取指定店铺的详情和评论
python main.py --normal 0 --detail 1 --review 1 --shop_id k30YbaScPKFS0hfP --need_more False

# 爬取更多详细数据（需要登录Cookie，风险较高）
python main.py --normal 0 --detail 1 --review 1 --shop_id k30YbaScPKFS0hfP --need_more True
```

--------------------------------

### Main Configuration (config.ini)

Source: https://context7.com/sniper970119/dianping_spider/llms.txt

Primary configuration file for setting up core parameters like cookie pool usage, API keys (uuid, tcv), user agent, data save mode (MongoDB), and request interval throttling.

```ini
[config]
# 是否使用cookie池，使用为True，反之为False
use_cookie_pool = False

# Cookie信息（从浏览器直接复制）
Cookie: fspop=test; cy=19; cye=dalian; _lxsdk_cuid=17a12f40183c8...

# uuid信息，用于加密接口
uuid : e5f18ed2-0f94-a5c1-6eba-496cdaa569fc.1623815619

# tcv信息，用于加密接口
tcv = zj9r0md0w5

# 浏览器UA
user-agent = Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:87.0) Gecko/20100101 Firefox/87.0

# 保存方式（目前只支持mongo/mongodb）
save_mode = mongo

# MongoDB连接地址
mongo_path = mongodb://127.0.0.1:27017

# 请求间隔配置：次数,秒数;次数,秒数
# 例：每请求1次休息2秒，每3次休息5秒，每10次休息50秒
requests_times = 1,2;3,5;10,50

[detail]
# 搜索关键字
keyword = 自助餐

# 位置代号（上海=1，北京=2，广州=4，大连=8）
location_id = 8

# 频道号
channel_id = 0

# 自定义搜索URL（可选，填写后上述参数无效）
search_url = 

# 是否只需要搜索页首条
need_first = False

# 需要搜索的页数
need_pages = 5

[proxy]
# 是否使用代理
use_proxy = False

# IP重复使用次数
repeat_nub = 5

# 代理模式：http提取
http_extract = True

# 代理模式：秘钥访问
key_extract = False

# http提取接口链接
http_link = http://your-proxy-api.com/get

# 秘钥模式服务器配置
proxy_host = proxy.example.com
proxy_port = 8080
key_id = your_key_id
key_key = your_key_secret
```

--------------------------------

### Search Module Initialization and Usage

Source: https://context7.com/sniper970119/dianping_spider/llms.txt

Initializes the `Search` class and performs a search using a provided URL. The `request_type` parameter specifies whether to use proxy and cookie. The function returns a list of dictionaries, each representing a shop with its basic information.

```python
from function.search import Search

# 初始化搜索类
searcher = Search()

# 执行搜索，返回商家列表
# search_url: 搜索页URL
# request_type: 请求类型，支持 'proxy, cookie' / 'no proxy, no cookie'
search_results = searcher.search(
    search_url='http://www.dianping.com/search/keyword/8/0_自助餐/p1',
    request_type='proxy, cookie'
)

# 返回数据结构示例
# [
#     {
#         '店铺id': 'k30YbaScPKFS0hfP',
#         '店铺名': '某某自助餐厅',
#         '评论总数': '1234条评价',
#         '人均价格': '￥128',
#         '标签1': '自助餐',
#         '标签2': '大连',
#         '店铺地址': '中山区某某路123号',
#         '详情链接': 'http://www.dianping.com/shop/k30YbaScPKFS0hfP',
#         '图片链接': 'http://p1.meituan.net/xxx.jpg',
#         '店铺均分': '口味 8.5 环境 8.2 服务 8.0',
#         '推荐菜': '三文鱼 生蚝 帝王蟹',
#         '店铺总分': '4.5'
#     },
#     ...
# ]

for shop in search_results:
    print(f"店铺: {shop['店铺名']}, 评分: {shop['店铺总分']}, 人均: {shop['人均价格']}")
```

--------------------------------

### Detail Module Initialization and Usage

Source: https://context7.com/sniper970119/dianping_spider/llms.txt

Initializes the `Detail` class and retrieves detailed information for a specific shop using its `shop_id`. The `request_type` parameter controls whether to use proxy and cookie. The function returns a dictionary containing comprehensive details about the shop.

```python
from function.detail import Detail

# 初始化详情类
detail_spider = Detail()

# 获取店铺详情
# shop_id: 16位店铺ID
# request_type: 请求类型
detail_info = detail_spider.get_detail(
    shop_id='k30YbaScPKFS0hfP',
    request_type='proxy, cookie'
)

# 返回数据结构示例
# {
#     '店铺id': 'k30YbaScPKFS0hfP',
#     '店铺名': '某某自助餐厅',
#     '评论总数': '1234',
#     '人均价格': '￥128',
#     '店铺地址': '辽宁省大连市中山区某某路123号',
#     '店铺电话': '0411-12345678',
#     '其他信息': '营业时间: 11:00-21:00'
# }

print(f"店铺: {detail_info['店铺名']}")
print(f"地址: {detail_info['店铺地址']}")
print(f"电话: {detail_info['店铺电话']}")
```

--------------------------------

### Run Full Scraping Process

Source: https://github.com/sniper970119/dianping_spider/blob/master/README.md

Executes the complete scraping workflow, including searching, fetching details (optional), and collecting reviews (optional).

```bash
python main.py
```

--------------------------------

### Run Customized Scraping - Details and Reviews

Source: https://github.com/sniper970119/dianping_spider/blob/master/README.md

Scrapes both shop details and reviews without performing a search. Requires specifying the shop_id. Set --normal to 0.

```bash
python main.py --normal 0  --detail 1 --review 1  --shop_id k30YbaScPKFS0hfP --need_more False
```

--------------------------------

### Run Full Spider Workflow

Source: https://context7.com/sniper970119/dianping_spider/llms.txt

Executes the complete spider workflow, including search, detail fetching, and review collection, based on configurations in `config.ini` and `require.ini`. Uses the `controller.main()` method.

```python
from utils.spider_controller import controller

# 完整流程：自动执行搜索 -> 详情 -> 评论
# 根据config.ini和require.ini配置自动爬取
controller.main()
```

--------------------------------

### Run Customized Scraping - Reviews Only

Source: https://github.com/sniper970119/dianping_spider/blob/master/README.md

Scrapes only shop reviews without performing a search. Requires specifying the shop_id. Set --normal and --detail to 0.

```bash
python main.py --normal 0 --detail 0 --review 1 --shop_id k30YbaScPKFS0hfP --need_more False
```

--------------------------------

### Run Customized Scraping - Details Only

Source: https://github.com/sniper970119/dianping_spider/blob/master/README.md

Scrapes only shop details without performing a search. Requires specifying the shop_id. Set --normal and --review to 0.

```bash
python main.py --normal 0 --detail 1 --review 0  --shop_id k30YbaScPKFS0hfP --need_more False
```

--------------------------------

### Scraping Strategy Configuration (require.ini)

Source: https://context7.com/sniper970119/dianping_spider/llms.txt

Defines which types of data to scrape, such as shop phone numbers (including full details), user reviews (including more details and pagination), and shop coordinates. Note that detailed phone numbers require login cookies and may increase the risk of account suspension.

```ini
[shop_phone]
# 是否需要店铺电话
need = True
# 是否需要完整电话（False为12345678**，True为完整号码）
# 注意：需要登录Cookie，频繁请求会导致封号
need_detail = False

[shop_review]
# 是否需要店铺评论
need = False
# 是否需要更多评论（False只有10条精选）
more_detail = False
# 需要多少页评论（一页30条）
need_pages = 4

[shop_location]
# 是否需要店铺经纬度
need = False
```

--------------------------------

### Fetch Specific Shop Reviews

Source: https://context7.com/sniper970119/dianping_spider/llms.txt

Enables fetching reviews for a specific shop by its ID. Use `detail=True` for full review data (requires cookies) or `detail=False` to utilize the encrypted API.

```python
# 单独爬取指定店铺评论
# shop_id: 店铺ID
# detail: True=获取完整评论（需Cookie），False=使用加密接口
controller.get_review(shop_id='k30YbaScPKFS0hfP', detail=False)
```

--------------------------------

### Save Shop Details to MongoDB

Source: https://context7.com/sniper970119/dianping_spider/llms.txt

Persists detailed shop information, such as ID, name, address, and phone number, into the 'detail' collection in MongoDB. Data should be provided as a dictionary.

```python
# 保存详情到 dianping.info_detail 集合
detail_data = {
    '店铺id': 'k30YbaScPKFS0hfP',
    '店铺名': '某某自助餐厅',
    '店铺地址': '辽宁省大连市中山区某某路123号',
    '店铺电话': '0411-12345678',
    # ... 其他字段
}
saver.save_data(detail_data, 'detail')
```

--------------------------------

### Initialize MongoDB Saver

Source: https://context7.com/sniper970119/dianping_spider/llms.txt

Initializes the `MongoSaver` class, which automatically connects to the MongoDB database configured in `config.ini`. This is used for persisting scraped data.

```python
from utils.saver.mongo_saver import MongoSaver

# 初始化MongoDB保存器（自动连接config.ini中配置的数据库）
saver = MongoSaver()
```

--------------------------------

### Save Review Data to MongoDB

Source: https://context7.com/sniper970119/dianping_spider/llms.txt

Stores review-related data, including shop ID, total review count, good review count, and featured reviews, into the 'review' collection in MongoDB. Data is expected as a dictionary.

```python
# 保存评论到 dianping.review 集合
review_data = {
    '店铺id': 'k30YbaScPKFS0hfP',
    '评论总数': 1234,
    '好评个数': '1100',
    '精选评论': [...],
    # ... 其他字段
}
saver.save_data(review_data, 'review')
```

--------------------------------

### Cookie Pool Configuration (cookies.txt)

Source: https://context7.com/sniper970119/dianping_spider/llms.txt

Stores multiple cookies, one per line, to enable cookie rotation. This helps in reducing the risk of being blocked by Dianping due to frequent requests from a single cookie.

```text
fspop=test; cy=19; cye=dalian; _lxsdk_cuid=xxx; dper=xxx; dplet=xxx
fspop=test; cy=19; cye=shanghai; _lxsdk_cuid=yyy; dper=yyy; dplet=yyy
fspop=test; cy=19; cye=beijing; _lxsdk_cuid=zzz; dper=zzz; dplet=zzz
```

--------------------------------

### Print Extracted Shop Information

Source: https://context7.com/sniper970119/dianping_spider/llms.txt

Prints the shop's name, score, average price, and location coordinates after fetching them using the previously defined functions.

```python
print(f"店铺: {hidden_info['店铺名']}")
print(f"评分: {score_info['店铺总分']}, 人均: {score_info['人均价格']}元")
print(f"位置: ({location['店铺纬度']}, {location['店铺经度']})")
```

--------------------------------

### Dianping Review Data Structure

Source: https://context7.com/sniper970119/dianping_spider/llms.txt

Represents the data structure for user reviews on Dianping. Includes review summaries, counts, and detailed selected reviews.

```python
{
    '店铺id': 'k30YbaScPKFS0hfP',
    '评论摘要': [
        {'描述': '环境好', '个数': '256'},
        {'描述': '服务好', '个数': '189'}
    ],
    '评论总数': 1234,
    '好评个数': '1100',
    '中评个数': '100',
    '差评个数': '34',
    '带图评论个数': '456',
    '精选评论': [
        {
            '店铺id': 'k30YbaScPKFS0hfP',
            '评论id': '123456789',
            '用户id': '987654321',
            '用户名': '美食达人',
            '用户总分': '4.5',
            '用户打分': {'口味': '4.5', '环境': '4.0', '服务': '5.0'},
            '评论内容': '非常棒的自助餐...',
            '点赞个数': 56,           # 接口特有
            '回复个数': 3,            # 接口特有
            '浏览次数': 1234,         # 接口特有
            '人均价格': '138',
            '喜欢的菜': ['三文鱼', '生蚝'],
            '发布时间': '2023-06-15',
            '商家回复': '感谢您的光临！',
            '评论图片': ['http://...', 'http://...']
        }
    ],
    '推荐菜': ['三文鱼', '生蚝', '帝王蟹']  # 接口特有
}
```

--------------------------------

### Print Dianping Review Data

Source: https://context7.com/sniper970119/dianping_spider/llms.txt

Prints the total review count and categorizes reviews (good, neutral, bad). Iterates through the top 3 featured reviews to display username and a snippet of the review content.

```python
print(f"总评论数: {review_data['评论总数']}")
print(f"好评: {review_data['好评个数']}, 中评: {review_data['中评个数']}, 差评: {review_data['差评个数']}")
for review in review_data['精选评论'][:3]:
    print(f"用户: {review['用户名']}, 评分: {review['用户总分']}")
    print(f"内容: {review['评论内容'][:50]}...")
```

--------------------------------

### Supported JSON Proxy Format

Source: https://github.com/sniper970119/dianping_spider/blob/master/docs/proxy.md

This is the expected JSON format for IP proxy lists. Ensure your proxy provider adheres to this structure for compatibility.

```json
[
    {
        "ip":"xxx.xxx.xxx.xxx",
        "port":"xxxxx"
    },
    {
        "ip":"xxx.xxx.xxx.xxx",
        "port":"xxxxx"
    },
]
```

--------------------------------

### Dianping Search URL Format

Source: https://github.com/sniper970119/dianping_spider/blob/master/docs/location.md

This is the standard URL format for searching on Dianping. You need to replace placeholders with specific IDs and page numbers.

```text
http://www.dianping.com/search/keyword/(location_id)/(channel_id)_(key_word)/p(page_nub)
```

--------------------------------

### Save Search Results to MongoDB

Source: https://context7.com/sniper970119/dianping_spider/llms.txt

Saves search-related data, including shop ID, name, review count, and price, to the 'search' collection in MongoDB. The data is passed as a dictionary.

```python
# 保存搜索结果到 dianping.info 集合
search_data = {
    '店铺id': 'k30YbaScPKFS0hfP',
    '店铺名': '某某自助餐厅',
    '评论总数': '1234条评价',
    '人均价格': '￥128',
    # ... 其他字段
}
saver.save_data(search_data, 'search')
```

--------------------------------

### Secret Key Mode Proxy Configuration

Source: https://github.com/sniper970119/dianping_spider/blob/master/docs/proxy.md

This format is used for authenticated proxy access, often referred to as 'tunnel mode'. Replace placeholders with your actual credentials and host information. This is typically configured in `requests_utils.py`.

```python
proxyMeta = "http://%(user)s:%(pass)s@%(host)s:%(port)s" % {
                "host": host,
                "port": port,
                "user": user（id）,
                "pass": pass(key),
            }

    proxies = {
        "http": proxyMeta,
        "https": proxyMeta,
    }
```

--------------------------------

### Dianping Search Result Data Structure

Source: https://context7.com/sniper970119/dianping_spider/llms.txt

Represents the data structure for a single search result from Dianping. Includes shop ID, name, ratings, address, and URLs.

```python
{
    '店铺id': 'k30YbaScPKFS0hfP',      # 16位店铺唯一标识
    '店铺名': '某某自助餐厅',            # 店铺名称
    '评论总数': '1234条评价',           # 评论数量
    '人均价格': '￥128',                # 人均消费
    '标签1': '自助餐',                  # 分类标签
    '标签2': '大连',                    # 地区标签
    '店铺地址': '中山区某某路123号',     # 简略地址
    '详情链接': 'http://...',          # 详情页URL
    '图片链接': 'http://...',          # 店铺图片URL
    '店铺均分': '口味 8.5 环境 8.2',    # 各项评分
    '推荐菜': '三文鱼 生蚝',            # 推荐菜品（接口特有）
    '店铺总分': '4.5',                  # 综合评分
    '店铺电话': '0411-12345678',       # 店铺电话
    '其他信息': '营业时间...',          # 其他信息（网页特有）
    '优惠券信息': '满100减20',          # 优惠信息（接口特有）
    '店铺纬度': 38.9123,               # 纬度坐标
    '店铺经度': 121.6234               # 经度坐标
}
```

--------------------------------

### Proxy API Response Format

Source: https://context7.com/sniper970119/dianping_spider/llms.txt

Specifies the expected JSON format for responses from a proxy API, which should be an array of objects, each containing an 'ip' and 'port'.

```json
[
    {"ip": "192.168.1.100", "port": "8080"},
    {"ip": "192.168.1.101", "port": "8080"}
]
```

--------------------------------

### Tampermonkey Script Metadata

Source: https://context7.com/sniper970119/dianping_spider/llms.txt

Metadata block for a Tampermonkey user script designed to fetch Dianping reviews and handle dynamic font encryption. It specifies the script's name, version, description, and the website URLs it should run on.

```javascript
// ==UserScript==
// @name         大众点评评论
// @namespace    http://tampermonkey.net/
// @version      0.10
// @description  获取大众点评网页评论,解决动态字体加密
// @match        http://www.dianping.com/shop*
// @match        https://www.dianping.com/shop*
// ==/UserScript==

// 使用方法：
// 1. 安装Tampermonkey浏览器插件
// 2. 添加此脚本（或从GreasyFork安装：https://greasyfork.org/zh-CN/scripts/435145）
// 3. 访问店铺评论页：http://www.dianping.com/shop/H3CxO06M73m8Kmvc/review_all
// 4. 脚本自动解析加密字体并显示原始文本

// 注意：Firefox + Tampermonkey 效果最佳
// Chrome浏览器可能存在页面刷新问题
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.