### DNS Verification Example: Reverse and Forward Lookup Source: https://ziyuan.baidu.com/college/articleinfo?id=2855 This example demonstrates the two-step DNS verification process. First, a reverse lookup on an IP address, followed by a forward lookup on the resolved hostname to confirm the IP address. ```bash > host 111.206.198.69 69.198.206.111. in-addr.arpa domain name pointer baiduspider-111-206-198-69.crawl.baidu.com. > host baiduspider-111-206-198-69.crawl.baidu.com baiduspider-111-206-198-69.crawl.baidu.com has address 111.206.198.69 ``` -------------------------------- ### Install Baidusubmit Plugin for Discuz! Source: https://ziyuan.baidu.com/wiki/726 Follow these steps to install the Baidusubmit plugin. Ensure you select the correct version based on your Discuz! platform's encoding (Simplified Chinese or Simplified Chinese UTF-8). After installation, enable the plugin from the plugin list. ```plaintext Step one: Download the appropriate version of the plugin based on your Discuz! platform environment. Extract the downloaded file to the /home/work/discuz/source/plugin directory of your Discuz! platform. Step two: Log in to your Discuz! platform with an administrator account and navigate to the Management Center. Step three: Go to the "Applications" page in the Management Center, click "Install New Plugin", select the "baidusubmit" plugin, and install it. Step four: Navigate to "Plugins" in the left-hand menu, select "Plugin List", and click "Enable" for the plugin. Step five: After enabling the Baidu structured data plugin, verify the corresponding forum address. Step six: Upon successful verification, real-time forum data will be pushed to Baidu. You can check the recent data push status at any time. Other: If you no longer wish to use the plugin, find it in the plugin list and click "Close" on the right. To uninstall, click "Uninstall". ``` -------------------------------- ### Experience Gain Example Source: https://ziyuan.baidu.com/college/courseinfo?id=759&page=1 Shows examples of 'experience gain' features that enhance user experience, such as navigation aids and interactive elements. ```html ``` -------------------------------- ### Robots.txt File Format Example Source: https://ziyuan.baidu.com/wiki/2579 This example demonstrates the basic structure of a robots.txt file, including comments and directives. Ensure correct syntax for proper interpretation by search engine robots. ```robots.txt # robots.txt for example.com # This file is to prevent the crawling of # some files or folders. User-agent: * Disallow: /help Disallow: /private/ Allow: /private/public.html ``` -------------------------------- ### DNS Verification Example: Reverse and Forward Lookup Source: https://ziyuan.baidu.com/college/articleinfo?id=3378 This example demonstrates the two-step DNS verification process. First, a reverse lookup confirms the hostname format, and then a forward lookup verifies that the resolved domain points back to the original IP address, ensuring authenticity. ```text > host 111.206.198.69 69.198.206.111. in-addr.arpa domain name pointer baiduspider-111-206-198-69.crawl.baidu.com. > host baiduspider-111-206-198-69.crawl.baidu.com baiduspider-111-206-198-69.crawl.baidu.com has address 111.206.198.69 ``` -------------------------------- ### URL Pattern and Mobile Adaptation Rule Example Source: https://ziyuan.baidu.com/college/articleinfo?id=802 This example shows a PC URL pattern and its corresponding mobile adaptation rule. It illustrates how non-standard URL components can interfere with Baidu's adaptation calculation, preventing correct functionality. ```regex http://www.123456.cn/(\w+)/teacher/p1ll(\d+)—— http://m.123456.cn/teacher/${1}/s${2} ``` -------------------------------- ### Case Sensitivity in URLs Source: https://ziyuan.baidu.com/college/articleinfo?id=376 Examples showing different cases for the same URL path, highlighting the potential for duplicate content issues. ```plaintext http://www.a.com/newyork/ ``` ```plaintext http://www.a.com/Newyork/ ``` ```plaintext http://www.a.com/NewYork/ ``` -------------------------------- ### Page Browsing Experience Example Source: https://ziyuan.baidu.com/college/courseinfo?id=759&page=1 Demonstrates best practices for text and image presentation, link spacing, and clickable area sizes to improve mobile page browsing. ```html ``` -------------------------------- ### Page Structure Example Source: https://ziyuan.baidu.com/college/courseinfo?id=759&page=1 Illustrates good page structure for mobile, ensuring main content is prominent and free from intrusive elements like ads. ```html ``` -------------------------------- ### URL Redirection Example Source: https://ziyuan.baidu.com/college/courseinfo?id=267&page=5 This example demonstrates how to check for a single-level 301 redirect using curl. Ensure your old links redirect to new links using a 301 status code. ```bash curl -I -L+改版前旧链接 ``` -------------------------------- ### Protocol Dead Link Example Source: https://ziyuan.baidu.com/college/articleinfo?id=1136 This is an example of a protocol dead link where the server returns an HTTP 404 status code. ```text ‘谁在伪善’http://news.sina.com.cn/movie/music/1999-06-15/2965.shtml ``` -------------------------------- ### Recommended URL Structure with Category and ID Source: https://ziyuan.baidu.com/college/articleinfo?id=376 An example of a structured URL for e-commerce sites, including category and ID for better organization. ```plaintext http://www.a.com/hotel/beijing/123/ ``` -------------------------------- ### PC User-Agent Examples Source: https://ziyuan.baidu.com/college/articleinfo?id=335 These User-Agent strings are used by Baidu's spiders for desktop (PC) access. Verify these patterns in your server logs. ```text Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html) ``` ```text Mozilla/5.0 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html) ``` -------------------------------- ### TXT Dead Link List Format Source: https://ziyuan.baidu.com/college/articleinfo?id=671 This is an example of a TXT file format for submitting dead links. Each line in the file should contain a URL of a dead link. ```text URL1 URL2 URL3 ``` -------------------------------- ### URL with Product Name in Path (Discouraged) Source: https://ziyuan.baidu.com/college/articleinfo?id=376 An example of a URL that includes product names in the path, which is discouraged due to potential changes and maintenance costs. ```plaintext http://www.amazon.cn/博集典藏馆043•基督山伯爵-亚历山大•仲马/dp/B005TZHJEQ/ ``` -------------------------------- ### Content Dead Link Example Source: https://ziyuan.baidu.com/college/articleinfo?id=1136 This is an example of a content dead link where the server returns an HTTP 200 status code, but the content is unavailable and cannot be set to 404. ```text http://news.sina.com.cn/c/2015-04-23/184531753084.shtml ``` -------------------------------- ### Baidu Mobile Sitemap for Adaptive Webpages Source: https://ziyuan.baidu.com/college/articleinfo?id=3380 This XML example illustrates how to define an adaptive webpage in a Baidu mobile sitemap. The `` tag indicates that the URL serves content for both PC and mobile devices. ```xml http://www.abc.com/autoadapt.html 2009-12-14 daily 0.8 ``` -------------------------------- ### Allow only Baiduspider and Googlebot to access your website Source: https://ziyuan.baidu.com/college/articleinfo?id=1516 This example permits Baiduspider and Googlebot to crawl the site, while blocking all other user agents. ```robots.txt User-agent: Baiduspider Allow: / User-agent: Googlebot Allow: / User-agent: * Disallow: / ``` -------------------------------- ### Submit URLs via HTTP POST Source: https://ziyuan.baidu.com/college/articleinfo?id=267&page=2 An example of submitting URLs using an HTTP POST request. This format is suitable for programmatic submission. ```http POST /urls?site=www.58.com&token=edk7ychrEZP9pDQD HTTP/1.1 User-Agent: curl/7.12.1 Host: data.zz.baidu.com Content-Length: 83 http://www.example.com/1.html http://www.example.com/2.html ``` -------------------------------- ### Regular Expression for URL Redesign Source: https://ziyuan.baidu.com/college/courseinfo?id=267&page=5 This example shows how to define a regular expression pattern for URL redesigns. It illustrates converting a specific old URL structure to a new one using capture groups and backreferences. ```regex http://news.a.com /(\d+)/(\d+)/(\d+)/(\w+).html http://a.com/news/${1}/${2}/${3}/${4}.html ``` -------------------------------- ### URL with Query Parameters for Different States Source: https://ziyuan.baidu.com/college/articleinfo?id=376 Examples of URLs using query parameters to display different states of the same entity, track traffic sources, or for debugging. ```plaintext http://www.travel.com/hotel/123/?checkindate=2015-06-09 &checkoutdate=2015-06-10 ``` ```plaintext http://www.a.com/?tracking=website_a ``` ```plaintext http://www.a.com/?tracking=website_a &click_spot=zone_abc ``` ```plaintext http://www.a.com/product/item123/?debug=true ``` ```plaintext http://www.amazon.cn/abc/dp/B005TZHJEQ/ref=lp_2130608051_1_1_ ``` -------------------------------- ### XML Data File Structure with Extended Tags Source: https://ziyuan.baidu.com/wiki/729 This example illustrates the basic structure of an XML data file for Baidu site search, highlighting the fixed tags. It includes urlset, url, loc, lastmod, changefreq, priority, and the mandatory data and display tags for extended information. Note that the display tag is for extended tags and should be properly closed. ```xml http://example.com/.../page1.html 2013-08-12 daily 0.8 ``` -------------------------------- ### Sitemap Index File Structure for Baidu Site Search Source: https://ziyuan.baidu.com/wiki/729 This example shows the structure of a sitemap index file used to list multiple XML data files for submission. It includes the sitemapindex tag and individual sitemap tags, each containing a loc for the data file and an optional lastmod. Ensure UTF-8 encoding. ```xml http://example.com/.../aaa.xml 2013-08-20 ``` -------------------------------- ### Mobile User-Agent Examples Source: https://ziyuan.baidu.com/college/articleinfo?id=335 These User-Agent strings are used by Baidu's spiders for mobile applications. Ensure your server logs match these patterns for correct identification. ```text Mozilla/5.0 (Linux;u;Android 4.2.2;zh-cn;) AppleWebKit/534.46 (KHTML,like Gecko)Version/5.1 Mobile Safari/10600.6.3 (compatible; Baiduspider/2.0;+http://www.baidu.com/search/spider.html) ``` ```text Mozilla/5.0 (iPhone;CPU iPhone OS 9_1 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko)Version/9.0 Mobile/13B143 Safari/601.1 (compatible; Baiduspider-render/2.0;+http://www.baidu.com/search/spider.html) ``` -------------------------------- ### XML Data File Structure for Baidu Site Search Source: https://ziyuan.baidu.com/wiki/729 This is an example of a single XML data file for submitting a URL to Baidu site search. It includes essential tags like urlset, url, loc, and optional tags for lastmod, changefreq, and priority, along with a data section for extended tags. Ensure UTF-8 encoding. ```xml http://example.com/.../page1.html 2013-08-12 daily 0.8 ``` -------------------------------- ### Submit URLs via POST Request to Baidu API Source: https://ziyuan.baidu.com/college/articleinfo?id=3380 This example demonstrates how to submit URLs to Baidu using an HTTP POST request. The URLs are included in the request body, with each URL on a new line. ```http POST /urls?site=58.com&token=edk7ychrEZP9pDQD HTTP/1.1 Host: data.zz.baidu.com User-Agent: curl/7.12.1 Content-Length: 83 http://www.example.com/1.html http://www.example.com/2.html ``` -------------------------------- ### Dead Link from Repeated Online/Offline Cycles Source: https://ziyuan.baidu.com/college/articleinfo?id=1136 This example illustrates a page that has been repeatedly taken offline and then brought back online, leading to Baidu misinterpreting it as a dead link even after being re-published. ```text http://news.sina.com.cn/2015-12-05/doc-ifxmhqaa9933763.shtml ``` -------------------------------- ### Pure Number Replacement with Prefix Pattern Example Source: https://ziyuan.baidu.com/wiki/525 This pattern handles URLs with a prefix before the numeric part, converting a desktop URL to a mobile URL. The captured numeric group is referenced using ${1}. ```regex http://www.abc.com/t([0-9]+).html-> http://m.abc.com/${1}.html ``` -------------------------------- ### XML Dead Link List Format Source: https://ziyuan.baidu.com/college/articleinfo?id=671 This is an example of an XML file format for submitting dead links. It includes the URL, last modified date, change frequency, and priority. ```xml http://www.example.com/ 2010-01-01 hourly 1.0 ``` -------------------------------- ### Successful API Push Response Example Source: https://ziyuan.baidu.com/college/courseinfo?id=267&page=2 This JSON structure represents a successful response from the Baidu API Push. It indicates the number of successfully pushed URLs, remaining quota, and lists of non-site or invalid URLs. ```json { "remain":4999998, "success":2, "not_same_site":[], "not_valid":[] } ``` -------------------------------- ### JSON-LD Time Factor Submission Example Source: https://ziyuan.baidu.com/college/articleinfo?id=2210 Use JSON-LD format to submit time factors like publication date, update date, and latest reply date. Ensure dates and times are in 'YYYY-MM-DDThh:mm:ss' format. ```json "pubDate": "2017-06-15T08:00:01", ``` ```json "upDate": "2017-06-16T01:02:03", ``` ```json "lrDate": "2017-06-17T09:10:11" ``` -------------------------------- ### Failed API Push Response Example Source: https://ziyuan.baidu.com/college/courseinfo?id=267&page=2 This JSON structure indicates a failed submission to the Baidu API Push. It includes an error code and a descriptive message explaining the reason for the failure. ```json { "error":400, "message":"site error" } ``` -------------------------------- ### Pure Number Replacement Pattern Example Source: https://ziyuan.baidu.com/wiki/525 Use this pattern to replace numeric parts of a URL, converting a desktop URL to a mobile URL. The captured numeric group is referenced using ${1}. ```regex http://www.abc.com/([0-9]+).html-> http://m.abc.com/${1}.html ``` -------------------------------- ### Robots.txt Disallow Directive Source: https://ziyuan.baidu.com/wiki/2579 The Disallow directive specifies URL paths that should not be accessed by robots. URLs starting with the specified value will be blocked. An empty Disallow allows all URLs. ```robots.txt Disallow: /help ``` ```robots.txt Disallow: /private/ ``` ```robots.txt Disallow: ``` -------------------------------- ### First Baidu Submission Attempt Source: https://ziyuan.baidu.com/college/articleinfo?id=771 This command attempts to submit original content to Baidu using a specified token and file. Ensure the file 'xingHuoYuanChuang_100.txt' exists and contains the URLs to be submitted. ```bash curl -H 'Content-Type:text/plain' --data-binary @xingHuoYuanChuang_100.txt "http://data.zz.baidu.com/urls?site=news.hexun.com&token=3******njSrku0bFx&type=original" > returnInfo_yc_100.xml ``` -------------------------------- ### Standardized File URL Source: https://ziyuan.baidu.com/college/articleinfo?id=376 Shows the recommended way to standardize file URLs by appending '.html' to avoid ambiguity. ```plaintext http://www.a.com/product/123.html ``` -------------------------------- ### 通用PC与移动链接正则匹配规则 Source: https://ziyuan.baidu.com/college/articleinfo?id=267&page=15 演示了如何根据PC链接和移动链接的结构，提取可替换参数并生成适配规则的pattern。 ```regex http://news.a.com/(\d+)/(\d+)/(\d+)/(\w+).html http://m.a.com/news/${1}/${2}/${3}/${4}.html ``` -------------------------------- ### TXT格式Sitemap示例 Source: https://ziyuan.baidu.com/college/articleinfo?id=267&page=2 使用TXT文本文件列明需要提交的网址，每行一个完整网址。文件需遵循UTF-8或GBK编码，大小不超过10MB，最多包含50,000个网址。 ```text http://www.example.com/repaste/101562698_5230191316.html http://www.example.com/repaste/101586283_5230215075.html http://www.example.com/repaste/101639435_5230310576.html ``` -------------------------------- ### Traffic and Keyword Tools Introduction Source: https://ziyuan.baidu.com/wiki/734 This section introduces the Baidu Webmaster Platform's Traffic and Keyword Tools, which provide precise keyword traffic data including popular keywords, click-through rates, search result rankings, and trends. ```text **流量与关键词工具介绍** ``` -------------------------------- ### Submit URLs via cURL to Baidu API Source: https://ziyuan.baidu.com/college/articleinfo?id=3380 Use this cURL command to push a list of URLs from a local file to Baidu for immediate indexing. Ensure the file contains one URL per line and replace the placeholder with your actual site and token. ```bash curl -H 'Content-Type:text/plain' --data-binary @urls.txt \ "http://data.zz.baidu.com/urls?site=www.example.com&token=edk7yc4rEZP9pDQD" ``` -------------------------------- ### Optimized URLs Using '#' for Parameters Source: https://ziyuan.baidu.com/college/articleinfo?id=376 Demonstrates how to replace '?' with '#' for URL parameters to prevent them from being crawled by search engines, while still allowing JavaScript to process them. ```plaintext http://www.travel.com/hotel/123/#checkindate=2015-06-09 &checkoutdate=2015-06-10 ``` ```plaintext http://www.a.com/#tracking=website_a ``` ```plaintext http://www.a.com/#tracking=website_a&click_spot=zone_abc ``` ```plaintext http://www.a.com/product/item123/#debug=true ``` -------------------------------- ### Submit URLs via cURL Source: https://ziyuan.baidu.com/college/articleinfo?id=267&page=2 Use this cURL command to submit a list of URLs from a local file to Baidu for indexing. Ensure the file contains one URL per line. ```bash curl -H 'Content-Type:text/plain' --data-binary @urls.txt "http://data.zz.baidu.com/urls?site=www.example.com&token=edk7yc4rEZP9pDQD" ``` -------------------------------- ### Dead Link Submission Tool Announcement Source: https://ziyuan.baidu.com/wiki/536 This is a plain text announcement regarding an update to the dead link submission tool. It highlights new features like rule-based submission and content dead link template submission. No specific code examples are provided in this announcement. ```plaintext 亲爱的网站管理员： 百度站长平台在收集了广大网站管理员的需求后，请工程师对死链提交工具进行了升级，升级后的死链提交工具提供规则提交功能，极大减轻了网站管理员的收集汇总工作，同时提高了工具效率。 **以下便是本次死链工具此次升级的两大亮点：** 1.支持死链规则提交。若网站出现某一目录或规则下链接全部死链的情况，可以提交相应的规则，即pattern形式提交死链，无需再汇总全部死链。协议死链、内容死链、跳转死链都可以通过规则的方式进行提交。 2.支持内容死链模版提交。 1）模版提交功能发布了内容死链规范代码，若网站中存在内容死链，可以在页面中统一添加百度官方提供的内容死链规范代码，帮助百度蜘蛛进行更好地识别。 2）提交站内内容死链模板。内容死链过多的站点，网站管理员可以通过工具提交内容死链的模板，便于后期百度更好地识别站内的内容死链页面。 需要说明的是，网站提交协议死链和内容死链的有效数量及比例都将视为对百度友好的程度，在后续其他合作中都将考虑到。具体关于标准死链和内容死链等说明，请参考《百度搜索标准死链官方文档》。更多讨论和意见建议请移步至站长社区专帖，相关工程师在专帖中恭候您。 ``` -------------------------------- ### HTML 标签验证代码 Source: https://ziyuan.baidu.com/college/articleinfo?id=1003 将此 meta 标签复制到网站首页的 标签之间，用于 HTML 标签验证。 ```html ``` -------------------------------- ### 分页对应移动页面生成pattern示例 Source: https://ziyuan.baidu.com/college/articleinfo?id=267&page=15 展示了如何处理PC端带有分页的URL，并生成对应的移动端URL的pattern。 ```regex http://www.a.com/([0-9]+)-([0-9]+).htm -> http://m.a.com/${1}.htm ``` -------------------------------- ### 智能体调优案例-角色类 Source: https://ziyuan.baidu.com/vipevent/detail?id=102 在智能体(Agent)的prompt中植入角色信息，可提高多轮对话中的角色稳定性。 ```text 在智能体(Agent) 的prompt中多个部分植入智能体角色信息，可有效提高智能体在多轮对话中结果反馈的角色稳定性。 ``` -------------------------------- ### Submit URLs using Ruby Net::HTTP Source: https://ziyuan.baidu.com/college/articleinfo?id=267&page=2 This Ruby script shows how to submit URLs to Baidu using the Net::HTTP library for POST requests. It joins URLs with newlines for submission. ```ruby require 'net/http' urls = ['http://www.example.com/1.html', 'http://www.example.com/2.html'] uri = URI.parse('http://data.zz.baidu.com/urls site=www.xxx.com&token=eTk7ychrWZP1pDQD') req = Net::HTTP::Post.new(uri.request_uri) req.body = urls.join("\n") req.content_type = 'text/plain' res = Net::HTTP.start(uri.hostname, uri.port) { |http| http.request(req) } puts res.body ``` -------------------------------- ### 移动Sitemap协议示例 Source: https://ziyuan.baidu.com/college/articleinfo?id=267&page=2 在标准XML Sitemap基础上增加了标签，用于区分移动网页、PC网页、自适应网页和代码适配网页。该标签有四种取值：'mobile'、'pc,mobile'、'htmladapt'，无该标签则表示PC网页。 ```xml http://m.example.com/index.html 2009-12-14 daily 0.8 http://www.example.com/index.html 2009-12-14 daily 0.8 http://www.example.com/autoadapt.html 2009-12-14 daily 0.8 http://www.example.com/htmladapt.html 2009-12-14 daily 0.8 ``` -------------------------------- ### Searching for PHP Malware Patterns Source: https://ziyuan.baidu.com/college/articleinfo?id=1249 Use these grep commands to search for common PHP malware functions like 'eval' and 'create_function' within your website's core code directory. This helps identify potential security breaches. ```bash grep "eval(" /var/www/* -r ``` ```bash grep "create_function(" /var/www/* -r ``` -------------------------------- ### 正则式 (W+) 与 ([a-zA-Z]+) 的区别 Source: https://ziyuan.baidu.com/college/articleinfo?id=3382 展示了在正则表达式中，\w+ 和 [a-zA-Z]+ 的匹配范围不同，前者包含数字和下划线，后者仅包含字母。 ```regex (\w+)与([a-zA-Z]+) ，(\d+)与([0-9]+) ``` -------------------------------- ### 中文字符串正则替换生成pattern示例 Source: https://ziyuan.baidu.com/college/articleinfo?id=267&page=15 演示了如何使用正则表达式匹配URL中的中文字符串（经过URL编码），并生成对应的pattern。 ```regex http://www.abc.com/((?:%[a-zA-Z0-9]{2,})+)/-> http://m.abc.com/${1}/ ``` -------------------------------- ### Allow all robots to access the website Source: https://ziyuan.baidu.com/college/articleinfo?id=1516 This configuration permits all search engine crawlers to access and index your site. An empty robots.txt file also achieves this. ```robots.txt User-agent: * Allow: / ``` -------------------------------- ### Baidu Webmaster Platform vs. Baidu Statistics Keyword Tools Source: https://ziyuan.baidu.com/wiki/734 This section clarifies the difference between the Baidu Webmaster Platform's keyword tool and Baidu Statistics' keyword tool, explaining that the Webmaster Platform's tool is more suitable for SEO analysis because it analyzes Baidu's own logs, providing first-hand data. ```text **百度站长平台关键词工具与百度统计搜索关键词工具的区别** ``` -------------------------------- ### Linux: Reverse DNS Lookup with 'host' Source: https://ziyuan.baidu.com/college/articleinfo?id=2855 On Linux systems, use the 'host' command to perform a reverse DNS lookup on an IP address. This helps determine if the IP belongs to a Baidu spider by checking its hostname format. ```bash host ip ``` -------------------------------- ### 字母和数字混合字符串替换生成pattern示例 Source: https://ziyuan.baidu.com/college/articleinfo?id=267&page=15 演示了如何匹配URL中字母和数字混合的字符串，并生成对应的pattern。 ```regex http://www.abc.com/((?:[a-zA-Z]+[0-9]+|[0-9]+[a-zA-Z]+)[a-zA-Z0-9]+)/ -> http://m.abc.com/${1}/ ``` ```regex http://news.abc.com/([0-9]+)/([0-9]+)/([0-9]+)/([ a-zA-Z0-9]+).html -> http://m.abc.com/news/${1}/${2}/${3}/${4}.html ``` -------------------------------- ### Second Baidu Submission Attempt with New Token Source: https://ziyuan.baidu.com/college/articleinfo?id=771 This command attempts to resubmit data after changing the token and adjusting the filename. It's used when the initial submission fails. Note that this specific attempt resulted in a connection error. ```bash curl -H 'Content-Type:text/plain' --data-binary @urls.txt "http://data.zz.baidu.com/urls?site=news.hexun.com&token=o******VomX2W2gDp&type=original" ``` -------------------------------- ### Windows/OS: Reverse DNS Lookup with 'nslookup' Source: https://ziyuan.baidu.com/college/articleinfo?id=2855 For Windows or IBM OS/2 platforms, use the 'nslookup' command to perform a reverse DNS lookup. Input the IP address to check if the resolved hostname indicates a Baidu spider. ```bash nslookup xxx.xxx.xxx.xxx ``` -------------------------------- ### Common Linux File Permissions Source: https://ziyuan.baidu.com/college/articleinfo?id=1249 These are common numerical representations for file permissions in Linux. Use them to set appropriate access levels for files and directories. ```bash 600 ``` ```bash 644 ``` ```bash 700 ``` ```bash 666 ``` ```bash 777 ``` -------------------------------- ### 带连接符的字符串替换生成pattern示例 Source: https://ziyuan.baidu.com/college/articleinfo?id=267&page=15 展示了如何匹配URL中由'-'或'_'连接的字母数字字符串，并生成对应的pattern。 ```regex http://www.abc.com/([a-zA-Z]+)-([a-zA-Z]+)([0-9]+)/->http://m.abc.com/${1}-${2}${3}/ ``` -------------------------------- ### Mini Program Baidu Spider User-Agent Source: https://ziyuan.baidu.com/college/articleinfo?id=2855 This User-Agent identifies Baidu spiders specifically crawling for mini-programs. It's a specialized string for this application context. ```text Mozilla/5.0 (iPhone;CPU iPhone OS 9_1 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko)Version/9.0 Mobile/13B143 Safari/601.1 (compatible; Baiduspider-render/2.0;Smartapp; +http://www.baidu.com/search/spider.html) ``` -------------------------------- ### Directory vs. File URL Ambiguity Source: https://ziyuan.baidu.com/college/articleinfo?id=376 Illustrates how URLs can be ambiguous, referring to either a file or a directory index, potentially leading to duplicate content. ```plaintext http://www.a.com/product/123 ``` ```plaintext http://www.a.com/product/123/ ``` -------------------------------- ### 字母与数字交替出现替换生成pattern示例 Source: https://ziyuan.baidu.com/college/articleinfo?id=267&page=15 展示了如何分别匹配URL中字母和数字交替出现的字符串，并生成对应的pattern。 ```regex http://www.abc.com/([a-zA-Z]+)([0-9]+)/-> http://m.abc.com/${1}${2}/ ``` -------------------------------- ### Submit URLs using PHP cURL Source: https://ziyuan.baidu.com/college/articleinfo?id=267&page=2 This PHP script demonstrates how to submit a list of URLs to Baidu using cURL. It sets the necessary options for a POST request with plain text content. ```php $urls = array( 'http://www.example.com/1.html', 'http://www.example.com/2.html', ); $api = 'http://data.zz.baidu.com/urls site=www.58.com&token=edk7ychrEZP9pDQD'; $ch = curl_init(); $options = array( CURLOPT_URL => $api, CURLOPT_POST => true, CURLOPT_RETURNTRANSFER => true, CURLOPT_POSTFIELDS => implode("\n", $urls), CURLOPT_HTTPHEADER => array('Content-Type: text/plain'), ); curl_setopt_array($ch, $options); $result = curl_exec($ch); echo $result; ``` -------------------------------- ### Recommended URL Structure with ID Source: https://ziyuan.baidu.com/college/articleinfo?id=376 The preferred URL strategy using only essential elements like IDs for better stability and uniqueness. ```plaintext http://www.a.com/hotel/12345678/ ``` -------------------------------- ### 纯数字替换生成pattern示例 Source: https://ziyuan.baidu.com/college/articleinfo?id=267&page=15 展示了如何使用正则表达式匹配URL中的纯数字部分，并进行替换生成pattern。 ```regex http://www.abc.com/([0-9]+).html-> http://m.abc.com/${1}.html ``` ```regex http://www.abc.com/t([0-9]+).html-> http://m.abc.com/${1}.html ``` -------------------------------- ### 纯字母替换生成pattern示例 Source: https://ziyuan.baidu.com/college/articleinfo?id=267&page=15 展示了如何使用正则表达式匹配URL中的纯字母部分，并进行替换生成pattern。 ```regex http://www.abc.com/([a-zA-Z]+)/ -> http://m.abc.com/${1}/ ``` -------------------------------- ### 参数部分正则替换生成pattern示例 Source: https://ziyuan.baidu.com/college/articleinfo?id=267&page=15 演示了如何匹配URL的查询参数部分，并生成对应的pattern。 ```regex http://www.abc.com/article\.html?act=([^&]+)&id=([^&]+) -> http://m.abc.com/article.html?act=${1}&id=${2} ``` -------------------------------- ### Network Connectivity Test Source: https://ziyuan.baidu.com/college/articleinfo?id=771 This command is used to test network connectivity to a Baidu service. Successful execution indicates that the server can reach Baidu's domain, which is crucial for data submission. ```bash ping data.zz.baidu.com ``` -------------------------------- ### Steps to Recover from Robots.txt Block Source: https://ziyuan.baidu.com/college/articleinfo?id=1179 Follow these steps to resolve issues caused by an incorrectly blocked robots.txt file. This process requires using Baidu Webmaster Tools and involves several key actions to restore search engine indexing. ```text 1. Modify Robots封禁 to allow, then detect and update Robots in Baidu Webmaster Tools. 2. In Baidu Webmaster Tools, use crawl detection. If it shows crawl failure, click several more times to trigger spider crawling. 3. In Baidu Webmaster Tools, go to Crawl Frequency and apply to increase crawl frequency. 4. Provide feedback in the Baidu Feedback Center, explaining that the situation occurred due to an operational error. 5. Set up data active push (real-time) in the Baidu Webmaster Tools link submission section. 6. Update the sitemap, resubmit it to Baidu, and manually submit it daily. ``` -------------------------------- ### Allow access to specific URLs within directories Source: https://ziyuan.baidu.com/college/articleinfo?id=1516 This configuration allows access to specific sub-paths within directories that are otherwise disallowed. Note the order of directives is important. ```robots.txt User-agent: * Allow: /cgi-bin/see Allow: /tmp/hi Allow: /~joe/look Disallow: /cgi-bin/ Disallow: /tmp/ Disallow: /~joe/ ``` -------------------------------- ### XML格式Sitemap示例 Source: https://ziyuan.baidu.com/college/articleinfo?id=267&page=2 标准的XML格式Sitemap，包含根元素和多个条目。每个可定义（网址）、（最后修改时间）、（更新频率）和（优先级）。 ```xml http://www.yoursite.com/yoursite.html 2009-12-14 daily 0.8 http://www.yoursite.com/yoursite2.html 2010-05-01 daily 0.8 ``` -------------------------------- ### Baidu Mobile Sitemap for Code-Adapted Webpages Source: https://ziyuan.baidu.com/college/articleinfo?id=3380 This XML snippet demonstrates the Baidu mobile sitemap format for pages using code adaptation. The `` tag specifies that the HTML is adapted based on the user's browser (UA). ```xml http://www.abc.com/htmladapt.html 2009-12-14 daily 0.8 ``` -------------------------------- ### CSS Media Query for Responsive Design Source: https://ziyuan.baidu.com/college/articleinfo?id=2759 Implement CSS media queries to apply different styles based on screen size and device characteristics. This is crucial for responsive web design. ```css @media screen and (max-width: 600px) { body { background-color: lightblue; } } ``` -------------------------------- ### Robots.txt Allow Directive Source: https://ziyuan.baidu.com/wiki/2579 The Allow directive specifies URL paths that should be accessed by robots. It is often used in conjunction with Disallow to create exceptions. All URLs are allowed by default. ```robots.txt Allow: /private/public.html ``` -------------------------------- ### Submit URLs via Ruby to Baidu API Source: https://ziyuan.baidu.com/college/articleinfo?id=3380 This Ruby script uses the Net::HTTP library to send a list of URLs to Baidu's data submission API via a POST request. It formats the URLs and sets the appropriate content type. ```ruby require 'net/http' urls = ['http://www.example.com/1.html', 'http://www.example.com/2.html'] uri = URI.parse('http://data.zz.baidu.com/urls?site=www.xxx.com&token=eTk7ychrWZP1pDQD') req = Net::HTTP::Post.new(uri.request_uri) req.body = urls.join("\n") req.content_type = 'text/plain' res = Net::HTTP.start(uri.hostname, uri.port) { |http| http.request(req) } puts res.body ``` -------------------------------- ### Allow Baiduspider to crawl web pages and .gif images only Source: https://ziyuan.baidu.com/college/articleinfo?id=1516 This configuration allows Baiduspider to crawl web pages and .gif files, while disallowing other image formats. ```robots.txt User-agent: Baiduspider Allow: /*.gif$ Disallow: /*.jpg$ Disallow: /*.jpeg$ Disallow: /*.png$ Disallow: /*.bmp$ ``` -------------------------------- ### macOS: Reverse DNS Lookup with 'dig' Source: https://ziyuan.baidu.com/college/articleinfo?id=2855 On macOS, the 'dig' command with the '-x' flag can be used for reverse DNS lookups. This verifies if the IP address's hostname matches the expected Baidu spider format. ```bash dig -x xxx.xxx.xxx.xxx ``` -------------------------------- ### 智能体调优案例-美化输出结果 Source: https://ziyuan.baidu.com/vipevent/detail?id=102 使用markdown语法对智能体输出结果进行格式限制和排版，可使回答遵循特定范式。 ```text 智能体输出结果可用 markdown语法 进行严格限制，可按照开发者个人意愿对回答格式进行排版。用markdown语法对智能体 prompt 进行修饰，可以让智能体(Agent)按照一定范式规范回答。 ``` -------------------------------- ### Allow only Baiduspider to access your website Source: https://ziyuan.baidu.com/college/articleinfo?id=1516 This configuration allows Baiduspider to crawl the site, while disallowing all other user agents. ```robots.txt User-agent: Baiduspider Allow: / User-agent: * Disallow: / ``` -------------------------------- ### HTML 页面元信息示例 Source: https://ziyuan.baidu.com/college/articleinfo?id=781 展示了包含标题、关键词和描述的HTML元信息。这些信息对于搜索引擎优化至关重要，但如果内容空洞，可能导致负面影响。 ```html 【备胎罩】-备胎罩价格-备胎罩批发-备胎罩报价-XXX汽车用品网 ``` -------------------------------- ### Robots.txt Wildcard Usage Source: https://ziyuan.baidu.com/wiki/2579 Baiduspider supports wildcard characters '*' and '$' for flexible URL matching. '*' matches zero or more characters, and '$' matches the end of the line. ```robots.txt User-agent: Baiduspider Disallow: /*.mp4$ ``` -------------------------------- ### 零代码开发-基础配置 Source: https://ziyuan.baidu.com/vipevent/detail?id=102 通过输入名称和一句话设定，自动生成智能体基础配置。支持手动添加知识库、上传图片、选择声音和工具等高级配置。 ```text 输入名称及一句话设定，即可自动生成基础配置； ``` -------------------------------- ### Submit Dead Links via Baidu Webmaster Tools Source: https://ziyuan.baidu.com/college/articleinfo?id=1136 Use the dead link submission tool on Baidu Webmaster Tools to proactively submit dead links for faster processing by Baidu. This is applicable for protocol dead links. ```text /college/courseinfo?id=267&page=4#h2_article_title18 ``` -------------------------------- ### PC与手机页对应关系示例 Source: https://ziyuan.baidu.com/college/articleinfo?id=267&page=15 展示了PC页URL与手机页URL之间正确的对应关系，用于指导适配工作。 ```text PC页http://www.58.com/mmmshandongrencai/ 手机页http://m.58.com/w/mmmshandongrencai/ ``` -------------------------------- ### URL with Non-Essential Elements in Path (Discouraged) Source: https://ziyuan.baidu.com/college/articleinfo?id=376 Illustrates a URL structure that includes translated names and IDs, leading to frequent changes and SEO burdens. ```plaintext http://www.a.com/Shangrila_International_Hotel-12345678-hotel/ ``` -------------------------------- ### Robots.txt File Structure and Directives Source: https://ziyuan.baidu.com/college/articleinfo?id=1516 The robots.txt file uses a simple field-value format to specify crawling rules. Key directives include User-agent, Disallow, and Allow. Comments can be added using '#'. ```robots.txt User-agent: * Disallow: /help Allow: /help/index.html ``` ```robots.txt User-agent: Baiduspider Disallow: "" Allow: /image/jpg ``` -------------------------------- ### Add rel="alternate" to PC Page Source: https://ziyuan.baidu.com/wiki/545 On the PC version of a webpage, add this link tag to point to the corresponding mobile version. This helps Baidu discover the mobile version of your site. ```html ``` -------------------------------- ### 页面浏览体验示例 Source: https://ziyuan.baidu.com/college/documentinfo?id=759 展示了页面浏览体验的优化建议，包括文本与背景的区分度、段落排版、正文字号、行间距、图片统一、文字链间距及可点击区域大小。 ```html