### Install Python dependencies with pip Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/node_vm2_pycaller.ipynb Installs the packages listed in the requirements file `../pycjs/requirements.txt`. The output shows all dependencies are already satisfied, so no changes are made. ```python !pip install -r ../pycjs/requirements.txt ``` -------------------------------- ### getAttention example for Chinese foundations Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/handbook.ipynb Calls openDigger.index.attention.getAttention to retrieve attention for Chinese foundation repos grouped by Foundation and by year from 2015 to 2021. The result is rendered as a scatter plot. ```javascript var startYear = 2015, startMonth = 1, endYear = 2021, endMonth = 12, x = []; for (let y = startYear; y <= endYear; y++) x.push(`${y}`); openDigger.index.attention.getAttention({labelIntersect: ['Foundation', ':regions/CN'], startYear, startMonth, endYear, endMonth, groupBy: 'Foundation', groupTimeRange: 'year', limit: -1 }).then(data => { openDigger.render.plotly(data.map(row => { return { x, y: row.attention, mode: 'scatter', name: row.name }; }), {title: `Attention for Chinese foundation repos ${startYear} - ${endYear}`, xaxis: {type: 'category'}}); }); ``` -------------------------------- ### Get GitHub Global Top 20 Repos by OpenRank Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/China_open_source_blue_paper_2022.ipynb Fetches the top 20 GitHub repositories by OpenRank for a given year and prints a table. The output shows the resulting table with repository names and their open_rank values. ```javascript openDigger.index.openrank.getRepoOpenrank({ startYear: year, endYear: year, startMonth: 1, endMonth: 12, groupTimeRange: 'year', limit: 20, order: 'DESC', }).then(data => console.table(data.map(d => { return { name: d.name, open_rank: d.openrank[0], }; }))); ``` ```text ┌─────────┬───────────────────────────────────┬───────────┐ │ (index) │ name │ open_rank │ ├─────────┼───────────────────────────────────┼───────────┤ │ 0 │ 'flutter/flutter' │ 24308.7 │ │ 1 │ 'home-assistant/core' │ 20570.64 │ │ 2 │ 'NixOS/nixpkgs' │ 20413.31 │ │ 3 │ 'microsoft/vscode' │ 18565.14 │ │ 4 │ 'pytorch/pytorch' │ 16227.35 │ │ 5 │ 'MicrosoftDocs/azure-docs' │ 15987.61 │ │ 6 │ 'dotnet/runtime' │ 15063.68 │ │ 7 │ 'kubernetes/kubernetes' │ 13484.14 │ │ 8 │ 'odoo/odoo' │ 11960.45 │ │ 9 │ 'elastic/kibana' │ 11731.84 │ │ 10 │ 'rust-lang/rust' │ 10332.07 │ │ 11 │ 'tensorflow/tensorflow' │ 9423.25 │ │ 12 │ 'grafana/grafana' │ 9388.1 │ │ 13 │ 'godotengine/godot' │ 8911.33 │ │ 14 │ 'DefinitelyTyped/DefinitelyTyped' │ 7596.26 │ │ 15 │ 'microsoft/PowerToys' │ 7493.91 │ │ 16 │ 'Homebrew/homebrew-core' │ 7403.37 │ │ 17 │ 'elementor/elementor' │ 7310.76 │ │ 18 │ 'dotnet/roslyn' │ 7179.64 │ │ 19 │ 'dotnet/aspnetcore' │ 6966.19 │ └─────────┴───────────────────────────────────┴───────────┘ ``` -------------------------------- ### getRepoActivity example for Chinese companies Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/handbook.ipynb Calls openDigger.index.activity.getRepoActivity to retrieve activity for Chinese projects grouped by Company and by year from 2015 to 2021. The result is rendered as a scatter plot using openDigger.render.plotly. ```javascript var startYear = 2015, startMonth = 1, endYear = 2021, endMonth = 12, x = []; for (let y = startYear; y <= endYear; y++) x.push(`${y}`); openDigger.index.activity.getRepoActivity({labelIntersect: ['Company', ':regions/CN'], startYear, startMonth, endYear, endMonth, groupBy: 'Company', groupTimeRange: 'year', limit: -1, order: 'DESC' }).then(data => { openDigger.render.plotly(data.map(row => { return { x, y: row.activity, mode: 'scatter', name: row.name }; }), {title: `Activity for companies from ${startYear} to ${endYear}`, xaxis: {type: 'category'}}); }); ``` -------------------------------- ### Load OpenDigger Clickhouse driver Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/clickhouse_demo.ipynb This JavaScript code loads the OpenDigger Clickhouse driver and sets up a query helper for executing Clickhouse queries. This is the foundation for the subsequent examples. ```javascript var openDigger = require('../src/open_digger') var query = q => openDigger.driver.clickhouse.query(q, { format: 'JSONEachRow' }); ``` -------------------------------- ### Query and visualize developer active hours distribution Source: https://github.com/x-lab2017/open-digger/blob/master/cooperations/paddle_hackathon_3rd/88_Swift.ipynb Queries GitHub events for a specific developer, calculates the proportion of events per hour (adjusted for timezone), and renders a line chart. The example uses 'wangkuiyi' as the actor_login and includes a commented-out filter for event types. ```javascript //活跃时间('wangkuiyi'所在的时区为-3) query( `WITH ( SELECT count() FROM github_log.events where actor_login = 'wangkuiyi' ) AS total_times select (24+toHour(created_at) -3) % 24 as hour,(count() / total_times)*100 as propotion from github_log.events where actor_login = 'wangkuiyi' group by hour order by hour ` ).then(data => { var hours=[],propotion = []; data.forEach(d => { hours.push(d.hour); propotion.push(d.propotion); }); openDigger.render.plotly([ {x: hours, y: propotion, type: 'scatter',mode:'line', name: '数量'}], { title: '开发者wangkuiyi一天的工作时间', xaxis: {type: 'category'}, height: 600, }); }); ``` -------------------------------- ### Output table of top 20 users by shared star ratio Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/clickhouse_demo.ipynb Example output table from the ClickHouse query, showing the top 20 users ranked by the ratio of shared stars. The table includes columns for actor_login, stars_my, stars_other, and ratio. The output is presented as a console.table result. ```text ┌─────────┬────────────────┬──────────┬─────────────┬───────┐ │ (index) │ actor_login │ stars_my │ stars_other │ ratio │ ├─────────┼────────────────┼──────────┼─────────────┼───────┤ │ 0 │ 'frank-zsy' │ '41' │ '0' │ 1.079 │ │ 1 │ 'hx94113' │ '25' │ '1' │ 0.641 │ │ 2 │ 'daydate' │ '22' │ '2' │ 0.55 │ │ 3 │ 'blindpirate' │ '58' │ '151' │ 0.307 │ │ 4 │ 'lidongze0629' │ '33' │ '74' │ 0.295 │ │ 5 │ 'wtfil' │ '27' │ '75' │ 0.239 │ │ 6 │ 'Tara-Fuki' │ '8' │ '1' │ 0.205 │ │ 7 │ 'RayProud' │ '9' │ '7' │ 0.2 │ │ 8 │ 'Jrmy-rbr' │ '9' │ '8' │ 0.196 │ │ 9 │ 'hcl0908' │ '7' │ '1' │ 0.179 │ │ 10 │ 'chenDSEE' │ '7' │ '2' │ 0.175 │ │ 11 │ 'wzl12356' │ '9' │ '15' │ 0.17 │ │ 12 │ 'Sophie36' │ '7' │ '5' │ 0.163 │ │ 13 │ 'fireber' │ '6' │ '0' │ 0.158 │ │ 14 │ 'oooooutsider' │ '6' │ '0' │ 0.158 │ │ 15 │ 'akselsoft' │ '6' │ '1' │ 0.154 │ │ 16 │ 'dplrwj' │ '6' │ '2' │ 0.15 │ │ 17 │ 'dpsisodia' │ '6' │ '2' │ 0.15 │ │ 18 │ 'Wammy1995' │ '6' │ '6' │ 0.136 │ │ 19 │ 'steven-wan' │ '6' │ '7' │ 0.133 │ └─────────┴────────────────┴──────────┴─────────────┴───────┘ ``` -------------------------------- ### Get Linux Foundation China Top 20 Repos by Activity Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/China_open_source_report_2021.ipynb Queries the OpenDigger activity API for repositories labeled with ':regions/China' and ':foundations/linux_foundation', limiting to the top 20 by activity. The output is a console table showing activity, participants, issue comments, open issues, open pulls, review comments, and merged pulls for each repository. ```javascript openDigger.index.activity.getRepoActivity({labelIntersect: [':regions/China', ':foundations/linux_foundation'], startYear, endYear, startMonth, endMonth, groupTimeRange, order, limit: 20}).then(data => { console.table(data); }); ``` ```text ┌─────────┬─────────────┬─────────────────────────────────────────────┬─────────────┬──────────────┬───────────────┬────────────┬────────────┬────────────────┬─────────────┐ │ (index) │ id │ name │ activity │ participants │ issue_comment │ open_issue │ open_pull │ review_comment │ merged_pull │ ├─────────┼─────────────┼─────────────────────────────────────────────┼─────────────┼──────────────┼───────────────┼────────────┼────────────┼────────────────┼─────────────┤ │ 0 │ '208728772' │ 'milvus-io/milvus' │ [ 5047.83 ] │ [ '967' ] │ [ '30913' ] │ [ '1990' ] │ [ '6119' ] │ [ '906' ] │ [ '5617' ] │ │ 1 │ '123983554' │ 'projectacrn/acrn-hypervisor' │ [ 1982.36 ] │ [ '394' ] │ [ '4447' ] │ [ '377' ] │ [ '874' ] │ [ '1566' ] │ [ '815' ] │ │ 2 │ '184341774' │ 'cloud-hypervisor/cloud-hypervisor' │ [ 1206.85 ] │ [ '232' ] │ [ '1700' ] │ [ '181' ] │ [ '1079' ] │ [ '1075' ] │ [ '955' ] │ │ 3 │ '254367182' │ 'sodafoundation/delfin' │ [ 744.04 ] │ [ '151' ] │ [ '421' ] │ [ '117' ] │ [ '198' ] │ [ '977' ] │ [ '151' ] │ │ 4 │ '157422219' │ 'occlum/occlum' │ [ 605.7 ] │ [ '163' ] │ [ '456' ] │ [ '133' ] │ [ '289' ] │ [ '430' ] │ [ '251' ] │ │ 5 │ '149651594' │ 'sodafoundation/multi-cloud' │ [ 316.77 ] │ [ '67' ] │ [ '246' ] │ [ '90' ] │ [ '63' ] │ [ '463' ] │ [ '45' ] │ │ 6 │ '271742278' │ 'baetyl/baetyl-cloud' │ [ 310.72 ] │ [ '72' ] │ [ '127' ] │ [ '5' ] │ [ '197' ] │ [ '140' ] │ [ '186' ] │ │ 7 │ '191105638' │ 'sodafoundation/documentation' │ [ 248.73 ] │ [ '51' ] │ [ '56' ] │ [ '12' ] │ [ '45' ] │ [ '421' ] │ [ '37' ] │ │ 8 │ '195003588' │ 'emqx/kuiper' │ [ 190.17 ] │ [ '67' ] │ [ '194' ] │ [ '63' ] │ [ '78' ] │ [ '24' ] │ [ '78' ] │ │ 9 │ '256235585' │ 'cloud-hypervisor/libvirt' │ [ 163.33 ] │ [ '41' ] │ [ '146' ] │ [ '22' ] │ [ '49' ] │ [ '157' ] │ [ '41' ] │ │ 10 │ '304559463' │ 'occlum/ngo' │ [ 150.38 ] │ [ '21' ] │ [ '27' ] │ [ '11' ] │ [ '117' ] │ [ '173' ] │ [ '96' ] │ │ 11 │ '118551918' │ 'sodafoundation/architecture-analysis' │ [ 145.67 ] │ [ '43' ] │ [ '21' ] │ [ '8' ] │ [ '19' ] │ [ '146' ] │ [ '17' ] │ │ 12 │ '147453621' │ 'TarsCloud/TarsCpp' │ [ 144.74 ] │ [ '72' ] │ [ '109' ] │ [ '20' ] │ [ '42' ] │ [ '0' ] │ [ '27' ] │ │ 13 │ '129361076' │ 'sodafoundation/installer' │ [ 140.52 ] │ [ '43' ] │ [ '43' ] │ [ '27' ] │ [ '39' ] │ [ '110' ] │ [ '37' ] │ │ 14 │ '221669123' │ 'baetyl/baetyl-go' │ [ 139.88 ] │ [ '49' ] │ [ '51' ] │ [ '1' ] │ [ '80' ] │ [ '9' ] │ [ '73' ] │ │ 15 │ '155232169' │ 'sodafoundation/dashboard' │ [ 139.73 ] │ [ '41' ] │ [ '88' ] │ [ '68' ] │ [ '60' ] │ [ '54' ] │ [ '52' ] │ │ 16 │ '181578726' │ 'cloud-hypervisor/rust-hypervisor-firmware' │ [ 134.66 ] │ [ '47' ] │ [ '100' ] │ [ '8' ] │ [ '68' ] │ [ '12' ] │ [ '59' ] │ │ 17 │ '148293609' │ 'baetyl/baetyl' │ [ 129.29 ] │ [ '47' ] │ [ '38' ] │ [ '6' ] │ [ '63' ] │ [ '28' ] │ [ '56' ] │ │ 18 │ '148576574' │ 'TarsCloud/TarsGo' │ [ 118.67 ] │ [ '59' ] │ [ '108' ] │ [ '21' ] │ [ '23' ] │ [ '9' ] │ [ '18' ] │ │ 19 │ '108890846' │ 'openmessaging/benchmark' │ [ 100.63 ] │ [ '37' ] │ [ '55' ] │ [ '11' ] │ [ '33' ] │ [ '43' ] │ [ '31' ] │ └─────────┴─────────────┴─────────────────────────────────────────────┴─────────────┴──────────────┴───────────────┴────────────┴────────────┴────────────────┴─────────────┘ ``` -------------------------------- ### Run ClickHouse container with sample data Source: https://github.com/x-lab2017/open-digger/blob/master/sample_data/README.md Run the ClickHouse container with the extracted data mounted into /data/. Use $(pwd) on Linux/MacOS and %cd% on Windows to ensure the host-src is an absolute path. At least 8 GB memory is required. ```bash docker run -d --name container_name -m 8G -p 8123:8123 -p 9000:9000 --ulimit nofile=262144:262144 --volume=$(pwd)/folder_name/:/data/ registry.cn-beijing.aliyuncs.com/open-digger/open-digger-clickhouse-base:v2 ``` ```bash docker run -d --name container_name -m 8G -p 8123:8123 -p 9000:9000 --ulimit nofile=262144:262144 --volume=%cd%/folder_name/:/data/ registry.cn-beijing.aliyuncs.com/open-digger/open-digger-clickhouse-base:v2 ``` -------------------------------- ### Console log for processing month 24-07 Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/GOSD2025/Chapter3_Global Open Source Influence Ranking/script/Fig_3.2.3_China_Tech_Enterprise_OpenRank_Influence_Decadal_Change.ipynb Console output indicating the start of processing for the month 24-07. ```text 📅 正在处理 24-07 数据... ``` -------------------------------- ### Initialize SHAP and Compute SHAP Values Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/openrank_fitting/main.ipynb Initializes SHAP, sets the LightGBM objective to 'reg:linear', creates a TreeExplainer, and computes SHAP values for the training data. ```python # 导入并初始化shap import shap shap.initjs() # 设置模型的目标 best_model_lgb.params['objective'] ='reg:linear' # shap解释器 explainer_lgb = shap.TreeExplainer(best_model_lgb) # shap value shap_values_lgb = explainer_lgb(X_train) ``` -------------------------------- ### Get label data and count types Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/handbook.ipynb Calls openDigger.label.getLabelData() to retrieve all labels and logs the total number of labels and distinct types. ```javascript var labelData = openDigger.label.getLabelData(); console.log(`OpenDigger has ${labelData.length} labels with ${new Set(labelData.map(l => l.type)).size} types`); ``` ```text Output: OpenDigger has 296 labels with 11 types ``` -------------------------------- ### Process logos with node lib/local/process_logos.js Source: https://github.com/x-lab2017/open-digger/blob/master/AGENTS.md Run from the repository root to process all logos in local_files/logos_origin. The processor walks all of logos_origin, preserves relative paths, and writes 240 x 240 RGBA PNGs to local_files/logos/ using contain-fit centering on a transparent canvas. ```bash node lib/local/process_logos.js ``` -------------------------------- ### Get platform data for Foundation and Company labels Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/handbook.ipynb Calls openDigger.label.getPlatformData(['Foundation', 'Company']) and logs the number of repos and orgs on GitHub for those labels. ```javascript var data = openDigger.label.getPlatformData(['Foundation', 'Company']); console.log(`There are ${data.find(p => p.name === 'GitHub').repos.length} repos on GitHub and ${data.find(p => p.name === 'GitHub').orgs.length} orgs on GitHub for all foundation and company labels`); ``` ```text Output: There are 134 repos on GitHub and 392 orgs on GitHub for all foundation and company labels ``` -------------------------------- ### Import NodeVM and Open Digger modules Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/node_vm2_pycaller.ipynb Imports NodeVM, Open Digger export module, VM context manager, and MultiMime. Also imports copy and pandas for data manipulation. ```python import copy import pandas as pd from node_vm2 import NodeVM from pycjs.open_digger import vm_option_open_digger, get_export_module_open_digger from pycjs.vm_context_manager import VMContext from pycjs.metrics import MultiMime ``` -------------------------------- ### Label data structure example Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/handbook.ipynb Shows the structure of an item returned by openDigger.label.getLabelData(), including identifier, type, name, and platforms with orgs, repos, and users. ```javascript { identifier: string; // The identifier of the label, like :companies/alibaba or :foundations/apache type: string; // The type of the label, like Company or Foundation name: string; // Name of the label, like Alibaba or Apache Software Foundation platforms: { // Platforms data of the label name: string; // Platform name, like GitHub, Gitee and etc. type: string; // Platform type, support `Code Hosting` right now. orgs: { id: number; name: string; }[]; repos: { id: number; name: string; }[]; users: { id: number; name: string; }[]; }[]; } ``` -------------------------------- ### Extract sample data archive with tar Source: https://github.com/x-lab2017/open-digger/blob/master/sample_data/README.md Extract the downloaded sample data archive into a folder. The extraction produces a 'table' and a 'data' file. ```bash tar -zxvf data.tar.gz -C ./folder_name ``` -------------------------------- ### GET /metric/chaoss/issuesClosed Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/handbook.ipynb Documentation for the Issues Closed metric endpoint, which returns the number of issues closed over a specified time period, optionally grouped by company or other dimensions. ```APIDOC ## [GET] /metric/chaoss/issuesClosed ### Description Retrieves the Issues Closed metric from CHAOSS, which measures the number of issues closed over a specified time period. The metric can be grouped by company or other dimensions and supports time-based aggregation. ### Method GET ### Endpoint /metric/chaoss/issuesClosed ### Parameters #### Query Parameters - **labelIntersect** (array) - Optional - Filters results to intersect with the given labels (e.g., ['Company']). - **startYear** (integer) - Required - The starting year of the time range. - **startMonth** (integer) - Required - The starting month (1-12) of the time range. - **endYear** (integer) - Required - The ending year of the time range. - **endMonth** (integer) - Required - The ending month (1-12) of the time range. - **groupBy** (string) - Optional - The dimension to group results by (e.g., 'Company'). - **groupTimeRange** (string) - Optional - The time granularity for grouping (e.g., 'month'). - **limit** (integer) - Optional - Maximum number of results to return. Use -1 for no limit. - **order** (string) - Optional - Sort order, either 'ASC' or 'DESC'. ### Request Example ```json { "labelIntersect": ["Company"], "startYear": 2015, "startMonth": 1, "endYear": 2015, "endMonth": 12, "groupBy": "Company", "groupTimeRange": "month", "limit": -1, "order": "DESC" } ``` ### Response #### Success Response (200) - **data** (array) - List of metric results, each containing: - **name** (string) - The name of the group (e.g., company name). - **ratio** (array) - The ratio values for each time bucket. - **count** (array) - The count values for each time bucket. #### Response Example ```json [ { "name": "Google", "ratio": ["4.6%","4.4%","6.0%","7.4%","6.6%","15%","8.5%","9.6%","9.0%","9.5%","9.2%","10%"] } ] ``` ``` -------------------------------- ### Query repositories with steady growth using gh_events Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/clickhouse_demo.ipynb Queries the gh_events table for WatchEvents, groups by repository and day, then computes the ratio of total stars to daily stars to rank repositories by steady growth. The output is a console.table of the top 20 repositories with their daily_stars, total_stars, and rate. ```python query(` SELECT repo_name, MAX(stars) AS daily_stars, SUM(stars) AS total_stars, ROUND(total_stars / daily_stars, 3) AS rate FROM ( SELECT repo_name, toDate(created_at) AS day, COUNT() AS stars FROM gh_events WHERE type = 'WatchEvent' GROUP BY repo_name, day ) GROUP BY repo_name ORDER BY rate DESC LIMIT 20 `).then(console.table); ``` ```text Output: ┌─────────┬─────────────────────────────────────┬─────────────┬─────────────┬─────────┐ │ (index) │ repo_name │ daily_stars │ total_stars │ rate │ ├─────────┼─────────────────────────────────────┼─────────────┼─────────────┼─────────┤ │ 0 │ 'auth0/node-jsonwebtoken' │ '18' │ '16602' │ 922.333 │ │ 1 │ 'powerline/fonts' │ '28' │ '24756' │ 884.143 │ │ 2 │ 'fxsjy/jieba' │ '34' │ '29750' │ 875 │ │ 3 │ 'zsh-users/zsh-syntax-highlighting' │ '18' │ '15665' │ 870.278 │ │ 4 │ 'cheeriojs/cheerio' │ '29' │ '23578' │ 813.034 │ │ 5 │ 'WordPress/WordPress' │ '20' │ '15414' │ 770.7 │ │ 6 │ 'jaredhanson/passport' │ '24' │ '18286' │ 761.917 │ │ 7 │ 'VundleVim/Vundle.vim' │ '25' │ '18812' │ 752.48 │ │ 8 │ 'square/retrofit' │ '56' │ '41681' │ 744.304 │ │ 9 │ 'tornadoweb/tornado' │ '21' │ '15625' │ 744.048 │ │ 10 │ 'aws/aws-cli' │ '16' │ '11881' │ 742.562 │ │ 11 │ 'tpope/vim-surround' │ '14' │ '10379' │ 741.357 │ │ 12 │ 'pxb1988/dex2jar' │ '15' │ '11079' │ 738.6 │ │ 13 │ 'mybatis/mybatis-3' │ '27' │ '19730' │ 730.741 │ │ 14 │ 'expressjs/multer' │ '15' │ '10851' │ 723.4 │ │ 15 │ 'Modernizr/Modernizr' │ '21' │ '15028' │ 715.619 │ │ 16 │ 'coreybutler/nvm-windows' │ '37' │ '26125' │ 706.081 │ │ 17 │ 'oblador/react-native-vector-icons' │ '25' │ '17480' │ 699.2 │ │ 18 │ 'Alamofire/Alamofire' │ '54' │ '37718' │ 698.481 │ │ 19 │ 'tymondesigns/jwt-auth' │ '17' │ '11859' │ 697.588 │ └─────────┴─────────────────────────────────────┴─────────────┴─────────────┴─────────┘ ``` -------------------------------- ### Set up OpenDigger and define monthly iteration for XSOSI report Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/community_analysis/xsosi-annual-report.ipynb Initializes the OpenDigger environment, defines the monthly iteration helper `forEveryMonth` for the report period (2022-07 to 2023-06), lists the repository IDs to analyze, and creates the `monthes` array and `getEmptyArray` helper for zero-filled trend arrays. ```javascript var openDigger = require('../../src/open_digger'); var now = new Date(); var startYear = 2022, startMonth = 7, endYear = 2023, endMonth = 6; var forEveryMonth = f => { var index = 0; for (var y = startYear; y <= endYear; y++) for (var m = (y === startYear ? startMonth : 1); m <= (y === endYear ? endMonth : 12); m++) f(y, m, index++); }; var repoIds = [288431943, 470364208, 467768946, 461374162, 471252840, 477937218, 428629051, 262909684, 221379769, 259035151, 358757145, 502755745, 508325123, 533339106, 572845686, 639923640 ]; var monthes = []; forEveryMonth((y, m) => monthes.push(`${y}-${m}`)); var getEmptyArray = () => new Array(12).fill(0); ``` -------------------------------- ### Get OpenDigger export module and access openDigger Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/node_vm2_pycaller.ipynb Retrieves the VM instance and the OpenDigger export module, then assigns the openDigger object for subsequent API calls. ```python vm = vmc.vm export_module = get_export_module_open_digger(vm) openDigger = export_module.openDigger ``` -------------------------------- ### Initialize OpenDigger and base options Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/China_open_source_report_2022.ipynb Sets up the OpenDigger client and defines base query options for the year 2022, grouping by year and ordering descending. Used as the foundation for subsequent metric queries in the report. ```javascript var openDigger = require('../src/open_digger'); var year = 2022, startMonth = 1, endMonth = 12; var baseOptions = { startYear: year, endYear: year, startMonth, endMonth, groupTimeRange: 'year', order: 'DESC' }; ``` -------------------------------- ### getUserActivity example for specific users Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/handbook.ipynb Calls openDigger.index.activity.getUserActivity to retrieve activity for the GitHub users k8s-ci-robot and vscode-triage-bot by year from 2015 to 2021. The result is rendered as a scatter plot. ```javascript var startYear = 2015, startMonth = 1, endYear = 2021, endMonth = 12, x = []; for (let y = startYear; y <= endYear; y++) x.push(`${y}`); openDigger.index.activity.getUserActivity({idOrNames: [{ platform: 'GitHub', userLogins: ['k8s-ci-robot', 'vscode-triage-bot'] }], startYear, startMonth, endYear, endMonth, groupTimeRange: 'year'}).then(data => { openDigger.render.plotly(data.map(row => { return { x, y: row.activity, mode: 'scatter', name: row.name }; }), {title: `Activity for users from ${startYear} to ${endYear}`, xaxis: {type: 'category'}}); }); ``` -------------------------------- ### Open the VM context Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/node_vm2_pycaller.ipynb Opens the VM context, returning a NodeVM instance. This must be called before accessing the VM's exports. ```python vmc.open() ``` -------------------------------- ### Create VMContext with open_digger options Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/node_vm2_pycaller.ipynb Initializes a VMContext instance using the vm_option_open_digger options. This is the first step to access the OpenDigger module in the Node.js VM. ```python # use vm_context_manager as a manual context manager vmc = VMContext(**vm_option_open_digger) ``` -------------------------------- ### Load metric tables for all projects Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/openrank_fitting/main.ipynb Iterates over all organizations and projects under data_path, builds a metric table for each project using get_project_metric_table, and appends it to a list. The commented line shows an optional correlation computation. ```python metric_table=[] # 数据表 # 对每个组织 for organization in tqdm(os.listdir(data_path)): org_path=os.path.join(data_path,organization) # 对组织的每个项目 for project in os.listdir(org_path): project_path=os.path.join(org_path,project) proj_metric_table=get_project_metric_table(project_path) metric_table.append(proj_metric_table) # corr_proj=proj_metric_table.corr() ``` -------------------------------- ### GET /metric/chaoss/busFactor Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/handbook.ipynb Documentation for the Bus Factor metric endpoint, which returns the bus factor for a project or group over a specified time period, with options for bot inclusion, percentage threshold, and calculation method. ```APIDOC ## [GET] /metric/chaoss/busFactor ### Description Retrieves the Bus Factor metric from CHAOSS, which identifies the minimum number of developers that would need to leave a project to make it unsustainable. The metric supports options for including bots, setting a percentage threshold, and choosing the calculation method. ### Method GET ### Endpoint /metric/chaoss/busFactor ### Parameters #### Query Parameters - **labelIntersect** (array) - Optional - Filters results to intersect with the given labels (e.g., ['Company']). - **startYear** (integer) - Required - The starting year of the time range. - **startMonth** (integer) - Required - The starting month (1-12) of the time range. - **endYear** (integer) - Required - The ending year of the time range. - **endMonth** (integer) - Required - The ending month (1-12) of the time range. - **groupTimeRange** (string) - Optional - The time granularity for grouping (e.g., 'quarter'). - **limit** (integer) - Optional - Maximum number of results to return. - **order** (string) - Optional - Sort order, either 'ASC' or 'DESC'. - **options** (object) - Optional - Additional options for the metric: - **withBot** (boolean) - Optional - Whether to include GitHub Apps (bots) in the calculation. Default: false. - **percentage** (number) - Optional - The percentage threshold to identify bus factor developers. Default: 0.5. - **by** (string) - Optional - The calculation method: 'commit', 'change request', or 'activity'. Default: 'activity'. ### Request Example ```json { "labelIntersect": ["Company"], "startYear": 2015, "startMonth": 1, "endYear": 2015, "endMonth": 12, "groupTimeRange": "quarter", "limit": 10, "options": { "withBot": true, "percentage": 0.2 }, "order": "DESC" } ``` ### Response #### Success Response (200) - **data** (array) - List of metric results, each containing: - **name** (string) - The name of the group (e.g., company name). - **bus_factor** (array) - The bus factor values for each time bucket. - **detail** (array) - Detailed list of bus factor developers for each time bucket. - **total_contributions** (array) - Total contributions count for each time bucket. #### Response Example ```json [ { "name": "docker/hub-feedback", "bus_factor": [1, 2, 3, 4], "detail": [ ["dev1", 2], ["dev2", 3] ], "total_contributions": [100, 200, 300, 617] } ] ``` ``` -------------------------------- ### 某标签下所有仓库的归一化贡献度排名 Source: https://github.com/x-lab2017/open-digger/blob/master/skills/open-digger-clickhouse-schema/SKILL.md Ranks repositories under a given label (e.g., ':companies/alibaba') by total normalized OpenRank for a specific month. Uses a subquery on flatten_labels to find repos with the label. Returns top 20 by total OpenRank. ```sql SELECT n.repo_name, sum(n.openrank) AS total_openrank FROM normalized_community_openrank AS n WHERE (n.platform, n.repo_id) IN ( SELECT platform, entity_id FROM flatten_labels WHERE id = ':companies/alibaba' AND entity_type = 'Repo' ) AND n.yyyymm = 202501 GROUP BY n.repo_name ORDER BY total_openrank DESC LIMIT 20; ``` -------------------------------- ### Using labelIntersect in getRepoActivity Source: https://github.com/x-lab2017/open-digger/blob/master/labeled_data/README.md Use the labelIntersect parameter in the getRepoActivity metric to filter activity data by labeled categories, such as 'Company' and ':divisions/CN'. This example shows the required parameters for time range, grouping, and ordering. ```javascript openDigger.index.activity.getRepoActivity({labelIntersect: ['Company', ':divisions/CN'], startYear, startMonth, endYear, endMonth, groupBy: 'Company', groupTimeRange: 'year', limit: -1, order: 'DESC' }) ``` -------------------------------- ### Configure OpenRank query options Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/node_vm2_pycaller.ipynb Sets up base options for querying OpenRank data for a range of years and months, and local options to limit the number of results. Merges them into a single options dictionary. ```python year = 2023 startYear = 2019 endYear = year startMonth = 1 endMonth = 12 years = [i + startYear for i in range(5)] baseOptions = { "startYear": startYear, "endYear": endYear, "startMonth": startMonth, "endMonth": endMonth, "groupTimeRange": 'year', "order": 'DESC' } localOptions = { "limit": 10 } options = copy.deepcopy(baseOptions) options.update(localOptions) ```