OpenDigger

https://github.com/x-lab2017/open-digger
Admin
OpenDigger is an open-source data analysis platform initiated by X-lab to analyze, measure, and...
Tokens:184,250
Snippets:594
Trust Score:9.5
License:Apache-2.0
Update:3 weeks ago
Show doc for...
Tokens:
Raw
### Install Python dependencies with pip

Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/node_vm2_pycaller.ipynb

Installs the packages listed in the requirements file `../pycjs/requirements.txt`. The output shows all dependencies are already satisfied, so no changes are made.

```python
!pip install -r ../pycjs/requirements.txt
```

--------------------------------

### getAttention example for Chinese foundations

Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/handbook.ipynb

Calls openDigger.index.attention.getAttention to retrieve attention for Chinese foundation repos grouped by Foundation and by year from 2015 to 2021. The result is rendered as a scatter plot.

```javascript
var startYear = 2015, startMonth = 1, endYear = 2021, endMonth = 12, x = [];
for (let y = startYear; y <= endYear; y++) x.push(`${y}`);
openDigger.index.attention.getAttention({labelIntersect: ['Foundation', ':regions/CN'], startYear, startMonth, endYear, endMonth, groupBy: 'Foundation', groupTimeRange: 'year', limit: -1 }).then(data => {
    openDigger.render.plotly(data.map(row => {
        return { x, y: row.attention, mode: 'scatter', name: row.name };
    }), {title: `Attention for Chinese foundation repos ${startYear} - ${endYear}`, xaxis: {type: 'category'}});
});
```

--------------------------------

### Get GitHub Global Top 20 Repos by OpenRank

Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/China_open_source_blue_paper_2022.ipynb

Fetches the top 20 GitHub repositories by OpenRank for a given year and prints a table. The output shows the resulting table with repository names and their open_rank values.

```javascript
openDigger.index.openrank.getRepoOpenrank({
    startYear: year, endYear: year, startMonth: 1, endMonth: 12,
    groupTimeRange: 'year', limit: 20, order: 'DESC',
}).then(data => console.table(data.map(d => {
    return {
        name: d.name,
        open_rank: d.openrank[0],
    };
})));
```

```text
┌─────────┬───────────────────────────────────┬───────────┐
│ (index) │               name                │ open_rank │
├─────────┼───────────────────────────────────┼───────────┤
│    0    │         'flutter/flutter'         │  24308.7  │
│    1    │       'home-assistant/core'       │ 20570.64  │
│    2    │          'NixOS/nixpkgs'          │ 20413.31  │
│    3    │        'microsoft/vscode'         │ 18565.14  │
│    4    │         'pytorch/pytorch'         │ 16227.35  │
│    5    │    'MicrosoftDocs/azure-docs'     │ 15987.61  │
│    6    │         'dotnet/runtime'          │ 15063.68  │
│    7    │      'kubernetes/kubernetes'      │ 13484.14  │
│    8    │            'odoo/odoo'            │ 11960.45  │
│    9    │         'elastic/kibana'          │ 11731.84  │
│   10    │         'rust-lang/rust'          │ 10332.07  │
│   11    │      'tensorflow/tensorflow'      │  9423.25  │
│   12    │         'grafana/grafana'         │  9388.1   │
│   13    │        'godotengine/godot'        │  8911.33  │
│   14    │ 'DefinitelyTyped/DefinitelyTyped' │  7596.26  │
│   15    │       'microsoft/PowerToys'       │  7493.91  │
│   16    │     'Homebrew/homebrew-core'      │  7403.37  │
│   17    │       'elementor/elementor'       │  7310.76  │
│   18    │          'dotnet/roslyn'          │  7179.64  │
│   19    │        'dotnet/aspnetcore'        │  6966.19  │
└─────────┴───────────────────────────────────┴───────────┘
```

--------------------------------

### getRepoActivity example for Chinese companies

Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/handbook.ipynb

Calls openDigger.index.activity.getRepoActivity to retrieve activity for Chinese projects grouped by Company and by year from 2015 to 2021. The result is rendered as a scatter plot using openDigger.render.plotly.

```javascript
var startYear = 2015, startMonth = 1, endYear = 2021, endMonth = 12, x = [];
for (let y = startYear; y <= endYear; y++) x.push(`${y}`);
openDigger.index.activity.getRepoActivity({labelIntersect: ['Company', ':regions/CN'], startYear, startMonth, endYear, endMonth, groupBy: 'Company', groupTimeRange: 'year', limit: -1, order: 'DESC' }).then(data => {
    openDigger.render.plotly(data.map(row => {
        return { x, y: row.activity, mode: 'scatter', name: row.name };
    }), {title: `Activity for companies from ${startYear} to ${endYear}`, xaxis: {type: 'category'}});
});
```

--------------------------------

### Load OpenDigger Clickhouse driver

Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/clickhouse_demo.ipynb

This JavaScript code loads the OpenDigger Clickhouse driver and sets up a query helper for executing Clickhouse queries. This is the foundation for the subsequent examples.

```javascript
var openDigger = require('../src/open_digger')
var query = q => openDigger.driver.clickhouse.query(q, { format: 'JSONEachRow' });
```

--------------------------------

### Query and visualize developer active hours distribution

Source: https://github.com/x-lab2017/open-digger/blob/master/cooperations/paddle_hackathon_3rd/88_Swift.ipynb

Queries GitHub events for a specific developer, calculates the proportion of events per hour (adjusted for timezone), and renders a line chart. The example uses 'wangkuiyi' as the actor_login and includes a commented-out filter for event types.

```javascript
//活跃时间（'wangkuiyi'所在的时区为-3）
query(    
    `WITH
    (
        SELECT count() 
        FROM github_log.events
        where actor_login = 'wangkuiyi'
    ) AS total_times
     select (24+toHour(created_at) -3) % 24 as hour,(count() / total_times)*100 as propotion
     from github_log.events 
     where actor_login = 'wangkuiyi'
     group by hour
     order by hour
    `
).then(data => {
    var hours=[],propotion = [];
    data.forEach(d => {
        hours.push(d.hour);
        propotion.push(d.propotion);
    });

openDigger.render.plotly([
    {x: hours, y: propotion, type: 'scatter',mode:'line', name: '数量'}], 
    {
        title: '开发者wangkuiyi一天的工作时间',
        xaxis: {type: 'category'},
        height: 600,
    });
});
```

--------------------------------

### Output table of top 20 users by shared star ratio

Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/clickhouse_demo.ipynb

Example output table from the ClickHouse query, showing the top 20 users ranked by the ratio of shared stars. The table includes columns for actor_login, stars_my, stars_other, and ratio. The output is presented as a console.table result.

```text
┌─────────┬────────────────┬──────────┬─────────────┬───────┐
│ (index) │  actor_login   │ stars_my │ stars_other │ ratio │
├─────────┼────────────────┼──────────┼─────────────┼───────┤
│    0    │  'frank-zsy'   │   '41'   │     '0'     │ 1.079 │
│    1    │   'hx94113'    │   '25'   │     '1'     │ 0.641 │
│    2    │   'daydate'    │   '22'   │     '2'     │ 0.55  │
│    3    │ 'blindpirate'  │   '58'   │    '151'    │ 0.307 │
│    4    │ 'lidongze0629' │   '33'   │    '74'     │ 0.295 │
│    5    │    'wtfil'     │   '27'   │    '75'     │ 0.239 │
│    6    │  'Tara-Fuki'   │   '8'    │     '1'     │ 0.205 │
│    7    │   'RayProud'   │   '9'    │     '7'     │  0.2  │
│    8    │   'Jrmy-rbr'   │   '9'    │     '8'     │ 0.196 │
│    9    │   'hcl0908'    │   '7'    │     '1'     │ 0.179 │
│   10    │   'chenDSEE'   │   '7'    │     '2'     │ 0.175 │
│   11    │   'wzl12356'   │   '9'    │    '15'     │ 0.17  │
│   12    │   'Sophie36'   │   '7'    │     '5'     │ 0.163 │
│   13    │   'fireber'    │   '6'    │     '0'     │ 0.158 │
│   14    │ 'oooooutsider' │   '6'    │     '0'     │ 0.158 │
│   15    │  'akselsoft'   │   '6'    │     '1'     │ 0.154 │
│   16    │    'dplrwj'    │   '6'    │     '2'     │ 0.15  │
│   17    │  'dpsisodia'   │   '6'    │     '2'     │ 0.15  │
│   18    │  'Wammy1995'   │   '6'    │     '6'     │ 0.136 │
│   19    │  'steven-wan'  │   '6'    │     '7'     │ 0.133 │
└─────────┴────────────────┴──────────┴─────────────┴───────┘
```

--------------------------------

### Get Linux Foundation China Top 20 Repos by Activity

Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/China_open_source_report_2021.ipynb

Queries the OpenDigger activity API for repositories labeled with ':regions/China' and ':foundations/linux_foundation', limiting to the top 20 by activity. The output is a console table showing activity, participants, issue comments, open issues, open pulls, review comments, and merged pulls for each repository.

```javascript
openDigger.index.activity.getRepoActivity({labelIntersect: [':regions/China', ':foundations/linux_foundation'], startYear, endYear, startMonth, endMonth, groupTimeRange, order, limit: 20}).then(data => {
    console.table(data);
});
```

```text
┌─────────┬─────────────┬─────────────────────────────────────────────┬─────────────┬──────────────┬───────────────┬────────────┬────────────┬────────────────┬─────────────┐
│ (index) │     id      │                    name                     │  activity   │ participants │ issue_comment │ open_issue │ open_pull  │ review_comment │ merged_pull │
├─────────┼─────────────┼─────────────────────────────────────────────┼─────────────┼──────────────┼───────────────┼────────────┼────────────┼────────────────┼─────────────┤
│    0    │ '208728772' │             'milvus-io/milvus'              │ [ 5047.83 ] │  [ '967' ]   │  [ '30913' ]  │ [ '1990' ] │ [ '6119' ] │   [ '906' ]    │ [ '5617' ]  │
│    1    │ '123983554' │        'projectacrn/acrn-hypervisor'        │ [ 1982.36 ] │  [ '394' ]   │  [ '4447' ]   │ [ '377' ]  │ [ '874' ]  │   [ '1566' ]   │  [ '815' ]  │
│    2    │ '184341774' │     'cloud-hypervisor/cloud-hypervisor'     │ [ 1206.85 ] │  [ '232' ]   │  [ '1700' ]   │ [ '181' ]  │ [ '1079' ] │   [ '1075' ]   │  [ '955' ]  │
│    3    │ '254367182' │           'sodafoundation/delfin'           │ [ 744.04 ]  │  [ '151' ]   │   [ '421' ]   │ [ '117' ]  │ [ '198' ]  │   [ '977' ]    │  [ '151' ]  │
│    4    │ '157422219' │               'occlum/occlum'               │  [ 605.7 ]  │  [ '163' ]   │   [ '456' ]   │ [ '133' ]  │ [ '289' ]  │   [ '430' ]    │  [ '251' ]  │
│    5    │ '149651594' │        'sodafoundation/multi-cloud'         │ [ 316.77 ]  │   [ '67' ]   │   [ '246' ]   │  [ '90' ]  │  [ '63' ]  │   [ '463' ]    │  [ '45' ]   │
│    6    │ '271742278' │            'baetyl/baetyl-cloud'            │ [ 310.72 ]  │   [ '72' ]   │   [ '127' ]   │  [ '5' ]   │ [ '197' ]  │   [ '140' ]    │  [ '186' ]  │
│    7    │ '191105638' │       'sodafoundation/documentation'        │ [ 248.73 ]  │   [ '51' ]   │   [ '56' ]    │  [ '12' ]  │  [ '45' ]  │   [ '421' ]    │  [ '37' ]   │
│    8    │ '195003588' │                'emqx/kuiper'                │ [ 190.17 ]  │   [ '67' ]   │   [ '194' ]   │  [ '63' ]  │  [ '78' ]  │    [ '24' ]    │  [ '78' ]   │
│    9    │ '256235585' │         'cloud-hypervisor/libvirt'          │ [ 163.33 ]  │   [ '41' ]   │   [ '146' ]   │  [ '22' ]  │  [ '49' ]  │   [ '157' ]    │  [ '41' ]   │
│   10    │ '304559463' │                'occlum/ngo'                 │ [ 150.38 ]  │   [ '21' ]   │   [ '27' ]    │  [ '11' ]  │ [ '117' ]  │   [ '173' ]    │  [ '96' ]   │
│   11    │ '118551918' │   'sodafoundation/architecture-analysis'    │ [ 145.67 ]  │   [ '43' ]   │   [ '21' ]    │  [ '8' ]   │  [ '19' ]  │   [ '146' ]    │  [ '17' ]   │
│   12    │ '147453621' │             'TarsCloud/TarsCpp'             │ [ 144.74 ]  │   [ '72' ]   │   [ '109' ]   │  [ '20' ]  │  [ '42' ]  │    [ '0' ]     │  [ '27' ]   │
│   13    │ '129361076' │         'sodafoundation/installer'          │ [ 140.52 ]  │   [ '43' ]   │   [ '43' ]    │  [ '27' ]  │  [ '39' ]  │   [ '110' ]    │  [ '37' ]   │
│   14    │ '221669123' │             'baetyl/baetyl-go'              │ [ 139.88 ]  │   [ '49' ]   │   [ '51' ]    │  [ '1' ]   │  [ '80' ]  │    [ '9' ]     │  [ '73' ]   │
│   15    │ '155232169' │         'sodafoundation/dashboard'          │ [ 139.73 ]  │   [ '41' ]   │   [ '88' ]    │  [ '68' ]  │  [ '60' ]  │    [ '54' ]    │  [ '52' ]   │
│   16    │ '181578726' │ 'cloud-hypervisor/rust-hypervisor-firmware' │ [ 134.66 ]  │   [ '47' ]   │   [ '100' ]   │  [ '8' ]   │  [ '68' ]  │    [ '12' ]    │  [ '59' ]   │
│   17    │ '148293609' │               'baetyl/baetyl'               │ [ 129.29 ]  │   [ '47' ]   │   [ '38' ]    │  [ '6' ]   │  [ '63' ]  │    [ '28' ]    │  [ '56' ]   │
│   18    │ '148576574' │             'TarsCloud/TarsGo'              │ [ 118.67 ]  │   [ '59' ]   │   [ '108' ]   │  [ '21' ]  │  [ '23' ]  │    [ '9' ]     │  [ '18' ]   │
│   19    │ '108890846' │          'openmessaging/benchmark'          │ [ 100.63 ]  │   [ '37' ]   │   [ '55' ]    │  [ '11' ]  │  [ '33' ]  │    [ '43' ]    │  [ '31' ]   │
└─────────┴─────────────┴─────────────────────────────────────────────┴─────────────┴──────────────┴───────────────┴────────────┴────────────┴────────────────┴─────────────┘
```

--------------------------------

### Run ClickHouse container with sample data

Source: https://github.com/x-lab2017/open-digger/blob/master/sample_data/README.md

Run the ClickHouse container with the extracted data mounted into /data/. Use $(pwd) on Linux/MacOS and %cd% on Windows to ensure the host-src is an absolute path. At least 8 GB memory is required.

```bash
docker run -d --name container_name -m 8G -p 8123:8123 -p 9000:9000 --ulimit nofile=262144:262144 --volume=$(pwd)/folder_name/:/data/ registry.cn-beijing.aliyuncs.com/open-digger/open-digger-clickhouse-base:v2
```

```bash
docker run -d --name container_name -m 8G -p 8123:8123 -p 9000:9000 --ulimit nofile=262144:262144 --volume=%cd%/folder_name/:/data/ registry.cn-beijing.aliyuncs.com/open-digger/open-digger-clickhouse-base:v2
```

--------------------------------

### Console log for processing month 24-07

Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/GOSD2025/Chapter3_Global Open Source Influence Ranking/script/Fig_3.2.3_China_Tech_Enterprise_OpenRank_Influence_Decadal_Change.ipynb

Console output indicating the start of processing for the month 24-07.

```text
📅 正在处理 24-07 数据...
```

--------------------------------

### Initialize SHAP and Compute SHAP Values

Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/openrank_fitting/main.ipynb

Initializes SHAP, sets the LightGBM objective to 'reg:linear', creates a TreeExplainer, and computes SHAP values for the training data.

```python
# 导入并初始化shap
import shap
shap.initjs()

# 设置模型的目标
best_model_lgb.params['objective'] ='reg:linear'

# shap解释器
explainer_lgb = shap.TreeExplainer(best_model_lgb)

# shap value
shap_values_lgb = explainer_lgb(X_train)
```

--------------------------------

### Get label data and count types

Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/handbook.ipynb

Calls openDigger.label.getLabelData() to retrieve all labels and logs the total number of labels and distinct types.

```javascript
var labelData = openDigger.label.getLabelData();
console.log(`OpenDigger has ${labelData.length} labels with ${new Set(labelData.map(l => l.type)).size} types`);
```

```text
Output:
OpenDigger has 296 labels with 11 types
```

--------------------------------

### Process logos with node lib/local/process_logos.js

Source: https://github.com/x-lab2017/open-digger/blob/master/AGENTS.md

Run from the repository root to process all logos in local_files/logos_origin. The processor walks all of logos_origin, preserves relative paths, and writes 240 x 240 RGBA PNGs to local_files/logos/ using contain-fit centering on a transparent canvas.

```bash
node lib/local/process_logos.js
```

--------------------------------

### Get platform data for Foundation and Company labels

Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/handbook.ipynb

Calls openDigger.label.getPlatformData(['Foundation', 'Company']) and logs the number of repos and orgs on GitHub for those labels.

```javascript
var data = openDigger.label.getPlatformData(['Foundation', 'Company']);
console.log(`There are ${data.find(p => p.name === 'GitHub').repos.length} repos on GitHub and ${data.find(p => p.name === 'GitHub').orgs.length} orgs on GitHub for all foundation and company labels`);
```

```text
Output:
There are 134 repos on GitHub and 392 orgs on GitHub for all foundation and company labels
```

--------------------------------

### Import NodeVM and Open Digger modules

Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/node_vm2_pycaller.ipynb

Imports NodeVM, Open Digger export module, VM context manager, and MultiMime. Also imports copy and pandas for data manipulation.

```python
import copy
import pandas as pd

from node_vm2 import NodeVM

from pycjs.open_digger import vm_option_open_digger, get_export_module_open_digger
from pycjs.vm_context_manager import VMContext
from pycjs.metrics import MultiMime
```

--------------------------------

### Label data structure example

Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/handbook.ipynb

Shows the structure of an item returned by openDigger.label.getLabelData(), including identifier, type, name, and platforms with orgs, repos, and users.

```javascript
{
    identifier: string;     // The identifier of the label, like :companies/alibaba or :foundations/apache
    type: string;           // The type of the label, like Company or Foundation
    name: string;           // Name of the label, like Alibaba or Apache Software Foundation
    platforms: {            // Platforms data of the label
        name: string;       // Platform name, like GitHub, Gitee and etc.
        type: string;       // Platform type, support `Code Hosting` right now.
        orgs: { id: number; name: string; }[];
        repos: { id: number; name: string; }[];
        users: { id: number; name: string; }[];
    }[];
}
```

--------------------------------

### Extract sample data archive with tar

Source: https://github.com/x-lab2017/open-digger/blob/master/sample_data/README.md

Extract the downloaded sample data archive into a folder. The extraction produces a 'table' and a 'data' file.

```bash
tar -zxvf data.tar.gz -C ./folder_name
```

--------------------------------

### GET /metric/chaoss/issuesClosed

Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/handbook.ipynb

Documentation for the Issues Closed metric endpoint, which returns the number of issues closed over a specified time period, optionally grouped by company or other dimensions.

```APIDOC
## [GET] /metric/chaoss/issuesClosed

### Description
Retrieves the Issues Closed metric from CHAOSS, which measures the number of issues closed over a specified time period. The metric can be grouped by company or other dimensions and supports time-based aggregation.

### Method
GET

### Endpoint
/metric/chaoss/issuesClosed

### Parameters
#### Query Parameters
- **labelIntersect** (array) - Optional - Filters results to intersect with the given labels (e.g., ['Company']).
- **startYear** (integer) - Required - The starting year of the time range.
- **startMonth** (integer) - Required - The starting month (1-12) of the time range.
- **endYear** (integer) - Required - The ending year of the time range.
- **endMonth** (integer) - Required - The ending month (1-12) of the time range.
- **groupBy** (string) - Optional - The dimension to group results by (e.g., 'Company').
- **groupTimeRange** (string) - Optional - The time granularity for grouping (e.g., 'month').
- **limit** (integer) - Optional - Maximum number of results to return. Use -1 for no limit.
- **order** (string) - Optional - Sort order, either 'ASC' or 'DESC'.

### Request Example
```json
{
  "labelIntersect": ["Company"],
  "startYear": 2015,
  "startMonth": 1,
  "endYear": 2015,
  "endMonth": 12,
  "groupBy": "Company",
  "groupTimeRange": "month",
  "limit": -1,
  "order": "DESC"
}
```

### Response
#### Success Response (200)
- **data** (array) - List of metric results, each containing:
  - **name** (string) - The name of the group (e.g., company name).
  - **ratio** (array) - The ratio values for each time bucket.
  - **count** (array) - The count values for each time bucket.

#### Response Example
```json
[
  {
    "name": "Google",
    "ratio": ["4.6%","4.4%","6.0%","7.4%","6.6%","15%","8.5%","9.6%","9.0%","9.5%","9.2%","10%"]
  }
]
```
```

--------------------------------

### Query repositories with steady growth using gh_events

Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/clickhouse_demo.ipynb

Queries the gh_events table for WatchEvents, groups by repository and day, then computes the ratio of total stars to daily stars to rank repositories by steady growth. The output is a console.table of the top 20 repositories with their daily_stars, total_stars, and rate.

```python
query(`
SELECT repo_name, MAX(stars) AS daily_stars, SUM(stars) AS total_stars, ROUND(total_stars / daily_stars, 3) AS rate
FROM (
    SELECT repo_name, toDate(created_at) AS day, COUNT() AS stars
    FROM gh_events
    WHERE type = 'WatchEvent'
    GROUP BY repo_name, day
)
GROUP BY repo_name
ORDER BY rate DESC
LIMIT 20
`).then(console.table);
```

```text
Output:
┌─────────┬─────────────────────────────────────┬─────────────┬─────────────┬─────────┐
│ (index) │              repo_name              │ daily_stars │ total_stars │  rate   │
├─────────┼─────────────────────────────────────┼─────────────┼─────────────┼─────────┤
│    0    │      'auth0/node-jsonwebtoken'      │    '18'     │   '16602'   │ 922.333 │
│    1    │          'powerline/fonts'          │    '28'     │   '24756'   │ 884.143 │
│    2    │            'fxsjy/jieba'            │    '34'     │   '29750'   │   875   │
│    3    │ 'zsh-users/zsh-syntax-highlighting' │    '18'     │   '15665'   │ 870.278 │
│    4    │         'cheeriojs/cheerio'         │    '29'     │   '23578'   │ 813.034 │
│    5    │        'WordPress/WordPress'        │    '20'     │   '15414'   │  770.7  │
│    6    │       'jaredhanson/passport'        │    '24'     │   '18286'   │ 761.917 │
│    7    │       'VundleVim/Vundle.vim'        │    '25'     │   '18812'   │ 752.48  │
│    8    │          'square/retrofit'          │    '56'     │   '41681'   │ 744.304 │
│    9    │        'tornadoweb/tornado'         │    '21'     │   '15625'   │ 744.048 │
│   10    │            'aws/aws-cli'            │    '16'     │   '11881'   │ 742.562 │
│   11    │        'tpope/vim-surround'         │    '14'     │   '10379'   │ 741.357 │
│   12    │          'pxb1988/dex2jar'          │    '15'     │   '11079'   │  738.6  │
│   13    │         'mybatis/mybatis-3'         │    '27'     │   '19730'   │ 730.741 │
│   14    │         'expressjs/multer'          │    '15'     │   '10851'   │  723.4  │
│   15    │        'Modernizr/Modernizr'        │    '21'     │   '15028'   │ 715.619 │
│   16    │      'coreybutler/nvm-windows'      │    '37'     │   '26125'   │ 706.081 │
│   17    │ 'oblador/react-native-vector-icons' │    '25'     │   '17480'   │  699.2  │
│   18    │        'Alamofire/Alamofire'        │    '54'     │   '37718'   │ 698.481 │
│   19    │       'tymondesigns/jwt-auth'       │    '17'     │   '11859'   │ 697.588 │
└─────────┴─────────────────────────────────────┴─────────────┴─────────────┴─────────┘
```

--------------------------------

### Set up OpenDigger and define monthly iteration for XSOSI report

Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/community_analysis/xsosi-annual-report.ipynb

Initializes the OpenDigger environment, defines the monthly iteration helper `forEveryMonth` for the report period (2022-07 to 2023-06), lists the repository IDs to analyze, and creates the `monthes` array and `getEmptyArray` helper for zero-filled trend arrays.

```javascript
var openDigger = require('../../src/open_digger');
var now = new Date();
var startYear = 2022, startMonth = 7, endYear = 2023, endMonth = 6;
var forEveryMonth = f => {
    var index = 0;
    for (var y = startYear; y <= endYear; y++)
        for (var m = (y === startYear ? startMonth : 1); m <= (y === endYear ? endMonth : 12); m++)
            f(y, m, index++);
};
var repoIds = [288431943, 470364208, 467768946, 461374162, 471252840, 477937218, 428629051, 262909684, 221379769, 259035151, 358757145, 502755745, 508325123, 533339106, 572845686, 639923640 ];
var monthes = [];
forEveryMonth((y, m) => monthes.push(`${y}-${m}`));
var getEmptyArray = () => new Array(12).fill(0);
```

--------------------------------

### Get OpenDigger export module and access openDigger

Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/node_vm2_pycaller.ipynb

Retrieves the VM instance and the OpenDigger export module, then assigns the openDigger object for subsequent API calls.

```python
vm = vmc.vm
export_module = get_export_module_open_digger(vm)
openDigger = export_module.openDigger
```

--------------------------------

### Initialize OpenDigger and base options

Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/China_open_source_report_2022.ipynb

Sets up the OpenDigger client and defines base query options for the year 2022, grouping by year and ordering descending. Used as the foundation for subsequent metric queries in the report.

```javascript
var openDigger = require('../src/open_digger');
var year = 2022, startMonth = 1, endMonth = 12;
var baseOptions = {
    startYear: year, endYear: year,
    startMonth, endMonth,
    groupTimeRange: 'year',
    order: 'DESC'
};
```

--------------------------------

### getUserActivity example for specific users

Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/handbook.ipynb

Calls openDigger.index.activity.getUserActivity to retrieve activity for the GitHub users k8s-ci-robot and vscode-triage-bot by year from 2015 to 2021. The result is rendered as a scatter plot.

```javascript
var startYear = 2015, startMonth = 1, endYear = 2021, endMonth = 12, x = [];
for (let y = startYear; y <= endYear; y++) x.push(`${y}`);
openDigger.index.activity.getUserActivity({idOrNames: [{ platform: 'GitHub', userLogins: ['k8s-ci-robot', 'vscode-triage-bot'] }], startYear, startMonth, endYear, endMonth, groupTimeRange: 'year'}).then(data => {
    openDigger.render.plotly(data.map(row => {
        return { x, y: row.activity, mode: 'scatter', name: row.name };
    }), {title: `Activity for users from ${startYear} to ${endYear}`, xaxis: {type: 'category'}});
});
```

--------------------------------

### Open the VM context

Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/node_vm2_pycaller.ipynb

Opens the VM context, returning a NodeVM instance. This must be called before accessing the VM's exports.

```python
vmc.open()
```

--------------------------------

### Create VMContext with open_digger options

Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/node_vm2_pycaller.ipynb

Initializes a VMContext instance using the vm_option_open_digger options. This is the first step to access the OpenDigger module in the Node.js VM.

```python
# use vm_context_manager as a manual context manager
vmc = VMContext(**vm_option_open_digger)
```

--------------------------------

### Load metric tables for all projects

Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/openrank_fitting/main.ipynb

Iterates over all organizations and projects under data_path, builds a metric table for each project using get_project_metric_table, and appends it to a list. The commented line shows an optional correlation computation.

```python
metric_table=[]     # 数据表

# 对每个组织
for organization in tqdm(os.listdir(data_path)):
    org_path=os.path.join(data_path,organization)

    # 对组织的每个项目
    for project in os.listdir(org_path):
        project_path=os.path.join(org_path,project)
        proj_metric_table=get_project_metric_table(project_path)
        metric_table.append(proj_metric_table)
        # corr_proj=proj_metric_table.corr()
```

--------------------------------

### GET /metric/chaoss/busFactor

Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/handbook.ipynb

Documentation for the Bus Factor metric endpoint, which returns the bus factor for a project or group over a specified time period, with options for bot inclusion, percentage threshold, and calculation method.

```APIDOC
## [GET] /metric/chaoss/busFactor

### Description
Retrieves the Bus Factor metric from CHAOSS, which identifies the minimum number of developers that would need to leave a project to make it unsustainable. The metric supports options for including bots, setting a percentage threshold, and choosing the calculation method.

### Method
GET

### Endpoint
/metric/chaoss/busFactor

### Parameters
#### Query Parameters
- **labelIntersect** (array) - Optional - Filters results to intersect with the given labels (e.g., ['Company']).
- **startYear** (integer) - Required - The starting year of the time range.
- **startMonth** (integer) - Required - The starting month (1-12) of the time range.
- **endYear** (integer) - Required - The ending year of the time range.
- **endMonth** (integer) - Required - The ending month (1-12) of the time range.
- **groupTimeRange** (string) - Optional - The time granularity for grouping (e.g., 'quarter').
- **limit** (integer) - Optional - Maximum number of results to return.
- **order** (string) - Optional - Sort order, either 'ASC' or 'DESC'.
- **options** (object) - Optional - Additional options for the metric:
  - **withBot** (boolean) - Optional - Whether to include GitHub Apps (bots) in the calculation. Default: false.
  - **percentage** (number) - Optional - The percentage threshold to identify bus factor developers. Default: 0.5.
  - **by** (string) - Optional - The calculation method: 'commit', 'change request', or 'activity'. Default: 'activity'.

### Request Example
```json
{
  "labelIntersect": ["Company"],
  "startYear": 2015,
  "startMonth": 1,
  "endYear": 2015,
  "endMonth": 12,
  "groupTimeRange": "quarter",
  "limit": 10,
  "options": {
    "withBot": true,
    "percentage": 0.2
  },
  "order": "DESC"
}
```

### Response
#### Success Response (200)
- **data** (array) - List of metric results, each containing:
  - **name** (string) - The name of the group (e.g., company name).
  - **bus_factor** (array) - The bus factor values for each time bucket.
  - **detail** (array) - Detailed list of bus factor developers for each time bucket.
  - **total_contributions** (array) - Total contributions count for each time bucket.

#### Response Example
```json
[
  {
    "name": "docker/hub-feedback",
    "bus_factor": [1, 2, 3, 4],
    "detail": [
      ["dev1", 2],
      ["dev2", 3]
    ],
    "total_contributions": [100, 200, 300, 617]
  }
]
```
```

--------------------------------

### 某标签下所有仓库的归一化贡献度排名

Source: https://github.com/x-lab2017/open-digger/blob/master/skills/open-digger-clickhouse-schema/SKILL.md

Ranks repositories under a given label (e.g., ':companies/alibaba') by total normalized OpenRank for a specific month. Uses a subquery on flatten_labels to find repos with the label. Returns top 20 by total OpenRank.

```sql
SELECT n.repo_name, sum(n.openrank) AS total_openrank
FROM normalized_community_openrank AS n
WHERE (n.platform, n.repo_id) IN (
    SELECT platform, entity_id
    FROM flatten_labels
    WHERE id = ':companies/alibaba' AND entity_type = 'Repo'
)
  AND n.yyyymm = 202501
GROUP BY n.repo_name
ORDER BY total_openrank DESC
LIMIT 20;
```

--------------------------------

### Using labelIntersect in getRepoActivity

Source: https://github.com/x-lab2017/open-digger/blob/master/labeled_data/README.md

Use the labelIntersect parameter in the getRepoActivity metric to filter activity data by labeled categories, such as 'Company' and ':divisions/CN'. This example shows the required parameters for time range, grouping, and ordering.

```javascript
openDigger.index.activity.getRepoActivity({labelIntersect: ['Company', ':divisions/CN'], startYear, startMonth, endYear, endMonth, groupBy: 'Company', groupTimeRange: 'year', limit: -1, order: 'DESC' })
```

--------------------------------

### Configure OpenRank query options

Source: https://github.com/x-lab2017/open-digger/blob/master/notebook/node_vm2_pycaller.ipynb

Sets up base options for querying OpenRank data for a range of years and months, and local options to limit the number of results. Merges them into a single options dictionary.

```python
year = 2023
startYear = 2019
endYear = year
startMonth = 1
endMonth = 12
years = [i + startYear for i in range(5)]
baseOptions = {
   "startYear": startYear,
   "endYear": endYear,
   "startMonth": startMonth,
   "endMonth": endMonth,
   "groupTimeRange": 'year',
   "order": 'DESC'
}

localOptions = {
   "limit": 10
}

options = copy.deepcopy(baseOptions)
options.update(localOptions)
```
Tokens:
Raw