### Install Development Dependencies Source: https://github.com/jannisborn/paperscraper/blob/main/CONTRIBUTING.md Use uv to synchronize development dependencies for the project. ```sh uv sync --group dev ``` -------------------------------- ### Install Project Dependencies Locally Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/getting-started.md Install project dependencies and development tools into a local virtual environment using uv sync. This command is typically used after cloning a project repository. ```console uv sync ``` -------------------------------- ### Quick Example: Dump Queries Source: https://github.com/jannisborn/paperscraper/blob/main/docs/README.md Demonstrates how to use the dump_queries function to search scholarly metadata across multiple categories. Nested lists define Boolean logic for queries. ```python from paperscraper import dump_queries ai = ["Artificial intelligence", "Machine learning"] qc = ["Quantum computing", "Quantum information", "Quantum algorithm"] chemistry = ["Chemistry", "Chemical", "Molecule", "Materials science"] dump_queries([[ai, qc, chemistry]], ".") ``` -------------------------------- ### Get Dumps Module Source: https://github.com/jannisborn/paperscraper/blob/main/docs/api/index.md Utilities for downloading metadata from bioRxiv, medRxiv, and chemRxiv. ```APIDOC ## Get Dumps Module [`paperscraper.get_dumps`](get_dumps.md) — Utilities to download bioRxiv, medRxiv & chemRxiv metadata ``` -------------------------------- ### Install paperscraper with uv Source: https://github.com/jannisborn/paperscraper/blob/main/docs/README.md Install the paperscraper package using uv, a fast Python package installer and resolver. This is an alternative to pip. ```sh uv add paperscraper ``` -------------------------------- ### Install paperscraper with pip Source: https://github.com/jannisborn/paperscraper/blob/main/docs/README.md Install the paperscraper package using pip. This is the standard way to add Python packages to your environment. ```sh pip install paperscraper ``` -------------------------------- ### Query Local arXiv Dump Source: https://github.com/jannisborn/paperscraper/blob/main/README.md Function to get and dump arXiv papers specifically from a local backend. This is used after downloading dumps. ```python from paperscraper.arxiv import get_and_dump_arxiv_papers get_and_dump_arxiv_papers(..., backend='local') ``` -------------------------------- ### Get Paper Citation Counts by DOI Source: https://github.com/jannisborn/paperscraper/blob/main/README.md Retrieve the number of citations for a paper using its DOI. This is a direct lookup function. ```python from paperscraper.citations import get_citations_by_doi get_citations_by_doi("10.1021/acs.jcim.3c00132") ``` -------------------------------- ### Analyze Researcher Self-Citation Tendencies Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/self-citation-analysis.md Aggregate self-reference and self-citation tendencies across an author's publication record using the Researcher class. This example limits analysis to a single paper for brevity. ```python from paperscraper.citations.entity import Researcher researcher = Researcher("2289839817") researcher.ssids = ["2c1edb95c07643a834c9d4f8f2acedfecfe894de"] _ = researcher.self_citations() result = researcher.self_references() print(result.name) print(result.self_citation_ratio) print(result.self_reference_ratio) print(result.num_citations) print(result.num_references) print(result.self_references) print(result) ``` -------------------------------- ### Get arXiv API Papers as DataFrame Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/paper-keyword-analysis.md Use `get_arxiv_papers_api` to retrieve arXiv API search results directly into a pandas DataFrame. Specify `max_results` for the number of papers. ```python from paperscraper.arxiv import get_arxiv_papers_api papers = get_arxiv_papers_api('all:"quantum machine learning"', max_results=25) ``` -------------------------------- ### Get PubMed Papers as DataFrame Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/paper-keyword-analysis.md Use `get_pubmed_papers` to retrieve PubMed search results directly into a pandas DataFrame. Supports custom fields like author emails. ```python from paperscraper.pubmed import get_pubmed_papers papers = get_pubmed_papers( "(machine learning) AND (zoology)", fields=["title", "doi", "emails"], max_results=50, ) ``` -------------------------------- ### Extract Researcher Self-Citations and References Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/self-citation-analysis.md Use SelfLinkClient with mode 'author' to extract self-citations and self-references for a researcher, identified by their Semantic Scholar ID. This example shows how to set a specific paper list for analysis. ```python >>> author_client = SelfLinkClient("2289839817", mode="author") >>> author_client.object.ssids = ["2c1edb95c07643a834c9d4f8f2acedfecfe894de"] >>> author_client.extract() >>> author_client.get_result() ResearcherResult( name="K. Wijk", self_reference_ratio=4.65, self_citation_ratio=0.0, num_references=43, num_citations=10, self_references={ "Diff-SPORT: Diffusion-based Sensor Placement Optimization and Reconstruction of Turbulent flows in urban environments": 4.65, }, self_citations={ "Diff-SPORT: Diffusion-based Sensor Placement Optimization and Reconstruction of Turbulent flows in urban environments": 0.0, }, ssaid=2289839817, orcid=None, ) ``` -------------------------------- ### Get Google Scholar Papers as DataFrame Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/paper-keyword-analysis.md Use `get_scholar_papers` to retrieve Google Scholar search results directly into a pandas DataFrame. Note that Google Scholar may prompt captchas during large automated runs. ```python from paperscraper.scholar import get_scholar_papers papers = get_scholar_papers("GT4SD") ``` -------------------------------- ### Batch Download with API Keys from File Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/pdf-retrieval.md When downloading from a dump, you can specify a file containing API keys for services like Wiley or Elsevier by passing the file path to `api_keys`. ```python save_pdf_from_dump( "pubmed_query_results.jsonl", pdf_path="papers", key_to_save="doi", api_keys="api_keys.txt", ) ``` -------------------------------- ### Format Code with Ruff Source: https://github.com/jannisborn/paperscraper/blob/main/CONTRIBUTING.md Use uv to run Ruff for code formatting. ```sh uv run ruff format paperscraper ``` -------------------------------- ### Check Code with Ruff Source: https://github.com/jannisborn/paperscraper/blob/main/CONTRIBUTING.md Use uv to run Ruff for code checking and linting. ```sh uv run ruff check paperscraper ``` -------------------------------- ### Sort Imports with isort Source: https://github.com/jannisborn/paperscraper/blob/main/CONTRIBUTING.md Use uv to run isort for sorting Python imports. ```sh uv run isort paperscraper ``` -------------------------------- ### Load and Use API Keys for Single Download Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/pdf-retrieval.md API keys can be loaded once using `load_api_keys` and then passed to `save_pdf` for subsequent downloads. ```python from paperscraper.pdf import load_api_keys, save_pdf api_keys = load_api_keys("api_keys.txt") save_pdf( {"doi": "10.1101/786871"}, filepath="taskload.pdf", api_keys=api_keys, ) ``` -------------------------------- ### Download arXiv Dumps with API Backend Source: https://github.com/jannisborn/paperscraper/blob/main/README.md Use the API backend for smaller, API-backed arXiv dumps within a specific date range. This is an alternative to the faster Kaggle backend. ```python from paperscraper.get_dumps import arxiv arxiv(start_date='2024-01-01',end_date='2024-01-04',backend='api') ``` -------------------------------- ### Download arXiv Dumps with Kaggle Backend Source: https://github.com/jannisborn/paperscraper/blob/main/README.md Use the Kaggle backend to bulk-download arXiv papers within a specified date range. This is the default and faster backend. ```python from paperscraper.get_dumps import arxiv arxiv(start_date='2019-01-01', end_date='2026-12-31') ``` -------------------------------- ### Run Python Code with PaperScraper Source: https://github.com/jannisborn/paperscraper/blob/main/CONTRIBUTING.md Execute Python code using uv, demonstrating the import of the paperscraper library. ```sh uv run python -c "import paperscraper" ``` -------------------------------- ### Download arXiv Metadata Dump using Kaggle Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/getting-started.md Download the entire arXiv dataset using the Kaggle backend. Ensure you have logged in to your Kaggle account using 'kaggle auth login' before running this command. ```python from paperscraper.get_dumps import arxiv arxiv(start_date="2019-01-01", end_date="2026-12-31") ``` -------------------------------- ### Configure xRxiv Download Parameters Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/getting-started.md Customize download parameters for bioRxiv, including retries, timeouts, backoff, workers, and window days. This is useful for tuning performance and handling API issues. ```python biorxiv( max_retries=12, request_timeout=(5.0, 45.0), retry_backoff_seconds=1.0, max_workers=8, window_days=30, ) ``` -------------------------------- ### Query Local xRxiv Dumps with XRXivQuery Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/paper-keyword-analysis.md Use `XRXivQuery` to search local xRxiv dumps (bioRxiv, medRxiv, chemRxiv) after downloading them. Saves results to a JSONL file. ```python from paperscraper.xrxiv.xrxiv_query import XRXivQuery querier = XRXivQuery("server_dumps/chemrxiv_2020-11-10.jsonl") querier.search_keywords(query, output_filepath="ai_quantum_chemistry.jsonl") ``` -------------------------------- ### Download chemRxiv dumps Source: https://github.com/jannisborn/paperscraper/blob/main/README.md Download the entire history of papers from chemRxiv. This process takes less than 15 minutes and adds over 50,000 papers. ```python from paperscraper.get_dumps import biorxiv, medrxiv, chemrxiv chemrxiv() ``` -------------------------------- ### Download arXiv Metadata Dump using API Backend Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/getting-started.md Download a smaller, API-backed metadata dump for arXiv for a specific date range. This is an alternative to the full Kaggle dump. ```python from paperscraper.get_dumps import arxiv arxiv(start_date="2024-01-01", end_date="2024-01-04", backend="api") ``` -------------------------------- ### Run Tests with Pytest Source: https://github.com/jannisborn/paperscraper/blob/main/CONTRIBUTING.md Use uv to execute Pytest for running project tests. ```sh uv run pytest paperscraper ``` -------------------------------- ### Search Local arXiv Dump Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/getting-started.md Search the locally downloaded arXiv dump using the 'local' backend. This allows for faster searching compared to using the arXiv API directly. ```python from paperscraper.arxiv import get_and_dump_arxiv_papers get_and_dump_arxiv_papers(..., backend="local") ``` -------------------------------- ### Dump Multiple Queries Across All Backends Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/paper-keyword-analysis.md Use `dump_queries` to run multiple search queries across all available backends. This function aggregates results from various sources. ```python from paperscraper import dump_queries physics = [ "Physics", "Physical", "Particle", "Condensed matter", "Many-body", "Fermion", "Hamiltonian", "Spin", ] biology = ["Biology", "Biological", "Cellular", "Genomics", "Gene", "Protein"] queries = [[ai, qc, chemistry], [ai, qc, physics], [ai, qc, biology]] dump_queries(queries, ".") ``` -------------------------------- ### Xrxiv Module Source: https://github.com/jannisborn/paperscraper/blob/main/docs/api/index.md Shared utilities for bioRxiv, medRxiv, and chemRxiv. ```APIDOC ## Xrxiv Module [`paperscraper.xrxiv`](xrxiv.md) — Shared utilities for {bio,med,chem}Rxiv ``` -------------------------------- ### Download xRxiv Dumps for Specific Dates Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/getting-started.md Download metadata dumps for a specific date range from medRxiv. The resulting file is labeled with the current date and used for subsequent local searches. ```python medrxiv(start_date="2023-04-01", end_date="2023-04-08") ``` -------------------------------- ### Authenticate Kaggle for arXiv Dumps Source: https://github.com/jannisborn/paperscraper/blob/main/README.md Authenticate your Kaggle account before using the Kaggle backend for downloading arXiv dumps. This is a command-line operation. ```sh kaggle auth login ``` -------------------------------- ### Download medRxiv dumps Source: https://github.com/jannisborn/paperscraper/blob/main/README.md Download the entire history of papers from medRxiv. This process takes less than 5 minutes and adds over 100,000 papers. ```python medrxiv() ``` -------------------------------- ### Paperscraper Main Package Source: https://github.com/jannisborn/paperscraper/blob/main/docs/api/index.md Entry point for the paperscraper package. This section details the top-level API and its functionalities. ```APIDOC ## Top-level API ::: paperscraper options: show_if_no_docstring: false show_submodules: false filters: - "!^_[^_]" ``` -------------------------------- ### Query Specific Preprint Servers via QUERY_FN_DICT Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/paper-keyword-analysis.md Access specific preprint server query functions through `QUERY_FN_DICT` for targeted searches on local dumps. Useful for querying only one server. ```python from paperscraper.load_dumps import QUERY_FN_DICT QUERY_FN_DICT["biorxiv"](query, output_filepath="biorxiv_ai_quantum_chemistry.jsonl") QUERY_FN_DICT["chemrxiv"](query, output_filepath="chemrxiv_ai_quantum_chemistry.jsonl") ``` -------------------------------- ### Download xRxiv Metadata Dumps Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/getting-started.md Download historical metadata dumps for bioRxiv, medRxiv, and chemRxiv. These dumps are stored locally in JSONL format and can be used for offline analysis. ```python from paperscraper.get_dumps import biorxiv, medrxiv, chemrxiv chemrxiv() # Takes <15min -> +50K papers (~30 MB file) medrxiv() # Takes <5min -> +100K papers (~200 MB file) biorxiv() # Takes <1h -> +450K papers (~800 MB file) ``` -------------------------------- ### Batch Download PDFs from Metadata Dump Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/pdf-retrieval.md Use `save_pdf_from_dump` to download multiple papers from a JSONL file. Specify the directory for PDFs and the key to use for identification. ```python from paperscraper.pdf import save_pdf_from_dump save_pdf_from_dump( "ai_quantum_chemistry.jsonl", pdf_path="papers", key_to_save="doi", ) ``` -------------------------------- ### Dump arXiv Papers with Date Bounds and Infer Backend Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/paper-keyword-analysis.md Save arXiv search results to a JSONL file, using local dumps if available (`backend="infer"`) or falling back to the API. Supports date filtering. ```python get_and_dump_arxiv_papers( query, output_filepath="arxiv_ai_quantum_chemistry_2024.jsonl", start_date="2024-01-01", end_date="2024-12-31", backend="infer", ) ``` -------------------------------- ### Retrieve and Save PDF by DOI Source: https://github.com/jannisborn/paperscraper/blob/main/README.md Download a PDF file from a given DOI and save it to a specified filepath. Returns True if the file was saved successfully. ```python from paperscraper.pdf import save_pdf save_pdf({"doi": "10.48550/arXiv.2207.03928"}, filepath="gt4sd_paper.pdf") ``` -------------------------------- ### Download bioRxiv dumps Source: https://github.com/jannisborn/paperscraper/blob/main/README.md Download the entire history of papers from bioRxiv. This process takes less than 1 hour and adds over 450,000 papers. ```python biorxiv() ``` -------------------------------- ### Download PDF with Metadata Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/pdf-retrieval.md To save paper metadata alongside the downloaded PDF, set `save_metadata=True` in the `save_pdf` function. ```python save_pdf(paper, filepath="gt4sd_paper.pdf", save_metadata=True) ``` -------------------------------- ### ArXiv Module Source: https://github.com/jannisborn/paperscraper/blob/main/docs/api/index.md Documentation for the ArXiv module, which handles scraping and keyword searches on ArXiv. ```APIDOC ## ArXiv Module [`paperscraper.arxiv`](arxiv.md) — ArXiv scraping & keyword search ``` -------------------------------- ### Set Semantic Scholar API Key Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/scholar-metrics-analysis.md Configure the Semantic Scholar API key for increased throughput. Set the SS_API_KEY environment variable. ```sh export SS_API_KEY=YOUR_API_KEY ``` -------------------------------- ### Download bioRxiv dumps with custom retry and timeout settings Source: https://github.com/jannisborn/paperscraper/blob/main/README.md Download bioRxiv papers with custom retry, timeout, and worker settings. This allows tuning for API connection issues. ```python biorxiv( max_retries=12, request_timeout=(5.0, 45.0), # connect timeout, read timeout retry_backoff_seconds=1.0, # initial retry backoff max_workers=8, # number of parallel date windows window_days=30, # smaller windows increase parallelism ) ``` -------------------------------- ### Return All Available Journal Fields Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/scholar-metrics-analysis.md This snippet shows how to retrieve all available fields for a journal search, including JCR quartile, NLM ID, ISSNs, and more. Set `return_all=True` to enable this. ```python >>> impactor.search("quantum information", threshold=90, return_all=True) [ { "factor": 8.3, "jcr": "Q1", "nlm_id": "101722857", "journal": "npj Quantum Information", "issn": ".", "zky": ".", "journal_abbr": "npj Quantum Inf", "eissn": "2056-6387", "score": 92, }, { "factor": 2.9, "jcr": "Q2", "nlm_id": "101703749", "journal": "Information", "issn": ".", "zky": ".", "journal_abbr": "Information (Basel)", "eissn": "2078-2489", "score": 95, }, { "factor": 1.3, "jcr": "Q2", "nlm_id": "9877123", "journal": "NATION", "issn": "0027-8378", "zky": ".", "journal_abbr": "Nation", "eissn": "0027-8378", "score": 91, }, { "factor": 1.1, "jcr": ".", "nlm_id": "138060", "journal": "Reformation", "issn": "1357-4175", "zky": ".", "journal_abbr": "Reformation", "eissn": "1752-0738", "score": 90, }, ] ``` -------------------------------- ### Dump arXiv Papers with Nested Queries Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/paper-keyword-analysis.md Use `get_and_dump_arxiv_papers` to save arXiv search results to a JSONL file. Accepts nested lists for complex queries. ```python from paperscraper.arxiv import get_and_dump_arxiv_papers get_and_dump_arxiv_papers(query, output_filepath="ai_quantum_chemistry.jsonl") ``` -------------------------------- ### PDF Module Source: https://github.com/jannisborn/paperscraper/blob/main/docs/api/index.md Module for downloading publications in PDF format. ```APIDOC ## PDF Module [`paperscraper.pdf`](pdf.md) — Download publications as pdfs ``` -------------------------------- ### Search Journals by Name with Threshold and Sort Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/scholar-metrics-analysis.md Use this snippet to search for journals by their name, applying a fuzzy match threshold and sorting the results. The `threshold` parameter defaults to 100 for exact matches. ```python >>> from paperscraper.impact import Impactor >>> impactor = Impactor() >>> impactor.search("Nat Comms", threshold=85, sort_by="impact") [ {"journal": "Nature Computational Science", "factor": 18.3, "score": 88}, {"journal": "Nature Communications", "factor": 15.7, "score": 94}, {"journal": "Natural Computing", "factor": 1.6, "score": 88}, ] ``` -------------------------------- ### Download Single PDF by DOI Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/pdf-retrieval.md Use `save_pdf` to download a single paper's PDF using its DOI. The function returns True on success. ```python from paperscraper.pdf import save_pdf paper = {"doi": "10.48550/arXiv.2207.03928"} save_pdf(paper, filepath="gt4sd_paper.pdf") ``` -------------------------------- ### Generate Venn Diagrams for Keyword Overlap Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/paper-keyword-analysis.md Creates Venn diagrams to visualize the overlap of keywords across different datasets and years. Supports two-set, three-set, and multiple combined Venn diagrams. ```python from paperscraper.plotting import plot_venn_two, plot_venn_three, plot_multiple_venn sizes_2024 = (18762, 6324, 624) sizes_2025 = (18774, 6966, 775, 28652, 1965, 597, 77) labels_2024 = ["Artificial\nIntelligence", "Quantum\nComputing"] labels_2025 = ("Artificial\nIntelligence", "Quantum\nComputing", "Chemistry") plot_venn_two( sizes_2024, labels_2024, title="2024", figpath="assets/ai_quantum_venn_2024.png", ) plot_venn_three( sizes_2025, labels_2025, title="2025", figpath="assets/ai_quantum_chemistry_venn_2025.png", ) plot_multiple_venn( [sizes_2024, sizes_2025], [labels_2024, labels_2025], titles=["2024", "2025"], suptitle="Keyword search comparison", gridspec_kw={"width_ratios": [1, 2]}, figsize=(10, 6), figpath="assets/ai_quantum_venn_both.png", ) ``` -------------------------------- ### Plot Comparison Bar Chart Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/paper-keyword-analysis.md Generates a comparison bar chart showing the distribution of papers across different fields over several years. This function is used after aggregating data for multiple queries. ```python from paperscraper.plotting import plot_comparison data_keys = [ "artificialintelligence_quantumcomputing_chemistry.jsonl", "artificialintelligence_quantumcomputing_physics.jsonl", "artificialintelligence_quantumcomputing_biology.jsonl", "artificialintelligence_quantumcomputing_medicine.jsonl", ] plot_comparison( data_dict, data_keys, x_ticks=[str(year) for year in range(2019, 2027)], title_text="'Artificial intelligence' AND 'Quantum computing' AND X", keyword_text=["Chemistry", "Physics", "Biology", "Medicine"], figpath="assets/ai_quantum_fields.png", ) ``` -------------------------------- ### Citations Module Source: https://github.com/jannisborn/paperscraper/blob/main/docs/api/index.md Documentation for the citations module, used to retrieve (self-)citations and (self-)references of papers and authors. ```APIDOC ## Citations Module [`paperscraper.citations`](citations.md) — Get (self-)citations & (self-)reference of papers and authors ``` -------------------------------- ### Search Journal Impact Metrics Source: https://github.com/jannisborn/paperscraper/blob/main/README.md Search for journal metrics, such as impact factor, based on journal name and a threshold. Results are sorted by impact. ```python from paperscraper.impact import Impactor Impactor().search("Nat Comms", threshold=85, sort_by="impact") ``` -------------------------------- ### Dump Google Scholar Papers with Plain Text Query Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/paper-keyword-analysis.md Use `get_and_dump_scholar_papers` to save Google Scholar search results to a JSONL file. Google Scholar uses a plain text query format. ```python from paperscraper.scholar import get_and_dump_scholar_papers get_and_dump_scholar_papers("Machine Learning") ``` -------------------------------- ### Aggregate Paper Data for Bar Plots Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/paper-keyword-analysis.md Aggregates paper data from JSONL files for specific queries and databases. It processes data for multiple fields (AI, Quantum Computing combined with Chemistry, Physics, Biology, Medicine) and stores the aggregated counts per year. ```python import os from paperscraper import QUERY_FN_DICT from paperscraper.postprocessing import aggregate_paper from paperscraper.utils import get_filename_from_query, load_jsonl medicine = ["Medicine", "Medical", "Clinical", "Disease", "Patient", "Health"] queries = [ [ai, qc, chemistry], [ai, qc, physics], [ai, qc, biology], [ai, qc, medicine], ] root = "../keyword_dumps" data_dict = {} for query in queries: filename = get_filename_from_query(query) data_dict[filename] = {} for db, _ in QUERY_FN_DICT.items(): data = load_jsonl(os.path.join(root, db, filename)) data_dict[filename][db], _filtered = aggregate_paper( data, 2019, bins_per_year=1, filtering=True, filter_keys=query, return_filtered=True, last_year=2026, ) ``` -------------------------------- ### Search Journals by Abbreviation, NLM ID, or E-ISSN Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/scholar-metrics-analysis.md This snippet demonstrates searching for journals using their abbreviation, NLM ID, or E-ISSN. These searches are typically more precise. ```python >>> impactor.search("Nat Rev Earth Environ") [{"journal": "Nature Reviews Earth & Environment", "factor": 71.5, "score": 100}] ``` ```python >>> impactor.search("101771060") [{"journal": "Nature Reviews Earth & Environment", "factor": 71.5, "score": 100}] ``` ```python >>> impactor.search("2662-138X") [{"journal": "Nature Reviews Earth & Environment", "factor": 71.5, "score": 100}] ``` -------------------------------- ### Plot Single Bar Chart Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/paper-keyword-analysis.md Generates a bar chart for a single query, useful for detailed analysis of one specific field. This function takes a subset of the aggregated data. ```python from paperscraper.plotting import plot_single plot_single( data_dict, [data_keys[0]], x_ticks=[str(year) for year in range(2019, 2027)], title_text="'Artificial intelligence' AND 'Quantum computing' AND Chemistry", figpath="assets/ai_quantum_chemistry_single.png", ) ``` -------------------------------- ### Dump PubMed Papers with Date Bounds and Custom Fields Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/paper-keyword-analysis.md Save PubMed search results to a JSONL file with specified date ranges and custom fields. Supports `start_date` and `end_date` parameters. ```python get_and_dump_pubmed_papers( query, output_filepath="pubmed_ai_quantum_chemistry_2024.jsonl", fields=["title", "authors", "date", "doi", "emails"], start_date="2024/01/01", end_date="2024/12/31", ) ``` -------------------------------- ### Analyze Paper Self-Citations and Self-References Source: https://github.com/jannisborn/paperscraper/blob/main/README.md Estimate paper-level self-citations and self-references using DOIs. This provides mean self-citation and self-reference percentages across paper authors. ```python from paperscraper.citations import self_citations_paper, self_references_paper doi = "10.1038/s41586-023-06600-9" self_citations_paper(doi).citation_score self_references_paper(doi).reference_score ``` -------------------------------- ### Scholar Module Source: https://github.com/jannisborn/paperscraper/blob/main/docs/api/index.md Module providing endpoints for Google Scholar. ```APIDOC ## Scholar Module [`paperscraper.scholar`](scholar.md) — Google Scholar endpoints ``` -------------------------------- ### Dump PubMed Papers with Nested Queries Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/paper-keyword-analysis.md Use `get_and_dump_pubmed_papers` to save PubMed search results to a JSONL file. Define search terms using nested lists for AND/OR logic. ```python from paperscraper.pubmed import get_and_dump_pubmed_papers ai = ["Artificial intelligence", "Machine learning"] qc = [ "Quantum computing", "Quantum computer", "Quantum information", "Quantum algorithm", "Quantum circuit", "Quantum simulation", "Quantum machine learning", "Qubit", "Quantum annealing", ] chemistry = ["Chemistry", "Chemical", "Molecule", "Molecular", "Materials science"] query = [ai, qc, chemistry] get_and_dump_pubmed_papers(query, output_filepath="ai_quantum_chemistry.jsonl") ``` -------------------------------- ### Retrieve Author Metrics from Semantic Scholar Source: https://github.com/jannisborn/paperscraper/blob/main/README.md Fetch author-level metrics like paper count, citation count, and h-index from Semantic Scholar using an ORCID. Requires conversion from ORCID to Semantic Scholar ID. ```python from paperscraper.citations.orcid import orcid_to_author_name from paperscraper.citations.utils import author_name_to_ssaid, semantic_scholar_requests_get ssaid, name = author_name_to_ssaid(orcid_to_author_name("0000-0001-8307-5670")) metrics = semantic_scholar_requests_get( f"https://api.semanticscholar.org/graph/v1/author/{ssaid}", params={"fields": "paperCount,citationCount,hIndex"}, ).json() ``` -------------------------------- ### Search PubMed Papers by Keywords Source: https://github.com/jannisborn/paperscraper/blob/main/README.md Search PubMed for papers matching complex keyword logic (AND for outer lists, OR for inner lists) and save results to a JSONL file. Requires PubMed records. ```python from paperscraper.pubmed import get_and_dump_pubmed_papers ai = ["Artificial intelligence", "Machine learning"] qc = ["Quantum computing", "Quantum information", "Quantum algorithm"] chemistry = ["Chemistry", "Chemical", "Molecule", "Materials science"] get_and_dump_pubmed_papers([ai, qc, chemistry], "ai_quantum_chemistry.jsonl") ``` -------------------------------- ### Extract Paper Self-Citations and References Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/self-citation-analysis.md Use SelfLinkClient with mode 'paper' to extract self-citations and self-references for a given paper DOI. The result includes citation counts, self-citation details, and reference details. ```python >>> from paperscraper.citations import SelfLinkClient >>> paper_client = SelfLinkClient("10.1038/s41586-023-06600-9", mode="paper") >>> paper_client.extract() >>> paper_client.get_result() PaperResult( ssid="10.1038/s41586-023-06600-9", title="Assembly theory explains and quantifies selection and evolution", num_citations=141, self_citations={ "Abhishek Sharma": 3.55, "Dániel Czégel": 0.71, "Michael Lachmann": 1.42, "C. Kempes": 3.55, "S. I. Walker": 4.96, "Leroy Cronin": 4.96, }, citation_score=3.192, num_references=33, self_references={ "Abhishek Sharma": 3.03, "Dániel Czégel": 0.0, "Michael Lachmann": 0.0, "C. Kempes": 0.0, "S. I. Walker": 6.06, "Leroy Cronin": 21.21, }, reference_score=5.05, ) ``` -------------------------------- ### Analyze Paper Self-References Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/self-citation-analysis.md Inspect a paper's references to determine how many include one of its own authors. This helps identify self-referential citation patterns. ```python from paperscraper.citations import self_references_paper doi = "10.1038/s41586-023-06600-9" self_references = self_references_paper(doi) print(self_references.num_references) print(self_references.reference_score) ``` ```python print(self_references.self_references) ``` -------------------------------- ### Analyze Paper Self-Citations Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/self-citation-analysis.md Examine papers that cite a given paper to see how often those citing papers share an author with the original paper. This quantifies self-citation. ```python from paperscraper.citations import self_citations_paper doi = "10.1038/s41586-023-06600-9" self_citations = self_citations_paper(doi) print(self_citations.num_citations) print(self_citations.citation_score) print(self_citations.self_citations) ``` -------------------------------- ### Fetch Researcher Metrics by Author ID Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/scholar-metrics-analysis.md Retrieve researcher metrics such as paper count, citation count, and h-index using their Semantic Scholar Author ID. Uses the Semantic Scholar API. ```python from paperscraper.citations.utils import semantic_scholar_requests_get ssaid = "2062641025" metrics = semantic_scholar_requests_get( f"https://api.semanticscholar.org/graph/v1/author/{ssaid}", params={"fields": "name,paperCount,citationCount,hIndex"}, ).json() ``` ```text { "authorId": "2062641025", "name": "Jannis Born", "paperCount": 63, "citationCount": 1910, "hIndex": 21, } ``` -------------------------------- ### PubMed Module Source: https://github.com/jannisborn/paperscraper/blob/main/docs/api/index.md Module for performing keyword searches on PubMed. ```APIDOC ## PubMed Module [`paperscraper.pubmed`](pubmed.md) — Pubmed keyword search ``` -------------------------------- ### Fetch Paper Citations by DOI Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/scholar-metrics-analysis.md Retrieve the citation count for a given paper using its DOI from Semantic Scholar. Requires the paperscraper library. ```python from paperscraper.citations import get_citations_by_doi get_citations_by_doi("10.1021/acs.jcim.3c00132") # Semantic Scholar citation count. ``` -------------------------------- ### Citation Source: https://github.com/jannisborn/paperscraper/blob/main/docs/api/index.md If you use paperscraper, please cite the following paper that motivated its development. ```APIDOC ## Citation If you use `paperscraper`, please cite a paper that motivated our development of this tool. ```bibtex @article{born2021trends, title={Trends in Deep Learning for Property-driven Drug Design}, author={Born, Jannis and Manica, Matteo}, journal={Current Medicinal Chemistry}, volume={28}, number={38}, pages={7862--7886}, year={2021}, publisher={Bentham Science Publishers} } ``` ``` -------------------------------- ### Fetch Paper Citations by Title Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/scholar-metrics-analysis.md Retrieve the citation count for a given paper using its title from Google Scholar. Requires the paperscraper library. Google Scholar queries may trigger captchas. ```python from paperscraper.citations import get_citations_from_title title = "GT4SD: Generative Toolkit for Scientific Discovery" get_citations_from_title(title) # Google Scholar citation count. ``` -------------------------------- ### Filter Journals by Impact Factor Range Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/scholar-metrics-analysis.md Use this snippet to find journals within a specified impact factor range. This is useful for identifying journals meeting certain impact criteria. ```python >>> impactor.search("Neural network", threshold=85, min_impact=1.5, max_impact=20) [ {"journal": "IEEE Transactions on Neural Networks and Learning Systems", "factor": 8.9, "score": 93}, {"journal": "NEURAL NETWORKS", "factor": 6.3, "score": 91}, {"journal": "Network", "factor": 3.1, "score": 92}, {"journal": "NETWORK-COMPUTATION IN NEURAL SYSTEMS", "factor": 1.6, "score": 92}, {"journal": "WORK-A Journal of Prevention Assessment & Rehabilitation", "factor": 1.5, "score": 86}, ] ``` -------------------------------- ### Fetch Paper IDs for an Author Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/scholar-metrics-analysis.md Retrieve a list of Semantic Scholar paper IDs associated with a given author ID. Requires the paperscraper library. ```python from paperscraper.citations.utils import get_papers_for_author paper_ids = get_papers_for_author("2062641025") len(paper_ids) # Number of papers linked to this Semantic Scholar author record. paper_ids[0] ``` -------------------------------- ### Resolve Author ID by Name Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/scholar-metrics-analysis.md Convert an author's name to their Semantic Scholar Author ID and name. Requires the paperscraper library. ```python from paperscraper.citations.utils import author_name_to_ssaid author_name_to_ssaid("Jannis Born") ``` -------------------------------- ### Cite Paperscraper Paper Source: https://github.com/jannisborn/paperscraper/blob/main/README.md BibTeX entry for citing the paperscraper paper in academic work. Include this in your bibliography if you use paperscraper for scraping papers. ```bibtex @article{born2021trends, title={Trends in Deep Learning for Property-driven Drug Design}, author={Born, Jannis and Manica, Matteo}, journal={Current Medicinal Chemistry}, volume={28}, number={38}, pages={7862--7886}, year={2021}, publisher={Bentham Science Publishers} } ``` -------------------------------- ### Resolve Author ID via ORCID Source: https://github.com/jannisborn/paperscraper/blob/main/docs/examples/scholar-metrics-analysis.md Resolve an author's Semantic Scholar ID by first converting their ORCID to their name, then to their Semantic Scholar ID. Requires the paperscraper library. ```python from paperscraper.citations.orcid import orcid_to_author_name from paperscraper.citations.utils import author_name_to_ssaid name = orcid_to_author_name("0000-0001-8307-5670") author_name_to_ssaid(name) ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.