AI-assisted literature curation quantifies the landscape of mathematics in oncology

Codebase for the analysis (see preprint in BioRxiv: https://doi.org/10.64898/2026.01.13.699306)

This repository contains a research pipeline for collecting, curating, cleaning, and visualizing publication metadata related to mathematical models in oncology. The current codebase is centered on main.py, with supporting modules in src/.

Setup

We used pipenv to manage the project dependencies. You can use the Pipfile shared with the repository to install the dependencies:

pip install --user pipenv
pipenv install

Or, for any other dependency manager, we also provide the requirements.txt file.

Demo / offline mode (recommended)

Full code execution requires access to the following APIs:

• Scopus (publication metadata)
• OpenAlex (pdf fulltext for context generation)
• OpenAI (AI-assisted curation)
• Anthropic (AI-assisted curation)
• Gemini (AI-assisted curation)

A demo version to execute offline data processing automatically pulls preprocessed data from Zenodo. This allows to check code execution without having to set credentials for the services we use, and without using tokens.

If you prefer to test the full pipeline, make sure to have access to all the services required (see next section), and to have enough tokens to run the LLM-curation validation, and full dataset curation. Then, set ZENODO_PULL=False in the main.py file, and execute (see next sections).

Credentials

This step is not required when running the pipeline in Demo / offline mode (see above).

Create config/config.json.

To setup access to Scopus, place your Scopus API key in the config file:

{
  "apikey": "PASTE_SCOPUS_API_KEY_HERE",
  "insttoken": ""
}

If you have access to multiple API keys, the pipeline will automatically rotate through those:

{
  "apikeys": [
    "PASTE_FIRST_SCOPUS_API_KEY_HERE",
    "PASTE_SECOND_SCOPUS_API_KEY_HERE",
    ...
  ],
  "insttoken": ""
}

To evaluate and run the LLM curation you need access to several LLM providers (Anthropic, OpenAI, Gemini). Paste your API Keys in the config.json following the provided format:

{
  "apikey": "PASTE_SCOPUS_API_KEY_HERE",
  "insttoken": "",
  "openalex_apikey": "PASTE_OPENALEX_KEY_HERE",
  "openai_apikey": "PASTE_OPENAI_KEY_HERE",
  "anthropic_apikey": "PASTE_ANTHROPIC_KEY_HERE",
  "gemini_apikey": "PASTE_GOOGLE_GEMINI_KEY_HERE"
}

Running the Pipeline

Activate the environment and run:

pipenv run python3 main.py

The script writes logs to log/ and generated analysis products to output/ (or demo/output if using Zenodo).

Input Data

Input data sit in data/:

• data/evaluation_human/: independent evaluations from human experts
• data/evaluation_llm/evaluation_test_dataset.xlsx: sample of 200 publications to test LLM-curation (the same evaluated by human experts)
• data/geo/: geographic reference files for world and US-state maps
• data/queries/scopus_query_str.txt: Scopus query used by the API search step
• data/regex/: regex dictionaries for cancer types, modeling methods, therapies, scientific areas, and author-keyword patterns
• data/twimo/missing_dois.txt: contains the DOIs of the reviews for which there was no PDF in OpenAlex
• data/TWIMO_Scopus.csv: Publications featured in TWiMO in csv format

Generated Outputs

The current pipeline writes to output/ (or demo/output if using Zenodo). Typical generated folders include:

• output/scopus_search_results_row/ and output/scopus_search_results_clean/: Scopus search result caches
• output/scopus_abstracts_row/: raw abstract metadata and converted CSVs
• output/scopus_abstracts_row_scored/: LLM-scored publication CSVs.
• output/scopus_abstracts_cleaned_scored/: cleaned, enriched, and filtered scored datasets
• output/scopus_affiliations/ and output/scopus_journal/: Scopus-derived affiliation and journal metadata
• output/evaluation_human/, output/evaluation_llm/, and output/evaluation_llm_comparison/: evaluator agreement and model comparison outputs
• output/figures/: Matplotlib figures and manuscript/supplementary figure layouts
• output/maps/: CSVs referenced by D3 world/US map pages
• output/chordcharts/: CSVs referenced by D3 chord chart pages
• output/vosviewer/: VOSviewer-style network outputs used by later visualization steps

The D3 map and network pages live in src/map/ and src/network/. They read configuration JSON files that point to generated CSVs under output/maps/ and output/chordcharts/.

To visualize them, serve the repository root:

python3 -m http.server

Then open:

• http://localhost:8000/src/map/map.html
• http://localhost:8000/src/map/map_US.html
• http://localhost:8000/src/network/chords.html
• http://localhost:8000/src/network/coauthorship.html

These pages require the corresponding generated output/ files to exist locally.

Cite

Preprint:

@article {Pradelli2026.01.13.699306,
	author = {Pradelli, Franco and Strobl, Maximilian and Marzban, Sadegh and de Kermenguy, Fran{\c c}ois and Barnett, Ari and Ganesan, Katyayni and Hormuth, David A. and Hamis, Sara and Bhaskar, Dhananjay and Lorenzo, Guillermo and Anderson, Alexander R. A. and West, Jeffrey},
	title = {140 Years of mathematical modeling in oncology through AI-assisted curation},
	elocation-id = {2026.01.13.699306},
	year = {2026},
	doi = {10.64898/2026.01.13.699306},
	publisher = {Cold Spring Harbor Laboratory},
	URL = {https://www.biorxiv.org/content/early/2026/06/16/2026.01.13.699306},
	eprint = {https://www.biorxiv.org/content/early/2026/06/16/2026.01.13.699306.full.pdf},
	journal = {bioRxiv}
}

License

This project is licensed under the MIT License.

Contact

Franco Pradelli (franco.pradelli94@gmail.com)