Keywords
Meta-analysis
Data extraction
DeepSeek
SciSpace
How to Cite
Abstract
The systematic review and meta‑analysis represent the pinnacle of evidence‑based environmental science, providing a rigorous framework for synthesizing disparate research findings into coherent, quantitative conclusions. However, the process is notoriously labor‑intensive, with the data extraction phase being a critical bottleneck that can delay projects for months. The advent of sophisticated artificial intelligence (AI) and large language models (LLMs) offers a promising solution to automate this tedious task. This article explores the transformative potential of AI in systematic reviews, with a specific focus on evaluating the accuracy of two prominent tools: the established platform SciSpace and the open‑source LLM DeepSeek‑V3.2. By analyzing their performance in extracting study design and review‑specific variables from 20 environmental studies, including qualitative and quantitative data, this analysis aims to determine their viability as ``second reviewers". Accuracy was assessed against human double‑extracted data as the gold standard. Overall accuracy for quantitative data was 95.38\% (95.63\% completely correct for DeepSeek and 95.12\% for SciSpace), while overall accuracy for qualitative data was 100\%. AI‑assisted data extraction already has the capability to enhance the efficiency and scalability under professional supervision in environmental analytic research.
References
Ajayi, K., 2025 SCITEUQ: Toward uncertainty-aware complex scientific table data extraction and understanding. Ph.D. thesis, Old Dominion University.
Bao, H., M. Sun, and M. Teplitskiy, 2025 Where there’s a will there’s a way: ChatGPT is used more for science in countries where it is prohibited. Quantitative Science Studies 6: 716–731.
Buscemi, N., L. Hartling, B. Vandermeer, L. Tjosvold, and T. P. Klassen, 2006 Single data extraction generated more errors than double data extraction in systematic reviews. Journal of Clinical Epidemiology 59: 697–703.
DeepSeek AI, D. Guo, et al., 2025 DeepSeek R1: Incentivizing reasoning capability in LLMs via reinforcement learning.
Deng, Z., W. Ma, Q. L. Han, W. Zhou, X. Zhu, et al., 2025 Exploring DeepSeek: A survey on advances, applications, challenges and future directions. IEEE/CAA Journal of Automatica Sinica 12: 872–893.
Ge, L., R. Agrawal, M. Singer, P. Kannapiran, J. A. De Castro Molina, et al., 2024 Leveraging artificial intelligence to enhance systematic reviews in health research: advanced tools and challenges. Systematic Reviews 13: 269.
Gurevitch, J., J. Koricheva, S. Nakagawa, and G. Stewart, 2018 Meta-analysis and the science of research synthesis. Nature 555: 175–182.
Gurevitch, J., L. L. Morrow, A. Wallace, and J. S. Walsh, 1992 A meta-analysis of competition in field experiments. The American Naturalist 140: 539–572.
Helms Andersen, T., T. M. Marcussen, A. D. Termannsen, T. W. H. Lawaetz, and O. Nørgaard, 2025 Using artificial intelligence tools as second reviewers for data extraction in systematic reviews: a performance comparison of two AI tools against human reviewers. Cochrane Evidence Synthesis and Methods 3: e70036.
Jarvinen, A., 1991 A meta-analytic study of the effects of female age on laying date and clutch size in the Great Tit Parus major and the Pied Flycatcher Ficedula hypoleuca. Ibis 133: 62–67.
Koricheva, J., J. Gurevitch, and K. Mengersen, 2013 Handbook of Meta-Analysis in Ecology and Evolution. Princeton University Press, Princeton.
Littell, J. H., J. Corcoran, and V. Pillai, 2008 Systematic Reviews and Meta-Analysis. Oxford University Press, Oxford.
Marshall, I. J., J. Kuiper, and B. C. Wallace, 2016 RobotReviewer: Evaluation of a system for automatically assessing bias in clinical trials. Journal of the American Medical Informatics Association 23: 193–201.
Mathes, T., P. Klaßen, and D. Pieper, 2017 Frequency of data extraction errors and methods to increase data extraction quality: A methodological review. BMC Medical Research Methodology 17: 152.
Nunes, G., V. Rolla, D. Pereira, V. Alves, A. Carreiro, et al., 2025 Benchmarking table extraction: multimodal LLMs vs traditional OCR. In Proceedings of the 1st Joint Workshop on Large Language Models and Structure Modeling (XLLM 2025).
Shi, B., M. Lu, R. Yang, M. Nie, and X. Zheng, 2025 A comparative meta-analysis of Cu, Ni, Cd, Zn, and As uptake by plants after inoculation with Pseudomonas or Bacillus. Environmental Science and Pollution Research 32: 27195–27207.
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.