Skip to main content
Springer Nature Link
Log in

The State of Reproducible Research in Computer Science

  • Conference paper
  • First Online:
17th International Conference on Information Technology–New Generations (ITNG 2020)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 1134))

  • 2115 Accesses

Abstract

Reproducible research is the cornerstone of cumulative science and yet is one of the most serious crisis that we face today in all fields. This paper aims to describe the ongoing reproducible research crisis along with counter-arguments of whether it really is a crisis, suggest solutions to problems limiting reproducible research along with the tools to implement such solutions by covering the latest publications involving reproducible research.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Popper, K.: The Logic of Scientific Discovery. Routledge, London (2005)

    Book  Google Scholar 

  2. Peng, R.D.: Reproducible research in computational science. Science 334(6060), 1226–1227 (2011)

    Article  Google Scholar 

  3. Barba, L.A.: Terminologies for reproducible research (2018). arXiv preprint:1802.03311

    Google Scholar 

  4. Fonseca Cacho, J.R., Taghva, K.: Reproducible research in document analysis and recognition. In: Information Technology-New Generations, pp. 389–395. Springer, Berlin (2018)

    Google Scholar 

  5. Leek, J.T., Peng, R.D.: Opinion: reproducible research can still be wrong: adopting a prevention approach. Proc. Natl. Acad. Sci. 112(6), 1645–1646 (2015)

    Article  Google Scholar 

  6. Baker, M.: 1500 scientists lift the lid on reproducibility. Nature News 533(7604), 452 (2016)

    Article  Google Scholar 

  7. Rampin, R., Chirigati, F., Steeves, V., Freire, J.: Reproserver: making reproducibility easier and less intensive (2018). arXiv preprint:1808.01406

    Google Scholar 

  8. Wickham, H., et al.: Tidy data. J. Stat. Softw. 59(10), 1–23 (2014)

    Article  Google Scholar 

  9. Hutson, M.: Artificial intelligence faces reproducibility crisis. American Association for the Advancement of Science 359(6377), 725–726 (2018), https://doi.org/10.1126/science.359.6377.725, https://science.sciencemag.org/content/359/6377/725

  10. Fonseca Cacho, J.R., Taghva, K., Alvarez, D.: Using the Google web 1t 5-gram corpus for OCR error correction. In 16th International Conference on Information Technology-New Generations (ITNG 2019), pp. 505–511. Springer, Berlin (2019)

    Google Scholar 

  11. Fonseca Cacho, J.R.: Improving OCR Post Processing with Machine Learning Tools. Ph.D. Dissertation, University of Nevada, Las Vegas (2019)

    Google Scholar 

  12. Fonseca Cacho, J.R., Taghva, K.: Aligning ground truth text with OCR degraded text. In: Intelligent Computing-Proceedings of the Computing Conference, pp. 815–833. Springer, Berlin (2019)

    Google Scholar 

  13. Nosek, B.A., Alter, G., Banks, G.C., Borsboom, D., Bowman, S.D., Breckler, S.J., Buck, S., Chambers, C.D., Chin, G., Christensen, G., et al.: Promoting an open research culture. Science 348(6242), 1422–1425 (2015)

    Article  Google Scholar 

  14. Sayre, F., Riegelman, A.: The reproducibility crisis and academic libraries. Coll. Res. Libr. 79(1), 2 (2018)

    Article  Google Scholar 

  15. Steeves, V.: Reproducibility librarianship. Collab. Librariansh. 9(2), 4 (2017)

    Google Scholar 

  16. Vines, T.H., Andrew, R.L., Bock, D.G., Franklin, M.T., Gilbert, K.J., Kane, N.C., Moore, J.-S., Moyers, B.T., Renaut, S., Rennison, D.J., et al.: Mandated data archiving greatly improves access to research data. FASEB J. 27(4), 1304–1308 (2013)

    Article  Google Scholar 

  17. Claerbout, J.F., Karrenbach, M.: Electronic documents give reproducible research a new meaning. In: SEG Technical Program Expanded Abstracts 1992. Society of Exploration Geophysicists, pp. 601–604 (1992)

    Google Scholar 

  18. Ram, K.: Git can facilitate greater reproducibility and increased transparency in science. Source Code Biol. Med. 8(1), 7 (2013)

    Article  Google Scholar 

  19. Patil, P., Peng, R.D., Leek, J.T.: A visual tool for defining reproducibility and replicability. Nat. Hum. Behav. 3(7), 650–652 (2019)

    Article  Google Scholar 

  20. Hung, L.-H., Kristiyanto, D., Lee, S.B., Yeung, K.Y.: Guidock: using docker containers with a common graphics user interface to address the reproducibility of research. PloS One 11(4), e0152686 (2016)

    Article  Google Scholar 

  21. Hosny, A., Vera-Licona, P., Laubenbacher, R., Favre, T.: AlgoRun, a Docker-based packaging system for platform-agnostic implemented algorithms. Bioinformatics 32(15), 2396–2398 (2016)

    Article  Google Scholar 

  22. Dalle, O.: Olivier dalle. should simulation products use software engineering techniques or should they reuse products of software engineering?–part 1. SCS Model. Simul. Mag. 2(3), 122–132 (2011)

    Google Scholar 

  23. Voelkl, B., Würbel, H.: Reproducibility crisis: are we ignoring reaction norms? Trends Pharmacol. Sci. 37(7), 509–510 (2016)

    Article  Google Scholar 

  24. Fanelli, D.: Opinion: is science really facing a reproducibility crisis, and do we need it to? Proc. Natl. Acad. Sci. 115(11), 2628–2631 (2018)

    Article  Google Scholar 

  25. Fanelli, D.: How many scientists fabricate and falsify research? a systematic review and meta-analysis of survey data. PloS One 4(5), e5738 (2009)

    Article  Google Scholar 

  26. Guthrie, M., Leblois, A., Garenne, A., Boraud, T.: Interaction between cognitive and motor cortico-basal ganglia loops during decision making: a computational study. J. Neurophysiol. 109 (12), 3025–3040 (2013)

    Article  Google Scholar 

  27. Topalidou, M., Leblois, A., Boraud, T., Rougier, N.P.: A long journey into reproducible computational neuroscience. Front. Comput. Neurosci. 9(30) (2015)

    Google Scholar 

Download references

Acknowledgements

Ben Cisneros for his contributions in helping run the survey and generating the graphics in this Publication.

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Nevada, Las Vegas, Las Vegas, NV, USA

    Jorge Ramón Fonseca Cacho & Kazem Taghva

Authors
  1. Jorge Ramón Fonseca Cacho
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kazem Taghva
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jorge Ramón Fonseca Cacho .

Editor information

Editors and Affiliations

  1. Department of Electrical and Computer Engineering, University of Nevada, Las Vegas, Las Vegas, NV, USA

    Shahram Latifi

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Cacho, J.R.F., Taghva, K. (2020). The State of Reproducible Research in Computer Science. In: Latifi, S. (eds) 17th International Conference on Information Technology–New Generations (ITNG 2020). Advances in Intelligent Systems and Computing, vol 1134. Springer, Cham. https://doi.org/10.1007/978-3-030-43020-7_68

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-43020-7_68

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-43019-1

  • Online ISBN: 978-3-030-43020-7

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics