Is Your Research Software Correct?

Resources from this presentation

• Slides: The slides are in presentation/index.html. On your local machine, Navigate to that folder and open index.html in a browser. See the online version at https://mikecroucher.github.io/reproducible_ML/
• PCA_demo The PCA demonstration used to motivate sharing of environments
• Bob's repo Repo created from the slide motivating the use of version control

External resources

Care about your code

....and those who write it

• Make your code citable
• Hidden REF - There's more to research than papers but the UK's official measure, The Research Excellence Framework (REF), doesn't reflect this. The Hidden REF aims to change that!
• Research Software Engineers Associaton - Supporting the people behind research software
• The Turing Way - Guidelines on reproducible research from the Alan Turing Institute

Academic Papers on reproducible research and software quality

• The (black) art of runtime evaluation: Are we comparing algorithms or implementations? - Demonstrating that the difference between the best and worst implementations of ML algorithms can be more than 10,000 times. Even when only considering mature frameworks
• Good enough practices in scientific computing

Coding problems leading to problems in research

• The Excel error that changed history
• The effects of Operating system and software version on Neuroscience results

Version Control - Learn git

• XKCD comic discusses git
• Software Carpentry lesson on git
• Download and install git
• GitLab - GitLab. If you need to host your own in-house replacement for GitHub.
• Oh Shit! Git - Hardly anyone knows how to use all of git. What we do is use the same few commands and google our way out of trouble when things go wrong. This site covers many cases that you'll run into in practice

Distribute your computational environment, not just the code

• My Conda cheatsheet - Conda: One way to do it in Python
• The MRAN - Daily snapshots of the R CRAN repository to allow you to version by date
• Docker and Mybinder - Containerisation for reproducible research

XKCD comics I used in the presentation

• Time taken to automate
• Python lets you fly

Numerical Algorithms Group

50 years of caring about algorithms from academia to industry.

• NAG Library for Python
• NAG Cloud HPC Migration Service

Note to self

The way you push the presentation folder to gh-pages to make the slide deck live is

git subtree push --prefix presentation origin gh-pages