Project directory for the clojure CI scripts.
This folder contains the first version of the CI, which was developed in Clojure, and was finished in February 2023.
A new version of the CI, in python, was developed in July 2023. It is in this same repo, in the folder ../python_CI_scripts/
The code structure is very close in both versions. Although the new one contains improvements, so it should be the one preferred.
Nevertheless, some functionalities remain only available through the clj version e.g.
- update the collection automatically
pending_matrix/only_collection.json
- set up fiji and deepimagej in different OS's
You need to perform this 2-step setup in order to use or develop this version of the code.
- change directory to
../bash_scripts/
and runlocal_setup.sh
which will clone:- the bioimage.io collection repository
- the numpy-tiffs repository
To run the scripts locally you need babashka installed. It is a standalone program (a 20Mb download) compatible with Linux, Mac, and Windows. Follow the installation instructions.
bb -m core.main -h
The output should be something like this:
DeepImageJ CI for models from the BioImage Model Zoo (https://bioimage.io/)
Usage: bb -m core.main [action] [options] (in the java_CI_scripts working directory)
Actions:
init (DEFAULT) Initial checks & generate folder structures and files for the compatible models to test.
download Populate model folders (download files). Build args for DeepImagej headless.
reproduce Run the models on Fiji with DeepImageJ headless. Create tests summaries.
Options:
-h, --help Show help
-j, --json-file FILE ./pending_matrix/use_cases.json Read input from json FILE
-s, --json-string STRING Read input from raw json STRING
-u, --unit-test false Run all unit tests
Please refer to the docs page for more information:
https://github.com/ivan-ea/CI-deepimagej-bioimage-io/blob/master/java_CI_scripts/Readme.md
bb -m core.main init -j pending_matrix/use_cases.json
bb -m core.main -s '<a valid json string literal here>'
example
bb -m core.main -s '{\"include\": [{\"resource_id\": \"10.5281/zenodo.7261974\",\"version_id\": \"7261975\"}, {\"resource_id\": \"deepimagej\",\"version_id\": \"DeepSTORMZeroCostDL4Mic/latest\"}]}'
bb -m core.main download -j pending_matrix/use_cases.json
Assumes model folders have been populated and their paths are written on models_to_test.txt
.
bb -m core.main reproduce
bb -m core.main -u
bb "(require 'collection) (collection/generate-pending-matrix-from-collection true)"
bb "(require 'collection-test 'core.cli-test) (clojure.test/run-tests 'collection-test 'core.cli-test)"
bb "(require '[reproduce.run-fiji-scripts :as r]) (r/build-bash-script)"
bb "(require '[reproduce.setup-fiji :as r]) (r/setup-fiji-&-deepimagej3)"
bb "(require '[reproduce.run-fiji-scripts :as r]) (r/grant-exec-permission) (r/run-seq-fiji-script)"
Simply change the beginning in all previous commands: bb -m core.main
for clj -M -m core.main
.
Example:
clj -M -m core.main -h
- This will use the dependencies from
deps.edn
instead of the ones that come with babashka. - Should give exactly the same results as running with babashka.
- You will need to have clojure installed, more information in the next section.
Conceptual organization of the code
- Install Java (recommended java 8)
- Install Clojure (instructions)
- Cursive
- Rainbow Brackets (recommended)
- IdeaVim (optional)
- Open the java_CI_scripts folder as a project (File > Open)
- After that, set up to work with babashka instructions.
- Go to File > Project Structure
- go to > Project
- select SDK java 8
- select outputs folder to \java_CI_scripts\out
- go to > Modules
- add [+] babashka module
- set the directory of the babashka binary
- go to > Project
- To avoid "symbol not resolved" warnings: right-click on each file and mark them as Babashka scripts
- Go to Run > Edit Configurations
- [+] add new configuration
- give it a name
- Press play (top right)