ARC dev branch (#4)

* Pull sample blobs code * Strip away everything except the open-able widget * Some basic structures that track the existing layers * BROKEN: Image layers don't play nice but I did combine the emitter objects * HACK: Napari will now load up, but this hasn't really fixed the naming issue. I've just hacked in a missing attribute to all of the GroupLayer children - they need to have a Node mixin to behave properly * I can see when layers are added, and have some kind of list. Still get SEG-faults or index errors when moving groups within groups, particularly empty group into empty group. * Recognises add/remove layers from viewer and auto-adds to the group layers view. Still seg-fault bug when nesting two empty group layers * Rename classes to be closer to development language * Rename and move items to break things up * Link up group layer selection with layer tab * Reliable testing framework * Fix our failing test suite (#13) * Edit file name to be specific to our use case * Rename single test * Add checks on napari `main` (#18) * Add workflow step to run tests on head of napari * Add optional dependency on napari-latest which fetches napari from the git repo directly * Create an optional install that uses HEAD of napari, add it to list of tests * Add note that napari-latest install is available to README * Update README.md Co-authored-by: Alessandro Felder <[email protected]> --------- Co-authored-by: Alessandro Felder <[email protected]> * Add conftest and some fixtures for testing (#15) * Add conftest.py for shared fixtures and do a quick model check for our QtGroupLayerModel * Typehints don't carry through fixtures * Allow renaming of layers / group layers (#19) * allow renaming of groups and layers * keep prefix for group layers and update setRoot * remove unused tree_model * add initial tests for renaming * add test for editing state on double click * move tests into test_group_layer_widget * Correctly Subclass our `GroupLayer` (#20) * Ensure GroupLayer subclasses from our own subclass of Node * Define ambiguously inherited attributes (IE ensure I don't break KM's tests) * Actually use the nested structure fixture so ruff doesn't get angry * Reorganise the test suite fixture layout since it's starting to get bloated * Basic tests to check is_group and strict typing * Self-review * Allow _check_if_already_tracking to be optionally recursive. Write test for method. * Update src/napari_experimental/group_layer.py Co-authored-by: Kimberly Meechan <[email protected]> * Dammit linter --------- Co-authored-by: Kimberly Meechan <[email protected]> * Adds independent layer controls to widget (#22) * add group layer controls * fix controls for latest changes to group layer * remove syncing of selection * add tests for group layer controls * fix TypeError * expand docstrings * react to sub-groups * add overview and todo sections to readme (#31) * Sync GroupLayer and Main Viewer layer orders (#26) * Allow group_layer to return a flat index * Enforce layer order updates through the GroupLayer events * Rework how items are added to GroupLayers to make things more explicit * Remove unused function * Docstring Tidy (#27) * Some docstrings for _widget * Docstrings for GroupLayerNode class * docstrings for group_layer_qt * Docstrings for group_layer.py * Add attributes and methods to classes where necessary * Missed a rename * Add tests for widget activity * Test for layer deletion sync * Ruff for 3.9 disagrees with 3.11 * strict keyword not in Py3.9 * Update test_and_deploy.yml (#32) (#33) * Fix Errors when Moving Layers (#28) * Allow group_layer to return a flat index * Enforce layer order updates through the GroupLayer events * Rework how items are added to GroupLayers to make things more explicit * Remove unused function * Docstring Tidy (#27) * Some docstrings for _widget * Docstrings for GroupLayerNode class * docstrings for group_layer_qt * Docstrings for group_layer.py * Add attributes and methods to classes where necessary * Missed a rename * Write method that fixes the min failing example * Hey look, a docstring! * Remove double-counting of positions after destination * Test revise index method, remove redundant variable passed * Maybe this is correct if I have a flat index sort a priori * Fix ordering issue and allow for Groups to be assigned a flatindex * Add 2nd test to check moving items out of different subgroups still tracks order correctly * Remove un-necessary variable * Remove commented-out experimental code * Apply suggestions from @K-Meech code review * Add `Implementation` section to README (#34) * MDLint + alessandrofelder -> brainglobe in README * Create implementation deatils section * Group and Node subclassing explanations * Notes on indexing conventions * Update README.md Co-authored-by: Kimberly Meechan <[email protected]> --------- Co-authored-by: Kimberly Meechan <[email protected]> * Add right click menu and styling (#29) * add group layer delegate * rename group layer delegate * add context for right click menu * working minimal right click menu * fix syncing of selection * update docstrings and remove unused functions * fix double click edit * simplify right click actions and context to fix tests * update docstrings * Force widget in tests to have a parent (#35) * Allows toggling visibility of group layers (#30) * toggle visiblity for group layers * add tests for visiblity * update docstrings * fix failing tests * remove logger and thumbnail role * fix view controls when switching from group to layer --------- Co-authored-by: Will Graham <[email protected]> * `main` and `ARC-dev-branch` merge commit reconcile (#37) * Fix bugs relating to empty `GroupLayers` (#38) * Fix seg-faults and broken drag-and-drop via overwriting index method * Switch to unique ID trackers over reimplementing method. Allows safe overwride of __eq__ and __hash__ * Remove one bug from the README broken features section --------- Co-authored-by: Alessandro Felder <[email protected]> * Add docs build and populate content (#39) * Basic Sphinx build and docs structure * Workflow to publish docs on pushes to main * Separate sphinx requirements from package requirements * Add building the docs section * Reorganise group_layers docs * Docs API for Groups & Nodes, some source docstring updates for text rendering * Placeholder for widget docs * Write widget docs page * Add note on hashing things * expand docs for group layers, delegates and context menu (#40) --------- Co-authored-by: Kimberly Meechan <[email protected]> --------- Co-authored-by: Alessandro Felder <[email protected]> Co-authored-by: Kimberly Meechan <[email protected]>
brainglobe · Jul 10, 2024 · e3235c1 · e3235c1
1 parent b5eec6f
commit e3235c1
Show file tree

Hide file tree

Showing 37 changed files with 2,998 additions and 246 deletions.
diff --git a/.github/workflows/build_docs.yaml b/.github/workflows/build_docs.yaml
@@ -0,0 +1,29 @@
+on:
+  push:
+    branches:
+      - main
+    tags:
+      - '*'
+  pull_request:
+  workflow_dispatch:
+
+jobs:
+  build_sphinx_docs:
+    name: Build Sphinx Docs
+    runs-on: ubuntu-latest
+    steps:
+      - uses: neuroinformatics-unit/actions/build_sphinx_docs@main
+        with:
+            check-links: false
+
+  deploy_sphinx_docs:
+    name: Deploy Sphinx Docs
+    needs: build_sphinx_docs
+    permissions:
+      contents: write
+    if: (github.event_name == 'push' || github.event_name == 'tag') && github.ref_name == 'main'
+    runs-on: ubuntu-latest
+    steps:
+      - uses: neuroinformatics-unit/actions/deploy_sphinx_docs@v2
+        with:
+            secret_input: ${{ secrets.GITHUB_TOKEN }}
diff --git a/.github/workflows/test_and_deploy.yml b/.github/workflows/test_and_deploy.yml
@@ -55,9 +55,35 @@ jobs:
           python-version: ${{ matrix.python-version }}
           secret-codecov-token: ${{ secrets.CODECOV_TOKEN }}
 
+  napari-latest-test:
+    needs: [linting, manifest]
+    name: Run tests using HEAD of napari main branch
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Setup qtpy libraries
+        uses: tlambert03/setup-qt-libs@v1
+
+      - name: Setup VTK headless display
+        uses: pyvista/setup-headless-display-action@v2
+
+      - name: Setup Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.11'
+          cache: "pip"
+
+      - name: Install tox
+        run: pip install tox
+
+      - name: Run tox on napari-latest
+        run: tox -vv -e napari-latest
+
   build_sdist_wheel:
     name: Build source distribution wheel
-    needs: [test]
+    needs: [test, napari-latest-test]
     if: github.event_name == 'push' && github.ref_type == 'tag'
     runs-on: ubuntu-latest
     steps:

diff --git a/.gitignore b/.gitignore
@@ -59,6 +59,7 @@ instance/
 
 # Sphinx documentation
 docs/_build/
+docs/build
 
 # MkDocs documentation
 /site/

diff --git a/MANIFEST.in b/MANIFEST.in
@@ -6,5 +6,10 @@ exclude .pre-commit-config.yaml
 recursive-exclude * __pycache__
 recursive-exclude * *.py[co]
 
+recursive-include src *.qss
+recursive-include src *.svg
+
+
 prune .napari-hub
+prune docs
 prune tests
diff --git a/README.md b/README.md
@@ -1,67 +1,148 @@
 # napari-experimental
 
-[![License BSD-3](https://img.shields.io/pypi/l/napari-experimental.svg?color=green)](https://github.com/alessandrofelder/napari-experimental/raw/main/LICENSE)
+[napari]: https://github.com/napari/napari
+[file an issue]: https://github.com/brainglobe/napari-experimental/issues
+[tox]: https://tox.readthedocs.io/en/latest/
+[pip]: https://pypi.org/project/pip/
+
+[![License BSD-3](https://img.shields.io/pypi/l/napari-experimental.svg?color=green)](https://github.com/brainglobe/napari-experimental/raw/main/LICENSE)
 [![PyPI](https://img.shields.io/pypi/v/napari-experimental.svg?color=green)](https://pypi.org/project/napari-experimental)
 [![Python Version](https://img.shields.io/pypi/pyversions/napari-experimental.svg?color=green)](https://python.org)
-[![tests](https://github.com/alessandrofelder/napari-experimental/workflows/tests/badge.svg)](https://github.com/alessandrofelder/napari-experimental/actions)
-[![codecov](https://codecov.io/gh/alessandrofelder/napari-experimental/branch/main/graph/badge.svg)](https://codecov.io/gh/alessandrofelder/napari-experimental)
+[![tests](https://github.com/brainglobe/napari-experimental/workflows/tests/badge.svg)](https://github.com/brainglobe/napari-experimental/actions)
+[![codecov](https://codecov.io/gh/brainglobe/napari-experimental/branch/main/graph/badge.svg)](https://codecov.io/gh/brainglobe/napari-experimental)
 [![napari hub](https://img.shields.io/endpoint?url=https://api.napari-hub.org/shields/napari-experimental)](https://napari-hub.org/plugins/napari-experimental)
 
-A place to experiment with core-independent features without breaking core napari
+## Overview
+
+### Features
+
+This plugin allows [napari] layers to be organised into 'layer groups'.
+Groups can hold any number of layers, in addition to further sub-groups.
+This follows suggestions [from the corresponding issue](https://github.com/napari/napari/issues/6345) on the main napari repository.
+
+Main features:
+
+- Creation of group layers
+- Drag and drop layers/groups to re-organise them
+- Sync changes in layer order from the plugin to the main napari `LayerList`
+- Toggle visibility of layers and entire groups through the 'eye' icon or right click menu
+
+### Ethos
+
+The aim of this plugin is to provide an entirely separate way to interact with layers in napari.
+While using it, the main `layer list` should only be used to add/remove layers, with all re-ordering,  re-naming etc done directly within the plugin itself.
+
+To aid this, the plugin contains its own independent layer controls, as well as right click menus and other features.
+
+### Outlook
+
+Ultimately, the goal of this plugin is to provide a way to experiment with group layers independent from the main napari codebase.
+Hopefully, parts of this plugin widget will be incorporated back into napari, replacing the existing `LayerList`.
+
+## Implementation Details
+
+This section is a halfway point between full developer documentation and throwing docstrings in a contributor's face.
+We have been diligent in adding docstrings to our methods, however we recommend you first read about [model/view programming](https://doc.qt.io/qt-6/model-view-programming.html) and in particular tree models and views.
+We also recommend you read the docstrings for `Group`s and `Node`s [in the napari codebase](https://github.com/napari/napari/blob/main/napari/utils/tree), for context on how napari currently handles such structures.
+You may also want to [review this article](https://refactoring.guru/design-patterns/composite) on composite design patterns.
+
+The docstrings, plus the explanations in this section, should then afford you enough information into how the plugin is operating.
+
+Additionally, the "enter debugger" button has been left within the plugin.
+This is intended for developer use and should be removed when the plugin is ready for release!
+Adding a breakpoint within the `GroupLayerWidget._enter_debug` method allows a developer to enter a debug context with the plugin's widget as `self`, whilst running napari.
+
+### Key Classes
+
+The key classes implemented in this plugin are
+
+- `GroupLayerNode`, `GroupLayer`: These are the Python classes that provide the tree-structure that group layers require. We expand on these below.
+- `QtGroupLayerControls`, `QtGroupLayerControlsContainer`: These classes are used to build the "control box" that the plugin provides when selecting a layer within the plugin. For all intents and purposes it mimics the existing napari layer viewer context window, but also reacts when selecting a group layer.
+- `QtGroupLayerModel`, `QtGroupLayerView`: These subclass from the appropriate Qt abstract classes, and provide the model/tree infrastructure for working with `GroupLayers`. Beyond this, they do not contain any remarkable functionality beyond patching certain methods for consistency with the data being handled / displayed.
+
+#### `Group`s and `Node`s
 
-----------------------------------
+`GroupLayerNode`  (GLN) and `GroupLayer`  (GL) inherit from the base napari classes `Node` and `Group` (respectively).
+The GLNs act as wrappers for `Layer`s - each one tracks (a pointer to) one particular `Layer` through the `.layer` attribute and uses its information when rendering in the GUI.
+GLs allow us to organise GLNs into groups, but they themselves must also be instances of GLN.
+For this reason, the convention we have adopted is that GLs have the `.layer` attribute set to `None` (since they do not track an individual layer that is associated to _them_ specifically).
+Furthermore, the `is_group` method is explicitly defined on both GLNs and GLs, which provides a way to distinguish between the two classes when traversing the tree structure.
 
-This [napari] plugin was generated with [Cookiecutter] using [@napari]'s [cookiecutter-napari-plugin] template.
+It should also be noted that we have made the decision not to make GLNs inherit from `Layer`, nor make the existing `Layer` class subclass from `Node`.
 
-<!--
-Don't miss the full getting started guide to set up your new package:
-https://github.com/napari/cookiecutter-napari-plugin#getting-started
+- The latter would require changes to core napari code, so is outside the scope of this plugin.
+- The former option then introduces event conflicts (within the constructors) when GL inherits from both `Group` and `Layer`. Again, fixing these issues requires either edits to the core napari codebase, or regurgitating a lot of the core napari codebase inside a new class for just a couple of line changes.
+- From a separation of concerns perspective, it is cleaner to have the group layer functionality impose a structure on top of the existing `LayerList` that napari tracks, rather than duplicate this information so it can be stored in the GL, and then have to deal with changes to one of the objects being reflected back on the "original" it came from.
 
-and review the napari docs for plugin developers:
-https://napari.org/stable/plugins/index.html
--->
+#### Differences in Indexing Conventions
+
+A key difference between a flat `LayerList` and our GL structure is how they are indexed, and how the structures are rendered in the napari viewer based on this indexing.
+`LayerList` is indexed from 0 ascending, and renders in the napari viewer with the layer at index 0 at the _bottom_ of the `LayerList`.
+`Layer`s with higher indices are rendered _on top_ of those with a lower index, with the item assigned the highest index appearing at the top of the layer viewer window.
+
+By contrast, GL uses `NestedIndex`es to track the position of GLNs; these are tuples of integers that can be of arbitrary length (so long as the GL has the structure to match), and should be interpreted as subsequent accesses of objects in the tree:
+
+- (0, 1) refers to the item at index 1 of the (sub)-GL at index 0 of the root GL.
+- (2,) refers to the item at index 2 of the root GL.
+
+Note that an "item" can be a GLN or a GL (use the `is_group` method to distinguish if necessary)!
+Again, items are added to a GL using the lowest available index.
+GLs can also assign a flat index order to their elements by starting from the root tree and counting upwards from 0, descending into sub-GLs and exhausting their items when they are encountered (see `GroupLayer.flat_index_order`).
+
+However GLs render with index 0 _at the top_ of the view, and higher indices below them.
+In order to preserve the user's intuitive understanding of "layers higher up in the display appear on top of lower layers", it is thus necessary to pass the _reversed_ order of layers back to the viewer after a rearrangement event in the GL viewer.
+
+## Todo
+
+### Desirable features
+
+- Expand right click menu options to include:
+  - Add selected items to new group
+  - Add selected items to existing group
+  - All existing right-click options for layers from the main `LayerList`
+- Add independent buttons to add/remove layers to the plugin (this would make it fully independent of the main `LayerList`). Also, style the `Add empty layer group` button to match these.
+
+### Known bugs and issues (non-breaking, but poor UX)
+
+- The open/closed state of group layers doesn't persist on drag and drop <https://github.com/brainglobe/napari-experimental/issues/23>
+
+### Known bugs and issues (breaking)
+
+### Development Tasks
+
+- Create a standalone docs site that expands on the implementation details section.
+  - Remove said section and transfer its contents to the docs site
+  - Link to the docs site in the README.
 
 ## Installation
 
 You can install `napari-experimental` via [pip]:
 
-    pip install napari-experimental
+```bash
+pip install napari-experimental
+```
 
+To install the latest development version:
 
+```bash
+pip install git+https://github.com/brainglobe/napari-experimental.git
+```
 
-To install latest development version :
-
-    pip install git+https://github.com/alessandrofelder/napari-experimental.git
+You can also install a version of the package that uses the latest version of napari (fetched from <https://github.com/napari/napari>):
 
+```bash
+pip install napari-experimental[napari-latest]
+```
 
 ## Contributing
 
-Contributions are very welcome. Tests can be run with [tox], please ensure
-the coverage at least stays the same before you submit a pull request.
+Contributions are very welcome.
+Tests can be run with [tox], please ensure the coverage at least stays the same before you submit a pull request.
 
 ## License
 
-Distributed under the terms of the [BSD-3] license,
-"napari-experimental" is free and open source software
+Distributed under the terms of the [BSD-3] license, "napari-experimental" is free and open source software
 
 ## Issues
 
 If you encounter any problems, please [file an issue] along with a detailed description.
-
-[napari]: https://github.com/napari/napari
-[Cookiecutter]: https://github.com/audreyr/cookiecutter
-[@napari]: https://github.com/napari
-[MIT]: http://opensource.org/licenses/MIT
-[BSD-3]: http://opensource.org/licenses/BSD-3-Clause
-[GNU GPL v3.0]: http://www.gnu.org/licenses/gpl-3.0.txt
-[GNU LGPL v3.0]: http://www.gnu.org/licenses/lgpl-3.0.txt
-[Apache Software License 2.0]: http://www.apache.org/licenses/LICENSE-2.0
-[Mozilla Public License 2.0]: https://www.mozilla.org/media/MPL/2.0/index.txt
-[cookiecutter-napari-plugin]: https://github.com/napari/cookiecutter-napari-plugin
-
-[file an issue]: https://github.com/alessandrofelder/napari-experimental/issues
-
-[napari]: https://github.com/napari/napari
-[tox]: https://tox.readthedocs.io/en/latest/
-[pip]: https://pypi.org/project/pip/
-[PyPI]: https://pypi.org/
diff --git a/docs/Makefile b/docs/Makefile
@@ -0,0 +1,20 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS    ?=
+SPHINXBUILD   ?= sphinx-build
+SOURCEDIR     = source
+BUILDDIR      = build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
diff --git a/docs/make.bat b/docs/make.bat
@@ -0,0 +1,35 @@
+@ECHO OFF
+
+pushd %~dp0
+
+REM Command file for Sphinx documentation
+
+if "%SPHINXBUILD%" == "" (
+	set SPHINXBUILD=sphinx-build
+)
+set SOURCEDIR=source
+set BUILDDIR=build
+
+%SPHINXBUILD% >NUL 2>NUL
+if errorlevel 9009 (
+	echo.
+	echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
+	echo.installed, then set the SPHINXBUILD environment variable to point
+	echo.to the full path of the 'sphinx-build' executable. Alternatively you
+	echo.may add the Sphinx directory to PATH.
+	echo.
+	echo.If you don't have Sphinx installed, grab it from
+	echo.https://www.sphinx-doc.org/
+	exit /b 1
+)
+
+if "%1" == "" goto help
+
+%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+goto end
+
+:help
+%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+
+:end
+popd
diff --git a/docs/requirements.txt b/docs/requirements.txt
@@ -0,0 +1 @@
+myst_parser
diff --git a/docs/source/conf.py b/docs/source/conf.py
@@ -0,0 +1,31 @@
+# Configuration file for the Sphinx documentation builder.
+#
+# For the full list of built-in configuration values, see the documentation:
+# https://www.sphinx-doc.org/en/master/usage/configuration.html
+
+# -- Project information -----------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
+
+project = "napari-experimental"
+copyright = "2024, Alessandro Felder, William Graham, Kimberly Meechan"
+author = "Alessandro Felder, William Graham, Kimberly Meechan"
+
+# -- General configuration ---------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration
+
+extensions = [
+    "myst_parser",
+    "sphinx.ext.autodoc",
+    "sphinx.ext.autosummary",
+    "sphinx.ext.napoleon",
+]
+
+templates_path = ["_templates"]
+exclude_patterns = []
+
+
+# -- Options for HTML output -------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output
+
+html_theme = "alabaster"
+html_static_path = ["_static"]