Extract and replace data in overlapping regions between tiles #20

IgorTatarnikov · 2024-11-11T17:08:32Z

Before submitting a pull request (PR), please read the contributing guide.

Please fill out as much of this template as you can, but if you have any problems or questions, just leave a comment and we will help out :)

Description

What is this PR

Bug fix
Addition of a new feature
Other

Why is this PR needed?
In order to correct for brightness differences, and interpolate over the stitches in the final fused images it's important to keep track of which tiles overlap and the coordinates of the overlap in both local (relative to the tile itself) and global (relative to the final fused image) coordinates.

What does this PR do?

brainglobe_stitch/tile.py: Introduced the Overlap class to store and manage information about the overlap between two tiles, including methods to calculate local overlap coordinates and extract/replace overlap data.
brainglobe_stitch/image_mosaic.py: Updated the ImageMosaic class to include an overlaps attribute and methods to calculate overlaps between tiles. This involved modifying the constructor, load_mesospim_directory, and stitch methods, and adding the calculate_overlaps method.
tests/test_unit/test_tile.py: Added new tests for the Overlap class, including tests for initialization, local coordinate calculation, overlap data extraction, and data replacement. These tests ensure that the overlap functionality works correctly.

How has this PR been tested?

Tests added to cover the new functionality.

NOTE:
To test the functionality locally you must install Fiji and also add BigStitcher to the update sites. This can be done using the GUI see here or via the command line after installing Fiji.

./ImageJ-linux64 --update add-update-sites "BigStitcherUpdate" "https://sites.imagej.net/BigStitcher/"
./ImageJ-linux64 --update update

Replace ImageJ-linux64 with the path to the local ImageJ executable.

Three different datasets are available for testing:

Tiny (~28 MB): this is mostly used for automated testing and doesn't work for visual inspection. It can be downloaded here.
Small (~1.8 GB): this dataset is more useful for visual inspection. It can be downloaded here.
A full size dataset is available (~130 GB): this dataset works best for testing overall performance, especially when reading or writing data. It's available by request.

Is this a breaking change?

No

Does this PR require an update to the documentation?

Yes, will come in subsequent PR.

Checklist:

The code has been tested locally
Tests have been added to cover all new functionality (unit & integration)
The code has been formatted with pre-commit

codecov · 2024-11-11T17:11:29Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.54%. Comparing base (ffa996a) to head (79fdbc3).
Report is 26 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #20      +/-   ##
==========================================
+ Coverage   95.20%   95.54%   +0.33%     
==========================================
  Files           6        6              
  Lines         626      673      +47     
==========================================
+ Hits          596      643      +47     
  Misses         30       30

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

lauraporta

Hi Igor, looks good to me. I'm just confused about how does get_local_overlap_coordinates work. Once I understand this bit, I'll complete the review with the tests.

lauraporta · 2024-11-14T16:11:37Z

brainglobe_stitch/image_mosaic.py

+                # Check for overlap in the x and y dimensions
+                # and that the tiles do not have the same tile_id
+                # and that the tiles are from the same channel
+                if (


Took me a while to parse these conditions 🤣

They're a mouthful!

lauraporta · 2024-11-14T16:13:01Z

brainglobe_stitch/tile.py

+        tile_j: Tile
+            The second tile.
+        """
+        self.coordinates: npt.NDArray = overlap_coordinates


Oh nice didn't know about numpy.typing.NDArray... I was just using np.ndarray

It's very handy! You can also specify the specific type the array is holding.

lauraporta · 2024-11-14T16:13:48Z

brainglobe_stitch/tile.py

+        self.coordinates: npt.NDArray = overlap_coordinates
+        self.size: List[npt.NDArray] = [overlap_size]
+        self.tiles: Tuple[Tile, Tile] = (tile_i, tile_j)
+        self.local_coordinates: List[Tuple[npt.NDArray, npt.NDArray]] = [


Is this datatype (List[Tuple[npt.NDArray, npt.NDArray]]) convenient for you somewhere else? 😮

I later understood that the tuple level is for the resolution 🙂
Can we have a comment unpacking the hierarchies of local_coordinates?

lauraporta · 2024-11-14T16:19:20Z

brainglobe_stitch/tile.py

+        resolution_pyramid = self.tiles[0].resolution_pyramid
+
+        for i in range(self.coordinates.shape[0]):
+            if self.tiles[0].position[i] < self.tiles[1].position[i]:


Why are we referring to the first and second tile? 🤔

I am understanding that the list of tiles is calculated once outside the creation of the Overlap object, and so they will have fixed positions... how do the first and second tile relate to the local coords of any overlap?

Each Overlap is made up of two tiles. There are two separate but connected sets of coordinates that we keep track of, the "global" coordinates representing where the overlap lies in the final fused image, and the "local" coordinates which hold information on where the overlap lies in the Tile it originates in. The get_local_overlap_coordinates method checks which tile is "first" in each dimension to see if the overlap is at the beginning or the end for that specific dimension. This later allows us to extract, and modify the data stored in each overlap prior to fusing the entire image.

Oh wow I would have never guessed that 😂

lauraporta · 2024-11-14T16:30:43Z

brainglobe_stitch/tile.py

+            )
+            self.size.append(self.size[0] // resolution_pyramid[j])
+
+    def extract_tile_overlaps(


Do you think that these two methods will be useful for the plugin as well (and not uniquely to testing)?

They'll be used in upcoming PRs for replacing overlapping areas after interpolating the stitched area.

lauraporta · 2024-11-14T16:46:07Z

tests/test_unit/test_tile.py

@@ -1,4 +1,106 @@
-from brainglobe_stitch.tile import Tile
+from typing import List, Tuple


Don't know if you only use tox for testing, but with pytest the tests do not run. I think the pyproject.toml file needs a change:

[tool.pytest.ini_options] addopts = ". --cov=brainglobe_stitch"

I can't seem to replicate this. What's the error?

No idea, now it works without any issue 🤷🏻‍♀️

lauraporta

LeGiTiMate ✅

IgorTatarnikov added 2 commits October 28, 2024 15:22

Overlap implementation and extraction from the image mosaic

9157ead

Updated tile tests

ce2c812

IgorTatarnikov requested a review from lauraporta November 11, 2024 17:08

Fixed imagej path check

6a2a380

lauraporta reviewed Nov 14, 2024

View reviewed changes

Address some review comments

79fdbc3

lauraporta self-requested a review November 29, 2024 11:47

lauraporta approved these changes Nov 29, 2024

View reviewed changes

IgorTatarnikov merged commit 41dfd95 into main Nov 29, 2024
13 checks passed

IgorTatarnikov deleted the extract-overlaps branch November 29, 2024 13:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Extract and replace data in overlapping regions between tiles #20

Extract and replace data in overlapping regions between tiles #20

IgorTatarnikov commented Nov 11, 2024 •

edited

Loading

codecov bot commented Nov 11, 2024 •

edited

Loading

lauraporta left a comment

lauraporta Nov 14, 2024

IgorTatarnikov Nov 29, 2024

lauraporta Nov 14, 2024

IgorTatarnikov Nov 29, 2024

lauraporta Nov 14, 2024

lauraporta Nov 14, 2024

IgorTatarnikov Nov 29, 2024

lauraporta Nov 14, 2024

lauraporta Nov 14, 2024

IgorTatarnikov Nov 29, 2024

lauraporta Nov 29, 2024

lauraporta Nov 14, 2024

IgorTatarnikov Nov 29, 2024

lauraporta Nov 14, 2024

IgorTatarnikov Nov 29, 2024

lauraporta Nov 29, 2024

lauraporta left a comment

		@@ -1,4 +1,106 @@
		from brainglobe_stitch.tile import Tile
		from typing import List, Tuple

Extract and replace data in overlapping regions between tiles #20

Extract and replace data in overlapping regions between tiles #20

Conversation

IgorTatarnikov commented Nov 11, 2024 • edited Loading

Description

How has this PR been tested?

Is this a breaking change?

Does this PR require an update to the documentation?

Checklist:

codecov bot commented Nov 11, 2024 • edited Loading

Codecov Report

lauraporta left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lauraporta left a comment

Choose a reason for hiding this comment

IgorTatarnikov commented Nov 11, 2024 •

edited

Loading

codecov bot commented Nov 11, 2024 •

edited

Loading