Making multiple VG Plots from large parquet file

[bbollen23]

Hi there,
So far, Mosaic has been great for us. But I'm currently having trouble getting the correct paradigm down for how to interact with Mosaic in the "correct" way.

The situation is that our application loads several large files when the application opens. Our idea was that if we can use "vg.loadParquet" to initialize the table, we can generate several different plots from this large table.

But I'm a bit confused on how to go about this. I've had some success with creating temp tables using coordinator.exec() and just filtering what I want out of the large table with a raw sql query, but this seems to go against what Mosaic is trying to do. My other guess was that each chart is essentially the same chart except with different data automatically filtered out.

So I guess all I'm asking is

1. given a table that is already generated in DuckDB, how can I generate several charts from this one dataset with varying "where" clauses
2. Is initializing a table before chart creation a smart way to go about generating charts or is this an anti-pattern in Mosaic's case?

Any help would be greatly appreciated. This may be a very simple question, but I've scoured the documentation and can't find any that seems to match what I'm trying to do. Thanks!

[domoritz]

I'm not sure I totally follow. Mosaic works best when you operate on a single dataset so that cross-filtering etc works across the various clients. Can you give a bit more detail on the use case? Do you want to have a a single source dataset (combining multiple parquet files) and then show different subsets on different charts?

[jheer]

Hi all, I would want to implement this by loading the main table once, and creating a grid of separate vgplot plot instances. Sounds like this is what you've initially tried to do.

As I understand it, the "tricky" bit is applying different fixed filtering criteria to these different plots. Right now all filtering criteria pushed to the database are passed in via the filterBy selection. We could add an additional data property for marks (maybe named where?) which includes a "hardwired" SQL filtering predicate that is always applied to that plot.

Do you think this would address your use case? It's certainly something I've seen requested in the past.

[domoritz]

Oh, so it's a facet/small multiple charts. We have that in https://idl.uw.edu/mosaic/examples/walmart-openings.html so you can look at how we do that there. Do you want to make this whole facet one mosaic client or each small chart a client?

[bbollen23]

Hi all, I would want to implement this by loading the main table once, and creating a grid of separate vgplot plot instances. Sounds like this is what you've initially tried to do.

As I understand it, the "tricky" bit is applying different fixed filtering criteria to these different plots. Right now all filtering criteria pushed to the database are passed in via the filterBy selection. We could add an additional data property for marks (maybe named where?) which includes a "hardwired" SQL filtering predicate that is always applied to that plot.

Do you think this would address your use case? It's certainly something I've seen requested in the past.

Yes! Thats what I was looking for. A lot of our charts do require user filtering, but they essentially "begin" as an already filtered dataset. So when working with Vue and React, its great to be able to pass these in as parameters to our component which can be passed to the plot for filtering.

Just like you said, our solution was still to generate separate VG plots (and we made our grid as just a Vue component which loops to create all the charts). Our way of doing that "fixed filtering" was to mimic an initial selection on the data.

[bbollen23]

Oh, so it's a facet/small multiple charts. We have that in https://idl.uw.edu/mosaic/examples/walmart-openings.html so you can look at how we do that there. Do you want to make this whole facet one mosaic client or each small chart a client?

To be honest, I'm not really clear on what the term "client" really means, but I can explain our desired workflow.

User selects an "experiment". For one experiment, one file is loaded using "vg.loadParquet()". This file contains all the information we need for the experiment.

For a single experiment, there are several different views, each rendering one or more individual plots (each plot would be one call of "vg.plot()"). Our eventual goal is to be able to link all these plots together. So, we may have a bar chart which shows the distribution of the masses of cells. When a user brushes over the bar chart, several other plots will be updated to show the filtering that the user applied.

In the view that @Dev-Lan showed, our goal is that a user can click on a single plot container (so not actually brushing on the vg.plot itself -- they are generally static) and by doing so, it would update a central store which says "Since the user clicked on this plot, we need to update all the filter by statements for all other Vg.plots".

My guess is that based on the documentation and based on the example we gave above, this is possible as long as there is a central truth of what the "filterBy" statements are applying to -- the "source". For example, if all vg.plots have the following filter by statement:

let filterBy = {
   source: "central_truth_source",
   predicate: "my_predicate"
}

Then each of the plots, regardless of where they are should be filtered correctly. Then, we can apply additional filtering if necessary to the individual plots (using something like vg.Selection.intersect() ).

Does that make sense?

[domoritz]

To be honest, I'm not really clear on what the term "client" really means

A client in Mosaic is this: https://idl.uw.edu/mosaic/core/#clients