-
Notifications
You must be signed in to change notification settings - Fork 54
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Better support for in-place operations on tables #116
Comments
Ok, I've been noodling on this for.......6 days (haha, actually longer, because people have brought it up on slack and stuff). @rofinn can you talk a little more about the use-case you have in mind for this? I have some ideas, but most of mine end in "oh, this actually wouldn't be useful for the most part", but I want to hear a solid case where someone wants to use it and how it would be helpful. Anyway, I can try to put some of my thoughts together, but in the mean time, I thought I'd ask for some more info from your side. |
My use case is in Impute.jl where I'm trying to mutate data in-place if possible by applying some operation over each column.
https://github.com/invenia/Impute.jl/blob/master/src/imputors.jl#L155 In this code, the passed in table will only sometimes mutate the data depending on table type passed in. It'd be nice if I could check that calling |
Hi, for my usecase -- trying to get Selections.jl easily available to the ecosystem -- I'd like to be have |
With #131, we're committing to enhancing the Tables.jl interface a bit, but also trying to keep it very minimal, to encourage adoption. As I've thought of this and a few other related issues, I think it would make sense to have a |
I think that there are three levels of this mutability, and we should be explicit at which level we target:
|
I think a separate package, which enhances the interface, would be the best approach for now, since it would allow experimentation with the mutable interface without affecting the API defined in this package (cf #133). |
@bkamins I would be tempted to try make these three seperate/orthogonal interfaces for perming different mutations, rather than “levels” or layers with some on top of the others. E.g. I’m imagining you could have 3 without 2 (data frame of static arrays) or 2 without 1 (functional programmers like to think of “append only” databases). |
Sure - they are largely orthogonal. I have this order in the back of my head, as it is natural in DataFrames.jl, but for other data structures clearly it is the way you say 😄. A particular cases is that 3 assumes allowing "replacing" of the column it is not the same as 1, which mostly assumes updating column in-place (however, for some data structures 1 would imply replacement - when in order to |
Yes it’s very interesting how mutating a column behaves somewhat the same as mutating the rows. Of course, you can tell the difference when you have access to the column references. The way I always imagined this playing out is (a) have two APIs/traits for mutation and insertion into data structures (and “upsert” for data structures that support both, this is the way it is done in Dictionaries.jl), and (b) have table modelled as a nested data structure (a relation is a collection of rows). All the different cases you mention simply fall out naturally. |
On a related note, should Tables.jl have a |
I wonder if there was any progress regarding experimentation of a trait system for mutable tables? At this point in time Tables.jl is the defacto standard for tables in Julia, and we are reaching applications where mutability and a basic |
Any progress here? Or any draft somewhere? |
Specifically, it'd be nice if I could use some traits to determine whether I can mutate the underlying data during row or column iteration (e.g., mutating values in DataFrameRow).
The text was updated successfully, but these errors were encountered: