Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix eachrow and eachcol indexing with CartesianIndex #3413

Merged
merged 1 commit into from
Jan 8, 2024

Conversation

bkamins
Copy link
Member

@bkamins bkamins commented Jan 6, 2024

Fixes #3412

@nalimilan - I add it to 1.7 release as probably there is still some time till Julia 1.11 release.

Having said that - I propose we go back to the discussion on which open PRs from https://github.com/JuliaData/DataFrames.jl/pulls we want to merge and make 1.7 release of DataFrames.jl.

The bug is uncovered due to JuliaLang/julia#52736, which starts using CartesianIndexing in broadcasting BitArray.

@nalimilan
Copy link
Member

Changes look good but I'm surprised that they are necessary, as the AbstractArray API doesn't require supporting CartesianIndex. Isn't there a bug in JuliaLang/julia#52736? Or is this due to how DataFrames.jl implements broadcasting?

@bkamins
Copy link
Member Author

bkamins commented Jan 7, 2024

These were also my thoughts, but having read https://docs.julialang.org/en/v1/manual/interfaces/#Working-with-Broadcasted-objects I concluded that broadcasting explicitly requires objects to support CartesianIndex directly (although getindex API does not require it). Quoting:

Iterating over the CartesianIndices of the axes(::Broadcasted) and using indexing with the resulting CartesianIndex object to compute the result.

And this is what JuliaLang/julia#52736 does (i.e. uses CartesianIndexing internally).

@mbauman - could you please confirm that this is a correct thinking?

The issue we currently have is that without the extra definition when we are passed a CartesianIndex when indexing we fall back to:

Base.@propagate_inbounds Base.getindex(itr::DataFrameRows, idx) =
    eachrow(@view parent(itr)[idx isa AbstractVector && !(eltype(idx) <: Bool) ? copy(idx) : idx, :])

which tries something like df[CartesianIndex(1), :] which is invalid.

@nalimilan
Copy link
Member

Ah OK so the failure is "our fault" as the the DataFrameRows method takes precedence over the fallback AbstractArray method.

@bkamins bkamins merged commit 3e29027 into main Jan 8, 2024
6 of 7 checks passed
@bkamins bkamins deleted the bk-fix_indexing_iterators branch January 8, 2024 09:23
@bkamins
Copy link
Member Author

bkamins commented Jan 8, 2024

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CartesianIndex error in Julia 1.11
2 participants