Updated Basic Usage of Manipulation Functions #3360

nathanrboyer · 2023-07-18T19:51:10Z

Replaces PR #2907 since it was too out of sync with the main branch.

docs/src/man/basics.md

nathanrboyer · 2023-09-15T13:44:18Z

Friendly bump 🙂

bkamins · 2023-09-15T17:20:36Z

Agreed. But first we need to make a decision on #3361. I will have a look at it and comment there.

bkamins · 2023-09-25T21:18:52Z

docs/src/man/basics.md

+| ------------ | -------------------------------- | -------------------------------------------- | ------------------------------------------------- |
+| `transform`  | Creates a new data frame.        | Retains both source and manipulated columns. | Retains same number of rows as source data frame. |
+| `transform!` | Modifies an existing data frame. | Retains both source and manipulated columns. | Retains same number of rows as source data frame. |
+| `select`     | Creates a new data frame.        | Retains only manipulated columns.            | Retains same number of rows as source data frame. |


"manipulated" or "created"?

bkamins · 2023-09-25T21:19:46Z

docs/src/man/basics.md

+| `transform!` | Modifies an existing data frame. | Retains both source and manipulated columns. | Retains same number of rows as source data frame. |
+| `select`     | Creates a new data frame.        | Retains only manipulated columns.            | Retains same number of rows as source data frame. |
+| `select!`    | Modifies an existing data frame. | Retains only manipulated columns.            | Retains same number of rows as source data frame. |
+| `subset`     | Creates a new data frame.        | Retains only source columns.                 | Number of rows is determined by the manipulation. |


maybe better say that by condition (to differentiate it from combine)

docs/src/man/basics.md

bkamins · 2023-09-25T21:24:41Z

docs/src/man/basics.md

+### Constructing Operation Pairs
+All of the functions above use the same syntax which is commonly
+`manipulation_function(dataframe, operation)`.
+The `operation` argument is a `Pair` which defines the


maybe add that Pair is constructed using =>?

Also - it does not have to be the pair, so maybe say it is usually a pair?
(your first example is not a pair below)

docs/src/man/basics.md

bkamins · 2023-09-25T21:28:31Z

docs/src/man/basics.md

-These rules are typically called transformation mini-language.
-
-Let us move to the examples of application of these rules
+## Basic Usage of Manipulation Functions


A larger question - maybe create a separate page for this tutorial?

I think that would be better. Maybe "Manipulation Functions" under "User Guide" before "Split-apply-combine"?

As commented below - I would put it as a "top level" with a name something along "A gentle introduction to manipulation functions" (so that we clearly signal that this material is less formal than the rest of the manual).

What would you think of making the new Top Level section something like "Beginner's Guide" or "User's Guide for Beginners" and then placing "Manipulation Functions" at a second level under that? I'm not volunteering to rewrite the entire User's Guide, but it could leave room for others to add similar "gentle" content to the documentation. It would also make the sidebar look cleaner by splitting up the current long name.

Makes sense. The other section that could go there is https://dataframes.juliadata.org/stable/man/basics/ as it has the same objective.

Oh, but Basics contains the existing "Basic Usage of Manipulation Functions". I don't know how to differentiate it from this new section if they live next to each other.

I initially intended to just clarify some topics within that section, but now the scope has grown.

I can maybe overwrite that section if I add these topics that I don't currently cover:

"Note that this time we use string column selectors because some of the column names have spaces in them."

"The benefit of select or combine over indexing is that it is easier to get the union of several column selectors."

"It is important to note that select always returns a data frame, even if a single column selected as opposed to indexing syntax."

"By default select copies columns of a passed source data frame. In order to avoid copying, pass the copycols=false keyword argument."

The other sections under Basics use the German dataset, but I think it is easier to understand what is going on with smaller data frames where you know all the data values.

I can maybe overwrite that section

Yes - I think it is OK just to expand that section (especially that it is top-level now already)

I think it is easier to understand what is going on with smaller data frames

Agreed. just please use different variable names than these already used there so that using different dataframes does not lead to confusion.

Thank you! (sorry for so many comments, but - unfortunately - writing documentation is hard)

Done (sort of). I did not use new variables names though. I had been and continued to just overwrite the definition of df. My data frames are so small and frequent that coming up with a new name each time would be a pain.

bkamins · 2023-09-25T21:29:40Z

docs/src/man/basics.md

+!!! Note
+      The Julia parser sometimes prevents `:` from being used by itself.
+      `ERROR: syntax: whitespace not allowed after ":" used for quoting`
+      means your `:` must be wrapped in either `(:)` or `Cols(:)`


I typically recommend All() which is easy to understand I think.

docs/src/man/basics.md

mortenpi · 2023-12-06T23:23:20Z

@mortenpi - how could @nathanrboyer reference e.g. this section

Sorry, missed this earlier. This actually looks like a bug. The fact that it's level 3 heading shouldn't matter. As a workaround, you might be able to use @id-s in the headers though.

nathanrboyer · 2024-05-30T16:25:56Z

I think I fixed my errors, but I am still getting some errors below which seem unrelated to me.

I get this error on main with julia --project make.jl:

ERROR: LoadError: ArgumentError: makedocs() got passed invalid keyword arguments:
  strict = true

I get this error on this branch with julia --project make.jl:

[ Info: Doctest: running doctests.
┌ Error: doctest failure in C:\Users\nboyer.AIP\.julia\packages\DataFrames\58MUJ\src\abstractdataframe\iteration.jl:38-73
│ 
│ ```jldoctest
│ julia> df = DataFrame(x=1:4, y=11:14)
│ 4×2 DataFrame
│  Row │ x      y
│      │ Int64  Int64
│ ─────┼──────────────
│    1 │     1     11
│    2 │     2     12
│    3 │     3     13
│    4 │     4     14
│ 
│ julia> eachrow(df)
│ 4×2 DataFrameRows
│  Row │ x      y
│      │ Int64  Int64
│ ─────┼──────────────
│    1 │     1     11
│    2 │     2     12
│    3 │     3     13
│    4 │     4     14
│ 
│ julia> copy.(eachrow(df))
│ 4-element Vector{NamedTuple{(:x, :y), Tuple{Int64, Int64}}}:
│  (x = 1, y = 11)
│  (x = 2, y = 12)
│  (x = 3, y = 13)
│  (x = 4, y = 14)
│ 
│ julia> eachrow(view(df, [4, 3], [2, 1]))
│ 2×2 DataFrameRows
│  Row │ y      x
│      │ Int64  Int64
│ ─────┼──────────────
│    1 │    14      4
│    2 │    13      3
│ ```
│ 
│ Subexpression:
│ 
│ copy.(eachrow(df))
│ 
│ Evaluated output:
│ 
│ 4-element Vector{@NamedTuple{x::Int64, y::Int64}}:
│  (x = 1, y = 11)
│  (x = 2, y = 12)
│  (x = 3, y = 13)
│  (x = 4, y = 14)
│ 
│ Expected output:
│ 
│ 4-element Vector{NamedTuple{(:x, :y), Tuple{Int64, Int64}}}:
│  (x = 1, y = 11)
│  (x = 2, y = 12)
│  (x = 3, y = 13)
│  (x = 4, y = 14)
│ 
│   diff =
│    4-element Vector{NamedTuple{(:x, :y), Tuple{Int64, Int64}}}:
│     Vector{@NamedTuple{x::Int64, y::Int64}}:
│     (x = 1, y = 11)
│     (x = 2, y = 12)
│     (x = 3, y = 13)
│     (x = 4, y = 14)
└ @ Documenter C:\Users\nboyer.AIP\.julia\packages\DataFrames\58MUJ\src\abstractdataframe\iteration.jl:38
ERROR: LoadError: `makedocs` encountered a doctest error. Terminating build

mortenpi · 2024-05-30T23:34:08Z

strict = true

If this is using Documenter 1.0+, then that keyword was removed (and strict=true is now the default). It looks like it got correctly removed on master though: https://github.com/JuliaData/DataFrames.jl/pull/3416/files#diff-4aae2d1c783cade58bd2cb13748da956e568b7f2aed5fafd9e2a46fb97daf613L45

The doctest failures seem to be due to different printing between Julia versions (i.e. make sure you use the same Julia version locally as is being used for the docs CI).

nathanrboyer · 2024-05-31T15:57:10Z

I thought that I already was in sync from this: 7614fc3
but I found another button to sync on my repository page which fixed the strict line.

I am still getting the second error, and I am using the juliaup release branch which seems to match the ci.yml.

version:
- '1.6'
- '1' # automatically expands to the latest stable 1.x release of Julia

…ion_basics

nathanrboyer · 2024-12-03T19:56:59Z

I am going to need help figuring out the build issue here. The tests were all passing until I merged main into this branch again today.

The failing doctest is one that appeared earlier on a docstring that is not part of this PR. I attempted to fix the issue in #3472, which worked on my local computer but apparently did not work when CI ran the same tests here. I'm not sure what to do now.

bkamins · 2024-12-11T20:02:05Z

Yes - let us hope that after StableRNGs.jk PR merge this one goes through cleanly. Thank you!

docs/src/man/basics.md

bkamins · 2024-12-13T11:47:15Z

Thank you!
Can you please double check the documentation on-line after it is deployed?

nathanrboyer · 2024-12-13T12:55:17Z

Reviewing now ...
The definition list doesn't seem to be supported. 😞
VSCode:

Online:

nathanrboyer · 2024-12-13T13:13:13Z

I also want to remind you that the terminology we decided on in #2907 (comment) still needs to be propagated to the rest of the manual and docstrings.

Initial commit

5785c72

nathanrboyer changed the title ~~Initial commit~~ Updated Basic Usage of Manipulation Functions Jul 18, 2023

nathanrboyer added 2 commits July 19, 2023 11:32

assumes requested method is added

f5cc15f

Typo: missing :z

7c3db8a

bkamins reviewed Jul 25, 2023

View reviewed changes

docs/src/man/basics.md Outdated Show resolved Hide resolved

nathanrboyer added 3 commits July 25, 2023 12:10

added subset(df, source_column_selector)

27d7e32

Added italics

be5fa9e

Moved note to main text

b1b3bab

nathanrboyer commented Aug 2, 2023

View reviewed changes

docs/src/man/basics.md Outdated Show resolved Hide resolved

nathanrboyer commented Aug 2, 2023

View reviewed changes

docs/src/man/basics.md Show resolved Hide resolved

nathanrboyer and others added 5 commits August 2, 2023 15:02

Added error example and removed ° symbol

da6607d

Moved Additional Resources to the end and cleaned

0c47d10

Capitalized Boolean

6f5dfc5

Merge branch 'main' into nb/manipulation_function_basics

e47cbdf

Removed extra space character

886d998

nathanrboyer added 3 commits September 18, 2023 12:02

Change function broadcasting to avoid old language

cabd73f

Made consistent with current proposal JuliaData#3361

2e9d2af

Change α to apple and make consistent with JuliaData#3380

b0777b1

bkamins reviewed Sep 25, 2023

View reviewed changes

docs/src/man/basics.md Outdated Show resolved Hide resolved

bkamins reviewed Sep 25, 2023

View reviewed changes

docs/src/man/basics.md Show resolved Hide resolved

bkamins reviewed Sep 25, 2023

View reviewed changes

docs/src/man/basics.md Show resolved Hide resolved

bkamins reviewed Sep 25, 2023

View reviewed changes

docs/src/man/basics.md Show resolved Hide resolved

bkamins reviewed Sep 25, 2023

View reviewed changes

docs/src/man/basics.md Show resolved Hide resolved

bkamins reviewed Sep 25, 2023

View reviewed changes

bkamins reviewed Oct 14, 2023

View reviewed changes

docs/src/man/basics.md Outdated Show resolved Hide resolved

bkamins reviewed Oct 14, 2023

View reviewed changes

docs/src/man/basics.md Outdated Show resolved Hide resolved

nathanrboyer and others added 3 commits October 14, 2023 21:42

Delete redundant expression

679f65f

Clean up new section and delete with reference

79a1171

Merge branch 'main' into nb/manipulation_function_basics

621f253

Merge branch 'main' into nb/manipulation_function_basics

7614fc3

nathanrboyer mentioned this pull request May 20, 2024

Cannot find level-three cross-references JuliaDocs/Documenter.jl#2507

Closed

nathanrboyer added 2 commits May 30, 2024 09:50

Fix admonitions

82d935c

Fix manipulation function reference

d9864ba

Merge remote-tracking branch 'origin/main' into nb/manipulation_funct…

efde542

…ion_basics

nathanrboyer mentioned this pull request Oct 10, 2024

Stabilize random number reproducibility in doctests #3472

Merged

Merge branch 'main' into nb/manipulation_function_basics

2abf894

Merge branch 'main' into nb/manipulation_function_basics

113d608

bkamins reviewed Dec 13, 2024

View reviewed changes

docs/src/man/basics.md Outdated Show resolved Hide resolved

bkamins reviewed Dec 13, 2024

View reviewed changes

docs/src/man/basics.md Outdated Show resolved Hide resolved

bkamins reviewed Dec 13, 2024

View reviewed changes

docs/src/man/basics.md Outdated Show resolved Hide resolved

Apply suggestions from code review

9d03a64

bkamins approved these changes Dec 13, 2024

View reviewed changes

bkamins merged commit dc59622 into JuliaData:main Dec 13, 2024
6 checks passed

nathanrboyer added a commit to nathanrboyer/DataFrames.jl that referenced this pull request Dec 13, 2024

Small formatting tweaks to JuliaData#3360 after reviewing online

261a1cc

bkamins pushed a commit that referenced this pull request Dec 13, 2024

Small formatting tweaks to #3360 after reviewing online (#3483)

84cd36b

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Updated Basic Usage of Manipulation Functions #3360

Updated Basic Usage of Manipulation Functions #3360

nathanrboyer commented Jul 18, 2023 •

edited

Loading

nathanrboyer commented Sep 15, 2023

bkamins commented Sep 15, 2023

bkamins Sep 25, 2023

bkamins Sep 25, 2023

bkamins Sep 25, 2023

bkamins Sep 25, 2023

bkamins Sep 25, 2023

nathanrboyer Sep 28, 2023

bkamins Sep 30, 2023

nathanrboyer Oct 4, 2023

bkamins Oct 9, 2023

nathanrboyer Oct 9, 2023

bkamins Oct 9, 2023

nathanrboyer Oct 13, 2023

bkamins Sep 25, 2023

mortenpi commented Dec 6, 2023

nathanrboyer commented May 30, 2024

mortenpi commented May 30, 2024 •

edited

Loading

nathanrboyer commented May 31, 2024

nathanrboyer commented Dec 3, 2024

bkamins commented Dec 11, 2024

bkamins commented Dec 13, 2024

nathanrboyer commented Dec 13, 2024

nathanrboyer commented Dec 13, 2024 •

edited

Loading

Updated Basic Usage of Manipulation Functions #3360

Updated Basic Usage of Manipulation Functions #3360

Conversation

nathanrboyer commented Jul 18, 2023 • edited Loading

nathanrboyer commented Sep 15, 2023

bkamins commented Sep 15, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mortenpi commented Dec 6, 2023

nathanrboyer commented May 30, 2024

mortenpi commented May 30, 2024 • edited Loading

nathanrboyer commented May 31, 2024

nathanrboyer commented Dec 3, 2024

bkamins commented Dec 11, 2024

bkamins commented Dec 13, 2024

nathanrboyer commented Dec 13, 2024

nathanrboyer commented Dec 13, 2024 • edited Loading

nathanrboyer commented Jul 18, 2023 •

edited

Loading

mortenpi commented May 30, 2024 •

edited

Loading

nathanrboyer commented Dec 13, 2024 •

edited

Loading