Skip to content

Commit

Permalink
how can i aggregate
Browse files Browse the repository at this point in the history
  • Loading branch information
ritchie46 committed Dec 12, 2020
1 parent 9a38b3c commit ce5890f
Show file tree
Hide file tree
Showing 4 changed files with 32 additions and 2 deletions.
7 changes: 5 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
SHELL=/bin/bash
PYTHON=.venv/bin/python
#PYTHON=.venv/bin/python
PYTHON=python

.venv:
@python3 -m venv .venv
Expand All @@ -13,15 +14,17 @@ data/: .venv
$(PYTHON) -m micro_bench.groupby_pandas
$(PYTHON) -m micro_bench.groupby_polars
$(PYTHON) -m micro_bench.join
$(PYTHON) -m micro_bench.plot_results

.PHONY: clean
@rm -r .venv
@rm -r data


run: data
$(PYTHON) -m micro_bench.plot_results
$(PYTHON) -m book.src.examples.lazy_chapter.data_head
$(PYTHON) -m book.src.examples.lazy_chapter.predicate_pushdown_0
$(PYTHON) -m book.src.examples.lazy_chapter.predicate_pushdown_1
$(PYTHON) -m book.src.examples.lazy_chapter.projection_pushdown_0
$(PYTHON) -m book.src.examples.how_can_i.groupby
$(PYTHON) -m book.src.examples.how_can_i.aggregate
1 change: 1 addition & 0 deletions book/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,4 @@
* [Other optimizations](lazy_polars/other_optimizations.md)
- [How can I?](how_can_i/intro.md)
* [GroupBy](how_can_i/groupby.md)
* [Aggregate](how_can_i/aggregate.md)
12 changes: 12 additions & 0 deletions book/src/examples/how_can_i/aggregate.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
import pypolars as pl
from pypolars.lazy import *

reddit = (
pl.scan_csv("data/reddit.csv")
.select([pl.sum("comment_karma"), pl.min("link_karma")])
)

if __name__ == "__main__":
df = reddit.collect()
with open("book/src/outputs/how_can_i_aggregate.txt", "w") as f:
f.write(str(df))
14 changes: 14 additions & 0 deletions book/src/how_can_i/aggregate.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# How can I aggregate?

Aggregations can be done in a `.select` or a `.with_column` method.

If you want to do a specific aggregation on all columns you can use the wildcard expression: `.select(col("*").sum())`

```python
{{#include ../examples/how_can_i/aggregate.py:1:8}}
reddit.collect()
```

```text
{{#include ../outputs/how_can_i_aggregate.txt}}
```

0 comments on commit ce5890f

Please sign in to comment.