Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set up basic CI, added tests, and implemented the mode function. #2

Merged
merged 5 commits into from
Sep 23, 2024

Conversation

demetribu
Copy link
Collaborator

Closes: #1

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very cool @dmitrybugakov -- thank you!

.gitignore Outdated
Cargo.lock
!datafusion-cli/Cargo.lock
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some of this looks like it may only be relevant for the datafusion project and not this one

<!-- [![CI](https://github.com/datafusion-contrib/datafusion-functions-json/actions/workflows/ci.yml/badge.svg?event=push)](https://github.com/datafusion-contrib/datafusion-functions-json/actions/workflows/ci.yml?query=branch%3Amain) -->
<!-- [![Crates.io](https://img.shields.io/crates/v/datafusion-functions-json?color=green)](https://crates.io/crates/datafusion-functions-json) -->

**Note:** This is not an official Apache Software Foundation release.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️ ❤️

README.md Outdated
@@ -0,0 +1,64 @@
# datafusion-functions-extra

<!-- [![CI](https://github.com/datafusion-contrib/datafusion-functions-json/actions/workflows/ci.yml/badge.svg?event=push)](https://github.com/datafusion-contrib/datafusion-functions-json/actions/workflows/ci.yml?query=branch%3Amain) -->
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These still refer to datafusion-functions-json which may be ok

tests/main.rs Outdated
async fn test_mode_date64() {
let sql = "SELECT MODE(date64_col) FROM test_table";
let batches = run_query(sql).await.unwrap();
assert_eq!(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some potential ideas to make testing easier:

  1. use assert_batches_eq
  2. integrate https://crates.io/crates/sqllogictest (we can avoid most of the stuff in the DataFusion runner, and it could be much simpler here I think) to assist in testing
  3. Use insta.rs (my personal suggestion) to compare expected and actual (it automates the capture and update of expected output). Here is an example

https://github.com/datafusion-contrib/datafusion-dft/blob/fbb039fab11a27133b2f06857007b9095d607e5c/tests/extension_cases/functions_json.rs#L35-L54

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alamb thanks for the review!

I’ve refactored the tests using insta, and I must say, it looks great now.

Regarding the sqllogictest, am I correct in understanding that we won’t be reusing the setup from DataFusion core and will instead implement it in place?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

am I correct in understanding that we won’t be reusing the setup from DataFusion core and will instead implement it in place?

that is one proposal -- but I think you should consider if that is worth while or if Insta is ok for this repo.

@demetribu demetribu requested a review from alamb September 22, 2024 18:30
Copy link
Collaborator

@austin362667 austin362667 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's really nice! Thank you

@demetribu
Copy link
Collaborator Author

@alamb and @austin362667 thanks for the review!

@demetribu demetribu merged commit f89f200 into datafusion-contrib:main Sep 23, 2024
3 checks passed
@demetribu demetribu deleted the init-project branch September 23, 2024 15:17
let actual = execution.run_and_format("SELECT MODE(utf8_col) FROM test_table").await;

insta::assert_yaml_snapshot!(actual, @r###"
- +---------------------------+
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Init project datafusion-functions-extra
3 participants