Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Pick-2183][AMORO-2168] Support hive commit protocol in mixed-hive format. #2220

Merged
merged 5 commits into from
Nov 1, 2023
Merged

[Pick-2183][AMORO-2168] Support hive commit protocol in mixed-hive format. #2220

merged 5 commits into from
Nov 1, 2023

Conversation

baiyangtx
Copy link
Contributor

@baiyangtx baiyangtx commented Oct 31, 2023 •
edited
Loading

Why are the changes needed?

Pick #2183 to 0.4.x branch

Brief change log

see #2183

How was this patch tested?

  • Add some test cases that check the changes thoroughly including negative and positive cases if possible

  • Add screenshots for manual tests if appropriate

  • Run test locally before making a pull request

Documentation

  • Does this pull request introduce a new feature? (yes )
  • If yes, how is the feature documented? ( docs )

baiyangtx and others added 2 commits October 31, 2023 16:06
@baiyangtx baiyangtx marked this pull request as ready for review October 31, 2023 08:13
@github-actions github-actions bot added type:docs Improvements or additions to documentation module:mixed-flink Flink moduel for Mixed Format module:mixed-spark Spark module for Mixed Format module:core Core module module:mixed-hive Hive moduel for Mixed Format labels Oct 31, 2023
@codecov
Copy link

codecov bot commented Oct 31, 2023 •
edited
Loading

Codecov Report

Attention: 20 lines in your changes are missing coverage. Please review.

Files Coverage Δ
...om/netease/arctic/data/file/FileNameGenerator.java 80.82% <ø> (ø)
...java/com/netease/arctic/table/TableProperties.java 0.00% <ø> (ø)
...a/com/netease/arctic/hive/HiveTableProperties.java 0.00% <ø> (ø)
...va/com/netease/arctic/hive/op/UpdateHiveFiles.java 85.92% <100.00%> (+0.42%) ⬆️
.../java/com/netease/arctic/spark/io/TaskWriters.java 92.30% <100.00%> (+0.17%) ⬆️
...e/io/writer/AdaptHiveGenericTaskWriterBuilder.java 66.33% <94.11%> (+2.89%) ⬆️
...tic/hive/io/writer/AdaptHiveOutputFileFactory.java 82.60% <90.00%> (+3.66%) ⬆️
.../netease/arctic/hive/op/ReplaceHivePartitions.java 67.12% <83.33%> (+0.45%) ⬆️
.../com/netease/arctic/hive/utils/HiveCommitUtil.java 90.00% <90.00%> (ø)
...com/netease/arctic/hive/op/OverwriteHiveFiles.java 41.66% <40.00%> (-4.34%) ⬇️
... and 1 more

... and 3 files with indirect coverage changes

📢 Thoughts on this report? Let us know!.

Aireed
Aireed approved these changes Nov 1, 2023
View reviewed changes
Copy link
Contributor

@Aireed Aireed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zhoujinsong zhoujinsong merged commit 0472fe2 into apache:0.4.x Nov 1, 2023
5 checks passed
@baiyangtx baiyangtx deleted the pick-2168-hive-consistent-write branch November 1, 2023 12:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Aireed Aireed Aireed approved these changes

Assignees
No one assigned
Labels
module:core Core module module:mixed-flink Flink moduel for Mixed Format module:mixed-hive Hive moduel for Mixed Format module:mixed-spark Spark module for Mixed Format type:docs Improvements or additions to documentation
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@baiyangtx @Aireed @zhoujinsong