Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: bump rocksdb to v9.2.1 #2327

Merged
merged 4 commits into from
Jun 3, 2024
Merged

Conversation

aleksraiden
Copy link
Contributor

@aleksraiden aleksraiden commented May 24, 2024

Bump rocksdb to v9.2.1. Full changelog: https://github.com/facebook/rocksdb/releases/tag/v9.2.1

Key features

  • Added a new API GetEntityFromBatchAndDB to WriteBatchWithIndex that can be used for wide-column point lookups with read-your-own-writes consistency. Similarly to GetFromBatchAndDB, the API can combine data from the write batch with data from the underlying database if needed
  • [Experimental] Introduce two new cross-column-family iterators - CoalescingIterator and AttributeGroupIterator. The CoalescingIterator enables users to iterate over multiple column families and access their values and columns. During this iteration, if the same key exists in more than one column family, the keys in the later column family will overshadow the previous ones. The AttributeGroupIterator allows users to gather wide columns per Column Family and create attribute groups while iterating over keys across all CFs.
  • Added a new API MultiGetEntityFromBatchAndDB to WriteBatchWithIndex that can be used for batched wide-column point lookups with read-your-own-writes consistency.
  • Add an option to WaitForCompactOptions - wait_for_purge to make WaitForCompact() API wait for background purge to complete
  • DeleteRange() will return NotSupported() if row_cache is configured since they don't work together in some cases.
  • Fix a bug causing VerifyFileChecksums() to return false-positive corruption under BlockBasedTableOptions::block_align=true
  • Fixed a bug in MultiGet() and MultiGetEntity() together with blob files (ColumnFamilyOptions::enable_blob_files == true). An error looking up one of the keys could cause the results to be wrong for other keys for which the statuses were Status::OK.

P.S. This update includes an 9.1.х releases too (see: https://github.com/facebook/rocksdb/releases/tag/v9.1.0)

@aleksraiden aleksraiden self-assigned this May 24, 2024
@aleksraiden
Copy link
Contributor Author

After a deep investigation, I'm found a broken commit - facebook/rocksdb@3104e55

@aleksraiden
Copy link
Contributor Author

All work OK if we use an option to disable using JEMALLOC at test docker container stage. So, adding a -DWITH_JEMALLOC=OFF to build args provide a successful build and all test are passed.

It's a magic :)))

@PragmaTwice PragmaTwice changed the title Bump rocksdb to v9.2.1 chore: bump rocksdb to v9.2.1 May 31, 2024
@PragmaTwice
Copy link
Member

All work OK if we use an option to disable using JEMALLOC at test docker container stage. So, adding a -DWITH_JEMALLOC=OFF to build args provide a successful build and all test are passed.

It's a magic :)))

Yeah it's because in alpine it uses musl libc instead of glibc, so there may be some problem.

I'll check this problem soon.

@aleksraiden
Copy link
Contributor Author

All work OK if we use an option to disable using JEMALLOC at test docker container stage. So, adding a -DWITH_JEMALLOC=OFF to build args provide a successful build and all test are passed.
It's a magic :)))

Yeah it's because in alpine it uses musl libc instead of glibc, so there may be some problem.

I'll check this problem soon.

But issue is only this test. Another problem - alpine 3.16 is too old (and musl too), but we cant upgrade into 3.20 because no libexecinfo (

@aleksraiden
Copy link
Contributor Author

If this PR #2348 will be OK, we can update to use latest rocksdb (as I hope)))

Copy link

sonarcloud bot commented Jun 3, 2024

Quality Gate Failed Quality Gate failed

Failed conditions
28.5% Coverage on New Code (required ≥ 50%)

See analysis details on SonarCloud

@aleksraiden aleksraiden marked this pull request as ready for review June 3, 2024 10:36
@aleksraiden
Copy link
Contributor Author

Now we are ready to merge (wow!)

@PragmaTwice PragmaTwice merged commit 9d65dce into apache:unstable Jun 3, 2024
35 of 36 checks passed
@mapleFU
Copy link
Member

mapleFU commented Jun 3, 2024

Public API Changes
DeleteRange() will return NotSupported() if row_cache is configured since they don't work together in some cases.

Would this matters on row_cache enabled user?

@PragmaTwice
Copy link
Member

Public API Changes
DeleteRange() will return NotSupported() if row_cache is configured since they don't work together in some cases.

Would this matters on row_cache enabled user?

🤔 We need to investigate this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants