Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DBZ-5071 Correctly handle NULL values in incremental snapshots #131

Merged
merged 1 commit into from
Feb 29, 2024

Conversation

james-johnston-thumbtack
Copy link
Contributor

This makes changes in the Db2 connector for the corresponding commit in the main Debezium project. The main thing we have to do is override the nullsSortLast function in Db2Connection to define how NULLs sort in Db2. We also have to update the test suite, since the base test suite now includes additional tests for working with keys that are nullable.

Related pull request: debezium/debezium#5158

@james-johnston-thumbtack
Copy link
Contributor Author

I ran the following command to test:

admin@ip-10-4-98-175:~/Thumbtack/debezium-connector-db2$ mvn clean install -Dit.test=IncrementalSnapshotIT

The tests pass.

I attached the full log if you are interested.
Db2Test.log

@james-johnston-thumbtack
Copy link
Contributor Author

From CI:

Error:    IncrementalSnapshotIT>AbstractIncrementalSnapshotTest.snapshotWithAdditionalCondition:811->AbstractSnapshotTest.consumeRecordsMixedWithIncrementalSnapshot:251->AbstractSnapshotTest.consumeMixedWithIncrementalSnapshot:182->AbstractSnapshotTest.consumeMixedWithIncrementalSnapshot:192->AbstractSnapshotTest.consumeMixedWithIncrementalSnapshot:211 [Too many no data record results, 0 < 10] 
Expecting:
 <6>
to be less than or equal to:
 <5> 

However, when I had tested the test, it passed - from the log that I posted:

2024-01-27 04:09:09,862 INFO   IncrementalSnapshotIT||test  Test io.debezium.connector.db2.IncrementalSnapshotIT#snapshotWithAdditionalCondition succeeded   [io.debezium.connector.db2.IncrementalSnapshotIT]

I'm assuming this is a previously-flaky test / unrelated to this PR...

There were also some other test failures in totally different test suites, again which seem unrelated:

Error:  Failures: 
Error:    Db2OnlineDefaultValueIT>AbstractDb2DefaultValueIT.shouldHandleDateTimeDefaultTypes:203->AbstractDb2DefaultValueIT.shouldHandleDefaultValuesCommon:217->AbstractDb2DefaultValueIT.testDefaultValuesAlterTableAdd:493->AbstractDb2DefaultValueIT.assertSchemaFieldWithDefaultCurrentDate:544->AbstractDb2DefaultValueIT.assertSchemaFieldValueWithDefault:540->AbstractDb2DefaultValueIT.lambda$assertSchemaFieldWithDefaultCurrentDate$2:552 [Unexpected field value: AVAL_DATE_SYSDATE_NONNULL] 
Expecting:
 <-719162>
to be greater than or equal to:
 <1> 

and

Error:  Errors: 
Error:  io.debezium.connector.db2.Db2ReselectColumnsProcessorIT.testNoColumnsReselectedWhenNullAndUnavailableColumnsAreDisabled
Error:    Run 1: Db2ReselectColumnsProcessorIT.beforeEach:32->AbstractReselectProcessorTest.beforeEach:60->createTable:85 » SQL ASNCAP server did not start.
Error:    Run 2: Db2ReselectColumnsProcessorIT.afterEach:41 » SqlSyntaxError DB2 SQL Error: SQLCODE=-204, SQLSTATE=42704, SQLERRMC=DB2INST1.DBZ4321, DRIVER=4.26.14

This makes changes in the Db2 connector for the corresponding commit in
the main Debezium project.  The main thing we have to do is override the
nullsSortLast function in Db2Connection to define how NULLs sort in Db2.
We also have to update the test suite, since the base test suite now
includes additional tests for working with keys that are nullable.
@jpechane jpechane merged commit 6885c33 into debezium:main Feb 29, 2024
2 checks passed
@james-johnston-thumbtack james-johnston-thumbtack deleted the james/DBZ-5071 branch February 29, 2024 20:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants