Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

*: fix several bugs of traffic replay #698

Merged
merged 18 commits into from
Oct 12, 2024
Merged

Conversation

djshow832
Copy link
Collaborator

@djshow832 djshow832 commented Oct 8, 2024

What problem does this PR solve?

Issue Number: ref #642

Problem Summary:

  • Reading data from gzip files fails because the reader is overwritten
  • fd number is not recovered after replay because backend connections are not closed
  • Paring ComPrepareStmt fails because the buffer is reset before fetching the data
  • The connections are not closed immediately because the ComQuit is not checked
  • The progress is wrong at the second replay because the replayedCmds is not reset
  • Prepared statements are not found because unmarshaling session states fails
  • Execution errors in the last 5 seconds may be not flushed
  • The parsed param num in session states is wrong
  • Paring ComExecuteStmt request is wrong
  • sample_replay_time in other_errors is NULL

What is changed and how it works:
Fix all the above bugs

Enhancements:

  • Disconnection errors during executions are summarized in table other_errors
  • Wait for 3 seconds before discarding a command
  • Upgrade golangci-lint version to support go 1.23 and fix some lint problems

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Notable changes

  • Has configuration change
  • Has HTTP API interfaces change
  • Has tiproxyctl change
  • Other user behavior changes

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

Copy link

ti-chi-bot bot commented Oct 8, 2024

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@ti-chi-bot ti-chi-bot bot added size/XL and removed size/L labels Oct 10, 2024
@djshow832 djshow832 marked this pull request as ready for review October 11, 2024 03:27
@djshow832 djshow832 marked this pull request as draft October 11, 2024 03:27
@djshow832 djshow832 marked this pull request as ready for review October 11, 2024 07:02
@codecov-commenter
Copy link

codecov-commenter commented Oct 11, 2024

Codecov Report

Attention: Patch coverage is 31.77083% with 131 lines in your changes missing coverage. Please review.

Please upload report for BASE (main@3b60b7e). Learn more about missing BASE report.

Files with missing lines Patch % Lines
pkg/proxy/net/protocol.go 0.00% 52 Missing ⚠️
pkg/proxy/net/mysql.go 34.24% 43 Missing and 5 partials ⚠️
pkg/sqlreplay/conn/backend_conn.go 28.00% 17 Missing and 1 partial ⚠️
pkg/sqlreplay/conn/conn.go 53.84% 5 Missing and 1 partial ⚠️
pkg/sqlreplay/report/report_db.go 55.55% 3 Missing and 1 partial ⚠️
pkg/sqlreplay/report/report.go 75.00% 1 Missing and 1 partial ⚠️
pkg/sqlreplay/replay/replay.go 75.00% 1 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main     #698   +/-   ##
=======================================
  Coverage        ?   65.85%           
=======================================
  Files           ?      122           
  Lines           ?    11066           
  Branches        ?        0           
=======================================
  Hits            ?     7288           
  Misses          ?     3246           
  Partials        ?      532           
Flag Coverage Δ
unit 65.85% <31.77%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@djshow832
Copy link
Collaborator Author

/retest

@djshow832
Copy link
Collaborator Author

/retest

@purelind
Copy link
Contributor

/test ?

Copy link

ti-chi-bot bot commented Oct 12, 2024

@purelind: The following commands are available to trigger required jobs:

  • /test build
  • /test check
  • /test integration-common-test
  • /test integration-ruby-orm-test
  • /test mysql-connector-test
  • /test unit-test

The following commands are available to trigger optional jobs:

  • /test integration-jdbc-test
  • /test integration-nodejs-test
  • /test integration-python-orm-test
  • /test mysql-test
  • /test sqllogic-test

Use /test all to run the following jobs that were automatically triggered:

  • pingcap/tiproxy/ghpr_build
  • pingcap/tiproxy/ghpr_check
  • pingcap/tiproxy/ghpr_unit_test
  • pingcap/tiproxy/pull_mysql_connector_test

In response to this:

/test ?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@purelind
Copy link
Contributor

/test mysql-connector-test

Copy link

ti-chi-bot bot commented Oct 12, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: xhebox

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added the lgtm label Oct 12, 2024
Copy link

ti-chi-bot bot commented Oct 12, 2024

[LGTM Timeline notifier]

Timeline:

  • 2024-10-12 03:55:41.543863024 +0000 UTC m=+70538.692772854: ☑️ agreed by xhebox.

@ti-chi-bot ti-chi-bot bot added the approved label Oct 12, 2024
@ti-chi-bot ti-chi-bot bot merged commit 31aa2e0 into pingcap:main Oct 12, 2024
5 checks passed
@djshow832 djshow832 deleted the gzip branch October 12, 2024 04:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants