Foundation of new syn2mas tool #3636
Conversation
| /// If you want to use this tool anyway, please pass this argument. | ||
| /// | ||
| /// If you want to migrate from Synapse to MAS today, please use the Node.js-based tool in the MAS repository. | ||
| #[clap(long = "i-swear-i-am-just-testing-in-a-staging-environment")] |
| const NUM_WRITER_CONNECTIONS: usize = 8; | ||
|
|
||
| impl Options { | ||
| pub async fn run(self, figment: &Figment) -> anyhow::Result<ExitCode> { |
There was a problem hiding this comment.
Generally, I would appreciate tracing spans on most operations, which would help a lot diagnose what takes time, but that can be introduced later
There was a problem hiding this comment.
yeah I still need to figure out tracing (in the sense of spans) to be honest. Can probably just add #[instrument]s without too much hassle though...
There was a problem hiding this comment.
we can always add more infos or better names on them later, but this is fine 👍
Deploying matrix-authentication-service-docs with
|
| Latest commit: |
e7172d6
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://765d99cc.matrix-authentication-service-docs.pages.dev |
| Branch Preview URL: | https://rei-syn2mas-pr1.matrix-authentication-service-docs.pages.dev |
reivilibre
force-pushed
the
rei/syn2mas_pr1
branch
from
fb1d512 to
64f5c65
Compare
| const NUM_WRITER_CONNECTIONS: usize = 8; | ||
|
|
||
| impl Options { | ||
| pub async fn run(self, figment: &Figment) -> anyhow::Result<ExitCode> { |
There was a problem hiding this comment.
we can always add more infos or better names on them later, but this is fine 👍
crates/syn2mas/src/mas_writer.rs
Outdated
| connection_rx: Receiver<Result<PgConnection, Error>>, | ||
|
|
||
| /// A sender handle to return a writer connection to the pool | ||
| /// The connection should still be mid-transaction! | ||
| connection_tx: Sender<Result<PgConnection, Error>>, |
There was a problem hiding this comment.
You're right. The transaction either olds a ref to a connection or a connection from a pool. Weird that there is no good way to have an owned connection in there
There was a problem hiding this comment.
This looks less scary after looking at it again
This is the start of a new version of the syn2mas tool.
Given it's missing a lot of stuff still, I'm intending to merge this to an intermediary branch where I can then build upon this with the missing features.
Notable inclusions
Notable missing features (for later PRs)
Performance
This currently runs a synthetic matrix.org-size migration under 15 seconds. (Bear in mind this is only one part of the migration, but it gives a rough idea of where we're going with this.)
The synthetic database was set up with the same number of real users as matrix.org, but guest and appservice users were not populated in the database so in reality this may run slower because the Synapse Postgres instance will need to read and skip rows corresponding to guests and appservice users.