Add an --initial-sql flag to InitDb.pl; allow configuring the REPLICATION_TYPE for create_test_db.sh
#3197
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mwiencek
force-pushed
the
initdb-initial-sql
branch
19 times, most recently
from
37842e3
to
54b817a
Compare
reosarevok
reviewed
Mar 8, 2024
| is($result, $f1_path, 'scalar file is found by direct path'); | ||
|
|
||
| $result = find_mbdump_file('artist', $dir1); | ||
| is($result, $f1_path, 'scalar file is found in directory'); |
There was a problem hiding this comment.
Is there a reason you prefer lowercase for these labels? I generally start them with uppercase because they look like full sentences to me the way we print them, but obviously this is mega minor anyway :)
There was a problem hiding this comment.
I guess because they read more like fragments than full sentences to me. :) And some of them start with undef or variable names which would be weird to capitalize...
I don't have a particular preference though.
reosarevok
reviewed
Mar 8, 2024
There was a problem hiding this comment.
LGTMBDNT, I'd like @yvanzo to check it too. I had no idea script/setup_development_db.sh even existed!
mwiencek
force-pushed
the
initdb-initial-sql
branch
from
54b817a
to
0db1479
Compare
yvanzo
reviewed
Mar 11, 2024
This merges the separate implementations of `find_file` contained in admin/MBImport.pl and admin/replication/ImportReplicationChanges into `Script::Utils`, and adds tests for said function. The main difference between the two was that MBImport.pl's implementation would return an array of matching files, rather than the first match. We now use Perl's `wantarray` built-in to distinguish between the two cases.
In case the search paths overlap (e.g., we may find the file via a direct path, and then again via a parent directory).
It works the same, except it doesn't check `mbdump/` subdirectories nor does it support returning a scalar result. `find_mbdump_file` calls it internally.
Support specifying a path prefix in the first `$file` paremeter: e.g., `caa/CreateTables.sql` in the added test case includes a `caa/` path prefix, but will now be found in the search path `/tmp/foo/sql/caa/CreateTables.sql`.
mwiencek
force-pushed
the
initdb-initial-sql
branch
from
0db1479
to
d7c4766
Compare
Allows specifying which SQL script to initialize a clean database with. Prior to this commit, admin/sql/InsertDefaultRows.sql was always used (and this could not be disabled!). One current use case of this is that we can initialize test databases with t/sql/initial.sql instead; and in that file, we no longer have to worry about disabling triggers (because the `initial-sql` script is run before triggers are added) or specifying `ON CONFLICT` handlers (which were to avoid conflicts with existing data that was in admin/sql/InsertDefaultRows.sql).
With the `--initial-sql` flag to InitDb.pl, we only have to drop the existing schemas if the database already exists (and otherwise create the pgtap extension). Thus, this commit delegates the majority of the work to InitDb.pl. One thing to be aware of is that the scripts run by InitDb.pl depend on the configured `REPLICATION_TYPE`. So if we want replication triggers, we have to set `REPLICATION_TYPE=1`,
`ON_ERROR_STOP` must be set in order for psql to return a non-zero exit code on statement failures. This is not much of an issue now that `IF NOT EXISTS` was added, but I thought I'd fix it anyway given that the intent of the code is to capture errors.
I've never once used this, and its purpose seems to have been overtaken by the sample database dumps.
mwiencek
force-pushed
the
initdb-initial-sql
branch
from
d7c4766
to
1092925
Compare
Mainly the commands I listed in the PR description: REPLICATION_TYPE=1 ./script/create_test_db.sh # master
REPLICATION_TYPE=2 ./script/create_test_db.sh # mirror
REPLICATION_TYPE=3 ./script/create_test_db.sh # standaloneand making sure the created databases have the appropriate customizations (e.g., mirrors should be without foreign keys/triggers, masters should have replication triggers). |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is based on top of #3194
Problem
This resolves two problems:
I'd like to fix these issues in order to improve the test suites that make use of replication packets (and thus require separate master/mirror test databases).
Solution
With this PR, you can set the
REPLICATION_TYPEenvironment variable before running./script/create_test_db.sh:See associated commits for details.
Testing
Just manual invocations of the create_test_db.sh and InitDb.pl commands, plus automated tests.