Skip to content

Commit

Permalink
Generate random run ID, track restarts
Browse files Browse the repository at this point in the history
Improve data provenance tracking, so that sets of runs can be linked
to each other, and the run used to generate data identified.

Each time the solver is run, generate a random number which is
broadcast to all processors. This is stored as "run_id" in:
- log files
- BOUT.settings, though at the moment only if the run finishes
- restart files
- dump files

When a simulation is restarted, the ID of the run it started from is
also recorded as "run_restart_from".

If restart files are used which don't have a run_id, then this will
either cause the run to fail, or set run_id to 0. The
`run_restart_from` ID therefore has two special values:
- 1 means no restart, the run was started from scratch
- 0 means restart from unknown run_id (missing -> run_id set to 0)

There may well be a better way to handle missing run_id values in
restart files.
  • Loading branch information
bendudson committed Nov 20, 2020
1 parent a332685 commit 56873bd
Show file tree
Hide file tree
Showing 2 changed files with 30 additions and 0 deletions.
6 changes: 6 additions & 0 deletions include/bout/solver.hxx
Original file line number Diff line number Diff line change
Expand Up @@ -417,6 +417,12 @@ protected:
/// Current iteration (output time-step) number
int iteration{0};

/// Randomly generated run ID
int run_id {1}; // 0 = unknown restart, 1 = not restarted
/// The run from which this was restarted.
/// Set to zero if no restart
int run_restart_from {1};

/// Run the user's RHS function
int run_rhs(BoutReal t);
/// Calculate only the convective parts
Expand Down
24 changes: 24 additions & 0 deletions src/solver/solver.cxx
Original file line number Diff line number Diff line change
Expand Up @@ -468,8 +468,28 @@ int Solver::solve(int NOUT, BoutReal TIMESTEP) {
throw BoutException(_("Failed to initialise solver-> Aborting\n"));
}

// Set the run ID to a random number
run_restart_from = run_id; // Restarting from the previous run ID

if (MYPE == 0) {
srand(static_cast<unsigned int>(time(nullptr)));
do {
run_id = rand(); // Different each time the simulation is run
} while((run_id == 0) || (run_id == 1)); // Ensure that run_id != 0 or 1
}
MPI_Bcast(&run_id, 1, MPI_INT, 0, BoutComm::get()); // All ranks have same run_id

// Put the run ID into the options tree
// Forcing in case the value has been previously set
Options::root()["run"]["run_id"].force(run_id, "Solver");
Options::root()["run"]["run_restart_from"].force(run_restart_from, "Solver");

/// Run the solver
output_info.write(_("Running simulation\n\n"));
output_info.write("Run ID: {:d}\n", run_id);
if (run_restart_from != 0) {
output_info.write("Restarting from ID: {:d}\n", run_restart_from);
}

time_t start_time = time(nullptr);
output_progress.write(_("\nRun started at : {:s}\n"), toString(start_time));
Expand Down Expand Up @@ -558,6 +578,10 @@ void Solver::outputVars(Datafile &outputfile, bool save_repeat) {
outputfile.addOnce(simtime, "tt");
outputfile.addOnce(iteration, "hist_hi");

// Add run information
outputfile.addOnce(run_id, "run_id");
outputfile.addOnce(run_restart_from, "run_restart_from");

// Add 2D and 3D evolving fields to output file
for(const auto& f : f2d) {
// Add to dump file (appending)
Expand Down

0 comments on commit 56873bd

Please sign in to comment.