Way to set a fixed number of training runs for reinforcement learning

[AsgerHB]

By default when using reinforcement learning to generate a strategy using minE or maxE queries, UPPAAL automatically chooses a number of traces to train on. On a very simple example, it seems to default to 9000 runs.

However, for experiments it can be beneficial to fix the number of training episodes while changing other variables. How can this be done? I have previously been trying something like these learning parameters to train for 10 traces only:

But this (seemingly) results in twice that amount of episodes being trained for.

Is the number in the result box correct, and should I put in half as many training episodes as I want in the boxes? Or is the reporting misleading somehow, and it actually trained for the 10 traces I wanted?

[thorulf4]

Unfortunately we don't have good documentation on these parameters internally, and I think we need help from @petergjoel to completely understand how your desired results.

[petergjoel]

Most of the parameters have a direct equivalent in Fig. 2 of On Time with Minimal Expected Cost!.

For your particular case, you need to limit to one iteration, as per your example.

1. limit the number of iterations to 1,
2. set Number of successful runs, Maximum number of runs and Number of good runs to the desired number of iterations, and
3. set evaluation runs to 1 (the evaluation is only used to discriminate between iterations).

This assume that your model ALWAYS reaches the trace-termination condition.
If not, then you will have to adjust the first three parameters to accommodate for the conditional learning.

[AsgerHB]

Setting number of runs to evaluate to 1 changes the number of runs reported from 20 to 11. The UI doesn't allow setting it to 0, but that hardly matters. This will save a good deal of time when running learning queries in the UI. (I see that when calling veriftya through CLI, I use the default 100 evaluation runs which is not so much.)