You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Performance/Memory wise, it makes sense to just append the results of generator-experiments to file. Right now it just overwrites the results, so you need to build a growing data structure in your experiments and yield the whole thing every time.
We should still leave in the old option though as not all generator-experiment will involve linearly growing data structures.
To do this properly, we should have a way of accessing the latest element without reading the whole file, so we need to make or find something like an IndexedPickleFile object that maintains both a data and an index file.
The text was updated successfully, but these errors were encountered:
Performance/Memory wise, it makes sense to just append the results of generator-experiments to file. Right now it just overwrites the results, so you need to build a growing data structure in your experiments and yield the whole thing every time.
We should still leave in the old option though as not all generator-experiment will involve linearly growing data structures.
To do this properly, we should have a way of accessing the latest element without reading the whole file, so we need to make or find something like an IndexedPickleFile object that maintains both a data and an index file.
The text was updated successfully, but these errors were encountered: