Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Serialisation of transactions #261

Open
jesper-friis opened this issue May 7, 2022 · 0 comments
Open

Serialisation of transactions #261

jesper-friis opened this issue May 7, 2022 · 0 comments
Labels
enhancement New feature or request

Comments

@jesper-friis
Copy link
Collaborator

jesper-friis commented May 7, 2022

Add support to serialise/deserialise transactions, as implemented in PR #260.

Both the parent_uuid and parent_hash should be included in serialisation of non-root instances belonging to a transaction.

For simplicity, it would be nice if would be able to avoid to also serialise the immutability flag... If the immutability flag is not serialised, we need to add the following heuristics:

  • when loading an instance, it is by default mutable (metadata instances can be immutable)
  • if inst->_parent->parent is NULL when calling dlite_instance_get_parent(), the parent is marked as immutable when retrieved with dlite_instance_get().

Implementation notes:

  • The parent uuid and hash should be considered an implementation detail and not exposed in the datamodel. The drawback is that each storage plugin must specifically add support for transactions.
  • Most python storages relies on the dict representation of instances. To support them, the dict representation should have an option to add the parent uuid and hash when serialising an instance. Likewise should the dict parser correctly interpret these.
  • Consider to base the dict representation on the json parsing/serialisation in C. It would simplify the python code and get a homogeneous behaviour regardless of whether you are using C or Python. Only drawback is that it might be slightly slower for datasets with large arrays - which would have to be converted back and forth from the json representation (which is a string). This should be an own issue, anyway.
  • Make a separate issue for adding needed support to each C plugin. For json, the support should be added directly to dlite-json.c.
  • Special attention should be put on collections and $ref properties. They should remember the hash of the instances they refer to. When you take a snapshot they should check the hash of the instance they refer to against the stored hash. If the referred instance has changed, a snapshot should be made of that and the new hash should be remembered. Implementation-wise this would require:
    • Collection: add (label, has_hash, <hash> triples to the triplestore. The has_hash relation should be a proper IRI defined in the datamodel ontology (see issue Ontologise the relations used in a collection #174).
    • $ref property: the dliteRef data type should be changed from just being a pointer to a DLiteInstance (DLiteInstance *) to be a struct
      struct _DLiteRef {
        DLiteInstance *inst;            /* Pointer to the referred instance. */
        uint8_t hash[DLITE_HASH_SIZE];  /* Hash of the referred instance. */
      } DLiteRef;
    These should be separate issues.
@jesper-friis jesper-friis added the enhancement New feature or request label May 7, 2022
@jesper-friis jesper-friis added question Further information is requested and removed question Further information is requested labels Aug 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant