Support entities

[florianm]

Feature

Support all API endpoints relevant to entities.

Test data

• Use the "report a problem" and "address a problem" examples from the entities tutorial
• Add these two forms as new test forms: use form IDs as defaults, provide env var as overrides
• Submit some test data
• Add gate check ru_msg_abort if the given ODKC_VERSION does not support the respective function

https://docs.getodk.org/central-api-dataset-management/

• https://docs.getodk.org/central-api-dataset-management/#datasets as ruODK::entitylist_list()
• https://docs.getodk.org/central-api-dataset-management/#dataset-metadata as ruODK::entitylist_detail()
• https://docs.getodk.org/central-api-dataset-management/#update-dataset-metadata as ruODK::entitylist_update()
• https://docs.getodk.org/central-api-dataset-management/#download-dataset as ruODK::entitylist_download()

https://docs.getodk.org/central-api-entity-management/

• https://docs.getodk.org/central-api-entity-management/#entities-metadata as ruODK::entity_list()
• https://docs.getodk.org/central-api-entity-management/#creating-an-entity as ruODK::entity_create()
• https://docs.getodk.org/central-api-entity-management/#getting-entity-details as ruODK::entity_detail()
• https://docs.getodk.org/central-api-entity-management/#deleting-an-entity as ruODK::entity_delete()
• https://docs.getodk.org/central-api-entity-management/#updating-an-entity as ruODK::entity_update()
• https://docs.getodk.org/central-api-entity-management/#listing-versions as ruODK::entity_versions()
• https://docs.getodk.org/central-api-entity-management/#getting-changes-between-versions as ruODK::entity_changes()
• https://docs.getodk.org/central-api-entity-management/#entity-audit-log as ruODK::entity_audit()

OData endpoints

• service https://docs.getodk.org/central-api-odata-endpoints/#odata-dataset-service as odata_entitylist_service_get
• Metadata Document /v1/projects/{projectId}/datasets/{name}.svc/$metadata as odata_entitylist_metadata_get
• (link to this from entity_list function help) Data Document /v1/projects/{projectId}/datasets/{name}.svc/Entities as odata_entitylist_data_get

Small jobs

• Adjust Rd documentation: structure with common subheadings, factor out shared text parts. See entity_detail.
• Extend yell_if_missing to include optionally did (entity list / dataset ID) and eid (entity ID). Update all functions using did or eid.
• Capitalise Dataset, Entity List, Entity in singular and plural forms.

Discussion points

• Return format: whatever makes most sense for each endpoint (current implementation but open to suggestions), or always a tibble (near impossible to create predictably without list columns / missing columns), or a custom list of lists (invent an S3 class)?

Approach: return a tibble if possible, keep unstructured = unpredictable data in list columns

[roammi]

thank you @florianm for all the great work for this, I have an use case and I will be looking forward for it to try it out

[florianm]

Oh that's great to hear, @roammi! I'm carving out some time for ruODK, so things are moving forward again.

Do you feel your use case (if appropriate, simplified and de-identified) could be used in a vignette as an example to other ruODK users?

[florianm]

Entity Lists are now supported, Entities coming soon.

[roammi]

Oh that's great to hear, @roammi! I'm carving out some time for ruODK, so things are moving forward again.

Do you feel your use case (if appropriate, simplified and de-identified) could be used in a vignette as an example to other ruODK users?

Yes I do, We will need to put it to work first, but it sounds great, I will be using the update entity function, I'll wait for the new release of ruodk to try it out, or is it a way to try [dataset_update] earlier?

[florianm]

@roammi you can try out entitylist_update right now if you install the latest version of ruODK (see Readme) and update your ODKC_VERSION (see NEWS).

In consultation with the core team ruODK uses the term entitylist for datasets.

Are you saying that you would like to use https://docs.getodk.org/central-api-entity-management/#updating-an-entity ? I will work on that one next then so you can try it out right away.

[roammi]

@roammi you can try out entitylist_update right now if you install the latest version of ruODK (see Readme) and update your ODKC_VERSION (see NEWS).

In consultation with the core team ruODK uses the term entitylist for datasets.

Are you saying that you would like to use https://docs.getodk.org/central-api-entity-management/#updating-an-entity ? I will work on that one next then so you can try it out right away.

thank you @florianm I have the latest version and it worked succesfully! thank you for the hard work! I've used ruODK::entitylist_list(), ruODK::entitylist_detail() ruODK::entitylist_download()

I also used bruno API and test it create, delete and update an entity succesfully so will be great to test it with ruODK. The challenge for update is the baseVersion, you have check the version and increase it by 1 each time making the update, it will be really great to do that natively in ruODK but it could be handled outside in R too

[lognaturel]

The challenge for update is the baseVersion, you have check the version and increase it by 1 each time making the update, it will be really great to do that natively in ruODK but it could be handled outside in R too

You also have the option of using the force query parameter as described in https://docs.getodk.org/central-api-entity-management/#updating-an-entity

If your updates are bringing in values from an external source of truth, it generally makes sense to force the update.

baseVersion exists to detect parallel updates from multiple clients which is especially likely in an offline context. If you’re running a script that is online, it should also generally be appropriate to use the force flag. If your updates are based on the prior version of an entity, you should be able to pull the entity and then update it almost immediately meaning you don’t have to worry about conflicting updates between the two actions unless your context is extremely dynamic.

If you intend to use baseVersion, you shouldn’t have to do any incrementing, you should be able to take the existing value for __version of the entity you want to update and use that as baseVersion for your update. If that’s what you plan on doing, I’d be interested in learning more about what you’re doing with the API and why forcing the update wouldn’t be appropriate for you.

(I’m an ODK maintainer. We can also take this to the ODK forum, sorry about going a bit off topic, @florianm!)

[florianm]

As always, the insight is highly appreciated, @lognaturel!

I'll have to digest the above a bit. Maybe ruODK could default to use the force=true flag.

[florianm]

https://github.com/ropensci/ruODK/releases/tag/v1.5.0 is out, pending support for the OData endpoints (after getting the naming right - see https://forum.getodk.org/t/ruodk-support-for-odata-entity-endpoints-naming/50037).