v2.38 "Välimerenkatu"

This release contains major changes to the REMS internal architecture regarding the caching of information. We have improved the caching behavior previously during the year, but now we drop the requirement for REMS to reload the application cache every hour or so. This was problematic, if the reload was slow, because it would stop REMS for a while and could lead to running out of memory. REMS got slower with more data, handlers and so on which made the problem worse.

The fundamental idea is that most information in REMS is cached, and can be served from the cache. We have tried to make this the fastest REMS release by going through all the previously sluggish actions. While this does not change the UI, the API or functionality, it is large enough to warrant this note. The release passes all our tests and has also been manually tested. Before deploying this release, consider doing some additional manual testing for your use cases.

About the REMS performance and test data. We have tested REMS performance using 1000+ catalogue items, 3000+ applications with 20+ largeish events each. The workflow contains 100 handlers each. We also test the forms with 200 fields. With these numbers, REMS performance is fine with any modern server with around 4 GB of memory.

Additions

• Catalogue tree now supports keyboard interactions for ARIA role="tree". Navigation works with arrow keys and Home/End, tree nodes can be opened/closed with Enter key, and default action (like add to cart) works with Enter key. (#3336)

Changes

• The logging of the simultaneous request count (rqc:) is now done for the request and response log lines only, and separately. Previously the same value was used for all logging in a request, which could be confusing as the actual request count can change between the lines.
• The performance has been improved when there are many handlers. (#3283)
• The error message "shake" animation has been removed. (#3298)
• The collapsible component no longer animates on open/close.
• If REMS is trying to send email, and if the address has an obvious problem, REMS will not try sending it again, but gives up immediately. This should avoid having many hopeless retries, if there is a typo in an email address.
• Form editor performance has been significantly improved. To give rough numbers, the editor now works smoothly with 200 form fields in test data. (#3105)
• The wording about experimental is removed from GA4GH API. It is already used in production. (#3299)
• Licenses are ordered consistently (alphabetically) wherever they appear. (#3302)

Fixes

• Empty the license attachment field if the upload fails. (#3292)
• Hooks set in extra scripts were overridden by initialization. The new place for them is rems_hooks. See docs/hooks.md for details. (#3351)

v2.37 "Laivapojankatu"

Additions

• The "Change Form" function in administration has been expanded to be able to change both forms and workflows. From now on it is called "Update catalogue item". The old change-form API still exists but has been deprecated. (#3271)

Changes

• Invitation links work multiple times. Previously, they were single-use. The invitation token is spent on the first use and it grants the rights then. However, the user may still use the link later for navigating into the application or workflow in question. (#3275)
• Invitation gives a more informative error when handling user tries to join application. (#3291)

v2.36 "Laivapojankuja"

Additions

• (Experimental) Voting options are expanded to have types, i.e. reviewer and handler voting. Use :enable-voting. (#3257)
• Email templates (:t.email/...) and event translations (:t.applications.events/...) now support named format parameters, and default english localizations have been updated for reference. While REMS moves towards fully supporting named parameters, all translations with index parameters should continue to work. Localizations with many parameters (like emails) have been burdensome to maintain with index parameters, so transitioning into named parameters is highly encouraged. (#3241)
• Processing states have been added. Now, in addition to the main states (draft, submitted, returned, approved, ...), the application tracks new "processing states" (i.e, "sub-states"). They can be added to facilitate for example standard bookkeeping of an application or technical integrations. The handler can change an application's processing state, when at least one has been configured in the application's workflow by the owner. Use :enable-processing-states. (#3259)

Changes

• Form fields in workflow admin pages are re-structured to show most often needed fields first.

v2.35.1 "Selkämerenkatu +1"

Fixes

• Fix issue with user roles after deletion. This was introduced in the previous release that optimizes cache updates. (#3243)
• Mark form as optional in catalogue item creation. Also consider categories optional unless the catalogue tree is enabled. (#3244)

v2.35 "Selkämerenkatu"

NB: This release removes the experimental application PDF export API. The non-experimental PDF export API is preferred instead. (#3098)

Additions

• By default administration pages show only "own organization" items. (#2046)
• The command-line use of REMS has been sped up. There are new commands dev-setup and perf-setup that dev_db.sh uses to start faster (a combination of migrate and data).
• Application expiration can be configured with :application-expiration-process-limit to process a subset of applications instead of everything at once. (#3225)
• Show organization in create workflow Forms dropdown (#3230)
• When copying an item, reset the organization, if it's not owned by the user. (#2880)
• Added malware scanning support. When :malware-scanner-path is set REMS will scan uploaded attachments and licenses with the executable at the provided path. Enabling :enable-malware-scanner-logging will pass the scanner executables output on STDERR to the REMS application log. (#2905)

Changes

• Application expiration now logs more, and more often. (#3225)
• Catalogue page has been sped up with optimizations. The API supports not joining organization data if not required (join-organization=false).
• Adding a user to blacklist does not reload the full cache anymore. This should make it faster. Also the user is directed to Applications page after delete, not to Catalogue.
• Deleting a (draft) application is now faster because it does not reload the full cache, only update it.
• Editing workflows (e.g. handlers) should now be faster because it does not reload the full cache, only update it.

Fixes

• Big improvements in performance from small improvements in how applications and events are processed and cached. REMS should be able to handle a 100k event DB without breaking a sweat. (#2783)
• Administration dropdown buttons should now respond to clicks more widely, and not only by directly clicking text. (#3167)
• Catalogue item unarchive should no longer fail when form does not exist. (#3217)
• Current page updates correctly. (#3218)
• Fixed faulty check in application expiration that prevents sending expiration notifications for applications, and expiring those applications. (#3225)

v2.34.2 "Santakatu +2"

Fixes

• DUO codes in draft saved event no longer cause schema validation error when :enable-duo config is false.

v2.34.1 "Santakatu +1"

NB: This release contains migrations!

Fixes

• A bug in event selection prevented migration (introduced in v2.34) from applying properly. This release contains a fixed version of that migration.

v2.34 "Santakatu"

NB: This release contains migrations!
NB: :application/public is renamed to :event/public (in application_event table eventdata column).
NB: :expires-on is renamed to :application/expires-on (in application_event table eventdata column).
NB: :last-activity is redundant and removed from the event (in application_event table eventdata column).

Changes

• "Show related events" has been removed from events. (#3156)
• Handler can now redact other handlers attachments. (#3190)
• The tables take advantage of a new paging feature. Especially, the handler Actions page and its handled applications table uses paging and only fetches handled applications when so requested. The first 50 rows are only fetched by default. Paging can be also configured per table, if the default is not good (page size 50 rows). (#3191)
• If reminder email is configured for the, expirer bot, it will delete an old draft application if and only if the reminder email is sent and the specified amount of time has passed. Previously it would just delete if the application was very old (esp. when enabling expiry for the first time).
• Allow leaving :form field away from catalogue item creation API calls. Previously the value was optional but now the key too.
• Application action buttons now wait until request has completed. This should prevent duplicate concurrent requests caused by repeated clicking. (#3204)

Additions

• (Experimental) Workflow can be configured to enable voting for the approval. Currently all handlers can vote (including bots). Use :enable-voting. (#3174)
• There is now a Danish language translation (#3176). We are considering supporting a limited set of languages officially, and improving support for community maintained translations (see #3179).
• Added experimental support for named format parameters in translations. (#3183)
• Added extended logging option (:enable-extended-logging), for additionally logging the content of mutating operations. (#3184)
• Cache reloading can be configured using the new :buzy-hours config. REMS will then try to avoid reloading during the specified time spans. (#3194)
• Application expiration respects :buzy hours config too.
• The handling users can now see whether an event is shown to the applicant from a small eye icon in the event history. (#3156)
• Event now shows which attachments were redacted when viewing as handling user. (#3190)
• Workflow has new option to anonymize handling users. When enabled, applying users may only see "Handler" in events where handling user's name would appear. Event emails to applying users similarly show only anonymized name where handling user's name would appear. Application API also returns only anonymized name to applying users. Anonymized name is customizable with translation key :t.roles/anonymous-handler, which defaults to :t.roles/handler.

Fixes

• Email template parameters for :application-expiration-notification event are now documented. The parameters are different from standard event email parameters, which may have caused confusion.

v2.33 "Santakuja"

Additions

• PDF output can be customized a little. This enables changing the font, which should fix missing diacritics. (#3158)
• Workflows can now disable commands dynamically by application state, user role, or both. Rules can be set in workflow administration page under disable commands field. (#3131)
• Plugins can be written to extend REMS functionality in select extension points. So far there is support for plugins written in Clojure embedded in Markdown files. See docs/plugins.md for details. (#3133)

Fixes

• Label and header fields have URLs made into links. (#3155)

v2.32 "Itämerenkatu"

NB: GET /entitlements.csv is deprecated and will be removed in a future release. Please use GET /api/entitlements/export-csv instead.
NB: Experimental application PDF export API is deprecated, and will be removed in a future release. The non-experimental PDF export API is preferred instead. (#3098)

Changes

• Application editing performance is improved. (#3106)
• Show form errors column only when some form has errors. (#3107)
• Latest event is now shown in the minimized application state. (#3119)
• Administration menu has been fine-tuned. Reports also shows only if
  the user can access it.
• The administration pages can now scale wider than before.
• The administration tables use a dropdown button for the modification actions.
  This allows more space for the table contents as there will be less buttons visible.
  Some of the buttons are only visible if the user has the permission. (#2814)
• The catalogue items page (in administration) is now sorted by the item creation time by default.
  Newly created items will now appear at the top instead of somewhere in the alphabetical order.
  The sort can still be changed by clicking on the column headers.
• The catalogue page margins have been adjusted to be more consistent. Also some of the default
  texts have been removed. They can still be added by extra translations.
• Catalogue "more info" is now styled as a link instead of a button.
• Actions page columns have been tuned, e.g. description, resource and applicant, so that they wrap nicely. (#3121, #3122)
• Accessibility tooling has been upgraded, the report improved, and serious issues fixed.
• Prefer and set language cookie so a db request is spared. (#3136)
• Logging has been adjusted:
  • Don't add static resources to audit log. (#3136)
  • Provide entry logging earlier (req >), even before user or roles are fetched.
  • Log a short request id for every request.
  • Log the simultaneous request count (rqc:).
  • Reduce duplicate logging from each line (user, request method).
  • Print the time it took to serve the request (ms).

Additions

• User attributes can now be retrieved from ID token and user_info endpoint. (#3028)
• User attributes are now formatted better (a boolean becomes a checkbox, etc.) (#3103)
• Most handler, reviewer, decider emails can be disabled with :enable-handler-emails false. (#3116)
  This does not include the invitation emails, which will always be sent, to be able to invite users.
  This also does not include any reminder emails that you can separately enable if you wish.
• Any individual email can be disabled by setting its translation to empty string "" (#3117).
• Handler, reviewer and decider can now redact application attachments. (#2491)
  • Redacted attachment exists in REMS like any other attachment, but it's file contents are permanently removed, and filename is shown to other non-handler users as redacted.
  • Attachments in field answers cannot be redacted, but the applicant can edit them as usual when application is draft or returned.
  • Handler can redact attachments also from other users.
  • Non-handler users can redact only their own attachments.
  • In decider workflow handler is not allowed to redact decider attachments.
• Database connection pool can be configured, see :hikaricp-extra-params in config-defaults.edn.
• The debug log level prints details of when the scheduled pollers run, e.g. email and event notification.
• Search bar automatically trims the whitespace form the left side, which is useful for copied values.
• The event notification can be configured to not send the application data with :send-application? false

Fixes

• Autosaving does not cause the focus to jump anymore (#3112)
• Fix form table default sort order to be internal name (was title). (#3107)
• Editing form of a catalogue item could fail due to categories.
• While clicking a button, link or menu item, a wrong color was sometimes displayed.
• Heading focus outline uses the correct blue color.