Add custom endpoints to extensions

[tschicke]

Thanks for merging #1600. This is a small WIP addon to that, that adds the ability for extensions to register their own endpoints that can be called at '/beancount/extension/<ext_name>/', so that the template html or the new js module can communicate with the extension backend, which I think opens up the possibility for much more advanced extensions. This is a pretty bare-bones version of the change that doesn't handle arguments particularly well, and might have issues with locking (I'm not sure how locking around requests works), but it has been working well for me in a few personal extensions that I've written in the past few weeks.

I'm not sure if this type of thing is something you'd want to support, so I wanted to see if this was interesting at all before spending too much time cleaning it up.

[yagebu]

I'm not sure if this type of thing is something you'd want to support

Sure, let's add this if you have a use case for it.

[yagebu]

This fixes #1530

[tschicke]

This fixes #1530

Oh thanks, I didn't notice there was already an issue for this.

[tschicke]

I made the extension_endpoint decorator the main way for extensions to register endpoints. This can be used either without arguments (endpoint name = function name, methods = ["GET"]):

@extension_endpoint
    def update_status(self):
        ...

Or with arguments (endpoint name, methods):

@extension_endpoint("update", methods=["POST"])
    def update_endpoint(self):
        ...

I also created an ExtensionApi class in the frontend. This gets set on each extension module, and had get, put, post, and delete functions to call endpoints within the extension.

The last thing might be overkill, but it looks through all elements with class name "extension-handler" and goes through each "data-handler-" attribute, converting the value to a handler for that event that has access to the extension module. This normally wouldn't be necessary, and you could just set an id on the required elements and init the event handlers in onExtensionPageLoad, but this makes it easier when the html template creates dynamic elements, like in a loop, where the handler for each element needs the loop variable, e.g. from one of my extensions:

{% for key, value in extension.status()|dictsort %}
    <tr>
      <td>{{ key }}</td>
      <td><pre>{{ value }}</pre></td>
      <td><button class="extension-handler" data-handler-click="this.api.post('disable', {name:'{{ key }}'})">Disable</button></td>
      <td><button class="extension-handler" data-handler-click="this.api.post('enable', {name:'{{ key }}'})">Enable</button></td>
    </tr>
{% endfor %}

Without something like the "data-handler" thing, it's more difficult to do something like {name:'{{ key }}'} and you need to do something like store the key somewhere and refer back to it later. I think it cleans some stuff up, but I understand if it's a little too hacky (like the need for // eslint-disable-next-line @typescript-eslint/no-implied-eval)

I still need up update the documentation and add an example of the functionality to an included extension, but I wanted to get your opinion on this direction before doing that.

[yagebu]

The last thing might be overkill, but it looks through all elements with class name "extension-handler" and goes through each "data-handler-" attribute, converting the value to a handler for that event that has access to the extension module.

I'm not a fan of that - I generally dislike inline event handlers. I also think the frontend code should be mostly agnostic of how the extension does frontend logic (unless it's reusing some existing Fava logic so I could imagine having some more direct access to the Svelte components or custom stuff like the router) - it could do so using a full-fledged frontend-rendering framework like React or Svelte or for ergonomics closer to HTML with inline handlers something like htmx or Stimulus.

[tschicke]

I'm not a fan of that - I generally dislike inline event handlers. I also think the frontend code should be mostly agnostic of how the extension does frontend logic (unless it's reusing some existing Fava logic so I could imagine having some more direct access to the Svelte components or custom stuff like the router) - it could do so using a full-fledged frontend-rendering framework like React or Svelte or for ergonomics closer to HTML with inline handlers something like htmx or Stimulus.

No problem, I figured it was probably too much of a hack. I can achieve most of that functionality in an extension that uses onPageLoad, since like I mentioned some sort of ability to specify things inline makes working with loops in extension html much easier for now.
I think making something like React or Svelte accessible to the extension frontend would be the ideal way to solve this, but Svelte would require shipping the Svelte compiler with fava, and invoking it at runtime, which seemed like too much, and adding a runtime framework like React or one of the others seemed confusing since the core frontend already uses Svelte.

[andreasgerstmayr]

I think making something like React or Svelte accessible to the extension frontend would be the ideal way to solve this, but Svelte would require shipping the Svelte compiler with fava, and invoking it at runtime, which seemed like too much, and adding a runtime framework like React or one of the others seemed confusing since the core frontend already uses Svelte.

Tangentially related: I once did a PoC of loading React in an extension: https://github.com/andreasgerstmayr/fava-dashboards/blob/react/frontend/src/app.tsx. Registering custom routes would be awesome, as currently it's writing JSON in a <script> tag and loading that to bootstrap the React app.

[tschicke]

I can achieve most of that functionality in an extension that uses onPageLoad
After playing around with this, the one extra thing I need in order to do this in an extension rather than fava is an ability to get other extensions from within an extension. I added the getExt function to the context so that any extension can call that to query for an instance of another extension if they need to.

[yagebu]

I think making something like React or Svelte accessible to the extension frontend would be the ideal way to solve this, but Svelte would require shipping the Svelte compiler with fava, and invoking it at runtime, which seemed like too much, and adding a runtime framework like React or one of the others seemed confusing since the core frontend already uses Svelte.

There's no need for invoking the Svelte compiler at runtime - if you want to build Svelte components, your extension should have a build process and if you want to use Fava's components, we can expose those to extensions. It's really up to the extension to pick a frontend framework and use it.

[tschicke]

I think making something like React or Svelte accessible to the extension frontend would be the ideal way to solve this, but Svelte would require shipping the Svelte compiler with fava, and invoking it at runtime, which seemed like too much, and adding a runtime framework like React or one of the others seemed confusing since the core frontend already uses Svelte.

There's no need for invoking the Svelte compiler at runtime - if you want to build Svelte components, your extension should have a build process and if you want to use Fava's components, we can expose those to extensions. It's really up to the extension to pick a frontend framework and use it.

Thanks, yeah I realized that was the intended way to do that after seeing how https://github.com/redstreet/fava_investor works. I have 5+ local extensions that I'm using/testing with so adding svelte/react to a bunch of different extensions would add a bunch of boilerplate, so I was trying to come up with a way that fava could do some of that work for me. But I agree that the flexibility of letting extensions use whatever frontend they want is better.