Memory allocation with every compiled template (to support l10n of templates) #493

wdolek · 2022-01-31T18:06:11Z

wdolek
Jan 31, 2022

We are using Handlebars for rendering localized templates. Having naive implementation of compiling each localized template separately we are getting to ~300MiB of compiled templates in memory.

Template compiled N times, each time with different "static" text:

<html>
<body>
  <p>Hello {{name}}</p>
</body>
</html>

We have decided to fix this by compiling template once and just vary localized text in it. Unfortunately even that localized text may contain Handlebars syntax, so we thought we can:

compile template itself
compile dictionary values separately, having "helper" which will inject it accordingly

... but this doesn't scale, because no matter how small compiled template is, it has over 30KiB.

We tried adding those dictionary entries as partial templates using Handlebars.RegisterTemplate - but there was no difference. When checked implementation it was obvious why - using partial template is the same as compiling it, but just registering it inside Handlebars Environment.

hb.RegisterTemplate("tpl1-dict-greetings-en", "Hello {{name}}");
hb.RegisterTemplate("tpl1-dict-greetings-fr", "Bonjour {{name}}");
// ...

<html>
<body>
  <p>{{> (append 'tpl1-dict-greetings-' culture) }}</p>
</body>
</html>

To the main point, we can see that every time we compile template, there's HandlebarsConfigurationAdapter configuration associated to it. This object takes ~25KiB and for every compiled template it is new instance (pointing to different place in memory) of seemingly same configuration object. Now, having several thousand of small templates we need to compile we are again getting to tens or hundreds of MiBs allocated.

Is there any way how to achieve:

localized templates, where translations may use Handlebars as well?
minimize memory footprint, optimize presence of HandlebarsConfigurationAdapter in every compiled template?

oformaniuk · 2022-01-31T21:24:08Z

oformaniuk
Jan 31, 2022
Maintainer

Hello @wdolek
Thanks for a detailed initial investigation 👍
Unfortunately, I do not see any way to tackle the issue without making changes. HandlebarsConfigurationAdapter is used to separate modifications done from inside a template to be propagated to the HandlebarsEnvironment and affecting other templates sharing the same environment.

I do see couple possible things that can be done:

Optimize HandlebarsConfigurationAdapter memory allocation. Can possibly be done by switching from Observable* collections to Cascade* versions. This should reduce memory footprint of each individual instance of HandlebarsConfigurationAdapter. In this case need to monitor what impact it may have on overall performance.
Provide a mechanism to provide another implementation and/or instance of ICompiledHandlebarsConfiguration that may inherit existing instance. May work the best for your case as i10n templates are tightly couple to the main template and should share things like helpers, partials, etc.

6 replies

oformaniuk Feb 1, 2022
Maintainer

@wdolek

Point 1

I tried to follow call stack of internal ctor (where it's possible to pass ICompiledHandlebarsConfiguration), but didn't really get anywhere.

Right now, it is in a code path of ViewEngine. The path and may be deprecated in v3 as part of #97.

Point 2

To grasp whole idea behind current implementation, can you elaborate more about:

... used to separate modifications done from inside a template ...

I wasn't even aware of this being possible

Possible scenario:

you use same environment for all templates compiling (assume static methods Handlebars.*)
you've compiled first template
in the second template you want to use some custom feature to provide helper override
Helper override from the feature will only affect templates created after feature is registered but not templates compiled before that

Point 3

but would it be big change to allow user explicitly say "hey, I know what I'm about to do - reuse this everywhere".

This should not be a big change and it corresponds to the second option I've proposed:

Provide a mechanism to provide another implementation and/or instance of ICompiledHandlebarsConfiguration

However, this change may require some new API design (e.g. how to extract and pass existing instance of ICompiledHandlebarsConfiguration between templates, etc.) in non-breaking way (if you want this to be part of v2).
I'd be happy to accept a PR with the change as well as help you with a design.

wdolek Feb 1, 2022
Author

@zjklee I would be more than happy to come with own PR, but I'm really struggling to understand internals of Handlebars.Net. It's my lack of knowledge what Handlebars can actually do. Is there any documentation describing architecture decisions and why things are connected as they are?

I'm probably happy modest user who just want to compile and render, perhaps register helpers but that's it - not expecting more - I'm missing a lot here.

Regarding to your 2nd point/suggestion, if I understand it correctly, I may try to create simple implementation of ICompiledHandlebarsConfiguration which will just use passed configuration directly (or at least as much possible), instead of creating all those observable collections over its values. Then find a way how to pass it to environment so it won't damage public API too much. Is that it?

oformaniuk Feb 2, 2022
Maintainer

@wdolek

I would be more than happy to come with own PR, but I'm really struggling to understand internals of Handlebars.Net.

It took me some time to understand all the internals as well. But project cannot sustain itself without external contributors :)

Is there any documentation describing architecture decisions and why things are connected as they are?

Unfortunately, there's not internal documentation. Most of the design choices were done to accommodate Handlebars features, provide an extension point or improve performance.

Regarding to your 2nd point/suggestion, if I understand it correctly, I may try to create simple implementation of ICompiledHandlebarsConfiguration which will just use passed configuration directly (or at least as much possible), instead of creating all those observable collections over its values. Then find a way how to pass it to environment so it won't damage public API too much. Is that it?

I was thinking about something like the following:

have env creation options:

public class CreationOptions
{
     public bool UseSharedConfiguration { get; set; } = false;
}

define new method

public static IHandlebars Create(CreationOptions options, HandlebarsConfiguration configuration = null) { ... }

create shared ICompiledHandlebarsConfiguration and use internal ctor in case UseSharedConfiguration is true

wdolek Feb 4, 2022
Author

@zjklee spend some time trying to figure out how to reproduce original and intended behavior. I still don't understand process you described. After all, once I call Compile, I don't expect anything to change on rendering if there's feature (or anything else) added between compilation and render - no matter of implementation. (Meaning, even if I shared configuration object, it still shouldn't affect compiled expression, should it?)

I tried to instantiate HandlebarsConfigurationAdapter once and pass it in, but then any change trough HandlebarsConfiguration (which is exposed) is not propagated to ICompiledHandlebarsConfiguration, leaving me with either null-ref exceptions, missing helpers or other errors.

Unfortunately, HandlebarsConfiguration.CompileTimeConfiguration isn't using observable collections, so whatever I add here after I instantiate Handlebars, I don't see a way how to propagate that change further to ICompiledHandlebarsConfiguration - without breaking change (changing IList to observable ones for example).

My other idea was to keep instance of ICompiledHandlebarsConfiguration in environment until something changes in HandlebarsConfiguration, but again, I don't see how to flag change easily when collections are accessed directly. Only option I can think of is to create check sum of everything and then compare those values in order to keep instance or create new. But that's additional load on CPU 🤷 - would it be feasible? Is that idea even correct? Actually, a) can't do checksum cheaply, b) won't help in my case - I am creating tons of new templates

Another simplistic idea coming to my mind is really to tell Handlebars "here, I'm creating environment with this settings, there's no way back, and despite there's interface where environment may be reconfigured, it won't have any affect" - which will be quite counter-intuitive. Not solving my issue again since I'm explicitly creating/registering new templates, and those are added to HandlebarsConfiguration

Do you have any suggestion? Maybe I overlooked something, or stuck on unrelated piece of implementation and got confused.

oformaniuk Feb 8, 2022
Maintainer

@wdolek ,

I tried to instantiate HandlebarsConfigurationAdapter once and pass it in, but then any change trough HandlebarsConfiguration (which is exposed) is not propagated to ICompiledHandlebarsConfiguration, leaving me with either null-ref exceptions, missing helpers or other errors.

I assume those reference different instances of HandlebarsConfiguration. I cannot say for sure without seeing the code.

Unfortunately, HandlebarsConfiguration.CompileTimeConfiguration isn't using observable collections, so whatever I add here after I instantiate Handlebars, I don't see a way how to propagate that change further to ICompiledHandlebarsConfiguration - without breaking change (changing IList to observable ones for example).

This limitation may be explicitly mentioned, stating that you should pass final version of HandlebarsConfiguration and that only some of the properties (e.g., Helpers) can be updated.

To summarize, create a PR and we can continue discussion there.

rexm · 2022-01-31T22:47:24Z

rexm
Jan 31, 2022
Maintainer

What kind of memory footprint are you seeking to achieve? How much reduction would make this no longer a problem in your use case?

4 replies

wdolek Feb 1, 2022
Author

@rexm Hi! We did first calculation/estimate and thought we can reduce 300 MiB to ~12 MiB (we are having 27 languages). But after initial work we realized we didn't count with dictionaries itself - still it was surprising for us to see having ~100 MiB in dictionary entries (not finishing processing of all dictionaries - so it will be even worse).

Basically any improvement will be beneficial :)

rexm Feb 1, 2022
Maintainer

Thanks for that info. I am trying to tackle your problem from a constraints side: if you can't reduce it from 300MB at all, does that make using Handlebars impossible for your use case? How much reduction would need to be achieved before your team said "ok, that's good enough, spending more effort on further reduction isn't worth it"? That will help aim our focus.

wdolek Feb 1, 2022
Author

I wouldn't say "impossible" to use - we would have to find other way how to use it, maybe by splitting/"sharding" load to multiple instances, but of course it would have impact on complexity of our solution.

What we want to achieve is to keep in memory ~10 times more content than we currently handle and still know that instance will survive that. (This is super-optimistic goal - not a hard limit)

There's probably another thing worth mentioning and having impact on overall memory usage - while I did memory analysis, I couldn't overlook huge amount of unmanaged memory being used as well. I couldn't identify what's responsible for it, but I assume it may be partially related to generated IL - I don't think there's anything to be optimized. (Just to have full picture, if I observe 300 MiB of compiled templates in memory, I also see several hundred MiBs of unmanaged memory being taken as well - not saying it's all related to Handlebars though)

oformaniuk Feb 2, 2022
Maintainer

@wdolek,

I couldn't overlook huge amount of unmanaged memory being used as well. I couldn't identify what's responsible for it, but I assume it may be partially related to generated IL - I don't think there's anything to be optimized

I cannot say for sure, but it indeed may be related to IL generation. However, Expression Trees are using DynamicMethod which is GC friendly. Handlebars does not use any unmanaged code explicitly so we cannot optimize anything in that direction.

wdolek · 2022-06-03T07:33:32Z

wdolek
Jun 3, 2022
Author

@zjklee yesterday we deployed our app with version 2.1.2 and enabled shared env - we got from ~900MiB to ~600MiB used. This will allow us to do further improvements as suggested in this discussion. Thank you 🙇 !

1 reply

oformaniuk Jun 3, 2022
Maintainer

Thanks for the update.
In case you'd spot places where memory footprint can be in the shared mode - let me know.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Memory allocation with every compiled template (to support l10n of templates) #493

{{title}}

Replies: 3 comments 11 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Memory allocation with every compiled template (to support l10n of templates) #493

wdolek Jan 31, 2022

Replies: 3 comments · 11 replies

oformaniuk Jan 31, 2022 Maintainer

oformaniuk Feb 1, 2022 Maintainer

Point 1

Point 2

Point 3

wdolek Feb 1, 2022 Author

oformaniuk Feb 2, 2022 Maintainer

wdolek Feb 4, 2022 Author

oformaniuk Feb 8, 2022 Maintainer

rexm Jan 31, 2022 Maintainer

wdolek Feb 1, 2022 Author

rexm Feb 1, 2022 Maintainer

wdolek Feb 1, 2022 Author

oformaniuk Feb 2, 2022 Maintainer

wdolek Jun 3, 2022 Author

oformaniuk Jun 3, 2022 Maintainer

wdolek
Jan 31, 2022

Replies: 3 comments 11 replies

oformaniuk
Jan 31, 2022
Maintainer

oformaniuk Feb 1, 2022
Maintainer

wdolek Feb 1, 2022
Author

oformaniuk Feb 2, 2022
Maintainer

wdolek Feb 4, 2022
Author

oformaniuk Feb 8, 2022
Maintainer

rexm
Jan 31, 2022
Maintainer

wdolek Feb 1, 2022
Author

rexm Feb 1, 2022
Maintainer

wdolek Feb 1, 2022
Author

oformaniuk Feb 2, 2022
Maintainer

wdolek
Jun 3, 2022
Author

oformaniuk Jun 3, 2022
Maintainer