Need best practices guides for performance and memory usage

[ClearScriptLib]

No description provided.

[danbopes]

I can't stress this enough. I'm trying to integrate this into my project, and I'd like to safely add usage and resource limits. What areas I'm struggling with:

• How the V8RuntimeConstraints worked, and some sample documentation on how to properly set reasonable size limits for user run code.
• How do I limit execution time? With jint, I can specify a cancellation token to cancel the currently running script.
• How to kill processes that hog the CPU:

engine.Execute(@"
    while (true) {}
");

• How the engine works with asynchronous code:

public class TestApi
{
    private delegate void Executor(dynamic resolve, dynamic reject);
    private static async Task<string> Delay(string msg)
    {
        await Task.Delay(2000);

        return msg;
    }
    public static object delay(string msg)
    {
        var task = Delay(msg);
        var ctor = (ScriptObject)ScriptEngine.Current.Script.Promise;
        return ctor.Invoke(true, new Executor((resolve, reject) => {
            task.ContinueWith(t => {
                if (t.IsCompleted)
                {
                    resolve(t.Result);
                }
                else
                {
                    reject(t.Exception);
                }
            });
        }));
    }
}

engine.AddHostType("Console", typeof(Console));
engine.AddHostType("api", typeof(TestApi));
engine.Execute(@"
    Console.WriteLine('start');

    const main = async () => {
      Console.WriteLine('main');
      var res = await api.delay('end');
      Console.WriteLine(res);
    }
    main();
");

If I wrap this in a using block, it will output:

start
main

and then dispose of the engine

If I execute this, and add a Console.ReadKey(); before the engine gets disposed, after 2 seconds it correctly outputs end. There's no documentation on how to wait until execution of all callbacks is completed.

[ClearScriptLib]

Hi @danbopes,

• How the V8RuntimeConstraints worked, and some sample documentation on how to properly set reasonable size limits for user run code.

We generally recommend that applications stay away from V8RuntimeConstraints.

Originally this API was somewhat suitable for sandboxing, but the V8 team decided years ago that constrained script execution isn't a goal for V8. Today, exceeding any of the specified limits causes V8 to crash instantly, and that's by design.

The API might still be useful for expanding V8's default limits, but we've found that V8 can then hit other limits that are inaccessible and vary unpredictably from version to version.

ClearScript does offer support for "soft limits" (see MaxRuntimeHeapSize, MaxRuntimeStackUsage), but as they require external monitoring via periodic timers, we can't guarantee their effectiveness in all cases.

The bottom line is that, unfortunately, if you must run untrusted script code with 100% safety, you'll need a Chromium-like multi-process architecture. The V8 team equates executing untrusted script code with invoking a function in an untrusted DLL.

V8's API is constantly changing though, and we'll continue to watch for new developments.

• How do I limit execution time? With jint, I can specify a cancellation token to cancel the currently running script.
• How to kill processes that hog the CPU

For these purposes you can use ScriptEngine.Interrupt.

There's no documentation on how to wait until execution of all callbacks is completed.

There's no special API for that, but you can always wait for a pre-arranged completion signal:

engine.AddHostType("Console", typeof(Console));
engine.AddHostType("api", typeof(TestApi));
engine.Script.done = new ManualResetEventSlim();
engine.Execute(@"
    Console.WriteLine('start');
    const main = async () => {
      Console.WriteLine('main');
      var res = await api.delay('end');
      Console.WriteLine(res);
      done.Set();
    }
    main();
");
engine.Script.done.Wait();

Please send any additional questions or comments our way.

Thanks!

[simontom]

Hey everyone, I'm wondering how to "correctly" set the memory limits and this seemed like the best place to ask based on what I've found. But copying it here as well....

What properties I've found, so far:

• V8ScriptEngine.MaxRuntimeHeapSize

• V8ScriptEngine.MaxRuntimeStackUsage

• V8RuntimeConstraints.MaxArrayBufferAllocation

• V8RuntimeConstraints.MaxOldSpaceSize

• V8RuntimeConstraints.MaxArrayBufferAllocation

• V8ScriptEngine.RuntimeHeapSizeSampleInterval

What properties I use + the values:

• V8ScriptEngine.MaxRuntimeHeapSize = 1GB
• V8RuntimeConstraints.MaxArrayBufferAllocation = 1GB

Properties with default:

• V8RuntimeConstraints.MaxOldSpaceSize = 1.5GB

Question:
Do I need to set explicitly the sample interval?

• V8ScriptEngine.RuntimeHeapSizeSampleInterval
  Or it works per se (has some value implicitly) when the
• V8ScriptEngine.MaxRuntimeHeapSize = 1GB
  is set?

The descriptions seem a bit vague to me, and I miss the default values in the documentation for them 😿

My current playground / flow:

• container (with dotnet) in AKS has a 2GB RAM limit (is killed when engine(s) exceed this limit) which runs the engine(s)
• engine is spawned for each custom JavaScript request and is disposed when the script finishes
• it can run up to 2 JavaScripts (engines) at the same time
• I currently use HeapConsumptionProbe that compares engine.GetRuntimeHeapInfo().TotalHeapSize > engine.MaxRuntimeHeapSize.ToUInt64() => throw which is run periodically (every 10s)

What problem I'm trying to kinda diminish:

• have correct/better settings for memory limits of ClearScript (V8Engine) => lower the number of containers's OOM kills
• stop the engine before it consumes all the RAM

Could give me a hand, please, and help me understand a bit more (find more proper values)?

With kind regards,
Šimon

P.s.: From what I've currently read here, we had wrong expectations / assumptions

[ClearScriptLib]

Hi Šimon,

Do I need to set explicitly the sample interval?

The default value (zero) selects an internally enforced minimum, which is currently 50ms. Our recommendation is not to increase unless you encounter performance issues.

Could give me a hand, please, and help me understand a bit more (find more proper values)?

This probably won't be very helpful, but our understanding is as follows:

• The V8 heap consists of two regions – the young generation, which holds recently allocated objects, and the old generation, which contains objects that have survived several rounds of garbage collection.
• The young generation is further subdivided equally into two semi-spaces and a large object space.
• The heap region and space in which an object resides determines various aspects of its handling by the garbage collector, which can move objects from one region to another.
• The young generation is typically much smaller than the old generation. On 64-bit systems, V8's per-runtime default limits are around 8MB for the former and 1.4GB for the latter, although optional features, when enabled, can alter them.

Ultimately, your ideal V8ResourceConstraints and MaxRuntimeHeapSize numbers depend on the scripts you need to run. We understand that the memory requirements of user-provided scripts are difficult to predict, especially if those scripts could be buggy or even malicious, but that's the unfortunate reality.

A potentially useful feature not mentioned above is on-demand heap expansion, which provides a way to increase the heap size limit when a script comes close to exceeding it. See HeapExpansionMultiplier.

Good luck!

[simontom]

Hello, thanks a lot!
Well, 50ms period seems fine so far. May be, I can/will try to tweak it (lower the period) a little to let it hit sooner.

What is the connection between V8ResourceConstraints.MaxArrayBufferAllocation and MaxRuntimeHeapSize?
Per the DOC, these are set independently (that's why I set both values).
How does is co-operate?
Does the V8ScriptEngine.RuntimeHeapSizeSampleInterval check both the heap and the ArrayBufferAllocation?
If not, how to see/check/validate the allocated memory for ArrayBufferAllocation?

What can be the max memory footprint of the engine if I have these values set:
V8ScriptEngine.MaxRuntimeHeapSize = 1GB
V8RuntimeConstraints.MaxArrayBufferAllocation = 1GB
V8RuntimeConstraints.MaxOldSpaceSize = 1.5GB

Is it recommended to explicitly call the GC on the V8 Engine before the instance gets dispose?

About the HeapExpansionMultiplier
Is the default value 1? As I understand it, 1 means it does not expand/change the size of the heap?
I think it is not usable for this solution, or ...?

And one more
Is there any difference creating the engine this way

then disposing the engine

vs

var runtime = new V8Runtime("xxx");
var engine = runtime.CreateScriptEngine(scriptEngineFlags);

then

engine.Dispose();
runtime.CollectGarbage(true);
engine = Runtime.CreateScriptEngine( .... );

?