Performance

[Todo18]

Hi,

First of all, thanks for providing (and maintaining) this really well-made, versatile scripting engine.

Our question is a general one about VBScript (and maybe JScript, since they're related) performance. We've noticed, since the integration with our software, that there's a significant difference in execution time between executing a script in ClearScript, and running that same script using the built-in interpreter of Windows (cscript.exe).

As we understand it, and judging from the source code, ClearScript talks to the same core libraries (ActiveScript) as the Windows Scripting Host (cscript/wscript), so there shouldn't be any (noticeable) difference in execution time between the two, especially for benchmark scripts that largely perform mathematical operations, for example? Any thoughts on why ClearScript takes on average about four times as long to execute such scripts? Maybe our integration is flawed? Although all we're doing is essentially instantiating a VBScriptEngine and calling Execute on this instance.

Kind regards

[ClearScriptLib]

Hi @Todo18,

First, thank you for your kind words!

We've noticed, since the integration with our software, that there's a significant difference in execution time between executing a script in ClearScript, and running that same script using the built-in interpreter of Windows (cscript.exe).

We tested a simple VBScript prime number sieve (2M capacity) in CScript and ClearScript, observing a penalty for the latter in the 1-5% range, depending on the run and the .NET runtime used, with greater variance for smaller capacities.

As this benchmark is 100% VBScript code without any calls to the host, the only variable is the runtime environment, including ClearScript's managed implementation of the ActiveScript site interfaces. Unfortunately the overhead appears to be intrinsic to the .NET COM interop environment, so we have no way to mitigate it without modifying the runtime or using a native assembly for ActiveScript siting.

Any thoughts on why ClearScript takes on average about four times as long to execute such scripts? Maybe our integration is flawed?

A difference that large is almost certainly due to the expensive boundary between managed and script code. What managed objects or types are you exposing? How often are you calling them from your script code? Are the affected scripts long-running? Please provide as much information as possible.

Thanks!

[Todo18]

Thanks for getting back to us in such short notice, appreciated!

Do you happen to be able to share the benchmark script you used in your tests, so we can try it out ourselves (the prime number sieve)? For completeness, we've attached our own benchmark script to this reply, which is a hasher (ext changed to txt, as to not alarm any scanners). You'll notice it doesn't talk to managed objects in any way (unless you count the .NET objects that are instantiated via CreateObject as such, but these are supported out of the box by the WSH as well...; in any case, no user-defined object or type is exposed). In our benchmark, the script is called one time, any number of times (multiple executions of the hosting app, not the script), and the time spent in script is output each time (and aggregated at the end). The affected scripts take about 20 seconds for cscript, and more than 100 seconds for ClearScript (in our tests).

If you need anything else, please, do tell.
Hasher.txt

[ClearScriptLib]

Hi @Todo18,

Do you happen to be able to share the benchmark script you used in your tests, so we can try it out ourselves (the prime number sieve)?

We used the simple implementation here, expanding the capacity to 2000000.

For completeness, we've attached our own benchmark script to this reply, which is a hasher

Thank you! We tested your script as follows:

public static void Main() {
    var sw = Stopwatch.StartNew();
    using (var engine = new VBScriptEngine()) {
        engine.Execute(File.ReadAllText(@"C:\Code\hasher.vbs"));
        Console.WriteLine(sw.Elapsed);
    }
    sw.Restart();
    using (var process = Process.Start("cscript", @"/nologo C:\Code\hasher.vbs")) {
        process.WaitForExit();
        Console.WriteLine(sw.Elapsed);
    }
}

On our test PC, this application produces the following output (with negligible differences from run to run):

00:00:30.3495580
00:00:27.2528805

As you can see, we aren't encountering anywhere near the overhead you're seeing.

As for the overhead we are seeing, we've traced most of it down to the fact that ClearScript's managed ActiveScript site implements ICustomQueryInterface, something we could eliminate with a bit of work. A quick test with this interface disabled yields:

00:00:27.8310104
00:00:27.0267728

One big difference between our benchmark and yours is that the latter extensively uses COM to access .NET resources without ClearScript's involvement. Is it possible that you have something installed that somehow alters COM interop behavior? We're not aware of any such tools, but no other explanation comes to mind.

Please don't hesitate to send additional thoughts or findings our way!

[Todo18]

Thanks, we can work with this! It's a bit busy at the moment, but I'll be sure to check it out in the coming days, and get back to you, regardless!

[Todo18]

Hi, it's been a while, but this was solved not in the least thanks to your help. As far as we're concerned this topic can be closed. :)