Add clearscript to benchmarks

[viceice]

WDYT about adding ClearScript V8 to benchmark comparision?

Jint is pretty fast and i think it will be faster when massive interop is used (our usecase).
I know ClearScript will be faster if we cache the compiled scripts.

---

BenchmarkDotNet v0.13.12, Windows 11 (10.0.22631.3007/23H2/2023Update/SunValley3)
AMD Ryzen 9 5950X, 1 CPU, 32 logical and 16 physical cores
.NET SDK 8.0.101
  [Host]     : .NET 8.0.1 (8.0.123.58001), X64 RyuJIT AVX2
  DefaultJob : .NET 8.0.1 (8.0.123.58001), X64 RyuJIT AVX2

Method	FileName	Mean	StdDev	Median	Rank	Allocated
ClearScript	array-stress	4,297.036 μs	249.4436 μs	4,241.306 μs	1	19.83 KB
Jint_ParsedScript	array-stress	5,426.846 μs	73.3077 μs	5,426.401 μs	2	6996.66 KB
Jint	array-stress	5,432.763 μs	55.6799 μs	5,447.937 μs	2	7017.84 KB
NilJS	array-stress	5,760.830 μs	58.5161 μs	5,748.725 μs	3	4533.76 KB
YantraJS	array-stress	9,345.675 μs	589.0515 μs	9,362.683 μs	4	8072.26 KB
Jurassic	array-stress	10,901.013 μs	176.4263 μs	10,886.944 μs	5	11647.14 KB
ClearScript	dromaeo-3d-cube	3,823.149 μs	306.8329 μs	3,709.306 μs	1	21.09 KB
YantraJS	dromaeo-3d-cube	5,605.018 μs	169.7980 μs	5,604.674 μs	2	11411.52 KB
NilJS	dromaeo-3d-cube	7,243.936 μs	85.7499 μs	7,213.765 μs	3	4684.03 KB
Jint	dromaeo-3d-cube	17,419.136 μs	236.3291 μs	17,314.303 μs	4	6241.11 KB
Jint_ParsedScript	dromaeo-3d-cube	17,861.981 μs	123.4090 μs	17,874.391 μs	5	5983.8 KB
Jurassic	dromaeo-3d-cube	49,914.862 μs	798.8504 μs	49,789.014 μs	6	10670.37 KB
NilJS	dromaeo-core-eval	1,522.418 μs	35.6018 μs	1,511.824 μs	1	1598.5 KB
Jint	dromaeo-core-eval	2,925.104 μs	48.4454 μs	2,915.036 μs	2	355.79 KB
Jint_ParsedScript	dromaeo-core-eval	2,986.116 μs	83.2087 μs	2,957.505 μs	2	338.74 KB
YantraJS	dromaeo-core-eval	6,398.902 μs	363.7110 μs	6,358.370 μs	3	36528.11 KB
ClearScript	dromaeo-core-eval	8,218.883 μs	244.4878 μs	8,152.984 μs	4	19.72 KB
Jurassic	dromaeo-core-eval	9,528.492 μs	232.2599 μs	9,498.659 μs	5	2884.01 KB
ClearScript	dromaeo-object-array	17,359.478 μs	114.1278 μs	17,330.634 μs	1	109.32 KB
Jint_ParsedScript	dromaeo-object-array	39,937.045 μs	1,758.0706 μs	39,722.164 μs	2	100371.29 KB
Jurassic	dromaeo-object-array	41,329.276 μs	1,686.4097 μs	41,178.115 μs	3	25812.48 KB
Jint	dromaeo-object-array	42,074.419 μs	2,453.5820 μs	41,200.331 μs	3	100411.23 KB
YantraJS	dromaeo-object-array	58,443.122 μs	1,877.8925 μs	58,328.062 μs	4	29477.88 KB
NilJS	dromaeo-object-array	62,419.378 μs	889.9446 μs	62,138.150 μs	5	17697.74 KB
ClearScript	droma(...)egexp [21]	121,264.127 μs	4,414.6544 μs	120,786.038 μs	1	53.62 KB
Jint_ParsedScript	droma(...)egexp [21]	177,079.692 μs	11,905.3247 μs	175,944.925 μs	2	166172.94 KB
Jint	droma(...)egexp [21]	184,284.979 μs	12,295.2187 μs	183,751.825 μs	3	163820.12 KB
NilJS	droma(...)egexp [21]	753,924.893 μs	32,404.9952 μs	751,876.600 μs	4	767870.95 KB
Jurassic	droma(...)egexp [21]	841,315.689 μs	18,105.0504 μs	845,173.200 μs	5	821895.44 KB
YantraJS	droma(...)egexp [21]	1,379,192.655 μs	41,920.7458 μs	1,383,068.000 μs	6	1153843.95 KB
ClearScript	droma(...)tring [21]	22,489.982 μs	381.7536 μs	22,370.125 μs	1	46.33 KB
Jint	droma(...)tring [21]	314,807.403 μs	24,308.3988 μs	316,365.800 μs	2	1321676.66 KB
Jint_ParsedScript	droma(...)tring [21]	319,565.765 μs	33,784.3437 μs	313,723.200 μs	2	1321521.97 KB
Jurassic	droma(...)tring [21]	324,314.952 μs	19,430.7187 μs	317,816.550 μs	2	1458269.8 KB
NilJS	droma(...)tring [21]	325,398.865 μs	37,695.4569 μs	317,606.700 μs	2	1377962.92 KB
YantraJS	droma(...)tring [21]	1,550,364.932 μs	128,556.8642 μs	1,551,380.200 μs	3	15731157.93 KB
ClearScript	droma(...)ase64 [21]	5,473.901 μs	160.6648 μs	5,486.563 μs	1	23.44 KB
NilJS	droma(...)ase64 [21]	33,540.831 μs	979.9409 μs	33,381.994 μs	2	19603.6 KB
Jint	droma(...)ase64 [21]	41,492.156 μs	500.7580 μs	41,322.204 μs	3	6125.05 KB
Jint_ParsedScript	droma(...)ase64 [21]	43,778.998 μs	458.2571 μs	43,856.675 μs	4	6039.12 KB
YantraJS	droma(...)ase64 [21]	59,688.858 μs	4,644.4929 μs	58,876.550 μs	5	760382.22 KB
Jurassic	droma(...)ase64 [21]	69,266.104 μs	1,307.5180 μs	69,012.400 μs	6	73296.59 KB
YantraJS	evaluation	NA	NA	NA	?	NA
Jint_ParsedScript	evaluation	11.369 μs	0.5189 μs	11.252 μs	1	32.28 KB
Jint	evaluation	23.423 μs	0.7708 μs	23.377 μs	2	40.85 KB
NilJS	evaluation	33.723 μs	0.4697 μs	33.866 μs	3	23.47 KB
ClearScript	evaluation	1,247.734 μs	22.7458 μs	1,244.796 μs	4	16.17 KB
Jurassic	evaluation	1,332.532 μs	57.3569 μs	1,316.846 μs	5	420.32 KB
Jint_ParsedScript	linq-js	87.262 μs	1.4026 μs	86.917 μs	1	224.88 KB
YantraJS	linq-js	460.583 μs	19.8288 μs	459.796 μs	2	1443.82 KB
Jint	linq-js	1,547.042 μs	32.4542 μs	1,541.576 μs	3	1273.64 KB
ClearScript	linq-js	2,250.900 μs	50.3497 μs	2,259.330 μs	4	16.17 KB
NilJS	linq-js	7,040.094 μs	302.9560 μs	6,975.407 μs	5	4121.1 KB
Jurassic	linq-js	35,692.093 μs	1,816.8446 μs	35,192.530 μs	6	9250.77 KB
Jint_ParsedScript	minimal	3.002 μs	0.1538 μs	2.981 μs	1	13.22 KB
NilJS	minimal	3.860 μs	0.1256 μs	3.838 μs	2	4.81 KB
Jint	minimal	4.364 μs	0.2130 μs	4.346 μs	3	14.6 KB
YantraJS	minimal	159.117 μs	12.9214 μs	155.202 μs	4	918.04 KB
Jurassic	minimal	253.600 μs	7.4086 μs	251.929 μs	5	386.21 KB
ClearScript	minimal	1,231.473 μs	74.7467 μs	1,217.730 μs	6	16.17 KB
ClearScript	stopwatch	21,853.332 μs	483.6832 μs	21,662.716 μs	1	37.38 KB
YantraJS	stopwatch	88,722.652 μs	4,060.0092 μs	88,472.533 μs	2	224269.37 KB
NilJS	stopwatch	161,377.975 μs	2,416.2827 μs	160,553.125 μs	3	97360.77 KB
Jurassic	stopwatch	176,721.709 μs	2,811.8136 μs	175,943.033 μs	4	156935.47 KB
Jint_ParsedScript	stopwatch	256,421.241 μs	5,035.0362 μs	255,563.575 μs	5	53017.38 KB
Jint	stopwatch	265,253.789 μs	3,844.1962 μs	265,389.575 μs	6	53042.05 KB

Benchmarks with issues:
EngineComparisonBenchmark.YantraJS: DefaultJob [FileName=evaluation]

[lahma]

This is a bit apples to oranges comparison as interpreters vs truly compiled and inter-process vs invokes...

[sebastienros]

@lahma what does it "inter-process" mean here?

I think it's good to represent compiled engines too, but maybe it should be an opportunity to call out the differences and introduce other benchmarks that will show the pros and cons. For instance the Allocated column is only for managed memory, and compilation-based engines use mostly native memory so this won't show up. The other thing could be to add a benchmark where a script is used once, or with a way that scripts are re-compiled or re-parsed on each call. Some application behave this way but running a script just once. The last thing to take into account is how scripts are disposed, for instance you keep compiling/parsing new scripts, would it lead to resource exhaustions? Or can we release the script resources automatically (in-memory assemblies, loaded code, ...)?

So maybe it's about splitting this whole table and explaining what the test is about and why different engines do better than others, with the extra ones. I'd also hope that would make the Jint/Jint_Parsed duplication not necessary.

[viceice]

yes, the table also shows that some tests are bad on clearscript and I assume it's because of the context switching between native V8 and dotnet. this is our main use case.

so we've a lot esm scripts which are also calling into dotnet a lot. so I think it's worth trying to switch to jint, or at least try it.

[lahma]

@lahma what does it "inter-process" mean here?

Just meant no interop between NET runtime and native code. It's the interop bridge which costs.

For V8 it might make sense to create a separate benchmark altogether comparing just it and Jint and keep V8 out from "interpreter" comparison.

Benchmark could include something like.

1. Small eval (from comparison benchmark)
2. Large eval (from comparison benchmark)
3. for/of loop against large native JsArray
4. for loop against injected value and setting values to each index which is should show values getting back to NET runtime's original array
5. tight loop of calling engine, like C# loop calling function(x) { return x; } and providing call value and reading result each time
6. JS calling back to c#, so test 5 backwards
7. maybe something more

[lahma]

There's also https://github.com/Taritsyn/JavaScriptEngineSwitcher if one wants to write their business code against abstraction and have possibility to change underlying engine later one (or on use case basic).

[viceice]

@lahma what does it "inter-process" mean here?

Just meant no interop between NET runtime and native code. It's the interop bridge which costs.

For V8 it might make sense to create a separate benchmark altogether comparing just it and Jint and keep V8 out from "interpreter" comparison.

Benchmark could include something like.

1. Small eval (from comparison benchmark)
2. Large eval (from comparison benchmark)
3. for/of loop against large native JsArray
4. for loop against injected value and setting values to each index which is should show values getting back to NET runtime's original array
5. tight loop of calling engine, like C# loop calling function(x) { return x; } and providing call value and reading result each time
6. JS calling back to c#, so test 5 backwards
7. maybe something more

Sounds good, will look into it later

There's also https://github.com/Taritsyn/JavaScriptEngineSwitcher if one wants to write their business code against abstraction and have possibility to change underlying engine later one (or on use case basic).

To limmited for our use cases 🙃