diff --git a/README.md b/README.md index 44deb16..a921b87 100644 --- a/README.md +++ b/README.md @@ -50,84 +50,132 @@ See [examples](./examples) for more ## Benchmark +### function selectors FP/FN - [False Positive/False Negative](https://en.wikipedia.org/wiki/False_positives_and_false_negatives) errors; smaller is better - + - + - + - + - + - + - + - - - - + + + + - + - + - + - + - + - - - - + + + + - + - + - + - + - + + + + + + + + + +
Dataset simpleevmole-js (py) whatsabi evm-hound-rsevmole-js (py)simple
largest1k
1000 contracts
24427 functions
largest1k
1000 contracts
24427 functions
FP/FN contracts:95 / 91 / 0 :1st_place_medal: 38 / 8 75 / 401 / 0 :1st_place_medal:95 / 9
FP/FN functions:749 / 12192 / 0 :2nd_place_medal: :1st_place_medal: 38 / 8 :1st_place_medal: :2nd_place_medal: 720 / 191192 / 0 :2nd_place_medal: :1st_place_medal:749 / 12
Time:1.97s3.8s1.34s2.03s (1.99s)1.6s (1.74s)3.54s1.1s1.85s
random50k
50000 contracts
1171102 functions
random50k
50000 contracts
1171102 functions
FP/FN contracts:4136 / 771 / 9 :1st_place_medal: 251 / 31 693 / 29031 / 9 :1st_place_medal:4136 / 77
FP/FN functions:14652 / 963 / 10 :1st_place_medal: 261 / 32 10798 / 35383 / 10 :1st_place_medal:14652 / 96
Time:35.84s69.91s11.97s25.02s (33.62s)18.81s (32.27s)67.13s11.95s34.39s
vyper
780 contracts
21244 functions
vyper
780 contracts
21244 functions
FP/FN contracts:185 / 4800 / 0 :1st_place_medal: 178 / 780 19 / 3000 / 0 :1st_place_medal:185 / 480
FP/FN functions:197 / 129710 / 0 :1st_place_medal: 181 / 21244 19 / 82730 / 0 :1st_place_medal:197 / 12971
Time:1.42s (1.28s)2.39s0.97s1.34s
+ +### function arguments +errors - when at least 1 argument is incorrect: `(uint256,string)` != `(uint256,bytes)`; smaller is better + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - - - - + +
Datasetevmole-js (py)simple
largest1k
1000 contracts
24427 functions
errors:15.1%, 3677 :1st_place_medal:58.3%, 14242
Time:6.84s (13.02s)1.17s
random50k
50000 contracts
1171102 functions
errors:5.4%, 63774 :1st_place_medal:54.9%, 643213
Time:185.03s (402.58s)8.76s
vyper
780 contracts
21244 functions
errors:52.3%, 11103 :1st_place_medal:56.8%, 12077
Time:1.62s2.55s1.29s1.44s (1.6s)7.77s (16.05s)0.98s