average normalized score | |
---|---|
fuzzer | |
aflplusplus | 98.66 |
fishpp_new | 95.23 |
fishpp_new_nocmp | 82.91 |
aflplusplus_nocmp | 82.57 |
average rank | |
---|---|
fuzzer | |
aflplusplus | 1.64 |
fishpp_new | 2.00 |
fishpp_new_nocmp | 2.86 |
aflplusplus_nocmp | 3.09 |
Note: The relative coverage summary table shows the median
relative performance of each fuzzer to the experiment maximum. Thus the
highest relative performance may not be 100%.
trial_relative_coverage = trial_coverage / experiment_max_coverage
fishpp_new | aflplusplus | fishpp_new_nocmp | aflplusplus_nocmp | |
---|---|---|---|---|
FuzzerMedian | 97.00 | 97.50 | 96.00 | 95.00 |
FuzzerMean | 95.67 | 94.73 | 83.38 | 83.00 |
bloaty_fuzz_target | 99.00 | 98.00 | 98.00 | 98.00 |
curl_curl_fuzzer_http | 96.00 | 96.00 | nan | 93.00 |
freetype2_ftfuzzer | 97.00 | 97.00 | 67.00 | 70.00 |
harfbuzz_hb-shape-fuzzer | 98.00 | 99.00 | 98.00 | 99.00 |
jsoncpp_jsoncpp_fuzzer | 100.00 | 100.00 | 99.00 | 99.00 |
lcms_cms_transform_fuzzer | 95.00 | 91.00 | 81.00 | 78.00 |
libjpeg-turbo_libjpeg_turbo_fuzzer | 99.00 | 99.00 | 99.00 | 99.00 |
libpcap_fuzz_both | 94.00 | 93.00 | 1.00 | 1.00 |
libpng_libpng_read_fuzzer | 96.00 | 96.00 | 96.00 | 96.00 |
libxml2_xml | 99.00 | 99.00 | 99.00 | 99.00 |
libxslt_xpath | nan | 98.00 | 97.00 | 97.00 |
mbedtls_fuzz_dtlsclient | 72.00 | 72.00 | 71.00 | 71.00 |
openssl_x509 | 99.00 | 99.00 | 99.00 | 99.00 |
openthread_ot-ip6-send-fuzzer | 94.00 | 84.00 | 78.00 | 78.00 |
proj4_proj_crs_to_crs_fuzzer | 94.00 | 96.00 | 10.00 | 10.00 |
re2_fuzzer | 99.00 | 99.00 | 99.00 | nan |
sqlite3_ossfuzz | 90.00 | 80.00 | 85.00 | 81.00 |
stb_stbi_read_fuzzer | 95.00 | 95.00 | 90.00 | 90.00 |
systemd_fuzz-link-parser | 99.00 | 99.00 | 94.00 | 94.00 |
vorbis_decode_fuzzer | 99.00 | 99.00 | 99.00 | 99.00 |
woff2_convert_woff2ttf_fuzzer | 98.00 | 97.00 | 94.00 | 95.00 |
zlib_zlib_uncompress_fuzzer | 97.00 | 98.00 | 97.00 | 97.00 |
count | mean | std | min | 25% | median | 75% | max | ||
---|---|---|---|---|---|---|---|---|---|
fuzzer | time | ||||||||
fishpp_new | 82800 | 9.0 | 6385.333333 | 64.212538 | 6240.0 | 6374.00 | 6404.0 | 6405.00 | 6468.0 |
fishpp_new_nocmp | 82800 | 8.0 | 6397.250000 | 34.820971 | 6329.0 | 6385.50 | 6401.5 | 6419.75 | 6435.0 |
aflplusplus_nocmp | 82800 | 12.0 | 6345.916667 | 96.939023 | 6057.0 | 6366.00 | 6374.0 | 6387.50 | 6413.0 |
aflplusplus | 82800 | 14.0 | 6350.571429 | 51.876650 | 6252.0 | 6315.75 | 6354.0 | 6371.75 | 6437.0 |
count | mean | std | min | 25% | median | 75% | max | ||
---|---|---|---|---|---|---|---|---|---|
fuzzer | time | ||||||||
fishpp_new | 19800 | 7.0 | 10698.571429 | 50.960302 | 10628.0 | 10662.00 | 10711.0 | 10728.00 | 10771.0 |
aflplusplus | 19800 | 18.0 | 10696.333333 | 47.300044 | 10605.0 | 10668.00 | 10690.5 | 10727.75 | 10786.0 |
aflplusplus_nocmp | 19800 | 16.0 | 10405.500000 | 34.871192 | 10335.0 | 10392.75 | 10411.0 | 10423.25 | 10479.0 |
count | mean | std | min | 25% | median | 75% | max | ||
---|---|---|---|---|---|---|---|---|---|
fuzzer | time | ||||||||
fishpp_new | 82800 | 11.0 | 11579.818182 | 266.421778 | 11089.0 | 11418.5 | 11680.0 | 11807.0 | 11866.0 |
aflplusplus | 82800 | 13.0 | 11520.076923 | 303.469510 | 10801.0 | 11400.0 | 11632.0 | 11694.0 | 11954.0 |
aflplusplus_nocmp | 82800 | 11.0 | 8293.909091 | 227.453492 | 7817.0 | 8290.0 | 8383.0 | 8423.5 | 8494.0 |
fishpp_new_nocmp | 82800 | 7.0 | 8145.857143 | 312.148698 | 7751.0 | 7998.0 | 8015.0 | 8342.0 | 8575.0 |
count | mean | std | min | 25% | median | 75% | max | ||
---|---|---|---|---|---|---|---|---|---|
fuzzer | time | ||||||||
aflplusplus_nocmp | 81900 | 13.0 | 10909.384615 | 166.422424 | 10476.0 | 10892.00 | 10948.0 | 11014.00 | 11040.0 |
aflplusplus | 81900 | 16.0 | 10927.062500 | 69.374797 | 10769.0 | 10874.75 | 10947.0 | 10966.25 | 11025.0 |
fishpp_new_nocmp | 81900 | 13.0 | 10911.076923 | 42.681888 | 10827.0 | 10888.00 | 10921.0 | 10945.00 | 10971.0 |
fishpp_new | 81900 | 12.0 | 10722.583333 | 592.705577 | 8846.0 | 10839.00 | 10890.0 | 10924.50 | 10961.0 |
count | mean | std | min | 25% | median | 75% | max | ||
---|---|---|---|---|---|---|---|---|---|
fuzzer | time | ||||||||
aflplusplus | 82800 | 15.0 | 520.000000 | 0.000000 | 520.0 | 520.00 | 520.0 | 520.0 | 520.0 |
fishpp_new | 82800 | 8.0 | 518.375000 | 4.206712 | 508.0 | 519.75 | 520.0 | 520.0 | 520.0 |
aflplusplus_nocmp | 82800 | 8.0 | 519.000000 | 0.000000 | 519.0 | 519.00 | 519.0 | 519.0 | 519.0 |
fishpp_new_nocmp | 82800 | 9.0 | 515.111111 | 6.173420 | 505.0 | 509.00 | 519.0 | 519.0 | 520.0 |
count | mean | std | min | 25% | median | 75% | max | ||
---|---|---|---|---|---|---|---|---|---|
fuzzer | time | ||||||||
fishpp_new | 81900 | 13.0 | 1897.000000 | 260.247318 | 1561.0 | 1571.00 | 2043.0 | 2109.00 | 2148.0 |
aflplusplus | 81900 | 12.0 | 1863.916667 | 230.849206 | 1525.0 | 1586.25 | 1963.5 | 2056.50 | 2084.0 |
fishpp_new_nocmp | 81900 | 15.0 | 1689.133333 | 134.141645 | 1435.0 | 1554.00 | 1753.0 | 1775.00 | 1860.0 |
aflplusplus_nocmp | 81900 | 16.0 | 1673.937500 | 111.654508 | 1500.0 | 1565.25 | 1685.0 | 1776.25 | 1807.0 |
count | mean | std | min | 25% | median | 75% | max | ||
---|---|---|---|---|---|---|---|---|---|
fuzzer | time | ||||||||
aflplusplus | 81900 | 16.0 | 3083.625000 | 1.892969 | 3080.0 | 3082.0 | 3084.0 | 3085.0 | 3086.0 |
fishpp_new_nocmp | 81900 | 15.0 | 3081.333333 | 5.752846 | 3064.0 | 3081.5 | 3083.0 | 3084.5 | 3086.0 |
aflplusplus_nocmp | 81900 | 16.0 | 3081.562500 | 3.846535 | 3072.0 | 3079.0 | 3082.5 | 3084.0 | 3086.0 |
fishpp_new | 81900 | 16.0 | 3073.750000 | 12.751471 | 3054.0 | 3061.5 | 3082.5 | 3084.0 | 3087.0 |
count | mean | std | min | 25% | median | 75% | max | ||
---|---|---|---|---|---|---|---|---|---|
fuzzer | time | ||||||||
fishpp_new | 82800 | 10.0 | 2967.400000 | 103.614885 | 2801.0 | 2888.75 | 2991.0 | 3031.0 | 3120.0 |
aflplusplus | 82800 | 13.0 | 2932.692308 | 118.893078 | 2713.0 | 2853.00 | 2955.0 | 2990.0 | 3176.0 |
fishpp_new_nocmp | 82800 | 9.0 | 40.222222 | 3.833333 | 33.0 | 42.00 | 42.0 | 42.0 | 43.0 |
aflplusplus_nocmp | 82800 | 8.0 | 36.375000 | 4.657943 | 33.0 | 33.00 | 33.0 | 42.0 | 42.0 |
count | mean | std | min | 25% | median | 75% | max | ||
---|---|---|---|---|---|---|---|---|---|
fuzzer | time | ||||||||
aflplusplus | 82800 | 17.0 | 2008.235294 | 13.544046 | 2001.0 | 2004.00 | 2005.0 | 2006.00 | 2060.0 |
fishpp_new | 82800 | 10.0 | 1995.600000 | 43.474641 | 1892.0 | 1993.75 | 2001.0 | 2005.00 | 2072.0 |
aflplusplus_nocmp | 82800 | 6.0 | 2000.500000 | 2.073644 | 1998.0 | 1999.25 | 2000.5 | 2001.00 | 2004.0 |
fishpp_new_nocmp | 82800 | 10.0 | 1971.900000 | 46.971504 | 1898.0 | 1933.00 | 2000.0 | 2001.75 | 2003.0 |
count | mean | std | min | 25% | median | 75% | max | ||
---|---|---|---|---|---|---|---|---|---|
fuzzer | time | ||||||||
aflplusplus | 82800 | 15.0 | 15740.933333 | 48.121077 | 15649.0 | 15708.00 | 15753.0 | 15767.0 | 15815.0 |
aflplusplus_nocmp | 82800 | 8.0 | 15724.375000 | 37.618147 | 15668.0 | 15688.75 | 15736.0 | 15753.0 | 15768.0 |
fishpp_new_nocmp | 82800 | 7.0 | 15734.571429 | 33.310945 | 15693.0 | 15710.00 | 15731.0 | 15757.5 | 15783.0 |
fishpp_new | 82800 | 11.0 | 15723.545455 | 38.011482 | 15662.0 | 15708.00 | 15717.0 | 15741.5 | 15794.0 |
count | mean | std | min | 25% | median | 75% | max | ||
---|---|---|---|---|---|---|---|---|---|
fuzzer | time | ||||||||
aflplusplus | 42300 | 15.0 | 11191.933333 | 105.500485 | 11015.0 | 11117.0 | 11229.0 | 11269.50 | 11338.0 |
fishpp_new_nocmp | 42300 | 10.0 | 11047.300000 | 82.460832 | 10912.0 | 10996.5 | 11039.0 | 11103.75 | 11175.0 |
aflplusplus_nocmp | 42300 | 17.0 | 11074.117647 | 81.904580 | 10992.0 | 11016.0 | 11034.0 | 11100.00 | 11247.0 |
count | mean | std | min | 25% | median | 75% | max | ||
---|---|---|---|---|---|---|---|---|---|
fuzzer | time | ||||||||
aflplusplus | 81900 | 18.0 | 2840.666667 | 184.337925 | 2725.0 | 2774.75 | 2795.0 | 2834.0 | 3565.0 |
fishpp_new | 81900 | 13.0 | 2769.000000 | 55.837562 | 2626.0 | 2750.00 | 2773.0 | 2804.0 | 2850.0 |
fishpp_new_nocmp | 81900 | 16.0 | 2795.187500 | 284.052394 | 2597.0 | 2716.00 | 2735.5 | 2753.0 | 3850.0 |
aflplusplus_nocmp | 81900 | 17.0 | 2823.470588 | 241.562134 | 2707.0 | 2720.00 | 2734.0 | 2774.0 | 3531.0 |
count | mean | std | min | 25% | median | 75% | max | ||
---|---|---|---|---|---|---|---|---|---|
fuzzer | time | ||||||||
fishpp_new | 81900 | 18.0 | 5824.333333 | 7.062744 | 5806.0 | 5820.75 | 5826.5 | 5828.75 | 5833.0 |
aflplusplus | 81900 | 15.0 | 5823.733333 | 3.058166 | 5818.0 | 5821.00 | 5825.0 | 5825.50 | 5829.0 |
aflplusplus_nocmp | 81900 | 15.0 | 5821.400000 | 6.288311 | 5811.0 | 5818.50 | 5823.0 | 5825.00 | 5831.0 |
fishpp_new_nocmp | 81900 | 12.0 | 5818.750000 | 10.323187 | 5799.0 | 5812.50 | 5821.5 | 5826.00 | 5831.0 |
count | mean | std | min | 25% | median | 75% | max | ||
---|---|---|---|---|---|---|---|---|---|
fuzzer | time | ||||||||
fishpp_new | 9900 | 1.0 | 3418.000000 | NaN | 3418.0 | 3418.00 | 3418.0 | 3418.0 | 3418.0 |
aflplusplus | 9900 | 19.0 | 3057.842105 | 78.942499 | 3011.0 | 3033.50 | 3039.0 | 3044.5 | 3372.0 |
aflplusplus_nocmp | 9900 | 18.0 | 2836.333333 | 57.578591 | 2798.0 | 2815.25 | 2820.5 | 2823.0 | 3032.0 |
fishpp_new_nocmp | 9900 | 4.0 | 2811.000000 | 3.915780 | 2806.0 | 2809.00 | 2811.5 | 2813.5 | 2815.0 |
count | mean | std | min | 25% | median | 75% | max | ||
---|---|---|---|---|---|---|---|---|---|
fuzzer | time | ||||||||
aflplusplus | 81900 | 15.0 | 7153.400000 | 197.377014 | 6676.0 | 7060.0 | 7172.0 | 7286.5 | 7419.0 |
fishpp_new | 81900 | 14.0 | 7073.714286 | 163.695509 | 6867.0 | 6956.0 | 7035.0 | 7154.0 | 7389.0 |
fishpp_new_nocmp | 81900 | 15.0 | 811.600000 | 4.954075 | 807.0 | 807.5 | 809.0 | 814.5 | 822.0 |
aflplusplus_nocmp | 81900 | 11.0 | 801.727273 | 22.786360 | 734.0 | 806.0 | 807.0 | 807.5 | 817.0 |
count | mean | std | min | 25% | median | 75% | max | ||
---|---|---|---|---|---|---|---|---|---|
fuzzer | time | ||||||||
aflplusplus | 82800 | 15.0 | 2877.333333 | 4.546061 | 2869.0 | 2875.50 | 2877.0 | 2881.00 | 2884.0 |
fishpp_new_nocmp | 82800 | 8.0 | 2864.625000 | 33.644942 | 2782.0 | 2873.75 | 2877.0 | 2878.25 | 2881.0 |
fishpp_new | 82800 | 8.0 | 2864.625000 | 32.723681 | 2784.0 | 2872.75 | 2875.5 | 2878.25 | 2880.0 |
count | mean | std | min | 25% | median | 75% | max | ||
---|---|---|---|---|---|---|---|---|---|
fuzzer | time | ||||||||
fishpp_new | 41400 | 3.0 | 19221.000000 | 833.433861 | 18623.0 | 18745.00 | 18867.0 | 19520.00 | 20173.0 |
fishpp_new_nocmp | 41400 | 3.0 | 16682.000000 | 3019.479922 | 13264.0 | 15529.50 | 17795.0 | 18391.00 | 18987.0 |
aflplusplus_nocmp | 41400 | 18.0 | 17079.333333 | 1554.172146 | 15317.0 | 15847.00 | 16933.5 | 17693.00 | 20052.0 |
aflplusplus | 41400 | 16.0 | 17266.875000 | 1455.793066 | 15442.0 | 16201.75 | 16689.0 | 18672.25 | 19591.0 |
count | mean | std | min | 25% | median | 75% | max | ||
---|---|---|---|---|---|---|---|---|---|
fuzzer | time | ||||||||
aflplusplus | 82800 | 13.0 | 2157.923077 | 49.614281 | 2111.0 | 2114.00 | 2121.0 | 2213.00 | 2219.0 |
fishpp_new | 82800 | 14.0 | 2134.571429 | 40.912128 | 2110.0 | 2113.00 | 2115.0 | 2116.00 | 2211.0 |
fishpp_new_nocmp | 82800 | 11.0 | 2009.545455 | 1.368476 | 2007.0 | 2009.00 | 2009.0 | 2010.50 | 2012.0 |
aflplusplus_nocmp | 82800 | 2.0 | 2007.500000 | 3.535534 | 2005.0 | 2006.25 | 2007.5 | 2008.75 | 2010.0 |
count | mean | std | min | 25% | median | 75% | max | ||
---|---|---|---|---|---|---|---|---|---|
fuzzer | time | ||||||||
aflplusplus | 81900 | 17.0 | 235.235294 | 6.077925 | 226.0 | 227.0 | 239.0 | 239.0 | 240.0 |
fishpp_new | 81900 | 13.0 | 234.307692 | 6.638080 | 222.0 | 226.0 | 238.0 | 239.0 | 240.0 |
aflplusplus_nocmp | 81900 | 16.0 | 225.937500 | 0.250000 | 225.0 | 226.0 | 226.0 | 226.0 | 226.0 |
fishpp_new_nocmp | 81900 | 14.0 | 225.928571 | 0.267261 | 225.0 | 226.0 | 226.0 | 226.0 | 226.0 |
count | mean | std | min | 25% | median | 75% | max | ||
---|---|---|---|---|---|---|---|---|---|
fuzzer | time | ||||||||
aflplusplus | 82800 | 17.0 | 1267.764706 | 3.562468 | 1262.0 | 1265.00 | 1268.0 | 1271.00 | 1274.0 |
fishpp_new_nocmp | 82800 | 15.0 | 1264.866667 | 13.542877 | 1217.0 | 1266.00 | 1267.0 | 1268.50 | 1276.0 |
aflplusplus_nocmp | 82800 | 2.0 | 1266.500000 | 2.121320 | 1265.0 | 1265.75 | 1266.5 | 1267.25 | 1268.0 |
fishpp_new | 82800 | 11.0 | 1266.090909 | 4.548726 | 1261.0 | 1262.50 | 1265.0 | 1267.50 | 1275.0 |
count | mean | std | min | 25% | median | 75% | max | ||
---|---|---|---|---|---|---|---|---|---|
fuzzer | time | ||||||||
fishpp_new | 81900 | 13.0 | 1156.384615 | 40.699178 | 1060.0 | 1158.0 | 1175.0 | 1181.00 | 1191.0 |
aflplusplus | 81900 | 15.0 | 1172.866667 | 19.171656 | 1131.0 | 1164.5 | 1171.0 | 1191.00 | 1195.0 |
aflplusplus_nocmp | 81900 | 18.0 | 1136.388889 | 5.689401 | 1128.0 | 1132.0 | 1136.0 | 1141.75 | 1147.0 |
fishpp_new_nocmp | 81900 | 13.0 | 1125.769231 | 18.934773 | 1065.0 | 1127.0 | 1130.0 | 1136.00 | 1137.0 |
count | mean | std | min | 25% | median | 75% | max | ||
---|---|---|---|---|---|---|---|---|---|
fuzzer | time | ||||||||
aflplusplus | 81900 | 16.0 | 460.375000 | 3.667424 | 456.0 | 457.0 | 460.0 | 462.25 | 469.0 |
aflplusplus_nocmp | 81900 | 16.0 | 459.687500 | 3.156343 | 456.0 | 457.0 | 459.5 | 460.50 | 469.0 |
fishpp_new_nocmp | 81900 | 16.0 | 459.687500 | 3.807339 | 455.0 | 457.0 | 459.5 | 462.00 | 468.0 |
fishpp_new | 81900 | 13.0 | 460.461538 | 4.701336 | 455.0 | 457.0 | 459.0 | 463.00 | 469.0 |