Fuzz introspector: fuzz_tokenizer
For issues and ideas: https://github.com/ossf/fuzz-introspector/issues

Fuzz blockers

The following nodes represent call sites where fuzz blockers occur.

Amount of callsites blocked Calltree index Parent function Callsite Largest blocked function
5 15 sacremoses.corpus.NonbreakingPrefixes.words call site: 00015 sacremoses.tokenize.MosesTokenizer.has_numeric_only
3 7 sacremoses.tokenize.MosesTokenizer.__init__ call site: 00007 sacremoses.corpus.NonbreakingPrefixes.words
2 3 ...fuzz_tokenizer.TestOneInput call site: 00003 fdp.ConsumeIntInRange
1 35 sacremoses.tokenize.MosesTokenizer.__init__ call site: 00035 sacremoses.tokenize.MosesTokenizer.penn_tokenize
1 44 sacremoses.tokenize.MosesTokenizer.handles_nonbreaking_prefixes call site: 00044 sacremoses.tokenize.MosesTokenizer.isanyalpha
1 48 sacremoses.tokenize.MosesTokenizer.isanyalpha call site: 00048 sacremoses.tokenize.MosesTokenizer.islower
1 51 sacremoses.tokenize.MosesTokenizer.islower call site: 00051 re.search

Fuzzer calltree

0 ...fuzz_tokenizer.TestOneInput [function] [call site] 00000
1 atheris.FuzzedDataProvider [function] [call site] 00001
1 fdp.ConsumeIntInRange [function] [call site] 00002
1 fdp.ConsumeUnicodeNoSurrogates [function] [call site] 00003
1 fdp.ConsumeIntInRange [function] [call site] 00004
1 fdp.ConsumeUnicodeNoSurrogates [function] [call site] 00005
1 sacremoses.tokenize.MosesTokenizer.__init__ [function] [call site] 00006
2 <builtin>.super [function] [call site] 00007
2 _nbp.strip [function] [call site] 00008
2 sacremoses.corpus.NonbreakingPrefixes.words [function] [call site] 00009
3 <builtin>.set [function] [call site] 00010
3 os.path.join [function] [call site] 00011
3 pkgutil.get_data [function] [call site] 00012
3 binary_data.decode [function] [call site] 00013
3 line.strip [function] [call site] 00014
3 line.startswith [function] [call site] 00015
2 <builtin>.open [function] [call site] 00016
2 line.strip [function] [call site] 00017
2 line.startswith [function] [call site] 00018
2 w.rpartition [function] [call site] 00019
2 sacremoses.tokenize.MosesTokenizer.has_numeric_only [function] [call site] 00020
3 re.search [function] [call site] 00021
3 <builtin>.bool [function] [call site] 00022
2 sacremoses.corpus.Perluniprops.chars [function] [call site] 00023
3 os.path.join [function] [call site] 00024
3 pkgutil.get_data [function] [call site] 00025
3 binary_data.decode [function] [call site] 00026
2 <builtin>.str [function] [call site] 00027
2 sacremoses.corpus.Perluniprops.chars [function] [call site] 00028
2 <builtin>.str [function] [call site] 00029
2 sacremoses.corpus.Perluniprops.chars [function] [call site] 00030
2 <builtin>.str [function] [call site] 00031
2 sacremoses.corpus.Perluniprops.chars [function] [call site] 00032
2 <builtin>.str [function] [call site] 00033
2 sacremoses.corpus.Perluniprops.chars [function] [call site] 00034
2 <builtin>.str [function] [call site] 00035
1 sacremoses.tokenize.MosesTokenizer.penn_tokenize [function] [call site] 00036
2 <builtin>.str [function] [call site] 00037
2 re.sub [function] [call site] 00038
2 sacremoses.tokenize.MosesTokenizer.handles_nonbreaking_prefixes [function] [call site] 00039
3 text.split [function] [call site] 00040
3 <builtin>.len [function] [call site] 00041
3 <builtin>.enumerate [function] [call site] 00042
3 re.search [function] [call site] 00043
3 token_ends_with_period.group [function] [call site] 00044
3 sacremoses.tokenize.MosesTokenizer.isanyalpha [function] [call site] 00045
4 <builtin>.set [function] [call site] 00046
4 <builtin>.set [function] [call site] 00047
4 <builtin>.any [function] [call site] 00048
3 sacremoses.tokenize.MosesTokenizer.islower [function] [call site] 00049
4 <builtin>.set [function] [call site] 00050
4 <builtin>.set [function] [call site] 00051
3 re.search [function] [call site] 00052
2 re.sub [function] [call site] 00053
2 text.split [function] [call site] 00054