Fuzz introspector: trainer_fuzzer.cc
For issues and ideas: https://github.com/ossf/fuzz-introspector/issues

Fuzz blockers

The following nodes represent call sites where fuzz blockers occur.

Amount of callsites blocked Calltree index Parent function Callsite Largest blocked function
452 0 EP call site: 00000 sentencepiece::SentencePieceProcessor::EncodeOptimized

Fuzzer calltree

0 LLVMFuzzerTestOneInput [function] [call site] 00000
1 FuzzedDataProvider::ConsumeIntegralInRange [function] [call site] 00001
1 FuzzedDataProvider::ConsumeIntegralInRange [function] [call site] 00002
1 std::to_string [function] [call site] 00003
1 FuzzedDataProvider::ConsumeBool [function] [call site] 00004
1 std::to_string [function] [call site] 00005
1 FuzzedDataProvider::ConsumeFloatingPointInRange [function] [call site] 00006
1 FuzzedDataProvider::ConsumeBool [function] [call site] 00007
1 std::to_string [function] [call site] 00008
1 FuzzedDataProvider::ConsumeIntegralInRange [function] [call site] 00009
1 FuzzedDataProvider::ConsumeBool [function] [call site] 00010
1 std::string [function] [call site] 00011
1 FuzzedDataProvider::ConsumeBool [function] [call site] 00012
1 FuzzedDataProvider::ConsumeBool [function] [call site] 00013
1 std::string [function] [call site] 00014
1 FuzzedDataProvider::ConsumeBool [function] [call site] 00015
1 FuzzedDataProvider::ConsumeBool [function] [call site] 00016
1 std::string [function] [call site] 00017
1 FuzzedDataProvider::ConsumeBool [function] [call site] 00018
1 FuzzedDataProvider::ConsumeBool [function] [call site] 00019
1 std::string [function] [call site] 00020
1 FuzzedDataProvider::ConsumeBool [function] [call site] 00021
1 FuzzedDataProvider::ConsumeBool [function] [call site] 00022
1 std::string [function] [call site] 00023
1 FuzzedDataProvider::ConsumeBool [function] [call site] 00024
1 FuzzedDataProvider::ConsumeIntegralInRange [function] [call site] 00025
1 FuzzedDataProvider::ConsumeRandomLengthString [function] [call site] 00026
1 std::string::empty [function] [call site] 00027
1 std::vector::push_back [function] [call site] 00028
1 std::vector::empty [function] [call site] 00029
1 std::vector::push_back [function] [call site] 00030
1 sentencepiece::SentencePieceTrainer::Train [function] [call site] 00031
2 normalizer_spec [function] [call site] 00032
2 RETURN_IF_ERROR [function] [call site] 00033
2 sentencepiece::SentencePieceTrainer::PopulateNormalizerSpec [function] [call site] 00034
3 RET_CHECK [function] [call site] 00035
3 NormalizerSpec*::normalization_rule_tsv [function] [call site] 00036
3 RET_CHECK [function] [call site] 00037
3 NormalizerSpec*::precompiled_charsmap [function] [call site] 00038
3 RETURN_IF_ERROR [function] [call site] 00039
3 sentencepiece::normalizer::Builder::LoadCharsMap [function] [call site] 00040
4 LOG [function] [call site] 00041
4 VectorBuffer::data [function] [call site] 00042
5 VectorBuffer::data [function] [call site] 00043
6 VectorBuffer::data [function] [call site] 00044
4 RET_CHECK [function] [call site] 00045
4 sentencepiece::filesystem::NewReadableFile [function] [call site] 00046
5 std::make_unique<DefaultReadableFile> [function] [call site] 00047
4 RETURN_IF_ERROR [function] [call site] 00048
4 sentencepiece::SentencePieceProcessor::status [function] [call site] 00049
5 RET_CHECK [function] [call site] 00050
5 RET_CHECK [function] [call site] 00051
5 RETURN_IF_ERROR [function] [call site] 00052
5 sentencepiece::SentencePieceProcessor::status [function] [call site] 00053
6 RET_CHECK [function] [call site] 00054
6 RET_CHECK [function] [call site] 00055
6 RETURN_IF_ERROR [function] [call site] 00056
6 sentencepiece::SentencePieceProcessor::status [function] [call site] 00057
6 RETURN_IF_ERROR [function] [call site] 00058
6 sentencepiece::SentencePieceProcessor::status [function] [call site] 00059
6 absl::OkStatus [function] [call site] 00060
5 RETURN_IF_ERROR [function] [call site] 00061
5 sentencepiece::SentencePieceProcessor::status [function] [call site] 00062
5 absl::OkStatus [function] [call site] 00063
4 Darts::DoubleArrayImpl::clear [function] [call site] 00064
4 sentencepiece::filesystem::PosixReadableFile::ReadLine [function] [call site] 00065
5 static_cast<bool> [function] [call site] 00066
5 std::getline [function] [call site] 00067
4 absl::StrSplit [function] [call site] 00068
4 absl::AllowEmpty [function] [call site] 00069
4 CHECK_GE [function] [call site] 00070
4 std::vector::size [function] [call site] 00071
4 std::vector::size [function] [call site] 00072
4 std::vector::emplace_back [function] [call site] 00073
4 absl::StrSplit [function] [call site] 00074
4 empty [function] [call site] 00075
4 absl::ConsumePrefix [function] [call site] 00076
4 std::vector::push_back [function] [call site] 00077
4 string_util::HexToInt<char32_t> [function] [call site] 00078
4 absl::StrSplit [function] [call site] 00079
4 empty [function] [call site] 00080
4 absl::ConsumePrefix [function] [call site] 00081
4 std::vector::push_back [function] [call site] 00082
4 string_util::HexToInt<char32_t> [function] [call site] 00083
4 RET_CHECK [function] [call site] 00084
4 std::vector::empty [function] [call site] 00085
4 absl::OkStatus [function] [call site] 00086
3 NormalizerSpec*::normalization_rule_tsv [function] [call site] 00087
3 RETURN_IF_ERROR [function] [call site] 00088
3 sentencepiece::normalizer::Builder::CompileCharsMap [function] [call site] 00089
4 RET_CHECK [function] [call site] 00090
4 RET_CHECK [function] [call site] 00091
4 empty [function] [call site] 00092
4 LOG [function] [call site] 00093
4 size [function] [call site] 00094
4 PyListStringViewVector::empty [function] [call site] 00095
5 PyListStringViewVector::empty [function] [call site] 00096
6 PyListStringViewVector::empty [function] [call site] 00097
4 absl::InvalidArgumentError [function] [call site] 00098
4 absl::InvalidArgumentError [function] [call site] 00099
4 std::string::size [function] [call site] 00100
4 sentencepiece::string_util::UnicodeTextToUTF8 [function] [call site] 00101
5 sentencepiece::string_util::EncodeUTF8 [function] [call site] 00102
6 static_cast<char> [function] [call site] 00103
5 std::string::append [function] [call site] 00104
4 RET_CHECK [function] [call site] 00105
4 sentencepiece::string_util::IsStructurallyValid [function] [call site] 00106
5 VectorBuffer::data [function] [call site] 00107
5 VectorBuffer::data [function] [call site] 00108
5 sentencepiece::random::ReservoirSampler::total_size [function] [call site] 00109
5 sentencepiece::string_util::DecodeUTF8 [function] [call site] 00110
6 static_cast<unsigned [call site] 00111
6 static_cast<unsigned [call site] 00112
6 sentencepiece::string_util::IsTrailByte [function] [call site] 00113
7 static_cast<signed [call site] 00114
6 sentencepiece::string_util::IsValidCodepoint [function] [call site] 00115
7 static_cast<uint32_t> [function] [call site] 00116
6 sentencepiece::string_util::IsTrailByte [function] [call site] 00117
6 sentencepiece::string_util::IsTrailByte [function] [call site] 00118
6 sentencepiece::string_util::IsValidCodepoint [function] [call site] 00119
6 sentencepiece::string_util::IsTrailByte [function] [call site] 00120
6 sentencepiece::string_util::IsTrailByte [function] [call site] 00121
6 sentencepiece::string_util::IsTrailByte [function] [call site] 00122
6 sentencepiece::string_util::IsValidCodepoint [function] [call site] 00123
5 sentencepiece::string_util::IsValidCodepoint [function] [call site] 00124
4 sentencepiece::string_util::UnicodeTextToUTF8 [function] [call site] 00125
4 RET_CHECK [function] [call site] 00126
4 std::string::empty [function] [call site] 00127
4 RET_CHECK [function] [call site] 00128
4 sentencepiece::string_util::IsStructurallyValid [function] [call site] 00129
4 std::vector::emplace_back [function] [call site] 00130
4 sentencepiece::port::FindOrDie [function] [call site] 00131
5 find [function] [call site] 00132
4 std::sort [function] [call site] 00133
4 std::vector::begin [function] [call site] 00134
4 std::vector::end [function] [call site] 00135
4 std::vector::size [function] [call site] 00136
4 std::vector::size [function] [call site] 00137
4 std::vector::size [function] [call site] 00138
4 first::c_str [function] [call site] 00139
4 RET_CHECK_EQ [function] [call site] 00140
4 Darts::Details::BitVector::build [function] [call site] 00141
5 reset [function] [call site] 00142
5 Darts::Details::BitVector::size [function] [call site] 00143
5 id_type::id_type [function] [call site] 00144
5 DARTS_THROW [function] [call site] 00145
5 Darts::Details::BitVector::size [function] [call site] 00146
5 Darts::Details::BitVector::pop_count [function] [call site] 00147
4 std::vector::size [function] [call site] 00148
4 const_cast<char**> [function] [call site] 00149
4 std::vector::data [function] [call site] 00150
4 std::vector::data [function] [call site] 00151
4 Darts::DoubleArrayImpl<A, [call site] 00152
5 Darts::Details::DoubleArrayUnit::offset [function] [call site] 00153
5 static_cast<uchar_type> [function] [call site] 00154
5 Darts::Details::DoubleArrayUnit::label [function] [call site] 00155
5 static_cast<uchar_type> [function] [call site] 00156
5 Darts::Details::DoubleArrayUnit::offset [function] [call site] 00157
5 Darts::Details::DoubleArrayUnit::has_leaf [function] [call site] 00158
5 set_result [function] [call site] 00159
5 static_cast<value_type> [function] [call site] 00160
5 value [function] [call site] 00161
5 static_cast<uchar_type> [function] [call site] 00162
5 Darts::Details::DoubleArrayUnit::label [function] [call site] 00163
5 static_cast<uchar_type> [function] [call site] 00164
5 Darts::Details::DoubleArrayUnit::offset [function] [call site] 00165
5 Darts::Details::DoubleArrayUnit::has_leaf [function] [call site] 00166
5 set_result [function] [call site] 00167
5 static_cast<value_type> [function] [call site] 00168
5 value [function] [call site] 00169
4 std::vector::data [function] [call site] 00170
4 std::vector::size [function] [call site] 00171
4 strlen [function] [call site] 00172
4 std::max [function] [call site] 00173
4 RET_CHECK_LT [function] [call site] 00174
4 static_cast<const [call site] 00175
4 Darts::DoubleArrayImpl::set_array [function] [call site] 00176
5 Darts::DoubleArrayImpl::clear [function] [call site] 00177
5 static_cast<const [call site] 00178
4 sentencepiece::random::ReservoirSampler::total_size [function] [call site] 00179
4 Darts::DoubleArrayImpl::unit_size [function] [call site] 00180
4 sentencepiece::normalizer::Normalizer::EncodePrecompiledCharsMap [function] [call site] 00181
5 std::string::append [function] [call site] 00182
5 string_util::EncodePOD<uint32_t> [function] [call site] 00183
5 sentencepiece::random::ReservoirSampler::total_size [function] [call site] 00184
5 std::string::append [function] [call site] 00185
5 VectorBuffer::data [function] [call site] 00186
5 sentencepiece::random::ReservoirSampler::total_size [function] [call site] 00187
5 sentencepiece::util::is_bigendian [function] [call site] 00188
5 reinterpret_cast<uint32_t*> [function] [call site] 00189
5 std::string::data [function] [call site] 00190
5 std::string::size [function] [call site] 00191
5 sentencepiece::util::Swap32 [function] [call site] 00192
6 absl::gbswap_32 [function] [call site] 00193
5 std::string::append [function] [call site] 00194
5 VectorBuffer::data [function] [call site] 00195
5 sentencepiece::random::ReservoirSampler::total_size [function] [call site] 00196
4 RETURN_IF_ERROR [function] [call site] 00197
4 IsValidNormalizerData [function] [call site] 00198
4 LOG [function] [call site] 00199
4 std::string*::size [function] [call site] 00200
4 absl::OkStatus [function] [call site] 00201
3 NormalizerSpec*::mutable_precompiled_charsmap [function] [call site] 00202
3 NormalizerSpec*::set_name [function] [call site] 00203
3 NormalizerSpec*::name [function] [call site] 00204
3 NormalizerSpec*::set_name [function] [call site] 00205
3 NormalizerSpec*::precompiled_charsmap [function] [call site] 00206
3 RETURN_IF_ERROR [function] [call site] 00207
3 sentencepiece::normalizer::Builder::GetPrecompiledCharsMap [function] [call site] 00208
4 RET_CHECK [function] [call site] 00209
4 std::string*::clear [function] [call site] 00210
4 absl::OkStatus [function] [call site] 00211
4 std::all_of [function] [call site] 00212
4 absl::string_view::begin [function] [call site] 00213
4 Darts::Details::AutoPool::append [function] [call site] 00214
5 resize_buf [function] [call site] 00215
5 sentencepiece::TEST [function] [call site] 00216
6 sentencepiece::GetTestDataPath [function] [call site] 00217
7 sentencepiece::util::JoinPath [function] [call site] 00218
8 VectorBuffer::data [function] [call site] 00219
8 sentencepiece::random::ReservoirSampler::total_size [function] [call site] 00220
7 ::testing::SrcDir [function] [call site] 00221
6 ASSERT_TRUE [function] [call site] 00222
6 ModelProto::ParseFromIstream [function] [call site] 00223
6 sentencepiece::TrainerComponents::mutable_normalizer_spec [function] [call site] 00224
6 auto*::precompiled_charsmap [function] [call site] 00225
6 ASSERT_GE [function] [call site] 00226
6 std::string::size [function] [call site] 00227
6 absl::little_endian::Store32 [function] [call site] 00228
6 const_cast<char*> [function] [call site] 00229
6 std::string::data [function] [call site] 00230
6 auto*::set_precompiled_charsmap [function] [call site] 00231
6 sentencepiece::SentencePieceProcessor::Load [function] [call site] 00232
7 std::make_unique<ModelProto> [function] [call site] 00233
7 RETURN_IF_ERROR [function] [call site] 00234
7 sentencepiece::io::LoadModelProto [function] [call site] 00235
8 PyListStringViewVector::empty [function] [call site] 00236
9 PyListStringViewVector::empty [function] [call site] 00237
8 absl::NotFoundError [function] [call site] 00238
8 sentencepiece::filesystem::NewReadableFile [function] [call site] 00239
8 RETURN_IF_ERROR [function] [call site] 00240
8 sentencepiece::SentencePieceProcessor::status [function] [call site] 00241
8 sentencepiece::filesystem::PosixReadableFile::ReadAll [function] [call site] 00242
9 LOG [function] [call site] 00243
9 std::string*::assign [function] [call site] 00244
9 std::istreambuf_iterator<char> [function] [call site] 00245
9 std::istreambuf_iterator<char> [function] [call site] 00246
8 absl::InternalError [function] [call site] 00247
8 absl::StrCat [function] [call site] 00248
8 ModelProto*::ParseFromArray [function] [call site] 00249
8 std::string::data [function] [call site] 00250
8 std::string::size [function] [call site] 00251
8 absl::InternalError [function] [call site] 00252
8 absl::StrCat [function] [call site] 00253
8 absl::OkStatus [function] [call site] 00254
7 PyThreadPool::get [function] [call site] 00255
8 PyThreadPool::get [function] [call site] 00256
9 PyThreadPool::get [function] [call site] 00257
7 sentencepiece::SentencePieceProcessor::Load [function] [call site] 00258
8 std::make_unique<ModelProto> [function] [call site] 00259
8 RETURN_IF_ERROR [function] [call site] 00260
8 sentencepiece::io::LoadModelProto [function] [call site] 00261
8 PyThreadPool::get [function] [call site] 00262
8 sentencepiece::SentencePieceProcessor::Load [function] [call site] 00263
8 std::move [function] [call site] 00264
7 std::move [function] [call site] 00265
6 EXPECT_FALSE [function] [call site] 00266
6 absl::Status::ok [function] [call site] 00267
6 EXPECT_EQ [function] [call site] 00268
6 sentencepiece::SentencePieceProcessor::Encode [function] [call site] 00269
7 sentencepiece::SentencePieceProcessor::EncodeOptimized [function] [call site] 00270
8 RET_CHECK_STATUS_STL [function] [call site] 00271
8 PyListStringViewVector::empty [function] [call site] 00272
8 std::vector<T>*::clear [function] [call site] 00273
8 sentencepiece::SentencePieceProcessor::ApplyExtraOptions [function] [call site] 00274
9 std::reverse [function] [call site] 00275
9 T*::mutable_pieces [function] [call site] 00276
9 T*::mutable_pieces [function] [call site] 00277
9 std::reverse [function] [call site] 00278
9 T*::begin [function] [call site] 00279
9 Darts::Details::AutoPool::append [function] [call site] 00280
10 resize_buf [function] [call site] 00281
10 sentencepiece::TEST [function] [call site] 00282
9 sentencepiece::SentencePieceProcessor::eos_id [function] [call site] 00283
9 T*::add_pieces [function] [call site] 00284
9 auto*::set_id [function] [call site] 00285
9 auto*::set_piece [function] [call site] 00286
9 eos_piece [function] [call site] 00287
9 eos_piece [function] [call site] 00288
9 auto*::set_begin [function] [call site] 00289
9 T*::text [function] [call site] 00290
9 auto*::set_end [function] [call site] 00291
9 T*::text [function] [call site] 00292
9 T*::emplace_back [function] [call site] 00293
9 T*::emplace_back [function] [call site] 00294
9 eos_piece [function] [call site] 00295
9 sentencepiece::SentencePieceProcessor::bos_id [function] [call site] 00296
9 T*::mutable_pieces [function] [call site] 00297
9 sentencepiece::unigram::BoundedPriorityQueue::Add [function] [call site] 00298
10 find [function] [call site] 00299
10 end [function] [call site] 00300
10 emplace [function] [call site] 00301
10 std::string [function] [call site] 00302
9 sentencepiece::random::ReservoirSampler::total_size [function] [call site] 00303
9 auto*::SwapElements [function] [call site] 00304
9 auto*::Mutable [function] [call site] 00305
9 auto*::set_id [function] [call site] 00306
9 auto*::set_piece [function] [call site] 00307
9 bos_piece [function] [call site] 00308
9 bos_piece [function] [call site] 00309
9 auto*::set_begin [function] [call site] 00310
9 auto*::set_end [function] [call site] 00311
9 T*::emplace [function] [call site] 00312
9 T*::begin [function] [call site] 00313
9 T*::emplace [function] [call site] 00314
9 T*::begin [function] [call site] 00315
9 bos_piece [function] [call site] 00316
9 T*::pieces_size [function] [call site] 00317
9 T*::mutable_pieces [function] [call site] 00318
9 sentencepiece::SentencePieceProcessor::IsUnknown [function] [call site] 00319
10 RET_CHECK_OR_RETURN_DEFAULT [function] [call site] 00320
10 sentencepiece::SentencePieceProcessor::IsUnknown [function] [call site] 00321
9 sentencepiece::string_util::IsStructurallyValid [function] [call site] 00322
10 VectorBuffer::data [function] [call site] 00323
10 VectorBuffer::data [function] [call site] 00324
10 sentencepiece::random::ReservoirSampler::total_size [function] [call site] 00325
10 sentencepiece::string_util::DecodeUTF8 [function] [call site] 00326
10 sentencepiece::string_util::IsValidCodepoint [function] [call site] 00327
9 auto*::set_piece [function] [call site] 00328
9 unk_piece [function] [call site] 00329
9 unk_piece [function] [call site] 00330
9 sentencepiece::unigram::Lattice::Clear [function] [call site] 00331
10 clear [function] [call site] 00332
10 clear [function] [call site] 00333
10 absl::string_view [function] [call site] 00334
10 clear [function] [call site] 00335
10 Free [function] [call site] 00336
9 Darts::DoubleArrayImpl::clear [function] [call site] 00337
9 absl::InternalError [function] [call site] 00338
9 absl::OkStatus [function] [call site] 00339
8 RETURN_IF_ERROR [function] [call site] 00340
8 sentencepiece::SentencePieceProcessor::Normalize [function] [call site] 00341
9 RET_CHECK [function] [call site] 00342
9 sentencepiece::SentencePieceProcessor::Normalize [function] [call site] 00343
8 sentencepiece::SentencePieceProcessor::Encode [function] [call site] 00344
9 sentencepiece::SentencePieceProcessor::EncodeOptimized [function] [call site] 00345
8 ByteFallbackEnabled [function] [call site] 00346
8 sentencepiece::SentencePieceProcessor::HasUnkPieceOption [function] [call site] 00347
8 std::vector<T>*::clear [function] [call site] 00348
8 std::vector<T>*::reserve [function] [call site] 00349
8 sentencepiece::random::ReservoirSampler::total_size [function] [call site] 00350
8 RET_CHECK [function] [call site] 00351
8 PyListStringViewVector::empty [function] [call site] 00352
8 sentencepiece::SentencePieceProcessor::IsControl [function] [call site] 00353
9 RET_CHECK_OR_RETURN_DEFAULT [function] [call site] 00354
9 sentencepiece::SentencePieceProcessor::IsControl [function] [call site] 00355
8 std::vector<T>*::emplace_back [function] [call site] 00356
8 std::vector<T>*::emplace_back [function] [call site] 00357
8 VectorBuffer::data [function] [call site] 00358
8 sentencepiece::random::ReservoirSampler::total_size [function] [call site] 00359
8 sentencepiece::SentencePieceProcessor::IsUnknown [function] [call site] 00360
8 sentencepiece::random::ReservoirSampler::total_size [function] [call site] 00361
8 sentencepiece::SentencePieceProcessor::PieceToId [function] [call site] 00362
9 RET_CHECK_OR_RETURN_DEFAULT [function] [call site] 00363
9 sentencepiece::SentencePieceProcessor::PieceToId [function] [call site] 00364
8 ByteToPiece [function] [call site] 00365
8 static_cast<uint8_t> [function] [call site] 00366
8 std::vector<T>*::emplace_back [function] [call site] 00367
8 std::vector<T>*::emplace_back [function] [call site] 00368
8 ByteToPiece [function] [call site] 00369
8 static_cast<uint8_t> [function] [call site] 00370
8 std::vector<T>*::emplace_back [function] [call site] 00371
8 std::vector<T>*::emplace_back [function] [call site] 00372
8 unk_piece [function] [call site] 00373
8 std::vector<T>*::emplace_back [function] [call site] 00374
8 VectorBuffer::data [function] [call site] 00375
8 sentencepiece::random::ReservoirSampler::total_size [function] [call site] 00376
8 std::vector<T>*::back [function] [call site] 00377
8 VectorBuffer::data [function] [call site] 00378
8 sentencepiece::random::ReservoirSampler::total_size [function] [call site] 00379
8 sentencepiece::SentencePieceProcessor::ApplyExtraOptions [function] [call site] 00380
6 EXPECT_EQ [function] [call site] 00381
6 absl::Status::message [function] [call site] 00382
4 absl::StatusBuilder [function] [call site] 00383
4 std::string*::assign [function] [call site] 00384
4 IsValidNormalizerData [function] [call site] 00385
4 absl::StrCat [function] [call site] 00386
4 sentencepiece::util::JoinPath [function] [call site] 00387
4 GetDataDir [function] [call site] 00388
4 sentencepiece::filesystem::NewReadableFile [function] [call site] 00389
4 sentencepiece::SentencePieceProcessor::status [function] [call site] 00390
4 sentencepiece::filesystem::PosixReadableFile::ReadAll [function] [call site] 00391
4 IsValidNormalizerData [function] [call site] 00392
4 absl::StatusBuilder [function] [call site] 00393
4 GetDataDir [function] [call site] 00394
3 NormalizerSpec*::name [function] [call site] 00395
3 NormalizerSpec*::mutable_precompiled_charsmap [function] [call site] 00396
3 absl::OkStatus [function] [call site] 00397
2 denormalizer_spec [function] [call site] 00398
2 RETURN_IF_ERROR [function] [call site] 00399
2 sentencepiece::SentencePieceTrainer::PopulateNormalizerSpec [function] [call site] 00400
2 sentencepiece::TrainerFactory::Create [function] [call site] 00401
3 TrainerSpec::model_type [function] [call site] 00402
3 std::make_unique<unigram::Trainer> [function] [call site] 00403
3 std::make_unique<bpe::Trainer> [function] [call site] 00404
3 std::make_unique<word::Trainer> [function] [call site] 00405
3 std::make_unique<character::Trainer> [function] [call site] 00406
3 LOG [function] [call site] 00407
3 TrainerSpec::model_type [function] [call site] 00408
3 std::make_unique<unigram::Trainer> [function] [call site] 00409
2 trainer_spec [function] [call site] 00410
2 absl::StrCat [function] [call site] 00411
2 PrintProto [function] [call site] 00412
2 trainer_spec [function] [call site] 00413
2 PrintProto [function] [call site] 00414
2 auto::precompiled_charsmap [function] [call site] 00415
2 PrintProto [function] [call site] 00416
2 LOG [function] [call site] 00417
2 RETURN_IF_ERROR [function] [call site] 00418
2 sentencepiece::character::Trainer::Train [function] [call site] 00419
3 RETURN_IF_ERROR [function] [call site] 00420
3 status [function] [call site] 00421
3 RET_CHECK [function] [call site] 00422
3 escape_whitespaces [function] [call site] 00423
3 RET_CHECK_EQ [function] [call site] 00424
3 model_type [function] [call site] 00425
3 RETURN_IF_ERROR [function] [call site] 00426
3 LoadSentences [function] [call site] 00427
3 vocab_size [function] [call site] 00428
3 size [function] [call site] 00429
3 RET_CHECK_GE [function] [call site] 00430
3 std::log [function] [call site] 00431
3 static_cast<float> [function] [call site] 00432
3 RET_CHECK [function] [call site] 00433
3 empty [function] [call site] 00434
3 Sorted [function] [call site] 00435
3 use_all_vocab [function] [call site] 00436
3 size [function] [call site] 00437
3 static_cast<size_t> [function] [call site] 00438
3 emplace_back [function] [call site] 00439
3 sentencepiece::string_util::UnicodeCharToUTF8 [function] [call site] 00440
4 sentencepiece::string_util::UnicodeTextToUTF8 [function] [call site] 00441
3 std::log [function] [call site] 00442
3 static_cast<float> [function] [call site] 00443
3 use_all_vocab [function] [call site] 00444
3 set_vocab_size [function] [call site] 00445
3 size [function] [call site] 00446
3 size [function] [call site] 00447
3 Save [function] [call site] 00448
2 ModelProto::SerializeAsString [function] [call site] 00449
2 RETURN_IF_ERROR [function] [call site] 00450
2 sentencepiece::character::Trainer::Train [function] [call site] 00451
2 absl::OkStatus [function] [call site] 00452