hYVSSKrSSKrSSKrSSKJr SSKrSSKJrJ r J r SSK J r S\ S\ S\ S\ S \R4 S jrS \ S\ S\ S \R4S jrS \ S\ 4SjrS,S\ S\ S\ S\ S\S\ S \R4SjjrS\S\\\R44SjrS\ S\ S\ S\ S\S\ S\ S \ S\ S\ S\S\ 4SjrS-S\ S\ S\ S\ S\S\ S \ S\ S\ S\S\ S\ 4SjjrSrS.S\ S\S-S \S-S!\S-S \\RS-\RS-\RS-44 S"jjrS.S#\S\S-S \S-S!\S-S \\RS-\RS-\RS-44 S$jjrS%rS&\S'\S\ S\ S\ S\ S\S\S-S \S-S!\S-S(\S\ S\S\ 4S)jrS*r\ S+:Xa\"5 gg)/N)Path) ModelProto TensorProto numpy_helper) OnnxModel input_ids batch_sizesequence_lengthdictionary_sizereturnc,URRR[R[R [R 4;de[RRX1U4[RS9nURRR[R:Xa[R"U5nU$URRR[R :Xa[R"U5nU$)a@Create input tensor based on the graph input of input_ids Args: input_ids (TensorProto): graph input of the input_ids input tensor batch_size (int): batch size sequence_length (int): sequence length dictionary_size (int): vocabulary size of dictionary Returns: np.ndarray: the input tensor created )sizedtype) type tensor_type elem_typerFLOATINT32INT64nprandomrandintint32float32int64)rr r r datas a/var/www/fran/franai/venv/lib/python3.13/site-packages/onnxruntime/transformers/bert_test_data.pyfake_input_ids_datars >> % % / /4   99  _3PXZX`X`  aD~~!!++{/@/@@zz$ K  # # - -1B1B Bxx~ K segment_idscURRR[R[R [R 4;de[R"X4[RS9nURRR[R:Xa[R"U5nU$URRR[R :Xa[R"U5nU$)aCreate input tensor based on the graph input of segment_ids Args: segment_ids (TensorProto): graph input of the token_type_ids input tensor batch_size (int): batch size sequence_length (int): sequence length Returns: np.ndarray: the input tensor created r) rrrrrrrrzerosrrr)r r r rs rfake_segment_ids_datar$1s    ' ' 1 16   88Z1 BD##--1B1BBzz$ K    % % / /;3D3D Dxx~ Krmax_sequence_lengthaverage_sequence_lengthcUS:aX::deSU-U:a[R"SU-U- U5$[R"SSU-S- 5$)N)rr)r%r&s rget_random_lengthr*Lsa "a ',C,ZZ Z ""%88~~a"99*QRUVQVtNQ,>'?$1u+'M\]!^$YZ]^Y^tNQ4F/G$STu+/UdeQ^a'!+,+:&1Q':>*'('B!^$/0/JQ^a'!+,+"",, 0A0AAzz$ K  $ $ . .+2C2C Cxx~ Kr directoryinputsc <[RRU5(d'[R"U5 [ SUS35 O[ SUS35 [ UR55Htunup4[R"XC5n[[RRUSUS35S 5nURUR55 S S S 5 Mv g ![ a [ SUS35 Nf=f!,(df  M=f) zOutput input tensors of test data to a directory Args: directory (str): path of a directory inputs (Dict[str, np.ndarray]): map from input name to value z#Successfully created the directory  zCreation of the directory z failedzWarning: directory z$ existed. Files will be overwritten.input_.pbwbN)ospathexistsmkdirprintOSError enumerateitemsr from_arrayopenjoinwriteSerializeToString)r8r9indexnamertensorfiles routput_test_datarPs 77>>) $ $ F HHY  7 {!D E #I;.RST(8|((4 "'',,yF5'*=> E JJv//1 2F E 9 C .ykA B CF EsC- D -D D  D test_casesverbose random_seedc Uce[RRU5 [R"U5 /n [U5Hn [ X`X5nUR U0nU(a[ XpU5XR 'U(a[XXX5XR 'U(a[U 5S:Xa [SU5 U RU5 M U $)aCreate given number of input data for testing Args: batch_size (int): batch size sequence_length (int): sequence length test_cases (int): number of test cases dictionary_size (int): vocabulary size of dictionary for input_ids verbose (bool): print more information or not random_seed (int): random seed input_ids (TensorProto): graph input of input IDs segment_ids (TensorProto): graph input of token type IDs input_mask (TensorProto): graph input of attention mask average_sequence_length (int): average sequence length excluding paddings random_sequence_length (bool): whether use uniform random number for sequence length mask_type (int): mask type 1 is mask index; 2 is 2D mask; 3 is key len, cumulated lengths of query and key Returns: List[Dict[str,numpy.ndarray]]: list of test cases, where each test case is a dictionary with input name as key and a tensor as value rzExample inputs) rrseedr1rrMr$r7lenrCappend)r r rQr rRrSrr r+r&r,r- all_inputs _test_caseinput_1r9s rfake_test_datar[sD   IINN; KK JJ' %i_^..'* '<[Ve'fF## $ &:Rh'F?? # s:!+ "F +&!( rrUc f[UUUU UUUUUUU U 5 n [U 5U:wa [S5 U $)auCreate given number of input data for testing Args: batch_size (int): batch size sequence_length (int): sequence length test_cases (int): number of test cases seed (int): random seed verbose (bool): print more information or not input_ids (TensorProto): graph input of input IDs segment_ids (TensorProto): graph input of token type IDs input_mask (TensorProto): graph input of attention mask average_sequence_length (int): average sequence length excluding paddings random_sequence_length (bool): whether use uniform random number for sequence length mask_type (int): mask type 1 is mask index; 2 is 2D mask; 3 is key len, cumulated lengths of query and key Returns: List[Dict[str,numpy.ndarray]]: list of test cases, where each test case is a dictionary with input name as key and a tensor as value z$Failed to create test data for test.)r[rVrC) r r rQrUrRrr r+r&r,r-r rXs rgenerate_test_datar]sPB   J :*$ 45 rcU[UR5:agURUnURU5nUcBURX5nUb.URS:XaURURS5nU$)NCastr)rVinputfind_graph_input get_parentop_type) onnx_model embed_node input_indexr` graph_input parent_nodes rget_graph_input_from_embed_noderi$sc***++   [ )E--e4K ++JD  "{':':f'D$55k6G6G6JKK rrdinput_ids_namesegment_ids_nameinput_mask_namecUR5nUbURU5nUc[SU35eSnU(a"URU5nUc[SU35eSnU(a"URU5nUc[SU35eSU(aSOS-U(aSOS-n[U5U:wa[SUS[U535eXVU4$[U5S:wa[S[U535eUR S 5n [U 5S:XakU Sn [ X S5n[ X S5n[ X S 5nUc-UH'n U R R5n S U ;dM%U nM) Uc [S 5eXVU4$SnSnSnUH9n U R R5n S U ;aU nM'S U ;dSU ;aU nM7U nM; U(aU(a U(aXVU4$[S5e)aFind graph inputs for BERT model. First, we will deduce inputs from EmbedLayerNormalization node. If not found, we will guess the meaning of graph inputs based on naming. Args: onnx_model (OnnxModel): onnx model object input_ids_name (str, optional): Name of graph input for input IDs. Defaults to None. segment_ids_name (str, optional): Name of graph input for segment IDs. Defaults to None. input_mask_name (str, optional): Name of graph input for attention mask. Defaults to None. Raises: ValueError: Graph does not have input named of input_ids_name or segment_ids_name or input_mask_name ValueError: Expected graph input number does not match with specified input_ids_name, segment_ids_name and input_mask_name Returns: Tuple[Optional[np.ndarray], Optional[np.ndarray], Optional[np.ndarray]]: input tensors of input_ids, segment_ids and input_mask Nz Graph does not have input named r(rzExpect the graph to have z inputs. Got r/z'Expect the graph to have 3 inputs. Got EmbedLayerNormalizationmaskz#Failed to find attention mask inputtokensegmentz?Fail to assign 3 inputs. You might try rename the graph inputs.)'get_graph_inputs_excluding_initializersra ValueErrorrVget_nodes_by_op_typerirMlower) rdrjrkrl graph_inputsrr r+expected_inputs embed_nodesrer`input_name_lowers rfind_bert_inputsr{1s24EEGL!//?  ??OPQ Q $556FGK" #CDTCU!VWW #44_EJ! #COCT!UVVKqQ7 1PQR |  /88IWZ[gWhVijk kz11 <AB3|CTBUVWW112KLK ;1 ^ 3JAN 5jaP 4ZQO  %#(::#3#3#5 --!&J&  BC Cz11IKJ ::++- % %J ' '98H+HKI[Zz11 V WWr onnx_filec[5n[US5nURUR55 SSS5 [ U5n[ XaX#5$!,(df  N%=f)aFind graph inputs for BERT model. First, we will deduce inputs from EmbedLayerNormalization node. If not found, we will guess the meaning of graph inputs based on naming. Args: onnx_file (str): onnx model path input_ids_name (str, optional): Name of graph input for input IDs. Defaults to None. segment_ids_name (str, optional): Name of graph input for segment IDs. Defaults to None. input_mask_name (str, optional): Name of graph input for attention mask. Defaults to None. Returns: Tuple[Optional[np.ndarray], Optional[np.ndarray], Optional[np.ndarray]]: input tensors of input_ids, segment_ids and input_mask rbN)rrHParseFromStringreadrr{)r|rjrkrlmodelrOrds rget_bert_inputsrsS( LE i $ diik* 5!J J8H ZZ  s A A$c[R"5nURSS[SS9 URSS[SSS9 URS S[S S S9 URS S[S SS9 URSS[SSS9 URSS[SSS9 URSS[SSS9 URSS[S SS9 URSS[SSS9 URSSSSS9 UR SS9 URSSSS S9 UR SS!9 URS"S#S$[S%S&9 URS'S(SSS)S9 UR SS*9 URS+S[S,S-S9 UR 5nU$).Nz--modelTzbert onnx model path.)requiredrhelpz --output_dirFz4output test data path. Default is current directory.)rrdefaultrz --batch_sizer(zbatch size of inputz--sequence_lengthz maximum sequence length of inputz--input_ids_namezinput name for input idsz--segment_ids_namezinput name for segment idsz--input_mask_namezinput name for attention maskz --samplesz$number of test cases to be generatedz--seedr/z random seedz --verbose store_truezprint verbose information)ractionr)rRz--only_input_tensorsz-only save input tensors and no output tensors)only_input_tensorsz-az--average_sequence_lengthz)average sequence length excluding padding)rrrz-rz--random_sequence_lengthz3use uniform random instead of fixed sequence length)r,z --mask_typer)z^mask type: (1: mask index, 2: raw 2D mask, 3: key lengths, cumulated lengths of query and key))argparseArgumentParser add_argumentstrint set_defaults parse_args)parserargss rparse_argumentsrs'  $ $ &F  DsAXY   C  S!Rgh   /    '    )    ,    3  5sAMZ  (   &  <   51  #  8   " B  u5   m     D Krr output_dirrc [XX5upn[UUUUUUUUU U U 5 n[U5H>unn[RR US[ U5-5n[UU5 M@ U (agSSKnSUR5;aSS/OS/nURUUS9nUR5Vs/sHnURPM nn[U5Hunn[RR US[ U5-5nURUU5n[U5Hunn[R"[ R""UU5U5n[%[RR USUS35S 5nUR'UR)55 SSS5 M M gs snf!,(df  M=f) a Create test data for a model, and save test data to a directory. Args: model (str): path of ONNX bert model output_dir (str): output directory batch_size (int): batch size sequence_length (int): sequence length test_cases (int): number of test cases seed (int): random seed verbose (bool): whether print more information input_ids_name (str): graph input name of input_ids segment_ids_name (str): graph input name of segment_ids input_mask_name (str): graph input name of input_mask only_input_tensors (bool): only save input tensors, average_sequence_length (int): average sequence length excluding paddings random_sequence_length (bool): whether use uniform random number for sequence length mask_type(int): mask type test_data_set_NrCUDAExecutionProviderCPUExecutionProvider) providersoutput_r=r>)rr]rEr?r@rIrrP onnxruntimeget_available_providersInferenceSession get_outputsrMrunrrGrasarrayrHrJrK)rrr r rQrUrRrjrkrlrr&r,r-rr r+rXr3r9r8rrsessionoutput output_namesresult output_name tensor_resultrOs rcreate_and_save_test_datar sD*9P`)r&IJ#  Jz* 6GGLL-=A-FG F++ #k&I&I&K K !"89$ % **5I*FG.5.A.A.CD.CFFKK.CLDz* 6GGLL-=A-FG \62' 5NA{(33BJJvay4I;WMbggll9s#.>?F$ =::<=GF6+EGFs;G G G c[5nURS::aURUlURnUcY[ UR 5n[ RRURSURSUR35nUb[ U5nURSSS9 O [S5 [UR UURURURURUR UR"UR$UR&UR(URUR*UR,5 [SU5 g)Nrbatch__seq_T)parentsexist_okz7Directory existed. test data files will be overwritten.z Test data is saved to directory:)rr&r rrrr?r@rIparentr rBrCrsamplesrUrRrjrkrlrr,r-)rrpr@s rmainrYs  D ##q('+';';$J  WW\\!((fT__4EU4K_K_J`,ab J 4$ / GH           $$ ## " ,j9r__main__)r))i')NNN)!rr?rpathlibrnumpyronnxrrrrdrrndarrayrr$r*boolr7rdictrPr[r]ritupler{rrrr__name__rrrs 66 (+>ATWZZ<{VY^`^h^h6B3BB  FFFF! F ! F  FZZFR33T#rzz/-B3.7777 7  7  7777!7!77L!1111  1  1  111!1!111h "&#'"& YXYX$JYXDjYX4Z YX  2:: bjj4/d1B BC YX|"&#'"& [[$J[Dj[4Z [  2:: bjj4/d1B BC [8aHI> I>I>I> I>  I>  I>I>$JI>DjI>4ZI>I>!I>!I>I>X$:N zFr