h3SSKrSSKrSSKrSSKrSSKJr SSjrSrSrSr Sr SSjr S r S r S r\S :Xa\"5r\"S \5 SSKJr \"\R(5 \R*(d!\R,(dS5e\"\5rO \R*rSSKJr \"\\5r\H r\"\5 M gg)N) TensorProtoc [R"5nURSSS[SS9 URSSS[SS9 URS S S[S S S 9 URSSS[SSS 9 URSS[S SS 9 URSS[S SS 9 URSS[SSS 9 URSS[ SSS 9 URSS[SSS 9 URSS[SS S 9 URS!S[SS"S 9 URS#S[SS$S 9 URS%SS&/S'QS(S)9 URS*S+SS,S-S.9 UR SS/9 URS0S[S1S2S 9 URS3SS,S4S.9 UR SS59 URS6SS,S7S.9 UR SS89 URS9S:SS,S;9 UR SS<9 URU5$)=Nz-iz--inputFz2Set the input file for reading the profile results)requiredtypehelpz-mz--modelzIonnx model path to run profiling. Required when --input is not specified.z-bz --batch_sizezbatch size of input)rrdefaultrz-sz--sequence_length zsequence length of inputz--past_sequence_lengthzpast sequence length for gpt2z--global_lengthz&number of global tokens for longformerz --samplesiz\number of samples to test. Set it large enough to reduce the variance of performance result.z --thresholdg{Gz?zfThreshold of run time ratio among all nodes. Nodes with larger ratio will show in top expensive nodes.z --thread_numznumber of threads to usez--input_ids_namez"input name for input IDs, for bertz--segment_ids_namez$input name for segment IDs, for bertz--input_mask_namez'input name for attention mask, for bertz--dummy_inputsr )bertgpt2 longformerr zEType of model inputs. The default will create dummy inputs with ones.)rr choicesrz-gz --use_gpu store_truezuse GPU)ractionr)use_gpuz --providercudazExecution provider to usez--basic_optimizationz_Enable only basic graph optimizations. By default, all optimizations are enabled in OnnxRuntime)basic_optimizationz--kernel_time_onlyz.Only include the kernel time and no fence time)kernel_time_onlyz-vz --verbose)rr)verbose)argparseArgumentParser add_argumentstrintfloat set_defaults parse_args)argvparsers [/var/www/fran/franai/venv/lib/python3.13/site-packages/onnxruntime/transformers/profiler.pyparse_argumentsr"s  $ $ &F    A     X     "     '     ,    5    k    u    '    1    3    6  9 T  kE,U^_ &   (   n   51  =   / kE,O &   T ""c SSKJn U"UUUU(+USS9nUHnURSU5n M UR5n U $)Nr)create_onnxruntime_sessionT)enable_all_optimization num_threadsenable_profiling)benchmark_helperr%run end_profiling) onnx_model_pathrproviderr thread_num all_inputsr%sessioninputs_ profile_files r! run_profiler4sV;($6 6 G KKf %((*L r#c~[URS55[:Xa[XRS55$S$)Nvalue)r WhichOneofrgetattr)dims r!get_dim_from_type_protor:s2489P4QUX4X73w/ 0b^bbr#cURRRVs/sHn[U5PM sn$s snfN) tensor_typeshaper9r:) type_protods r!get_shape_from_type_protorAs40:0F0F0L0L0P0P Q0P1 #A &0P QQ Qs;c80nUR5GHcn[UR5n/n[U5H-up[ U [ 5(dMUR U5 M/ [U5S:a g[U5S:aXUS'[U5S:aX&US'URRRn U [R[R[R4;deU [R:Xa[RO3U [R:Xa[R O[R"n [R$"XkS9n XUR&'GMf [)U5V s/sHoPM nn U$s sn f)zCreate dummy inputs for ONNX model. Args: onnx_model (OnnxModel): ONNX model batch_size (int): batch size sequence_length (int): sequence length samples (int): number of samples Returns: List[Dict]: list of inputs Nrrdtype)'get_graph_inputs_excluding_initializersrAr enumerate isinstancerappendlenr= elem_typerFLOATINT32INT64numpyfloat32int64int32onesnamerange) onnx_model batch_sizesequence_lengthsamples dummy_inputs graph_inputr> symbol_dimsir9rK data_typedatar2r/s r!create_dummy_inputsr`s[L!IIK )+*:*:; &FA#s##""1%' { a  { a $.+a. ! { a $3+a. !$$00:: [.. 0A0A;CTCTUUUUK--- MM!*k.?.?!?%++U[[  zz%1)-[%%&/L2).g71,J7 8s Fc JSSKJnJn U"XXV5upn U"UUUSSU U U SS9 n U $)aCreate dummy inputs for BERT model. Args: onnx_model (OnnxModel): ONNX model batch_size (int): batch size sequence_length (int): sequence length samples (int): number of samples input_ids_name (str, optional): Name of graph input for input IDs. Defaults to None. segment_ids_name (str, optional): Name of graph input for segment IDs. Defaults to None. input_mask_name (str, optional): Name of graph input for attention mask. Defaults to None. Returns: List[Dict]: list of inputs r)find_bert_inputsgenerate_test_data{F) test_casesseedr input_ids segment_ids input_maskrandom_mask_length)bert_test_datarbrc) rVrWrXrYinput_ids_namesegment_ids_nameinput_mask_namerbrcrgrhrir/s r!create_bert_inputsrosG.D)9*Vf)x&IJ#   J r#cUUUX#-S.n0nUR5GH,n[UR5n[U5H5up[ U [ 5(dMX;a[ SU 35eXZX'M7 URRRn U [R[R[R4;deU [R:Xa[RO3U [R:Xa[RO[R n [R""XS9n XUR$'GM/ ['U5Vs/sHoPM nnU$s snf)aCreate dummy inputs for GPT-2 model. Args: onnx_model (OnnxModel): ONNX model batch_size (int): batch size sequence_length (int): sequence length past_sequence_length (int): past sequence length samples (int): number of samples Raises: RuntimeError: symbolic is not supported. Use the tool convert_to_onnx.py to export ONNX model instead. Returns: List[Dict]: list of inputs )rWseq_len past_seq_len total_seq_lensymbol is not supported: rD)rFrArrGrHr RuntimeErrorr=rKrrLrMrNrOrPrQrRrSrTrU)rVrWrXpast_sequence_lengthrYsymbolsrZr[r>r]r9rKr^r_r2r/s r!create_gpt2_inputsrx s@$!",(? GL!IIK )+*:*:;&FA#s##%&)B3%'HII&|EH ' $$00:: [.. 0A0A;CTCTUUUUK--- MM!*k.?.?!?%++U[[  zz%1)-[%%&#L&).g71,J7 8s E)c2XS.n0nUR5GH\n[UR5n[U5H5up[ U [ 5(dMX;a[ SU 35eXZX'M7 URRRn U [R[R[R4;deU [R:Xa[RO3U [R:Xa[RO[R n SUR";a [R$"XS9n SU SS2SU24'O[R&"XS9n XUR"'GM_ [)U5Vs/sHoPM nnU$s snf)aCreate dummy inputs for Longformer model. Args: onnx_model (OnnxModel): ONNX model batch_size (int): batch size sequence_length (int): sequence length global_length (int): number of global tokens samples (int): number of samples Raises: RuntimeError: symbolic is not supported. Use the tool convert_longformer_to_onnx.py to export ONNX model instead. Returns: List[Dict]: list of inputs )rWrXrtglobalrDrN)rFrArrGrHrrur=rKrrLrMrNrOrPrQrRrTzerosrSrU)rVrWrX global_lengthrYrwrZr[r>r]r9rKr^r_r2r/s r!create_longformer_inputsr};s_ (LGL!IIK )+*:*:;&FA#s##%&)B3%'HII&|EH ' $$00:: [.. 0A0A;CTCTUUUUK--- MM!*k.?.?!?%++U[[  {'' ';;u6D&'DN]N" #::e5D)-[%%&-L0).g71,J7 8s Fc URS:a URO[R"SS9nS[R;a[ U5[RS'SSKJn SSKJ n U"U"UR55nSnURS:XaN[UURURURUR UR"UR$5nOURS:Xa8['UURURUR(UR5nOsURS :Xa8[+UURURUR,UR5nO+[/X@RURUR5n[1URUR2UR4UR6URU5nU$) NrF)logicalOMP_NUM_THREADS)load) OnnxModelr r r)r.psutil cpu_countosenvironronnxrrVrmodelrZrorWrXrYrlrmrnrxrvr}r|r`r4rr-r)argsr'rrrVr/r3s r!r*r*js%)__q%8$//f>N>NW\>]K *(+K(8 $%$4 +,JJ F"'  OO  LL     ! !      f $'  OO   % % LL     l *-  OO     LL  )__dFZFZ\`\h\hi       L r#__main__ Arguments) setup_loggerzMrequires either --model to run profiling or --input to read profiling results)process_resultsr<)NNN)rrrOrrrr"r4r:rAr`rorxr}r*__name__ argumentsprintr)rrinputrr3profile_result_processorrresultsliner#r!rs I#X&cR'^&R-`,^3l z!I +y!-""# ??o oo9~   8lI6G d #r#