U “±Ëh¾ ã@s`ddlZddlZddlZddlZddlmZe e¡Zdd„Z dd„Z dd„Z Gd d „d ƒZ dS) éN)ÚConv1DcCs<|jj\}}tj ||¡}|jjj ¡|j_|jj|j_|S)N) ZweightÚshapeÚtorchÚnnÚLinearÚdataÚTÚ contiguousZbias)ÚmoduleZin_sizeZout_sizeÚlinear©r úL/tmp/pip-unpacked-wheel-socb9apf/onnxruntime/transformers/quantize_helper.pyÚ_conv1d_to_linears   rcCsNt d¡t|jƒD]4}|j|}t|tƒr@t|ƒ}||j|<qt|ƒqdS)zsin-place This is for Dynamic Quantization, as Conv1D is not recognized by PyTorch, convert it to nn.Linear zreplace Conv1D with LinearN)ÚloggerÚdebugÚlistZ_modulesÚ isinstancerrÚconv1d_to_linear)ÚmodelÚnamer r r r r rs    rcCs.t | ¡d¡tj d¡d}t d¡|S)Nztemp.pé)rÚsaveZ state_dictÚosÚpathÚgetsizeÚremove)rÚsizer r r Ú_get_size_of_pytorch_model's rc@s,eZdZeejfdd„ƒZeddd„ƒZdS)ÚQuantizeHelpercCsLt|ƒtjj|tjjh|d}t dt|ƒ›¡t dt|ƒ›¡|S)z{ Usage: model = quantize_model(model) TODO: mix of in-place and return, but results are different )Údtypez'Size of full precision Torch model(MB):z"Size of quantized Torch model(MB):) rrZ quantizationÚquantize_dynamicrrrÚinfor)rrZquantized_modelr r r Úquantize_torch_model/s z#QuantizeHelper.quantize_torch_modelFcCs†ddlm}ddlm}||ƒjjdddt dtj   |¡d›¡||||dt d |›¡t d tj   |¡d›¡dS) Nr)ÚPath)r T)ÚparentsÚexist_okz&Size of full precision ONNX model(MB):r)Úuse_external_data_formatzquantized model saved to:z!Size of quantized ONNX model(MB):) Úpathlibr#Zonnxruntime.quantizationr ÚparentÚmkdirrr!rrr)Zonnx_model_pathZquantized_model_pathr&r#r r r r Úquantize_onnx_model<s  ýz"QuantizeHelper.quantize_onnx_modelN)F)Ú__name__Ú __module__Ú __qualname__Ú staticmethodrZqint8r"r*r r r r r.s r) ÚloggingrZonnxrZtransformers.modeling_utilsrÚ getLoggerr+rrrrrr r r r Ús