U qh(-@s UddlmZddlZddlmZmZmZddlmZm Z m Z ddl m Z ddl mZddlmZdd d d d Zd d ddddZgaded<gaded<dd d ddddZdd d ddddZdd d ddddZd ddddZddd d!Zed"kredS)#) annotationsN)IOAnySequence)AttributeProtodefsload)collect_snippets)load_model_tests)RunnerzSequence[defs.OpSchema]bool)schemasreturncCstdd|DS)Ncss|]}|jdkVqdS ai.onnx.mlN)domain.0srC/tmp/pip-unpacked-wheel-xnis5xre/onnx/backend/test/stat_coverage.py szis_ml..)any)r rrris_mlsrzIO[Any]None)fmlrcCsP|d|r|dn |d|d|d|d|ddS)Nz# Test Coverage Reportz (ONNX-ML Operators) z (ONNX Core Operators) z ## Outlines z,* [Node Test Coverage](#node-test-coverage) z.* [Model Test Coverage](#model-test-coverage) z2* [Overall Test Coverage](#overall-test-coverage) write)rrrrr gen_outliness      rz Sequence[str]common_coveredexperimental_covered)r rrrc stdddddhttfdd|Datfdd|D}tfd d|D}tfd d|Datfd d|D}tfd d|D}ttt|t|}ttt|t|}|d |d|r.|dttd|dttt|dddt|d n |d|rn|dtt|ttt|dt|n |dddddg} t|t|g} | D]:} |d| ddd| dd d d!d"q|d#t | | D]\} } |d$| d#| D]} |d%| | kr|d&t| d't| D]B\}}|d(|d)|d*|d+|d,|d-qFn | kr|d.n |d/|d0q|d1qdS)2NZ MultinomialZ RandomNormalZRandomNormalLikeZ RandomUniformZRandomUniformLikec3s<|]4}|jkr|jtjjjkr|jdkkr|jVqdSrname support_levelrOpSchema SupportTypeZCOMMONrrr node_testsrrr4s  z)gen_node_test_coverage..c3s<|]4}|jkr|jtjjjkr|jdkkr|jVqdSrr"rr'rrr;s  c3s|]}|kr|VqdSNrrr# generatorsrrrBsc3s<|]4}|jkr|jtjjjkr|jdkkr|jVqdSrr#r$rr%r&Z EXPERIMENTALrrr'rrrCs  c3s<|]4}|jkr|jtjjjkr|jdkkr|jVqdSrr-rr'rrrJs  c3s|]}|kr|VqdSr)rr*r+rrrQsz# Node Test Coverage z ## Summary zNode tests have covered /z (dz.2fz%, z) generators excluded) common operators. z6Node tests have covered 0/0 (N/A) common operators. zYNode tests have covered {}/{} ({:.2f}%, {} generators excluded) experimental operators. z;Node tests have covered 0/0 (N/A) experimental operators. z!💚Covered Common Operatorsz"💔No Cover Common Operatorsz'💚Covered Experimental Operatorsz(💔No Cover Experimental Operatorsz* [ z](# -z)  ## z### z There are z" test cases, listed as following:
z z ```python z ```
z (random generator operator) z (call for test cases) z z
) setr sortedr r!lenrfloatformatlowerreplacezip)r rrZcommon_no_coverZcommon_generatorZexperimental_no_coverZexperimental_generatorZ num_commonZnum_experimentaltitlesZ all_liststlrsummarycoder)r,rr(rgen_node_test_coverage%s    6  8      rEcCsH|di}|D]}|||j<qi}g}tddD]|}|jdrtjtjtj t ddd|j}tj |st d|d| |q4t|} | tj| dq4|d } |D]h} t| } |rd } | jD]}|jd krd } q| sqnd } |d | jjd d}| jjD]}|jtks>|jtkr |d7}|jD]}|j|krhi||j<|j||jkrg||j|j<|jtjkr|j||j|jkr||j|j |jqL|jtjkr|j||j|jkr||j|j |jqL|jtj krP|j!||j|jkr||j|j |j!qL|jtj"kr|j#||j|jkr||j|j |j#qL|jtj$kr|j%||j|jkr||j|j |j%qL|jtj&kr|j'||j|jkr||j|j |j'n|jtj(krT|j)||j|jkr||j|j |j)n|jtj*kr|j+||j|jkr||j|j |j+n~|jtj,kr|j-||j|jkr||j|j |j-n>|jtj.krL|j/||j|jkrL||j|j |j/qLq |d | jjd|dt0| jjddt1|t1t0| jjd |d|dt2|D]}|d|d|dt0||dt0||j3dt2||j3D]H}|||kr||dt0|||d n||dq|dq~|dq| sD|rD|ddS)Nz# Model Test Coverage real)kindzonnx/backend/test/data/light/z..zUnable to find model .z model.onnxFrTr4r3rz has z nodes. Of these, z are covered by node tests (gY@z%) r5znodes r6z: z out of z attributes covered z: 0 r7z z+No model tests present for selected domain )4rr#r url startswithospathnormpathjoindirname__file__existsFileNotFoundErrorappendr Zprepare_model_datasortrZ opset_importrgraphnodeZop_typer r! attributetyperFLOATrINTiSTRINGrZTENSORrAZGRAPHgZFLOATSZfloatsZINTSZintsZSTRINGSstringsZTENSORSZtensorsZGRAPHSZgraphsr:r;r9 attributes)r rrZ schema_dictZschemaattrsZ model_pathsrtZ model_nameZ model_dirZ model_writtenZ model_pb_pathmodelZ ml_presentZopsetZ num_coveredrWattroprXrrrgen_model_test_coverages             @    ($  rfcCs|d|ddS)Nz# Overall Test Coverage z## To be filled. r)r rrrrrgen_overall_test_coverages rg)rrcCs|ddS)Nz. r)rrrrgen_spdxsrh)rc Cstjtjtjtjtjt}tj|d}t}t|}tj|d}t |dddd<}t |t |dt ||dt ||dt||dW5QRX|rtj|d}t |dddd<}t |t |d t ||d t ||d t||d W5QRXdS) NZdocszTestCoverage.mdzw+zutf-8)newlineencodingFzTestCoverage-ml.mdT)rLrMrPrealpathrQrOrZget_all_schemasropenrhrrErfrg)base_dirZdocs_dirr Zhas_mlfnamerrrrmain s*"      rp__main__) __future__rrLtypingrrrZonnxrrrZonnx.backend.test.caser Zonnx.backend.test.loaderr Zonnx.backend.test.runnerr rrr __annotations__r!rErfrgrhrp__name__rrrrs$      pi