U
hb @ sN d dl Z d dlZd dlZd dlZd dlZd dlmZ d dlmZ d dl m
Z
d dl Z ddl m
Z
mZmZmZmZ G dd dZG d d
d
Zd$e jeee
e d d
dZd%e
e dddZd&e
e dddZd'e
e dddZd(e je
ej dddZ ejdddZ d)ej ee
ej d d d Z d d Z d!d" Z!e"d#krJe! dS )* N)deque)IntEnum)Optional )get_producer_consumer_mapsis_fixed_size_tensor iterate_graph_per_graph_funciterate_graph_per_node_funcoptimize_modelc @ s( e Zd ZdZdd Zdd Zdd ZdS ) _SupportedOpsCheckerz
Class to process the md file with list of supported ops and caveats for an execution provider.
e.g. /tools/ci_build/github/android/nnapi_supported_ops.md
/tools/ci_build/github/apple/coreml_supported_ops.md
c C s || _ i | _t | _t|f}| D ]V}|dr&| d}t |dkr&|d }|d }|
dd}d|kr&|| j|< q&W 5 Q R X d S )N| r z
:) _filename_opsset _ops_seenopen readlines
startswithstripsplitlenreplace)selffilenameflinepieces domain_opcaveat r# V/tmp/pip-unpacked-wheel-socb9apf/onnxruntime/tools/mobile_helpers/usability_checker.py__init__ s
z _SupportedOpsChecker.__init__c C s< |j r |j nd}|d |j }|| jk}|r8| j| |S )Nai.onnxr )domainop_typer r add)r noder' r! is_supportedr# r# r$ is_op_supported3 s
z$_SupportedOpsChecker.is_op_supportedc C s: g }t | jD ]&}| j| }|r|| d| q|S )Nr )sortedr r append)r caveatsopr" r# r# r$ get_caveats= s
z _SupportedOpsChecker.get_caveatsN)__name__
__module__ __qualname____doc__r% r, r1 r# r# r# r$ r s
r c @ s> e Zd ZG dd deZdd Zdd Zeje ddd Z
d
S ) PartitioningInfoc @ s e Zd ZdZdZdZdS )zPartitioningInfo.TryWithEP)r )r r N)r2 r3 r4 NOMAYBEYESr# r# r# r$ TryWithEPH s r: c C s< d| _ d| _d| _d| _d | _g | _t | _d| _d| _ d S )N)
num_nodesnum_supported_nodesnum_partitionsnum_nodes_in_subgraphssupported_ops_checkersupported_groupsr unsupported_opsnodes_unsupported_due_to_op&nodes_unsupported_due_to_dynamic_input)r r# r# r$ r% M s zPartitioningInfo.__init__c C sv | j | j }| j| d }| jdkrL|dkr4tjjS |dkrDtjjS tjjS | jdkrn|dkrftjjS tjjS tjjS )Nd r K 2 r ) r< r? r= r> r6 r: r9 r8 r7 )r r< Z
pct_supportedr# r# r$ suitabilityX s
z PartitioningInfo.suitability)loggerep_namec
s | j | j }|| j d| j d| d| d | jrJ|| j d | jr|dddd | jD d
|d | j | jr|d | j | t
j kr| j r|d
dt
| j | j }|rd |dd fdd |D | j| d }| jdkr0|| d n| jdkr|dkr`|| d|dd n<|dkr|| d |dd n|| d |d d nP| jd!kr|dkr|| d"|dd# n || d$| j d%|dd& d'S )(z
Analyze the partitioning information and log the analysis
:param logger: Logger to use
:param ep_name: Execution provider name to use in the log messages
z partitions with a total of /z nodes can be handled by the z EP.z9 nodes are in subgraphs, which are currently not handled.zPartition sizes: [z, c S s g | ]}t t|qS r# )strr ).0 partitionr# r# r$
s z2PartitioningInfo.dump_analysis..]z"Unsupported nodes due to operator=z8Unsupported nodes due to input having a dynamic shape=%dzUnsupported ops: ,z zRCaveats that have not been checked and may result in a node not being supported: c s g | ]}t j | qS r# )oslinesep)rM r" indentr# r$ rO s rE r z$ cannot run any nodes in this model.r rF zD should work well for this model as there is one partition covering z.1fz % of the nodes in the model.rG z, may work well for this model, however only zD% of nodes will use it. Performance testing is required to validate.z4 will probably not work will for this model as only z.2fz% of nodes will use it.r zG can be considered for this model as there are two partitions covering z<% of the nodes. Performance testing is required to validate.z1 is not recommended with this model as there are z partitions covering zj% of the nodes in the model. This will most likely result in worse performance than just using the CPU EP.N)r< r? infor> r= rA joinrC rD getEffectiveLevelloggingDEBUGrB r- r@ r1 debug)r rI rJ r< r/ Zpct_nodes_using_epr# rU r$
dump_analysiss sZ $
z PartitioningInfo.dump_analysisN) r2 r3 r4 r r: r% rH rZ LoggerrL r] r# r# r# r$ r6 G s r6 F)graphr@ require_fixed_input_sizes
value_infoc s |rst dt| \}}dd | jD fdd dg i }t }t }| jD ]6} | |krpt|| nd}
|
|| < |
dkrX|| qXtjtjt gdd d
} dg} t
| | | | d g t d}
d}d}t }fd d
}|s|r8|s| |}t }q| } |
| }| p>t fdd| jD }|oH|}|s| krf|| q|s|| jr~| jnd d| j |d7 }n|d7 }|r|
d7 }
| | krΈ| | |kr|| D ]}| q| |kr|| D ]2}|| }|d8 }|dkr*|| |||< qq| t| |j
t| j}t}t }||_|
|_||_| d |_||_|_||_ ||_ ||_ |S )ah
Estimate the partitions the graph will be split into for nodes that is_node_supported_fn returns true for.
The check on whether a node is supported is purely based on the operator type. Additional limitations
(e.g. NNAPI EP only supports 2D Conv) are not checked, so partitions may not be 100% accurate. The limitations
for operators in the partitions are printed so the user can manually check.
:param graph: Graph to process
:param supported_ops_checker: Checker with info on supported ops.
:param require_fixed_input_sizes: If True, require that the inputs to a potentially supported node are
fixed size tensors for it to be considered as supported.
If True, onnx.shape_inference.infer_shapes should have been run on the model
to populate the shape information.
:param value_info: Map of value name to ValueInfoProto. Required if require_fixed_input_sizes is True to lookup
the shape of a value.
:return PartitioningInfo instance with details
zAvalue_info must be provided if require_fixed_input_sizes is True.c S s g | ]
}|j qS r# )namerM ir# r# r$ rO s z&check_partitioning..c s$ | krt | S | kr dS dS )NTF)r )value) initializersra r# r$ _is_fixed_shape_value s
z1check_partitioning.._is_fixed_shape_valueNr Z cur_graphoriginal_graphcountc S s" | |kr |d t | j7 < d S )Nr )r r* rh r# r# r$ _count_subgraph_nodes s z1check_partitioning.._count_subgraph_nodes)ri rj c s8 r4 p } | r$ d S N)r. copyclear)Zkeep_partition)on_group_closed_fnsupported_groupsupported_group_borderrA r# r$ close_group s z'check_partitioning..close_groupc 3 s | ]} |V qd S rl r# rc )rg r# r$ ( s z%check_partitioning..r& r r )
ValueErrorr Z initializerr r* r r. onnx
GraphProtointr r popleftr, allinputr) r' r( remover r6 r< r= r> r? r@ rA rB rC rD )r_ r@ r` ra Znode_to_producersZnode_to_consumersZ in_degreeZnodes_to_processZ nodes_to_process_with_next_groupr* Znode_input_edge_countrk Znodes_in_subgraphsr= Z num_unsupported_nodes_due_to_opZ*num_unsupported_nodes_due_to_dynamic_inputrB rr r, Zis_input_shape_supportedZis_node_supportedZconsumerZconsumer_node_in_degreer< r> rW r# )rg rf ro rp rq rA ra r$ check_partitioning s
$
r| )ra c C s t |}t| j||d k |}|S rl )r r| r_ )modelZsupported_ops_configra Z
supported_opspartition_infor# r# r$ _check_ep_partitioningh s r c C sP t tj}|d }| r"|}n"|jd }|d d d d d }t| ||S )Nznnapi_supported_ops.md toolsci_buildgithubZandroidpathlibPath__file__parentexistsparentsr r} ra Z
script_dirZ local_configZ config_pathZort_rootr# r# r$ check_nnapi_partitionsn s
r c C sP t tj}|d }| r"|}n"|jd }|d d d d d }t| ||S )Nzcoreml_supported_ops.mdr r r r Zappler r r# r# r$ check_coreml_partitions| s
r )r_ rI c C sR d}d}g }| j D ]J}t|sT|| |rJ|ddt| |d7 }q|d7 }qg }| jD ]J}t|s|| |r|ddt| |d7 }qh|d7 }qh| jst | j
dkst |t | j ks| d | jD ] }t|r|d7 }q|d7 }q|rJ|d| d| |rJ|r@|d n
|d
||fS ) a
Check the shapes of graph inputs, values and graph outputs to determine if they have static or dynamic sizes.
NNAPI and CoreML do not support dynamically sized values.
:param graph: Graph to check. If shape inferencing has been run the checks on values will be meaningful.
:param logger: Optional logger for diagnostic information.
:return: Tuple of List of inputs with dynamic shapes, Number of dynamic values found
r z"Input is not a fixed size tensor: r r z#Output is not a fixed size tensor: ziUnable to check shapes within model. ONNX shape inferencing should be run on the model prior to checking.z Num values with fixed shape=z . Num values with dynamic shape=zModel has dynamic inputs and outputs. Consider re-exporting model with fixed sizes if NNAPI or CoreML can be used with this model.ao Model has dynamically sized inputs but fixed sized outputs.
If the sizes become fixed early in the model (e.g. pre-processing of a dynamic input size
results in a fixed input size for the majority of the model) performance with NNAPI and CoreML,
if applicable, should not be significantly impacted.) rz r r. rW rX rL r outputra r r* warning) r_ rI Znum_fixed_valuesnum_dynamic_valuesdynamic_inputsrd Zdynamic_outputsovir# r# r$ check_shapes sN
&
r )rI c s t | }t j|i jjD ]}||j< q"jjD ]}||j< q:jjD ]}||j< qRt j\ } fdd}|dt
}|dt }|t j
jks|t j
jkrƈ tjkrƈd d |t j
jkp|t j
jkS )Nc s d| |} tjkr4|| | } d| d|j |tjj kr r d |} tjkr d || | } d| d|j |tjj
krЈ d |j |j kr|}|S )N Checking z Model should perform well with z as is: zHChecking if model will perform better if the dynamic shapes are fixed...zHPartition information if the model was updated to make the shapes fixed:z) if modified to have fixed input shapes: zPShapes can be altered using python -m onnxruntime.tools.make_dynamic_shape_fixed) rW rY rZ r[ r] rH rb r6 r: r9 r7 re )rJ Z checker_funcr~ rH Z partition_info_with_fixed_shapesZfixed_shape_suitabilityr rI Zmodel_with_shape_infoZvalue_to_shaper# r$ check_ep s*
zchecker..check_epZNNAPIZCoreMLzKRe-run with log level of DEBUG for more details on the NNAPI/CoreML issues.z---------------)ru loadZshape_inferenceZ infer_shapesr_ rz rb r ra r r r r6 r: r9 rY rZ r[ rW r7 )
model_pathrI r} vr r Znnapi_suitabilityZcoreml_suitabilityr# r r$ checker s,
r )r
skip_optimizerI c C s| |st d}|t j |d| d t >}|sXt|| j }t
| | |} t t | j
dd|}W 5 Q R X |S )a
Analyze the provided model to determine if it's likely to work well with the NNAPI or CoreML Execution Providers
:param model_path: Model to analyze.
:param skip_optimize: Skip optimizing to BASIC level before checking. When exporting to ORT format we will do this
optimization..
:param logger: Logger for output
:return: True if either the NNAPI or CoreML Execution Providers may work well with this model.
Zusability_checkerr z for usability with ORT Mobile.T)strict)rZ getLoggersetLevelINFOrW tempfileTemporaryDirectoryr r rb r
r rL resolve)r r rI tmpZtmp_pathZtry_epsr# r# r$
analyze_model s
r c C sZ t jtjtdd} | jdddddgddd | jd
d d d
| jdtjdd | S )Nz3Analyze an ONNX model for usage with the ORT mobile) descriptionz --log_levelr\ rW r errorz
Logging level)choicesdefaulthelpz--skip_optimize
store_truezDon't optimize the model to BASIC level prior to analyzing. Optimization will occur when exporting the model to ORT format, so in general should not be skipped unless you have a specific reason to do so.)actionr r zProvide path to ONNX model)typer )
argparseArgumentParserrS pathbasenamer add_argumentr r
parse_args)parserr# r# r$ r ( s"
r c C s t } td}| jdkr(|tj n<| jdkr@|tj n$| jdkrX|tj n |tj | j
}t || j | d S )Nr r\ rW r )
r rZ r Z log_levelr r[ r WARNINGERRORr r r r )argsrI r r# r# r$ run_analyze_model< s
r __main__)FN)N)N)N)N)FN)#r rZ rS r r collectionsr enumr typingr ru Zonnx_model_utilsr r r r r
r r6 rv booldictr| r r r r^ r r r r r r r2 r# r# r# r$ s< /x -I;