模型可用性檢查器

模型可用性檢查器會分析 ONNX 模型，評估其與 ORT Mobile、NNAPI 和 CoreML 一起使用的適用性。

目錄

• 用法
• 與 NNAPI 和 CoreML 一起使用
• 建議

用法

python -m onnxruntime.tools.check_onnx_model_mobile_usability --help
usage: check_onnx_model_mobile_usability.py [-h] [--log_level {debug,info}] model_path

Analyze an ONNX model to determine how well it will work in mobile scenarios.

positional arguments:
  model_path            Path to ONNX model to check

optional arguments:
  -h, --help            show this help message and exit
  --log_level {debug,info}
                        Logging level (default: info)

與 NNAPI 和 CoreML 一起使用

該指令碼將檢查模型中的運算子是否受 ORT 的 NNAPI 執行提供程式 (EP) 和 CoreML EP 支援。根據支援的運算子數量及其在模型中的位置，它將評估使用 NNAPI 或 CoreML 是否可能帶來益處。始終建議進行效能測試以進行驗證。

此檢查的示例輸出如下：

INFO:  Checking resnet50-v1-7.onnx for usability with ORT Mobile.
INFO:  Checking NNAPI
INFO:  1 partitions with a total of 121/122 nodes can be handled by the NNAPI EP.
INFO:   Partition sizes: [121]
INFO:  Unsupported nodes due to operator=0
INFO:   Caveats that have not been checked and may result in a node not actually being supported:
     ai.onnx:Conv:Only 2D Conv is supported. Weights and bias should be constant.
     ai.onnx:Gemm:If input B is not constant, transB should be 1.
     ai.onnx:GlobalAveragePool:Only 2D Pool is supported.
     ai.onnx:MaxPool:Only 2D Pool is supported.
INFO:  Unsupported nodes due to input having a dynamic shape=1
INFO:  NNAPI should work well for this model as there is one partition covering 99.2% of the nodes in the model.
INFO:  Model should perform well with NNAPI as is: YES

如果模型具有動態輸入形狀，則會進行額外檢查，以評估將形狀設定為固定大小是否有助於效能。有關更多資訊，請參閱 onnxruntime.tools.make_dynamic_shape_fixed。

此檢查的示例輸出：

INFO:  Checking resnet50-v1-7.onnx for usability with ORT Mobile.
...
INFO:  Checking CoreML MLProgram
INFO:  2 partitions with a total of 120/122 nodes can be handled by the CoreML MLProgram EP.
INFO:   Partition sizes: [119, 1]
INFO:  Unsupported nodes due to operator=1
INFO:   Unsupported ops: ai.onnx:Flatten
INFO:   Caveats that have not been checked and may result in a node not actually being supported:
     ai.onnx:Conv:Only 1D/2D Conv is supported. Bias if provided must be constant.
     ai.onnx:Gemm:Input B must be constant.
     ai.onnx:GlobalAveragePool:Only 2D Pool is supported currently. 3D and 5D support can be added if needed.
     ai.onnx:MaxPool:Only 2D Pool is supported currently. 3D and 5D support can be added if needed.
INFO:  Unsupported nodes due to input having a dynamic shape=1
INFO:  CoreML MLProgram can be considered for this model as there are two partitions covering 98.4% of the nodes. Performance testing is required to validate.
INFO:  Model should perform well with CoreML MLProgram as is: MAYBE
INFO:  --------
INFO:  Checking if model will perform better if the dynamic shapes are fixed...
INFO:  Partition information if the model was updated to make the shapes fixed:
INFO:  2 partitions with a total of 121/122 nodes can be handled by the CoreML MLProgram EP.
INFO:   Partition sizes: [120, 1]
INFO:  Unsupported nodes due to operator=1
INFO:   Unsupported ops: ai.onnx:Flatten
INFO:   Caveats that have not been checked and may result in a node not actually being supported:
     ai.onnx:Conv:Only 1D/2D Conv is supported. Bias if provided must be constant.
     ai.onnx:Gemm:Input B must be constant.
     ai.onnx:GlobalAveragePool:Only 2D Pool is supported currently. 3D and 5D support can be added if needed.
     ai.onnx:MaxPool:Only 2D Pool is supported currently. 3D and 5D support can be added if needed.
INFO:  CoreML MLProgram can be considered for this model as there are two partitions covering 99.2% of the nodes. Performance testing is required to validate.
INFO:  Model should perform well with CoreML MLProgram if modified to have fixed input shapes: MAYBE
INFO:  Shapes can be altered using python -m onnxruntime.tools.make_dynamic_shape_fixed

診斷輸出提供了有關提出這些建議的深入資訊。

這包括：

• 有關 NNAPI 和 CoreML EP 支援或不支援的單個運算子的資訊
• 有關支援的運算子被分成多少組（也稱為分割槽）的資訊
  • 組越多，效能越差，因為每次在支援和不支援的節點組之間切換時，我們都必須在 NPU（神經網路處理單元）和 CPU 之間切換。

建議

最後，指令碼將提供使用哪個 EP 的建議。

INFO:  As NNAPI or CoreML may provide benefits with this model it is recommended to compare the performance of the model using the NNAPI EP on Android, and the CoreML EP on iOS, against the performance using the CPU EP.