-
80-NL315-14 A
MAY CONTAIN U.S. AND INTERNATIONAL EXPORT CONTROLLED INFORMATION
|
Snapdragon Neural Processing Engine SDK
Reference Guide
|
This chapter describes the various SDK tools and features.
snpe-net-run loads a DLC file, loads the data for the input tensor(s), and executes the network on the specified runtime.
DESCRIPTION:
------------
Example application demonstrating how to load and execute a neural network
using the SNPE C++ API.
REQUIRED ARGUMENTS:
-------------------
--container <FILE> Path to the DL container containing the network.
--input_list <FILE> Path to a file listing the inputs for the network.
OPTIONAL ARGUMENTS:
-------------------
--use_gpu Use the GPU runtime for SNPE.
--use_dsp Use the DSP fixed point runtime for SNPE.
--use_aip Use the AIP fixed point runtime for SNPE.
--debug Specifies that output from all layers of the network
will be saved.
--output_dir <DIR> The directory to save output to. Defaults to ./output
--storage_dir <DIR> The directory to store SNPE metadata files
--encoding_type <VAL> Specifies the encoding type of input file. Valid settings are "nv21".
Cannot be combined with --userbuffer*.
--userbuffer_float [EXPERIMENTAL] Specifies to use userbuffer for inference, and the input type is float.
Cannot be combined with --encoding_type.
--userbuffer_tf8 [EXPERIMENTAL] Specifies to use userbuffer for inference, and the input type is tf8exact0.
Cannot be combined with --encoding_type.
--perf_profile <VAL> Specifies perf profile to set. Valid settings are "system_settings" , "power_saver" , "balanced" ,
"default" , "high_performance" , "sustained_high_performance" , and "burst".
NOTE: "balanced" and "default" are the same. "default" is being deprecated in the future.
--profiling_level <VAL>
Specifies the profiling level. Valid settings are "off", "basic", "moderate" and "detailed".
Default is detailed.
--enable_cpu_fallback Enables cpu fallback functionality. Defaults to disable mode.
--input_name <INPUT_NAME>
Specifies the name of input for which dimensions are specified.
--input_dimensions <INPUT_DIM>
Specifies new dimensions for input whose name is specified in input_name. e.g. "1,224,224,3".
For multiple inputs, specify --input_name and --input_dimensions multiple times.
--gpu_mode <VAL> Specifies gpu operation mode. Valid settings are "default", "float16".
default = float32 math and float16 storage (equiv. use_gpu arg).
float16 = float16 math and float16 storage.
--udo_package_path <VAL,VAL>
Path to UDO package with registration library for UDOs.
Optionally, user can provide multiple packages as a comma-separated list.
--help Show this help message.
--version Show SNPE Version Number.This binary outputs raw output tensors into the output folder by default. Examples of using snpe-net-run can be found in Running AlexNet tutorial.
Additional details:
snpe-net-run is able to automatically batch the input data. The batch size is indicated in the model container (DLC file) but can also be set using the "input_dimensions" argument passed to snpe-net-run. Users do not need to batch their input data. If the input data is not batch, the input size needs to be a multiple of the size of the input data files. snpe-net-run would group the provided inputs into batches and pad the incomplete batches (if present) with zeros.
In the example below, the model is set to accept batches of three inputs. So, the inputs are automatically grouped together to form batches by snpe-net-run and padding is done to the final batch. Note that there are five output files generated by snpe-net-run:
…
Processing DNN input(s):
cropped/notice_sign.raw
cropped/trash_bin.raw
cropped/plastic_cup.raw
Processing DNN input(s):
cropped/handicap_sign.raw
cropped/chairs.raw
Applying padding snpe-net-run can take multiple input files as input data per iteration, and specify multiple output names, in an input list file formated as below:
#<output_name>[<space><output_name>]
<input_layer_name>:=<input_layer_path>[<space><input_layer_name>:=<input_layer_path>]
…The first line starting with a "#" specifies the output layers' names. If there is more than one output, a whitespace should be used as a delimiter. Following the first line, you can use multiple lines to supply input files, one line per iteration, and each line only supply one layer.If there is more than one input per line, a whitespace should be used as a delimiter.
Here is an example, where the layer names are "Input_1" and "Input_2", and inputs are located in the path "Placeholder_1/real_input_inputs_1/". Its input list file should look like this:
#Output_1 Output_2
Input_1:=Placeholder_1/real_input_inputs_1/0-0#e6fb51.rawtensor Input_2:=Placeholder_1/real_input_inputs_1/0-1#8a171b.rawtensor
Input_1:=Placeholder_1/real_input_inputs_1/1-0#67c965.rawtensor Input_2:=Placeholder_1/real_input_inputs_1/1-1#54f1ff.rawtensor
Input_1:=Placeholder_1/real_input_inputs_1/2-0#b42dc6.rawtensor Input_2:=Placeholder_1/real_input_inputs_1/2-1#346a0e.rawtensorNote: If the batch dimension of the model is greater than 1, the number of batch elements in the input file has to either match the batch dimension specified in the DLC or it has to be one. In the latter case, snpe-net-run will combine multiple lines into a single input tensor.
python script snpe_bench.py runs a DLC neural network and collects benchmark performance information.
usage: snpe_bench.py [-h] -c CONFIG_FILE [-o OUTPUT_BASE_DIR_OVERRIDE]
[-v DEVICE_ID_OVERRIDE] [-r HOST_NAME] [-a]
[-t DEVICE_OS_TYPE_OVERRIDE] [-d] [-s SLEEP]
[-b USERBUFFER_MODE] [-p PERFPROFILE] [-l PROFILINGLEVEL]
[-json] [-cache]
Run the snpe_bench
required arguments:
-c CONFIG_FILE, --config_file CONFIG_FILE
Path to a valid config file
Refer to sample config file config_help.json for more
detail on how to fill params in config file
optional arguments:
-o OUTPUT_BASE_DIR_OVERRIDE, --output_base_dir_override OUTPUT_BASE_DIR_OVERRIDE
Sets the output base directory.
-v DEVICE_ID_OVERRIDE, --device_id_override DEVICE_ID_OVERRIDE
Use this device ID instead of the one supplied in config
file. Cannot be used with -a
-r HOST_NAME, --host_name HOST_NAME
Hostname/IP of remote machine to which devices are
connected.
-a, --run_on_all_connected_devices_override
Runs on all connected devices, currently only support 1.
Cannot be used with -v
-t DEVICE_OS_TYPE_OVERRIDE, --device_os_type_override DEVICE_OS_TYPE_OVERRIDE
Specify the target OS type, valid options are
['android', 'android-aarch64', 'le', 'le64_gcc4.9',
'le_oe_gcc6.4', 'le64_oe_gcc6.4']
-d, --debug Set to turn on debug log
-s SLEEP, --sleep SLEEP
Set number of seconds to sleep between runs e.g. 20
seconds
-b USERBUFFER_MODE, --userbuffer_mode USERBUFFER_MODE
[EXPERIMENTAL] Enable user buffer mode, default to
float, can be tf8exact0
-p PERFPROFILE, --perfprofile PERFPROFILE
Set the benchmark operating mode (balanced, default,
sustained_high_performance, high_performance,
power_saver, system_settings)
-l PROFILINGLEVEL, --profilinglevel PROFILINGLEVEL
Set the profiling level mode (off, basic, moderate, detailed).
Default is basic.
-json, --generate_json
Set to produce json output.
-cache, --enable_init_cache
Enable init caching mode to accelerate the network
building process. Defaults to disable.
snpe-caffe-to-dlc converts a Caffe model into an SNPE DLC file.
usage: snpe-caffe-to-dlc [-h] [--input_network INPUT_NETWORK] [-o OUTPUT_PATH]
[--copyright_file COPYRIGHT_FILE]
[--model_version MODEL_VERSION]
[--disable_batchnorm_folding]
[--input_type INPUT_NAME INPUT_TYPE]
[--input_encoding INPUT_NAME INPUT_ENCODING]
[--validation_target RUNTIME_TARGET PROCESSOR_TARGET]
[--strict] [--debug [DEBUG]]
[-b CAFFE_BIN] [--udl UDL_MODULE FACTORY_FUNCTION]
Script to convert caffemodel into a DLC file.
optional arguments:
-h, --help show this help message and exit
required arguments:
--input_network INPUT_NETWORK, -i INPUT_NETWORK
Path to the source framework model.
optional arguments:
-o OUTPUT_PATH, --output_path OUTPUT_PATH
Path where the converted Output model should be
saved.If not specified, the converter model will be
written to a file with same name as the input model
--copyright_file COPYRIGHT_FILE
Path to copyright file. If provided, the content of
the file will be added to the output model.
--model_version MODEL_VERSION
User-defined ASCII string to identify the model, only
first 64 bytes will be stored
--disable_batchnorm_folding
If not specified, converter will try to fold batchnorm
into previous convolution layer
--input_type INPUT_NAME INPUT_TYPE, -t INPUT_NAME INPUT_TYPE
Type of data expected by each input op/layer. Type for
each input is |default| if not specified. For example:
"data" image.Note that the quotes should always be
included in order to handle special characters,
spaces,etc. For multiple inputs specify multiple
--input_type on the command line. Eg: --input_type
"data1" image --input_type "data2" opaque These
options get used by DSP runtime and following
descriptions state how input will be handled for each
option. Image: input is float between 0-255 and the
input's mean is 0.0f and the input's max is 255.0f. We
will cast the float to uint8ts and pass the uint8ts to
the DSP. Default: pass the input as floats to the dsp
directly and the DSP will quantize it. Opaque: assumes
input is float because the consumer layer(i.e next
layer) requires it as float, therefore it won't be
quantized.Choices supported:['image', 'default',
'opaque']
--input_encoding INPUT_NAME INPUT_ENCODING, -e INPUT_NAME INPUT_ENCODING
Image encoding of the source images. Default is bgr.
Eg usage: "data" rgba Note the quotes should always be
included in order to handle special characters,
spaces, etc. For multiple inputs specify
--input_encoding for each on the command line. Eg:
--input_encoding "data1" rgba --input_encoding "data2"
other. Use options: color encodings(bgr,rgb, nv21...)
if input is image; time_series: for inputs of rnn
models; other: if input doesn't follow above
categories or is unknown. Choices supported:['bgr',
'rgb', 'rgba', 'argb32', 'nv21', 'time_series',
'other']
--validation_target RUNTIME_TARGET PROCESSOR_TARGET
A combination of processor and runtime target against
which model will be validated. Choices for
RUNTIME_TARGET: {cpu, gpu, dsp}. Choices for
PROCESSOR_TARGET: {snapdragon_801, snapdragon_820,
snapdragon_835}.If not specified, will validate model
against {snapdragon_820, snapdragon_835} across all
runtime targets.
--strict If specified, will validate in strict mode whereby
model will not be produced if it violates constraints
of the specified validation target. If not specified,
will validate model in permissive mode against the
specified validation target.
--debug [DEBUG] Run the converter in debug mode.
-b CAFFE_BIN, --caffe_bin CAFFE_BIN
Input caffe binary file containing the weight data
--udl UDL_MODULE FACTORY_FUNCTION
Option to add User Defined Layers. Provide Filename,
Function name.1.Filename: Name of python module to
load for registering custom udl(note: must be in
PYTHONPATH). If file part of package list the
package.filename as you would when doing a python
import.2.Function name: Name of the udl factory
function that return a dictionary of key layer type
and value function callback.Examples of using this script can be found in Converting Models from Caffe to SNPE.
Additional details:
snpe-caffe2-to-dlc converts a Caffe2 model into an SNPE DLC file.
usage: snpe-caffe2-to-dlc [-h] -p PREDICT_NET -e EXEC_NET -i INPUT_DIM
INPUT_DIM [-d DLC] [--enable_preprocessing]
[--encoding {argb32,rgba,nv21,bgr}]
[--opaque_input [OPAQUE_INPUT [OPAQUE_INPUT ...]]]
[--model_version MODEL_VERSION]
[--reorder_list REORDER_LIST [REORDER_LIST ...]]
[--verbose]
Script to convert caffe2 networks into a DLC file.
optional arguments:
-h, --help show this help message and exit
required arguments:
-p PREDICT_NET, --predict_net PREDICT_NET
Input caffe2 binary network definition protobuf
-e EXEC_NET, --exec_net EXEC_NET
Input caffe2 binary file containing the weight data
-i INPUT_DIM INPUT_DIM, --input_dim INPUT_DIM INPUT_DIM
The names and dimensions of the network input layers
specified in the format "input_name" B,C,H,W. Ex "data"
1,3,224,224. Note that the quotes should always be
included in order to handle special characters,
spaces, etc. For multiple inputs specify multiple
--input_dim on the command line like: --input_dim
"data1" 1,3,224,224 --input_dim "data2" 1,3,50,100 We
currently assume that all inputs have 4 dimensions.
optional arguments:
-d DLC, --dlc DLC Output DLC file containing the model. If not
specified, the data will be written to a file with
same name and location as the predict_net file with a
.dlc extension
--copyright_file COPYRIGHT_FILE
Path to copyright file. If provided, the content of
the file will be added to the dlc.
--enable_preprocessing
If specified, the converter will enable image mean
subtraction and cropping specified by ImageInputOp. Do
NOT enable if there is not a ImageInputOp present in
the Caffe2 network.
--encoding {argb32,rgba,nv21,bgr}
Image encoding of the source images. Default is bgr if
not specified
--opaque_input [OPAQUE_INPUT [OPAQUE_INPUT ...]]
A space separated list of input blob names which
should be treated as opaque (non-image) data. These
inputs will be consumed as-is by SNPE. Any input blob
not listed will be assumed to be image data.
--model_version MODEL_VERSION
User-defined ASCII string to identify the model, only
first 64 bytes will be stored
--reorder_list REORDER_LIST [REORDER_LIST ...]
A list of external inputs or outputs that SNPE should
automatically reorder to match the specified Caffe2
channel ordering. Note that this feature is only
enabled for the GPU runtime.
--verbose Verbose printingsnpe-diagview loads a DiagLog file generated by snpe-net-run whenever it operates on input tensor data. The DiagLog file contains timing information information for each layer as well as the entire forward propagate time. If the run uses an input list of input tensors, the timing info reported by snpe-diagview is an average over the entire input set.
The snpe-net-run generates a file called "SNPEDiag_0.log", "SNPEDiag_1.log" ... , "SNPEDiag_n.log", where n corresponds to the nth iteration of the snpe-net-run execution.
usage: snpe-diagview --input_log DIAG_LOG [-h] [--output CSV_FILE]
Reads a diagnostic log and output the contents to stdout
required arguments:
--input_log DIAG_LOG
Diagnostic log file (required)
optional arguments:
--output CSV_FILE
Output CSV file with all diagnostic data (optional)
snpe-dlc-info outputs layer information from a DLC file, which provides information about the network model.
usage: snpe-dlc-info [-h] -i INPUT_DLC [-s SAVE]
required arguments:
-i INPUT_DLC, --input_dlc INPUT_DLC
path to a DLC file
optional arguments:
-s SAVE, --save SAVE
Save the output to a csv file. Specify a target file path.
snpe-dlc-diff compares two DLCs and by default outputs some of the following differences in them in a tabular format:
usage: snpe-dlc-diff [-h] -i1 INPUT_DLC_ONE -i2 INPUT_DLC_TWO [-c] [-l] [-p]
[-d] [-w] [-o] [-i] [-x] [-s SAVE]
required arguments:
-i1 INPUT_DLC_ONE, --input_dlc_one INPUT_DLC_ONE
path to the first dl container archive
-i2 INPUT_DLC_TWO, --input_dlc_two INPUT_DLC_TWO
path to the second dl container archive
optional arguments:
-h, --help show this help message and exit
-c, --copyrights compare copyrights between models
-l, --layers compare unique layers between models
-p, --parameters compare parameter differences between identically
named layers
-d, --dimensions compare dimension differences between identically
named layers
-w, --weights compare weight differences between identically named
layers.
-o, --outputs compare output_tensor name differences names between
identically named layers
-i, --diff_by_id Overrides the default comparison strategy for diffing
2 models components. By default comparison is made
between identically named layers. With this option the
models are ordered by id and diff is done in order as
long as no more than 1 consecutive layers have
different layer types.
-x, --hta compare HTA records differences in Models
-s SAVE, --save SAVE Save the output to a csv file. Specify a target file
path.
snpe-dlc-viewer visualizes the network structure of a DLC in a web browser.
usage: snpe-dlc-viewer [-h] -i INPUT_DLC [-s]
required arguments:
-i INPUT_DLC, --input_dlc INPUT_DLC
Path to a DLC file
optional arguments:
-s, --save Save HTML file. Specify a file name and/or target save path
-h, --help Shows this help message and exitsAdditional details:
The DLC viewer tool renders the specified network DLC in HTML format that may be viewed on a web browser.
On installations that support a native web browser a browser instance is opened on which the network is automatically rendered.
Users can optionally save the HTML content anywhere on their systems and open on a chosen web browser independently at a later time.
snpe-dlc-quantize converts non-quantized DLC models into quantized DLC models.
Command Line Options:
[ -h,--help ] Displays this help message.
[ --version ] Displays version information.
[ --verbose ] Enable verbose user messages.
[ --quiet ] Disables some user messages.
[ --silent ] Disables all but fatal user messages.
[ --debug=<val> ] Sets the debug log level.
[ --debug1 ] Enables level 1 debug messages.
[ --debug2 ] Enables level 2 debug messages.
[ --debug3 ] Enables level 3 debug messages.
[ --log-mask=<val> ] Sets the debug log mask to set the log level for one or more areas.
Example: ".*=USER_ERROR, .*=INFO, NDK=DEBUG2, NCC=DEBUG3"
[ --log-file=<val> ] Overrides the default name for the debug log file.
[ --log-dir=<val> ] Overrides the default directory path where debug log files are written.
[ --log-file-include-hostname ]
Appends the name of this host to the log file name.
--input_dlc=<val> Path to the dlc container containing the model for which fixed-point encoding
metadata should be generated. This argument is required.
--input_list=<val> Path to a file specifying the trial inputs. This file should be a plain text file,
containing one or more absolute file paths per line. These files will be taken to constitute
the trial set. Each path is expected to point to a binary file containing one trial input
in the 'raw' format, ready to be consumed by SNPE without any further modifications.
This is similar to how input is provided to snpe-net-run application.
[ --no_weight_quantization ]
Generate and add the fixed-point encoding metadata but keep the weights in
floating point. This argument is optional.
[ --output_dlc=<val> ]
Path at which the metadata-included quantized model container should be written.
If this argument is omitted, the quantized model will be written at <unquantized_model_name>_quantized.dlc.
[ --enable_hta ] Pack HTA information in quantized DLC.
[ --hta_partitions ] Specify subnet partitions to run on HTA.
Partitions are specified with an ASIC identifier,start and end layer IDs, like 0-20.
Multiple partitions can also be specified as a comma-separated string, like 0-20,40-60,80-100 after asic is specified.
Partitions can be specified for multiple ASICs by specifying this option multiple times.
For ex., 1) --hta_partitions sm8150:
2) --hta_partitions sm8150:0-20,40-60,80-100
3) --hta_partitions sm8150:0-120 --hta_partitions sm7250:0-120
Please refer to SNPE documentation for more details.
[ --use_enhanced_quantizer ]
Use the enhanced quantizer feature when quantizing the model. Regular quantization determines the range using the actual
values of min and max of the data being quantized. Enhanced quantization uses an algorithm to determine optimal range. It can be
useful for quantizing models that have long tails in the distribution of the data being quantized.
[ --use_adjusted_weights_quantizer ]
Use the adjusted tf quantizer for quantizing the weights only. This might be helpful for improving the accuracy of some models,
such as denoise model as being tested. This option is only used when quantizing the weights with 8 bit.
[ --optimizations ] Use this option to enable new optimization algorithms. Usage is:
--optimizations <algo_name1> --optimizations <algo_name2>
The available optimization algorithms are:
cle - Cross layer equalization includes a number of methods for equalizing weights and biases across layers in order to rectify imbalances that cause quantization errors.
bc - Bias correction adjusts biases to offset activation quantization errors. Typically used in conjunction with 'cle' to improve quantization accuracy.
[ --override_params ]
Use this option to override quantization parameters when quantization was provided from the original source framework (eg TF fake quantization)
[ --use_symmetric_quantize_weights ]
Use the symmetric quantizer feature when quantizing the weights of the model. It makes sure min and max have the
same absolute values about zero. Symmetrically quantized data will also be stored as int#_t data such that the offset is always 0.
[ --bias_bitwidth=<val> ]
Use the --bias_bitdwith option to select the bitwidth to use when quantizing the biases, either 8 (default) or 32. Using 32 bit biases may
sometimes provide a small improvement in accuracy. Can't mix with --bitwidth.
[ --act_bitwidth=<val> ]
Use the --act_bitwidth option to select the bitwidth to use when quantizing the activations, either 8 (default) or 16. 8w/16a is only supported
by the HTA currently. Can't mix with --bitwidth.
[ --weights_bitwidth=<val> ]
Use the --weights_bitwidth option to select the bitwidth to use when quantizing the weights, either 8 (default) or 16. 8w/16a is only supported
by the HTA currently. Can't mix with --bitwidth.
[ --bitwidth=<val> ]
Use the --bitwidth option to select the bitwidth to use when quantizing the weights/activation/bias, either 8 (default) or 16. Can't mix with
--weights_bitwidth or --act_bitwidth or --bias_bitdwith.
[ --udo_package_path=<val> ]
Use this option to specify path to the Registration Library for a UDO Package. Usage is:
--udo_package_path=<path_to_reg_lib>
This option must be specified for Networks with UDO. All UDO's in Network must have host-executable CPU Implementation
Description:
Generate 8 or 16 bit TensorFlow style fixed point weight and activations encodings for a floating point SNPE model.
Additional details:
snpe-tensorflow-to-dlc converts a TensorFlow model into an SNPE DLC file.
usage: snpe-tensorflow-to-dlc [-h] [--input_network INPUT_NETWORK]
[-o OUTPUT_PATH]
[--copyright_file COPYRIGHT_FILE]
[--model_version MODEL_VERSION]
[--disable_batchnorm_folding]
[--input_type INPUT_NAME INPUT_TYPE]
[--input_encoding INPUT_NAME INPUT_ENCODING]
[--validation_target RUNTIME_TARGET PROCESSOR_TARGET]
[--strict] [--debug [DEBUG]] -d INPUT_NAME
INPUT_DIM --out_node OUT_NODE
[--allow_unconsumed_nodes]
Script to convert tensorflowmodel into a DLC file.
optional arguments:
-h, --help show this help message and exit
required arguments:
--input_network INPUT_NETWORK, -i INPUT_NETWORK
Path to the source framework model.
-d INPUT_NAME INPUT_DIM, --input_dim INPUT_NAME INPUT_DIM
The names and dimensions of the network input layers
specified in the format "input_name" comma-separated-
dimensions, for example: "data" 1,224,224,3. Note that
the quotes should always be included in order to
handle special characters, spaces, etc. For multiple
inputs specify multiple --input_dim on the command
line like: --input_dim "data1" 1,224,224,3 --input_dim
"data2" 1,50,100,3.
--out_node OUT_NODE Name of the graph's output node.
optional arguments:
-o OUTPUT_PATH, --output_path OUTPUT_PATH
Path where the converted Output model should be
saved.If not specified, the converter model will be
written to a file with same name as the input model
--copyright_file COPYRIGHT_FILE
Path to copyright file. If provided, the content of
the file will be added to the output model.
--model_version MODEL_VERSION
User-defined ASCII string to identify the model, only
first 64 bytes will be stored
--disable_batchnorm_folding
If not specified, converter will try to fold batchnorm
into previous convolution layer
--input_type INPUT_NAME INPUT_TYPE, -t INPUT_NAME INPUT_TYPE
Type of data expected by each input op/layer. Type for
each input is |default| if not specified. For example:
"data" image.Note that the quotes should always be
included in order to handle special characters,
spaces,etc. For multiple inputs specify multiple
--input_type on the command line. Eg: --input_type
"data1" image --input_type "data2" opaque These
options get used by DSP runtime and following
descriptions state how input will be handled for each
option. Image: input is float between 0-255 and the
input's mean is 0.0f and the input's max is 255.0f. We
will cast the float to uint8ts and pass the uint8ts to
the DSP. Default: pass the input as floats to the dsp
directly and the DSP will quantize it. Opaque: assumes
input is float because the consumer layer(i.e next
layer) requires it as float, therefore it won't be
quantized.Choices supported:['image', 'default',
'opaque']
--input_encoding INPUT_NAME INPUT_ENCODING, -e INPUT_NAME INPUT_ENCODING
Image encoding of the source images. Default is bgr.
Eg usage: "data" rgba Note the quotes should always be
included in order to handle special characters,
spaces, etc. For multiple inputs specify
--input_encoding for each on the command line. Eg:
--input_encoding "data1" rgba --input_encoding "data2"
other. Use options: color encodings(bgr,rgb, nv21...)
if input is image; time_series: for inputs of rnn
models; other: if input doesn't follow above
categories or is unknown. Choices supported:['bgr',
'rgb', 'rgba', 'argb32', 'nv21', 'time_series',
'other']
--validation_target RUNTIME_TARGET PROCESSOR_TARGET
A combination of processor and runtime target against
which model will be validated. Choices for
RUNTIME_TARGET: {cpu, gpu, dsp}. Choices for
PROCESSOR_TARGET: {snapdragon_801, snapdragon_820,
snapdragon_835}.If not specified, will validate model
against {snapdragon_820, snapdragon_835} across all
runtime targets.
--strict If specified, will validate in strict mode whereby
model will not be produced if it violates constraints
of the specified validation target. If not specified,
will validate model in permissive mode against the
specified validation target.
--debug [DEBUG] Run the converter in debug mode.
--allow_unconsumed_nodes
Uses a relaxed graph node to layer mapping algorithm
which may not use all graph nodes during conversion
while retaining structural integrity.
--udo_config_paths UDO_CONFIG_PATHS [UDO_CONFIG_PATHS ...], -udo UDO_CONFIG_PATHS
[UDO_CONFIG_PATHS ...]
Path to the UDO configs (space separated, if multiple)Examples of using this script can be found in Converting Models from TensorFlow to SNPE.
Additional details:
The name of the last node in your TensorFlow graph which will represent the output layer of your network.
snpe-onnx-to-dlc converts a serialized ONNX model into a SNPE DLC file.
usage: snpe-onnx-to-dlc [-h] [--input_network INPUT_NETWORK] [-o OUTPUT_PATH]
[--copyright_file COPYRIGHT_FILE]
[--model_version MODEL_VERSION]
[--disable_batchnorm_folding]
[--input_type INPUT_NAME INPUT_TYPE]
[--input_encoding INPUT_NAME INPUT_ENCODING]
[--validation_target RUNTIME_TARGET PROCESSOR_TARGET]
[--strict] [--debug [DEBUG]]
[--dry_run [DRY_RUN]]
Script to convert onnxmodel into a DLC file.
optional arguments:
-h, --help show this help message and exit
required arguments:
--input_network INPUT_NETWORK, -i INPUT_NETWORK
Path to the source framework model.
optional arguments:
-o OUTPUT_PATH, --output_path OUTPUT_PATH
Path where the converted Output model should be
saved.If not specified, the converter model will be
written to a file with same name as the input model
--copyright_file COPYRIGHT_FILE
Path to copyright file. If provided, the content of
the file will be added to the output model.
--model_version MODEL_VERSION
User-defined ASCII string to identify the model, only
first 64 bytes will be stored
--disable_batchnorm_folding
If not specified, converter will try to fold batchnorm
into previous convolution layer
--input_type INPUT_NAME INPUT_TYPE, -t INPUT_NAME INPUT_TYPE
Type of data expected by each input op/layer. Type for
each input is |default| if not specified. For example:
"data" image.Note that the quotes should always be
included in order to handle special characters,
spaces,etc. For multiple inputs specify multiple
--input_type on the command line. Eg: --input_type
"data1" image --input_type "data2" opaque These
options get used by DSP runtime and following
descriptions state how input will be handled for each
option. Image: input is float between 0-255 and the
input's mean is 0.0f and the input's max is 255.0f. We
will cast the float to uint8ts and pass the uint8ts to
the DSP. Default: pass the input as floats to the dsp
directly and the DSP will quantize it. Opaque: assumes
input is float because the consumer layer(i.e next
layer) requires it as float, therefore it won't be
quantized.Choices supported:['image', 'default',
'opaque']
--input_encoding INPUT_NAME INPUT_ENCODING, -e INPUT_NAME INPUT_ENCODING
Image encoding of the source images. Default is bgr.
Eg usage: "data" rgba Note the quotes should always be
included in order to handle special characters,
spaces, etc. For multiple inputs specify
--input_encoding for each on the command line. Eg:
--input_encoding "data1" rgba --input_encoding "data2"
other. Use options: color encodings(bgr,rgb, nv21...)
if input is image; time_series: for inputs of rnn
models; other: if input doesn't follow above
categories or is unknown. Choices supported:['bgr',
'rgb', 'rgba', 'argb32', 'nv21', 'time_series',
'other']
--validation_target RUNTIME_TARGET PROCESSOR_TARGET
A combination of processor and runtime target against
which model will be validated. Choices for
RUNTIME_TARGET: {cpu, gpu, dsp}. Choices for
PROCESSOR_TARGET: {snapdragon_801, snapdragon_820,
snapdragon_835}.If not specified, will validate model
against {snapdragon_820, snapdragon_835} across all
runtime targets.
--strict If specified, will validate in strict mode whereby
model will not be produced if it violates constraints
of the specified validation target. If not specified,
will validate model in permissive mode against the
specified validation target.
--debug [DEBUG] Run the converter in debug mode.
--dry_run [DRY_RUN] Evaluates the model without actually converting any
ops, and returns unsupported ops/attributes as well as
unused inputs and/or outputs if any. Leave empty or
specify "info" to see dry run as a table, or specify
"debug" to show more detailed messages only"
--udo_config_paths UDO_CONFIG_PATHS [UDO_CONFIG_PATHS ...], -udo UDO_CONFIG_PATHS
[UDO_CONFIG_PATHS ...]
Path to the UDO configs (space separated, if multiple)For more information, see ONNX Model Conversion
DESCRIPTION: ------------ snpe-platform-validator checks the SNPE compatibility/capability of a device. This tool runs on the device, rather than on the host, and requires a few additional files to be pushed to the device besides its own executable. Additional details below. REQUIRED ARGUMENTS: ------------------- --runtime <RUNTIME> Specify the runtime to validate. <RUNTIME> : gpu, dsp, aip, all. OPTIONAL ARGUMENTS: ------------------- --coreVersion Query the runtime core descriptor. --libVersion Query the runtime core library API. --testRuntime Run diagnostic tests on the specified runtime. --targetPath <DIR> The directory to save output on the device. Defaults to /data/local/tmp/platformValidator/output. --debug Turn on verbose logging. --help Show this help message.
Additional details:
bin/snpe-platform-validator lib/libcalculator_domains.so lib/libcalculator.so lib/libsnpe_dsp_domains_v2.so lib/dsp/libcalculator_skel.so lib/dsp/libsnpe_dsp_v65_domains_v2_skel.so lib/dsp/libsnpe_dsp_v66_domains_v2_skel.so example: for pushing arm-android-clang6.0 variant to /data/local/tmp/platformValidator adb push $SNPE_ROOT/bin/arm-android-clang6.0/snpe-platform-validator /data/local/tmp/platformValidator/bin/snpe-platform-validator adb push $SNPE_ROOT/lib/arm-android-clang6.0 /data/local/tmp/platformValidator/lib adb push $SNPE_ROOT/lib/dsp /data/local/tmp/platformValidator/dsp
snpe-throughput-net-run concurrently runs multiple instances of SNPE for a certain duration of time and measures inference throughput. Each instance of SNPE can have its own model, designated runtime and performance profile. Please note that the "--duration" parameter is common for all instances of SNPE created.
DESCRIPTION:
------------
Example application demonstrating how to load concurrent SNPE objects
using the SNPE C++ API.
REQUIRED ARGUMENTS:
-------------------
--container <FILE> Path to the DL container containing the network.
--duration <VAL> Duration of time (in seconds) to run network execution.
--use_cpu Use the CPU runtime for SNPE.
--use_gpu Use the GPU float32 runtime for SNPE.
--use_gpu_fp16 Use the GPU float16 runtime for SNPE.
--use_dsp Use the DSP fixed point runtime for SNPE.
--perf_profile <VAL> Specifies perf profile to set. Valid settings are "balanced" , "default" , "high_performance" ,
"sustained_high_performance" , "burst" , "power_saver" and "system_settings".
NOTE: "balanced" and "default" are the same. "default" is being deprecated in the future.
--use_aip Use the AIP fixed point runtime for SNPE
OPTIONAL ARGUMENTS:
-------------------
--debug Specifies that output from all layers of the network
will be saved.
will be saved.
--storage_dir <DIR> The directory to store SNPE metadata files
--version Show SNPE Version Number.
--iterations <VAL> Number of times to iterate through entire input list
--verbose Print more debug information.
--skip_execute Don't do execution (just SNPE graph build/teardown)
--json <FILE> Generated JSON report.
--input_raw <FILE> Path to raw inputs for the network, seperated by ",".
--enable_cpu_fallback Enables cpu fallback functionality. Defaults to disable mode.
--udo_package_path <VAL,VAL>
Path to UDO package with registration library for UDOs.
Optionally, user can provide multiple packages as a comma-separated list.
--help Show this help message. DESCRIPTION:
------------
snpe-platform-validator-py checks the SNPE compatibility/capability of a device. The output is saved in a CSV file in the
"Output" directory, in a csv format. Basic logs are also displayed on the console.
REQUIRED ARGUMENTS:
-------------------
--runtime <RUNTIME> Specify the runtime to validate. <RUNTIME> : gpu, dsp, aip, all.
--directory <ARTIFACTS> Path to the root of the unpacked SDK directory containing the executable and library files.
OPTIONAL ARGUMENTS:
-------------------
--buildVariant <VARIANT> Specify the build variant (e.g: arm-android-clang6.0(default), aarch64-android-clang6.0) to be validated.
--deviceId Uses the device for running the adb command. Defaults to first device in the adb devices list.
--coreVersion Outputs the version of the runtime that is present on the target.
--libVersion Outputs the library version of the runtime that is present on the target.
--testRuntime Runs a small program on the runtime and Checks if SNPE is supported for runtime.
--targetPath <PATH> The path to be used on the device. Defaults to /data/local/tmp/platformValidator
NOTE that this directory will be deleted before proceeding with validation.
--remoteHost <REMOTEHOST> Run on remote host through remote adb server. Defaults to localhost.
--debug Set to turn on debug log.DESCRIPTION:
------------
This tool generates a UDO (User Defined Operation) package using a
user provided config file.
USAGE:
------------
snpe-udo-package-generator [-h] --config_path CONFIG_PATH [--debug]
[--output_path OUTPUT_PATH] [-f]
OPTIONAL ARGUMENTS:
-------------------
-h, --help show this help message and exit
--debug Returns debugging information from generating the package
--output_path OUTPUT_PATH, -o OUTPUT_PATH
Path where the package should be saved
-f, --force-generation
This option will delete the existing package
Note appropriate file permissions must be set to use
this option.
REQUIRED_ARGUMENTS:
-------------------
--config_path CONFIG_PATH, -p CONFIG_PATH
The path to a config file that defines a UDO.
80-NL315-14 A
MAY CONTAIN U.S. AND INTERNATIONAL EXPORT CONTROLLED INFORMATION