From eLinux.org
< TensorRT
Revision as of 19:05, 18 August 2021 by Zerollzeng (talk | contribs) (How to fix Error Code 10: Internal Error (Could not find any implementation for node xxx))
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

How to fix the error “Could not find scales for tensor xxxx” for INT8 mode?

Generally, after INT8 calibration is done, Int8Calibrator will save the scaling factors into a local file (through API writeCalibrationCache), so that it wouldn’t need to do calibration again for subsequent running and load the cached calibration table directly (through API readCalibrationCache).
If you change the network or update the network or run the network among different GPU platforms or different TensorRT versions, then you may probably get the error “Could not find scales for tensor xxxx”, that indicates builder couldn’t find corresponding scaling factor from local cached calibration table. It’s intended since the network graph after fusion would change among different GPU platform or different TensorRT version or modification to network itself. The solution is very simple that removes the local calibration table and does calibration again.

How to fix Error Code 10: Internal Error (Could not find any implementation for node xxx)

Try increase workspace size, eg: trtexec --onnx=model.onnx --workspace=1024

How to fix "LogicError: explicit_context_dependent failed" during running TRT Python in multi-thread?

If you are using the common.py of TRT/sample to do inference with multi-thread, and getting below error, this FAQ will help you to fix that.

 "pycuda._driver.LogicError: explicit_context_dependent failed: invalid device context - no currently active context?"

Refer to "PyCuda FAQ: How does PyCUDA handle threading", this error is caused by the missing active context in work thread.
Please make context as below before trigger the GPU task that reported the error:

   dev = cuda.Device(0) // 0 is your GPU number
   ctx = dev.make_context()

and cleans up after the GPU task using:

   del ctx

Why there is (Unnamed Layer* <N>) appearing in calibration table or verbose log?

For example, here is the calibration table for mnist,

data: 3c000889
conv1: 3c8954be
pool1: 3c8954be
conv2: 3dd33169
pool2: 3dd33169
(Unnamed Layer* 4) [Fully Connected]_output: 3dcbd455
ip1: 3daeff02
ip2: 3e7d50e9
prob: 3c010a14

(Unnamed Layer* 4) [Fully Connected]_output actually denotes ip1 layer and the next ip1 denotes relu1 layer.

How does it happen?

It's because we use top attribute to name layer or tensor name, but ip1 and relu1 in mnist.prototxt share the same top name ip1. Hence there must be either of them to use system assigned name.

layer {
  name: "ip1"
  type: "InnerProduct"
  bottom: "pool2"
  top: "ip1"
  param {
    lr_mult: 1.0
  param {
    lr_mult: 2.0
  inner_product_param {
    num_output: 500
    weight_filler {
      type: "xavier"
    bias_filler {
      type: "constant"
layer {
  name: "relu1"
  type: "ReLU"
  bottom: "ip1"
  top: "ip1"
layer {
  name: "ip2"
  type: "InnerProduct"
  bottom: "ip1"
  top: "ip2"
  param {
    lr_mult: 1.0

This wouldn't have any impact on the execution behavior of network.

And it can be avoided through removing all in-place nodes, for mnist, updating the top attribute of relu1 with its layer name relu1, other than ip1 (and updating bottom attribute of ip2 with 'relu1' accordingly).

UFF Parser Error Messages

The following table captures common UFF parser error messages and the way to fix it.
Other common UFF Parser Error can be found in TRT Devemop Guide

Error Message Description Fixing
uff/UffParser.cpp:1130: std::shared_ptr UffParser::parseConvTranspose(const uff::Node&, const Fields&, NodesMap&): Assertion `outSpatial[0] == padLayer->getOutput(0)->getDimensions().d[1]' failed. It's known issue found in TRT It fixed by TRT 6.0. Please update TRT to 6.0 if you got this error
UffParser: Parser error: XXX_LAYER: Order size is not matching the number dimensions of TensorRT It happened when Uffparser is trying to add transpose for the layer whose format is not NCHW and tensor is 5d Modify your model to make sure it's all NCHW format
UffParser: UFFParser: Parsing XXX_Conv_LAYER,python: ../builder/cudnnBuilderUtils.cpp:577: void nvinfer1::cudnn::updatePaddingAccordingToPaddingMode(nvinfer1::PaddingMode, nvinfer1::Dims&, nvinfer1::Dims&, nvinfer1::Dims, nvinfer1::Dims, nvinfer1::Dims, nvinfer1::Dims, bool): Assertion `inputDims.nbDims >= nbSpatialDims' failed. Aborted (core dumped) It might casued by using Asymmetric Padding. Uffparser do not support Asymmetric Padding If the Asymmetric Padding is must-to-have for your case, you have to convert pb to onnx, and use ONNX parser.
UffParser: Parser error: XXX_LAYER: Invalid Batchnorm inputs for layer XXX_LAYER It caused by the dims of BN input are not the same Set is_training to false, so that the mean and variance aren't NONE