❓ [Question] Error when exporting TorchScript model with dynamic input using Torch-TensorRT (aten::clamp issue)

[Mmmyyym]

Environment

Libtorch 2.5.0.dev (latest nightly) (built with CUDA 12.4)
CUDA 12.4
TensorRT 10.1.0.27
PyTorch 2.4.0+cu124
Torch-TensorRT 2.4.0
Python 3.12.8
Windows 10

Code

import torch
import torch_tensorrt

model = DeepLabv3+(num_classes=8,  backbone='mobilenet', output_stride=16)
model.eval().cuda()
dynamic_input = torch_tensorrt.Input(
        min_shape=[1, 3, 20, 20],
        opt_shape=[1, 3, 512, 512],
        max_shape=[1, 3, 2448, 2048],
        dtype=torch.float32 )
static_input = [torch.randn((1, 3, 512, 512)).cuda()]
trt_gm = torch_tensorrt.compile(model, ir="dynamo", inputs = dynamic_input)
torch_tensorrt.save(trt_gm, "trt_dynamic.ts", output_format="torchscript",inputs=static_input)

ERROR

torch_tensorrt.compile succeeds, but torch_tensorrt.save fails.

torch_tensorrt.save(trt_gm, "trt_dynamic.ts", output_format="torchscript",inputs=static_input)

    def __call__(self_, *args, **kwargs):  # noqa: B902
        # use `self_` to avoid naming collide with aten ops arguments that
        # are named "self". This way, all the aten ops can be called by kwargs.
        return self_._op(*args, **kwargs)

RuntimeError: aten::clamp() Expected a value of type 'Optional[number]' for argument 'max' but instead found type 'Tensor'.
Position: 2
Value: tensor([31], device='cuda:0')
Declaration: aten::clamp(Tensor self, Scalar? min=None, Scalar? max=None) -> Tensor
Cast error details: Cannot cast tensor([31], device='cuda:0') to number

[narendasan]

@apbose Please take a look at this, when I run with this implementation of DeepLabV3+ (https://github.com/VainF/DeepLabV3Plus-Pytorch) with dynamic shape I get the following converter error

  File "/home/naren/pytorch_org/tensorrt/.venv/lib/python3.13/site-packages/torch_tensorrt/dynamo/conversion/impl/upsample.py", line 38, in upsample
    layer.shape = shape
    ^^^^^^^^^^^
TypeError: (): incompatible function arguments. The following argument types are supported:
    1. (arg0: tensorrt_bindings.tensorrt.IResizeLayer, arg1: tensorrt_bindings.tensorrt.Dims) -> None

Invoked with: <tensorrt_bindings.tensorrt.IResizeLayer object at 0x7295eab9f130>, [1, 256, <tensorrt_bindings.tensorrt.ITensor object at 0x7295eab95f70>, <tensorrt_bindings.tensorrt.ITensor object at 0x7295eab8bcf0>]

Detection of dynamic shape here: if has_dynamic_shape(shape): does not seem to handle this case

[Mmmyyym]

    def __call__(self_, *args, **kwargs):
        try:
            return self_._op(*args, **kwargs)
        except RuntimeError as e:
            if "aten::clamp" in str(e):
                new_args = list(args)
                if len(new_args) > 2:
                    max_arg = new_args[2]
                    if (isinstance(max_arg, torch.Tensor) and max_arg.numel() == 1 and float(max_arg.item()) == 31.0):
                        new_args[2] = 31
                    if (isinstance(max_arg, torch.Tensor) and max_arg.numel() == 1 and float(max_arg.item()) == 127.0):
                        new_args[2] = 127
                return self_._op(*tuple(new_args), **kwargs)
            else:
                raise

Forcing the problematic input values to int allows successful .ts export. However, in C++ inference, if the input shape isn’t [1,3,512,512] (e.g., [1,3,1948,1083]), the output tensors can differ by up to 700 compared to Python, causing inconsistent results. With [1,3,512,512], the difference is only around 0.2.