[None][Doc] Rename TensorRT-LLM to TensorRT LLM.

Signed-off-by: nv-guomingz <[email protected]>

Author: nv-guomingz

docs/source/architecture/add-model.md

@@ -2,19 +2,19 @@
 
 # Adding a Model
 
-This document describes how to add a typical decoder-only model in TensorRT-LLM.
+This document describes how to add a typical decoder-only model in TensorRT LLM.
 
 ## Step 1. Write Modeling Part
 
-TensorRT-LLM provides different levels of APIs:
+TensorRT LLM provides different levels of APIs:
 
 - Low-level functions, for example, `concat`, `add`, and `sum`.
 - Basic layers, such as, `Linear` and `LayerNorm`.
 - High-level layers, such as, `MLP` and `Attention`.
 - Base class for typical decoder-only models, such as, `DecoderModelForCausalLM`.
 
 1. Create a model directory in `tensorrt_llm/models`, for example `my_model`.
-2. Write a `model.py` with TensorRT-LLM's APIs
+2. Write a `model.py` with TensorRT LLM's APIs
 
 ```python
 class MyDecoderLayer(Module):
@@ -52,7 +52,7 @@ class MyModelForCausalLM(DecoderModelForCausalLM):
 
 ## Step 2. Implement Weight Conversion
 
-The weights from source framework need to be converted and bound to the new added TensorRT-LLM model. Here is an example of converting HuggingFace weights:
+The weights from source framework need to be converted and bound to the new added TensorRT LLM model. Here is an example of converting HuggingFace weights:
 
 ```python
 class MyModelForCausalLM(DecoderModelForCausalLM):
@@ -62,8 +62,8 @@ class MyModelForCausalLM(DecoderModelForCausalLM):
             hf_model_dir,
             dtype='float16',
             mapping: Optional[Mapping] = None) -> MyModelForCausalLM
-        # create a TensorRT-LLM MyModelForCausalLM model object
-        # convert HuggingFace checkpoint to TensorRT-LLM expected weights dict
+        # create a TensorRT LLM MyModelForCausalLM model object
+        # convert HuggingFace checkpoint to TensorRT LLM expected weights dict
         # load the weights to MyModelForCausalLM object
 ```
 

docs/source/architecture/checkpoint.md

@@ -1,36 +1,36 @@
-# TensorRT-LLM Checkpoint
+# TensorRT LLM Checkpoint
 
 ## Overview
 
-The earlier versions (pre-0.8 version) of TensorRT-LLM were developed with a very aggressive timeline. For those versions, emphasis was not put on defining a unified workflow. Now that TensorRT-LLM has reached some level of feature richness, the development team has decided to put more effort into unifying the APIs and workflow of TensorRT-LLM. This file documents the workflow around TensorRT-LLM checkpoint and the set of CLI tools to generate checkpoint, build engines, and evaluate engines.
+The earlier versions (pre-0.8 version) of TensorRT LLM were developed with a very aggressive timeline. For those versions, emphasis was not put on defining a unified workflow. Now that TensorRT LLM has reached some level of feature richness, the development team has decided to put more effort into unifying the APIs and workflow of TensorRT LLM. This file documents the workflow around TensorRT LLM checkpoint and the set of CLI tools to generate checkpoint, build engines, and evaluate engines.
 
 There are three steps in the workflow:
 
-1. Convert weights from different source frameworks into TensorRT-LLM checkpoint.
-2. Build the TensorRT-LLM checkpoint into TensorRT engines with a unified build command.
-3. Load the engines to TensorRT-LLM model runner and evaluate with different evaluation tasks.
+1. Convert weights from different source frameworks into TensorRT LLM checkpoint.
+2. Build the TensorRT LLM checkpoint into TensorRT engines with a unified build command.
+3. Load the engines to TensorRT LLM model runner and evaluate with different evaluation tasks.
 
 ```
 NeMo -------------
                   |
 HuggingFace ------
                   |   convert                             build                    load
-Modelopt ---------  ----------> TensorRT-LLM Checkpoint --------> TensorRT Engine ------> TensorRT-LLM ModelRunner
+Modelopt ---------  ----------> TensorRT LLM Checkpoint --------> TensorRT Engine ------> TensorRT LLM ModelRunner
                   |
 JAX --------------
                   |
 DeepSpeed --------
 ```
 
-## Prepare the TensorRT-LLM Checkpoint
+## Prepare the TensorRT LLM Checkpoint
 
-TensorRT-LLM aims at supporting different sources:
+TensorRT LLM aims at supporting different sources:
 
 1. Trained models from NVIDIA NeMo, Microsoft DeepSpeed, and JAX
 2. Quantized models from NVIDIA Modelopt
 3. Popular models from HuggingFace
 
-TensorRT-LLM defines its own checkpoint format. A checkpoint directory includes:
+TensorRT LLM defines its own checkpoint format. A checkpoint directory includes:
 
 1. One config `json` file, which contains several model hyper-parameters.
 2. One or several rank weights files, each file contains a dictionary of tensors (weights).
@@ -107,7 +107,7 @@ Here is the model specific config list:
 ### Rank Weights
 
 Like PyTorch, the tensor (weight) name is a string containing hierarchical information,
-which is uniquely mapped to a certain parameter of a TensorRT-LLM model.
+which is uniquely mapped to a certain parameter of a TensorRT LLM model.
 
 For example, each transformer layer of the OPT model contains an `Attention` layer, an `MLP` layer. and two `LayerNorm` layers.
 
@@ -169,7 +169,7 @@ Here is the AWQ scaling factors of `mlp.fc` linear layer:
 - `transformer.layers.0.mlp.fc.prequant_scaling_factor`
 
     ```{note}
-    The linear weights in TensorRT-LLM checkpoint always follows (`out_feature`, `in_feature`) shape, whereas some quantized linear in TensorRT-LLM implemented by plugin may use (`in_feature`, `out_fature`) shape. The `trtllm-build` command adds a transpose operation to post-process it.
+    The linear weights in TensorRT LLM checkpoint always follows (`out_feature`, `in_feature`) shape, whereas some quantized linear in TensorRT LLM implemented by plugin may use (`in_feature`, `out_fature`) shape. The `trtllm-build` command adds a transpose operation to post-process it.
 
 ### Example
 
@@ -218,7 +218,7 @@ Here is the `config.json`:
 
 ## Build Checkpoint into TensorRT Engine
 
-TensorRT-LLM provides a unified build command: `trtllm-build`. Before using it,
+TensorRT LLM provides a unified build command: `trtllm-build`. Before using it,
 you may need to add it to the `PATH`.
 
 ```bash

docs/source/blogs/Best_perf_practice_on_DeepSeek-R1_in_TensorRT-LLM.md

@@ -1,18 +1,18 @@
-# How to get best performance on DeepSeek-R1 in TensorRT-LLM
+# How to get best performance on DeepSeek-R1 in TensorRT LLM
 
 NVIDIA has announced world-record DeepSeek-R1 inference performance at NVIDIA GTC 2025. A single NVIDIA DGX system with eight NVIDIA Blackwell GPUs can achieve over 250 tokens per second per user or a maximum throughput of over 30,000 tokens per second on the massive, state-of-the-art 671 billion parameter DeepSeek-R1 model. [NVIDIA Blackwell Delivers World-Record DeepSeek-R1 Inference Performance](https://developer.nvidia.com/blog/nvidia-blackwell-delivers-world-record-deepseek-r1-inference-performance/)
 
 In this blog, we share the configurations and procedures about how to reproduce the number on both B200 and H200 with PyTorch workflow.
 
 ## Table of Contents
 
-- [How to get best performance on DeepSeek-R1 in TensorRT-LLM](#how-to-get-best-performance-on-deepseek-r1-in-tensorrt-llm)
+- [How to get best performance on DeepSeek-R1 in TensorRT LLM](#how-to-get-best-performance-on-deepseek-r1-in-tensorrt-llm)
   - [Table of Contents](#table-of-contents)
-  - [Prerequisites: Install TensorRT-LLM and download models](#prerequisites-install-tensorrt-llm-and-download-models)
-      - [1. Download TensorRT-LLM](#1-download-tensorrt-llm)
+  - [Prerequisites: Install TensorRT LLM and download models](#prerequisites-install-tensorrt-llm-and-download-models)
+      - [1. Download TensorRT LLM](#1-download-tensorrt-llm)
       - [2. Download the DeepSeek R1 models](#2-download-the-deepseek-r1-models)
-      - [3. Build and run TensorRT-LLM container](#3-build-and-run-tensorrt-llm-container)
-      - [4. Compile and Install TensorRT-LLM](#4-compile-and-install-tensorrt-llm)
+      - [3. Build and run TensorRT LLM container](#3-build-and-run-tensorrt-llm-container)
+      - [4. Compile and Install TensorRT LLM](#4-compile-and-install-tensorrt-llm)
       - [5. Optional: Tune GPU clocks](#5-optional-tune-gpu-clocks)
       - [6. Dataset preparation](#6-dataset-preparation)
   - [Reproducing steps](#reproducing-steps)
@@ -34,13 +34,13 @@ In this blog, we share the configurations and procedures about how to reproduce
     - [Out of memory issues](#out-of-memory-issues)
 
 
-## Prerequisites: Install TensorRT-LLM and download models
+## Prerequisites: Install TensorRT LLM and download models
 
-This section can be skipped if you already have TensorRT-LLM installed and have already downloaded the DeepSeek R1 model checkpoint.
+This section can be skipped if you already have TensorRT LLM installed and have already downloaded the DeepSeek R1 model checkpoint.
 
-#### 1. Download TensorRT-LLM
+#### 1. Download TensorRT LLM
 
-**You can also find more comprehensive instructions to install TensorRT-LLM in this [TensorRT-LLM installation guide](https://nvidia.github.io/TensorRT-LLM/installation/build-from-source-linux.html), refer to that guide for common issues if you encounter any here.**
+**You can also find more comprehensive instructions to install TensorRT LLM in this [TensorRT LLM installation guide](https://nvidia.github.io/TensorRT-LLM/installation/build-from-source-linux.html), refer to that guide for common issues if you encounter any here.**
 
 ``` bash
 # Prerequisites
@@ -50,7 +50,7 @@ git lfs install
 # Replace with your actual path
 YOUR_WORK_PATH=<YOUR_WORK_PATH>
 
-# Clone the TensorRT-LLM repository
+# Clone the TensorRT LLM repository
 cd $YOUR_WORK_PATH
 git clone https://github.com/NVIDIA/TensorRT-LLM.git
 cd TensorRT-LLM
@@ -77,15 +77,15 @@ git clone https://huggingface.co/nvidia/DeepSeek-R1-FP4
 git clone https://huggingface.co/deepseek-ai/DeepSeek-R1
 ```
 
-#### 3. Build and run TensorRT-LLM container
+#### 3. Build and run TensorRT LLM container
 
 ``` bash
 cd TensorRT-LLM
 make -C docker run LOCAL_USER=1 DOCKER_RUN_ARGS="-v $YOUR_MODEL_PATH:$YOUR_MODEL_PATH:ro -v $YOUR_WORK_PATH:$YOUR_WORK_PATH"
 ```
 Here we set `LOCAL_USER=1` argument to set up the local user instead of root account inside the container, you can remove it if running as root inside container is fine.
 
-#### 4. Compile and Install TensorRT-LLM
+#### 4. Compile and Install TensorRT LLM
 Here we compile the source inside the container:
 
 ``` bash
@@ -122,11 +122,11 @@ The command to generate synthetic dataset will be attached to the max throughput
 
 This section provides the reproducing steps for NVIDIA Blackwell B200 and H200 GPUs, for both min-latency and max-throughput scenarios.
 
-All the benchmarking is done by the trtllm-bench command line tool provided in the TensorRT-LLM installation, see [TensorRT-LLM Benchmarking](https://nvidia.github.io/TensorRT-LLM/performance/perf-benchmarking.html) for details of this tool.
+All the benchmarking is done by the trtllm-bench command line tool provided in the TensorRT LLM installation, see [TensorRT LLM Benchmarking](https://nvidia.github.io/TensorRT-LLM/performance/perf-benchmarking.html) for details of this tool.
 
 For brevity, we only provide the commands to reproduce the perf numbers without detailed explanation of the tools and options in this doc.
 
-All these commands here are assumed to be running inside the container started by `make -C docker run ...` command mentioned in the [Build and run TensorRT-LLM container section](#3-build-and-run-tensorrt-llm-container)
+All these commands here are assumed to be running inside the container started by `make -C docker run ...` command mentioned in the [Build and run TensorRT LLM container section](#3-build-and-run-tensorrt-llm-container)
 
 ### B200 min-latency
 Our benchmark results are based on **Batch = 1, ISL = 1K, OSL = 2K, num_requests = 10 from real dataset**
@@ -158,7 +158,7 @@ trtllm-bench --model nvidia/DeepSeek-R1-FP4 \
 ```
 
 Explanation:
-- `trtllm-bench`: A CLI benchmarking utility that aims to make it easier for users to reproduce our officially published. See [TensorRT-LLM Benchmarking](https://nvidia.github.io/TensorRT-LLM/performance/perf-benchmarking.html) for details.
+- `trtllm-bench`: A CLI benchmarking utility that aims to make it easier for users to reproduce our officially published. See [TensorRT LLM Benchmarking](https://nvidia.github.io/TensorRT-LLM/performance/perf-benchmarking.html) for details.
 - `--dataset`: Prompt dataset used to benchmark. Our official benchmark dataset has ISL = 1K, OSL = 2K
 - `--num_requests`: Num requests used for the benchmark.
 - `--concurrency`: Total concurrency for the system.
@@ -186,7 +186,7 @@ Average request latency (ms):                     7456.1219
 
 Due to our evaluation found that FP8 KV cache does not introduce obvious accuracy drop compared to BF16 KV cache. See [Precision strategy](./tech_blog/blog3_Optimizing_DeepSeek_R1_Throughput_on_NVIDIA_Blackwell_GPUs.md#precision-strategy), the latest [DeepSeek-R1-0528-FP4](https://huggingface.co/nvidia/DeepSeek-R1-0528-FP4) checkpoint had enabled FP8 KV cache by-default.
 
-We are seeing meaningful speedup using FP8 KV cache, thus refreshing the numbers here. The results are reproduced with TensorRT-LLM commit b6261862419c33d6ce2313aff1e7116067d6037d.
+We are seeing meaningful speedup using FP8 KV cache, thus refreshing the numbers here. The results are reproduced with TensorRT LLM commit b6261862419c33d6ce2313aff1e7116067d6037d.
 
 !! Note that the exact command to reproduce numbers can change as the API/options are refactored, the option and numbers here is a reference at given exact commit.
 
@@ -239,7 +239,7 @@ Per GPU Output Throughput (tps/gpu):              5393.2755
 ### B200 max-throughput for R1 with FP16 KV cache
 Our benchmark results are based on **Batch = 3072, ISL = 1K, OSL = 2K, num_requests = 49152 from synthetic dataset**.
 
-The results are reproduced with TensorRT-LLM commit b6261862419c33d6ce2313aff1e7116067d6037d.
+The results are reproduced with TensorRT LLM commit b6261862419c33d6ce2313aff1e7116067d6037d.
 
 !! Note that the exact command to reproduce numbers can change as the API/options are refactored, the option and numbers here is a reference at given exact commit.
 
@@ -401,7 +401,7 @@ Average request latency (ms):                     181540.5739
 
 ## Exploring more ISL/OSL combinations
 
-To benchmark TensorRT-LLM on DeepSeek models with more ISL/OSL combinations, you can use `prepare_dataset.py` to generate the dataset and use similar commands mentioned in the previous section. TensorRT-LLM is working on enhancements that can make the benchmark process smoother.
+To benchmark TensorRT LLM on DeepSeek models with more ISL/OSL combinations, you can use `prepare_dataset.py` to generate the dataset and use similar commands mentioned in the previous section. TensorRT LLM is working on enhancements that can make the benchmark process smoother.
 ### WIP: Enable more features by default
 
 Currently, there are some features that need to be enabled through a user-defined file `extra-llm-api-config.yml`, such as CUDA graph, overlap scheduler and attention dp. We're working on to enable those features by default, so that users can get good out-of-the-box performance on DeepSeek models.

docs/source/blogs/Falcon180B-H200.md

@@ -1,13 +1,13 @@
 # Falcon-180B on a single H200 GPU with INT4 AWQ, and 6.7x faster Llama-70B over A100
 
-H200's large capacity & high memory bandwidth, paired with TensorRT-LLM's
+H200's large capacity & high memory bandwidth, paired with TensorRT LLM's
 optimizations, maximizes inference performance.
 
 ## Falcon-180B on a single H200 with INT4 AWQ
 [Falcon-180B](https://huggingface.co/tiiuae/falcon-180B), one of the largest &
 most accurate open source models available, can run on a *single* H200 GPU.
 
-The 141GB of memory on H200, paired with TensorRT-LLM running INT4 AWQ with
+The 141GB of memory on H200, paired with TensorRT LLM running INT4 AWQ with
 FP8, allows for the entire large language model to fit on a single GPU, where
 previously eight A100s were required. H200 Falcon-180B provides up to **800**
 tok/s and retains high accuracy.
@@ -30,7 +30,7 @@ BS: (in order) 256, 128 </sup>
 
 **Model Accuracy:**
 Often quantization can have adverse impacts on the accuracy of the model,
-however, TensorRT-LLM's AWQ decreases memory footprint of the model by **4x**
+however, TensorRT LLM's AWQ decreases memory footprint of the model by **4x**
 while maintaining high accuracy.
 
 <img src="https://github.com/NVIDIA/TensorRT-LLM/blob/rel/docs/source/blogs/media/Falcon180B-H200_acc.png?raw=true" alt="Falcon-180B accuracy comparison" width="600" height="auto">
@@ -52,18 +52,18 @@ retain higher accuracy than other 4bit methods and reduce memory usage, but
 requires special kernels capable of handling the change in precision
 performantly.
 
-TensorRT-LLM has implemented custom kernels for AWQ, and taken the technique a
+TensorRT LLM has implemented custom kernels for AWQ, and taken the technique a
 step further by performing FP8 computation on Hopper GPUs instead of the
 standard FP16.
 
-Similar examples running Falcon-180B with quantization in TensorRT-LLM are
+Similar examples running Falcon-180B with quantization in TensorRT LLM are
 available in [examples/models/contrib/falcon](/examples/models/contrib/falcon).
 
 ## Llama-70B on H200 up to 6.7x A100
 
-TensorRT-LLM has improved its Group Query Attention (GQA) kernels, in the
+TensorRT LLM has improved its Group Query Attention (GQA) kernels, in the
 generation phase, providing up to 2.4x improvement on Llama-70B over
-TensorRT-LLM v0.5, achieving over **3,800** tok/s/gpu at up to **6.7x** faster
+TensorRT LLM v0.5, achieving over **3,800** tok/s/gpu at up to **6.7x** faster
 than A100.
 
 **H200 6.7x A100**
@@ -106,7 +106,7 @@ BS 192 </sup>
 [**Grouped Query Attention (GQA)**](https://arxiv.org/abs/2305.13245v2)
 (Ainslie et al., 2023), used in Llama-70B, is a variant of Multihead Attention
 (MHA) which groups key-value (KV) heads together, resulting in fewer KV heads
-than query (Q) heads. TensorRT-LLM has a custom implementation of MHA which
+than query (Q) heads. TensorRT LLM has a custom implementation of MHA which
 supports GQA, multi-query attention (MQA) and standard MHA. It leverages Tensor
 Cores, including in the generation phase, and delivers great performance on
 NVIDIA GPUs.
@@ -116,7 +116,7 @@ NVIDIA GPUs.
 These improvements will be published in the `main` branch soon, and will be
 included in the v0.7 & v0.8 releases.
 
-Similar examples running Llama-70B in TensorRT-LLM are published in
+Similar examples running Llama-70B in TensorRT LLM are published in
 [examples/models/core/llama](/examples/models/core/llama).
 
 For more information about H200, please see the [H200 announcement blog](./H200launch.md).