Skip to content

Changelog

RF-DETR release notes are maintained on GitHub Releases. Use the release feed to review versioned package changes, migration notes, and model updates.

Changelog

All notable changes to RF-DETR are documented here.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[Unreleased]

Changed

  • Segmentation prediction with RFDETR.predict() and RFDETRInference.predict() no longer copies every full-resolution mask on the device a second time when all detections pass the threshold. Post-processing already drops masks below the score threshold, so decoding now selects mask rows only when it removes some, as with per-class thresholds or a custom post-processor. Returned masks stay independent of the post-processor's tensors. On a Tesla T4, decoding 100 masks at 1920×1080 peaked at 198 MiB less CUDA memory and took 15.8 ms instead of 17.6 ms; this covers mask decoding and the copy to the CPU, not the model forward pass or mask resizing. Predictions are unchanged. (#1688)

Fixed

  • RFDETRInference.predict() without batch now predicts a directory, glob or recorded video in batches of the fixed batch size of a predictor from RFDETR.inference(batch_size=N) or of a fixed-batch export, with either stream=True or stream=False. It used one image per batch, so any N above one failed on the first batch with Batch size mismatch unless the caller passed batch=N. A final batch with fewer than N images is still refused, not padded. Live captures and .streams manifests keep one frame per source in each simultaneous batch, regardless of batch; their source count must match the runtime's fixed batch size.

[1.12.0] — 2026-11-05

Added

  • Public rfdetr.RFDETRInference provides shared image prediction from a live RFDETR model, native checkpoint or supported full export, with common preprocessing and decoded Supervision detections or keypoints. (#1563)
    • Exported artifacts carry versioned inference metadata: embedded in ONNX files and written as <artifact-name>.rfdetr.json sidecars for other formats. RFDETRInference uses it to validate the artifact interface and recover prediction semantics; sidecars include an artifact digest to detect stale metadata. (#1563)
    • RFDETR.write_export_metadata(artifact, ...) writes inference metadata for an export produced by an older release, so RFDETRInference can load it. (#1563)
    • RFDETRInference(..., runtime_options=...) passes per-format runtime settings to the exported runtime: TensorRT cuda_graph, engine_host_code_allowed and verbose; OpenVINO cache_dir, inference_precision and config; TFLite and LiteRT num_threads. An unknown key raises ValueError, and native sources reject runtime_options. (#1563)
    • RFDETRInference.predict() accepts video files, directories, globs, CSV and .streams manifests, cameras, RTSP and HTTP live sources, YouTube URLs and screen capture. stream=True returns a generator; batch, vid_stride and stream_buffer control batching and frame reading, and collecting a stream's results in memory warns past 1 GiB of retained frames. (#1563) (#1577)
    • New rfdetr[stream] extra installs opencv-python-headless>=4.8.1.78, mss>=9 and yt-dlp>=2026.7.4 for camera, live-URL, YouTube and screen sources. (#1577)
  • RFDETR.predict() takes any iterable of images, path-like inputs and a BCHW tensor, which is split into its images and predicted as one batch. (#1577) (#1563)
  • RFDETR.predict(threshold=...) and RFDETRInference.predict(threshold=...) accept one threshold per class through the new rfdetr.ClassThresholds. Name the classes that need their own bar and let the rest take a default, as in ClassThresholds({"person": 0.35, "traffic light": 0.6}, default=0.5), or give one float per entry of class_names in the same order, as in ClassThresholds([0.4, 0.55, 0.45]); each detection is kept when its score exceeds the threshold of its own class, and a single float applies to every class as before. For a pretrained COCO checkpoint the thresholds still follow class_names, not the sparse COCO category IDs. Per-class thresholds drop the detections that belong to no entry of class_names (the no-object slot, unused COCO category IDs), which a single float keeps. A NaN, or a sequence that is not one-dimensional, raises ValueError when the ClassThresholds is built; an unknown class name, a sequence of the wrong length, or a model without classes raises ValueError before inference; and a threshold that is neither a number nor a ClassThresholds, such as a plain list or dict, raises TypeError. The per-class confidence thresholds cookbook shows how to choose the values and measures on COCO val2017 when they beat a single threshold and when they do not. (#1626) (#1635)
  • Keypoint training accepts Kornia augmentation on CUDA. The default backend remains cpu; users can select kornia/gpu explicitly, or select auto to use Kornia when available. Image geometry, boxes, padding masks and keypoints share each augmentation draw; keypoint visibility, dropped instances and paired-joint flips stay synchronized. (#1606)
  • TrainConfig.eval_backend="hotcoco_streaming" matches each validation batch with hotcoco's StreamingEval as it arrives, so the end-of-epoch COCO evaluation only accumulates and summarizes. Metrics are bit-identical to eval_backend="hotcoco". On an Apple M-series CPU with synthetic data, the end-of-epoch step dropped from 0.238 s to 0.025 s for 600,000 box detections and from 0.178 s to 0.005 s for a bbox+segm run of 20,000 detections. Total CPU time is higher for box-only runs (1.05 s vs 0.29 s) and about equal for bbox+segm (0.66 s vs 0.71 s), because matching moves into the validation step. Streaming applies to single-process runs whose labels fall within [0, num_classes]; under DDP, or after the metric is pickled mid-epoch, it evaluates in one batch as "hotcoco" does and logs why. (#1590) (#1614)
  • format="coreml" now embeds notes in the .mlpackage metadata (MLModel.user_defined_metadata, key rfdetr_notes, encoded as in the ONNX export) instead of warning and dropping it, and stamps the RF-DETR version under rfdetr_version. (#1024) (#1609)
    • format="coreml" now validates notes before the forward pass, as the other formats that embed it do. A value JSON cannot encode used to export with a warning that notes was dropped; it now raises ValueError (NaN/Infinity, a circular reference) or TypeError (a non-JSON object such as a pathlib.Path). (#1024) (#1609)
  • Extension points for new backbones and Plus models:
    • rfdetr.RFDETRAtto, rfdetr.RFDETRFemto and rfdetr.RFDETRPico: real-time detection models with a PE-Core-T backbone, provided by rfdetr_plus like the XLarge models (pip install "rfdetr[plus]", Platform Model License 1.0). RFDETR.from_checkpoint() resolves their checkpoints; an outdated rfdetr_plus raises an upgrade hint, and a broken rfdetr_plus install no longer blocks loading core checkpoints. (#1568)
    • rfdetr.models.backbone.register_backbone(encoder, backbone_cls): an extension package can provide a non-DINOv2 encoder for a ModelConfig.encoder name by subclassing Backbone and overriding _build_encoder. A registered encoder that defines set_export_shape(shape) gets its position embeddings frozen for export, like DINOv2. (#1568)
    • ModelConfig.dim_feedforward (default 2048) sets the decoder feed-forward width. (#1568)
    • RFDETRModelModule(..., load_encoder_weights=False) skips fetching the encoder's upstream pretrained weights when every weight is replaced right after construction; the default True keeps the previous behaviour. RFDETR.evaluate() passes it. (#1568)
  • RFDETR.export(format="tensorrt", quantization="int8", calibration_data=...) builds an INT8 TensorRT engine (*_int8.trt) for detection, segmentation and keypoint models. The mask head and keypoint head stay FP16. The 3-channel patch embedding of the detection models, which TensorRT cannot run in INT8 as a convolution without padding its input, is exported as a matrix multiply so that it runs in INT8 too; the 12-pixel patch embedding of the segmentation and keypoint models is a matrix multiply that stays FP16. For the detection models the export also smooths the backbone LayerNorm outputs (SmoothQuant) and corrects the bias of every backbone MLP layer for the mean shift INT8 adds, both measured on the calibration images and folded into the model's constants: on the RTX 5070 the box AP lost at batch 1 is 0.50, 0.47, 0.66 and 0.32 points (Nano, Small, Medium, Large), and on the Tesla T4 (TensorRT 11.4) the detection models lose 0.50, 0.36, 0.65 and 0.27 points; a detection export now reads the calibration images three times. Every row used TensorRT 11.4. On an RTX 5070 (native Linux) and a Tesla T4 (Colab), at batch 1, 4 and 8, INT8 is 1.11× to 1.42× faster than FP16 on the RTX 5070 and 1.08× to 1.41× on the T4 with a CUDA graph. At batch 1 it loses 0.32 to 1.01 AP points on the RTX 5070 and 0.27 to 0.97 on the T4 (box AP for detection, mask AP for segmentation, keypoint AP for keypoints, on all 5000 val2017 images, calibrated on 128 train2017 images). The INT8 engine is 24% to 47% smaller than the FP16 one on the RTX 5070 and 26% to 59% smaller on the T4, and on the T4 the segmentation and keypoint INT8 engines do not grow with the batch. Section 8.1 of the CUDA cookbook reproduces these numbers on your GPU. dynamic_batch=True and backbone_only=True are refused, and the engine needs fp16=True and TensorRT 10 or newer. Calibration runs calibration_data through the FP32 graph with onnxruntime on the CPU, so host memory grows with batch_size. The TensorRT INT8 section has the per-model tables and the limits. (#1024) (#1595) (#1627) (#1638) (#1642) (#1645)
  • Public export helper modules:
    • Public rfdetr.export.inference exposes TRTInference, load_executorch_method, preprocess_to_nchw, decode_detections and DecodedDetections next to OpenVINOInference; each resolves lazily, so importing the module does not require an optional runtime. (#1585)
    • Public rfdetr.export.benchmark exposes measure_latency, measure_memory, BenchmarkResult and MemoryResult, the helpers the per-hardware export cookbooks use. (#1585)
  • TRTInference(cuda_graph=True) (rfdetr.export.inference.TRTInference; python -m rfdetr.export.benchmark does not expose it) captures the engine's launch into a CUDA graph on the first call and replays it on every later call, which removes most of the per-call launch cost at small batch sizes. It is off by default. Measured as wall-clock time per call, including the copy of the input, against the same engine run with sync_mode=True (3 x 1000 calls, minimum of the medians, RTX 5070 under WSL2): with TensorRT 10.16 a Nano FP16 engine took 1.65 ms at batch 1 and a replay 0.99 ms (1.67x faster), Small 2.10 ms and 1.49 ms (1.41x); at batch 8 the gain was 1.09x for Nano and 1.05x for Small. With TensorRT 11.3 Nano gained 1.71x and Small 1.42x at batch 1. The default async path, which needs pycuda, was not measured. The outputs matched the plain call (bit for bit on the engines measured; repo tests check within 1e-3). Only an engine whose optimization profile fixes every input shape, the batch included, is accepted, so a runtime keeps one graph: an engine exported with dynamic_batch=True, or one whose image size varies, is refused with a ValueError at construction, because captured graphs would share one execution context that a change of shape modifies between replays, which TensorRT documents as undefined behaviour. The graph held 0.57 to 0.73 MiB of device memory and 1.6 to 2.1 MiB of host memory on the Nano and Small engines measured. A runtime is not safe to call from several threads. The option uses its own torch stream, so it does not need pycuda, and it raises instead of falling back when combined with sync_mode=True or when the engine cannot be captured. (#1583)
  • RFDETR.export(format="tensorrt") engine options:
    • RFDETR.export(format="tensorrt", trt_metadata=True) writes <engine>.json next to the engine. It lists what a program needs to run the engine without importing the model: the input name, size, layout and normalization (RGB order, divide by 255, mean and std, and the resize method), the output names in order, the batch profile (static size or dynamic min/opt/max), the precision actually built, the opset, the TensorRT version and GPU, notes, and the engine file's size and SHA-256, so a consumer can check that the description file belongs to the engine it loads. A .trt file has no slot for this, so a C++, Triton or DeepStream consumer had to be told by hand, and notes only reached the intermediate ONNX. The file is written atomically and, on Linux and macOS where permitted, takes the engine's permission bits and group. When a build writes no description file (the option off, or TensorRTExporter.build_engine), an earlier description file with the same name is reported with a warning, never deleted. Class names and the background slot are not recorded yet. Off by default; other formats warn and ignore it. (#1024) (#1581)
    • RFDETR.export(format="tensorrt") takes trt_hardware_compatibility ("ampere_plus" or "same_compute_capability") and trt_version_compatible=True. They ask TensorRT for an engine that other GPUs, or other releases of the same TensorRT major version, may load; every engine used to be tied to the GPU and TensorRT version that built it. We had one GPU, so running on a second one is untested. Version compatibility worked between TensorRT 11.2 and 11.3 in both directions, and didn't work across major versions or between 10.13 and 10.16. It needs TensorRT's lean runtime package, which the pip wheels don't include; without it the export stops early with an ImportError that names it. On an RTX 5070 with TensorRT 11.3 (static-batch RFDETRNano fp16, median of fresh-process build_engine calls, latency from CUDA events): "ampere_plus" took 130 s to build instead of 38 s, made a 99 MB engine instead of 62 MB and ran 10.9% slower (RFDETRSmall: 9.6%); "same_compute_capability" cost nothing measurable; trt_version_compatible added 105 MB and ran as fast (within 1%). COCO AP stayed within build-to-build noise for every option. A portable engine gets its own file name suffix, so it doesn't overwrite a default engine (unless output_name is set). TRTInference and rfdetr.export.benchmark take engine_host_code_allowed (off by default), which a TensorRT 11 version-compatible engine needs because it carries host code. Both options are off by default; other formats warn and ignore them. (#1024) (#1580)
    • RFDETR.export(format="tensorrt", trt_timing_cache="<path>") saves TensorRT's kernel timings to a file and reuses them on the next build. Every build used to time all kernels from scratch: rfdetr never gave Polygraphy a timing cache. Rebuilding the same model at the same precision and batch profile (after fine-tuning, say) now skips most of that work. On an RTX 5070 with TensorRT 11.3, build_engine for a static-batch RFDETRNano went from 37.9 s to 8.4 s in fp16 and from 12.9 s to 7.9 s in fp32 (median of three runs, each in a fresh process). A cache from the pretrained model did as well for a model fine-tuned to 3 classes (37.4 s to 9.0 s in fp16). The engine is the same: its outputs match the build that filled the cache bit for bit, and its latency is within 2% of a build without the cache. A cache from another TensorRT major version, or an empty or corrupt file, doesn't stop the build; TensorRT logs an error and the file is replaced. Off by default; other formats warn and ignore it. (part of #1024, #1579)
  • RFDETR.inference(compile_backend="cudagraph") adds a direct CUDA Graph replay backend for fixed batch size and resolution, the default on CUDA (see Changed). It freezes the TorchScript trace, captures its forward once, and clones graph-owned outputs so later calls cannot overwrite earlier results; compile_backend="cudagraph" on a non-CUDA device raises ValueError. On one NVIDIA L4 VM with RF-DETR Nano, FP16, batch 1, and the public PIL-image predict() path, seven sequential fresh-process paired runs reduced median end-to-end latency from 9.719 ms to 3.884 ms (60.03% lower, 2.50x throughput; paired throughput range 2.43-2.57x). Total setup took a 7.37 s median and graph capture added 29.1 MiB of allocated CUDA memory over the retained trace in the benchmark. One full COCO val2017 pass produced exactly the same AP/AP50 (0.480240/0.671360), with 1.5 million ranked detections per arm; its descriptive, single-process median predict() latency was 9.887 ms versus 4.051 ms. (#1547)
  • RFDETR.predict() accepts a keyword-only antialias flag (default False, unchanged behaviour). Pass antialias=True for checkpoints trained with torchvision resizing; exported models still resize without antialiasing. On MPS with PyTorch older than 2.7, antialiased resizing runs on the CPU. (#1525)
  • RFDETR.export(format="coreml", coreml_neural_engine=True) exports an Apple Neural Engine friendly graph (einsum and one-hot top-k rewrites). Off by default and refused with float32 precision. (#1586)
  • Static INT8 post-training quantization for format="onnx" and format="openvino", via quantization="int8" plus a required calibration_data (a directory of representative images, a .npy file of shape (N, C, H, W), or an equivalent array). ONNX writes {name}_int8.onnx beside the FP32 graph it was derived from; OpenVINO compresses the IR with NNCF (pip install nncf, deliberately outside the [openvino] extra) before writing it. Unlike TFLite's dynamic-range "int8", which ignores calibration data, static quantization reads activation ranges from it, so omitting it is refused rather than defaulted. Only MatMul/Gemm run in 8-bit: attention-score multiplies, the detection heads, normalization, softmax and the surrounding elementwise math stay in float, and the quantized ops' outputs are not requantized. Measured on a 500-image COCO val2017 subset, ONNX Runtime's default op coverage instead costs 9.65 mAP on RFDETRNano and 12.14 on RFDETRSmall, against 2.65 and 2.67 for this configuration, and is also slower — the conversions around non-matmul ops cost more than the 8-bit kernels save. Expect roughly 3x smaller artifacts and ~1.4x faster CPU inference at batch 1, at a real accuracy cost worth measuring on your own data. (#1560)
  • build_trainer(extra_callbacks=[...]) adds callbacks to the RF-DETR ones before the Trainer is built, so Lightning checks them together: a second progress bar raises, and a ModelSummary takes the place of the default summary. Passing callbacks= still replaces the RF-DETR callbacks. The rfdetr CLI hands --trainer.callbacks to it. (#1629)

Breaking Changes

  • Detection and segmentation training with any augmentation_backend that resolves to Kornia now raises ValueError during setup with square_resize_div_64=False or multi_scale="per-sample", as keypoint training already did. Those settings collate images of different sizes into a padded canvas, and the Kornia path normalizes boxes by that canvas, so smaller images trained on shrunken targets, and segmentation failed later with a tensor shape error. Use square_resize_div_64=True with multi_scale="per-batch" or "off", or augmentation_backend="torchvision" or "albumentations". (#1680)
  • RFDETR.inference() now returns an independent RFDETRInference predictor. Capture the return value and call predictor.predict(images); the source model is unchanged and later source mutations do not affect the predictor. The removed inplace keyword is rejected. Each predictor owns one copy of the current base weights; delete it when its runtime is no longer needed. The default batch size 1 and float32 settings are unchanged; the default compile backend now follows the device (CUDA graph on CUDA, inductor on MPS, TorchScript elsewhere), see Changed. Pass compile_backend="torchscript" to keep the previous default. (#1563)
  • format="coreml" with coreml_precision="float16" now returns float32 output arrays; the .mlpackage returned float16 outputs before, because it was converted as an iOS 16 program and is now converted as iOS 15; bundles now declare iOS 15 / macOS 12 as the minimum OS, down from iOS 16 / macOS 13. A consumer that reads the outputs as float16 (a typed buffer, a dtype check) must read float32, or cast the arrays with astype(np.float16). coreml_precision="float32" bundles are unchanged. (#1024) (#1604)
    • Accuracy: the float16 bundle lost about 3 box AP on the Apple Neural Engine. The bundle was converted for iOS 16, whose program runs the deformable-attention resample on the Neural Engine in fp16; it is now converted for iOS 15, where that op stays on the CPU, which also matches what the ExecuTorch CoreML delegate lowers. Measured on an Apple M3 Pro (macOS 27.0.1, pretrained models, all 5000 COCO val2017 images) under CPU_AND_NE: RFDETRNano 45.06 → 48.04 AP (eager 48.03) and RFDETRSmall 50.70 → 52.76 AP (eager 52.80); CPU_AND_GPU was already unaffected, and fp32 scores 48.03 on RFDETRNano either way. The cost is single-image latency on the Neural Engine, RFDETRNano fp16 under CPU_AND_NE (p50, batch 1): 10.8 → 12.7 ms back to back and 18.3 → 24.9 ms with 200 ms between passes; RFDETRSegNano 18.1 → 20.3 ms and 30.4 → 38.8 ms (the pairs were measured in the same session). (#1024) (#1604)
  • rfdetr[onnx] no longer installs polygraphy or onnx_graphsurgeon, so a host that exports to TensorRT or TFLite with rfdetr[onnx] plus a separately installed tensorrt or onnx2tf must install the missing package itself (pip install polygraphy onnx_graphsurgeon); see the matching entry under Removed. (#1588) (#1589)
  • Custom Affine configs on the Kornia augmentation backend behave differently from 1.11.2:
    • A custom Affine(translate_px=...) config on the Kornia augmentation backend behaves differently from 1.11.2. That release ignored translate_px: the config built, never shifted the image and, when rotate was omitted, rotated it by a random ±15°. The shift is now applied, the implicit rotation is gone, and combinations Kornia cannot reproduce, such as rotation, scale, shear, a nonzero fill or reflected borders, raise ValueError when the pipeline is built. Pass augmentation_backend="cpu" (Albumentations, rfdetr[augment]) to keep them; augmentation_backend="auto" resolves to Kornia on a CUDA machine with Kornia installed. Fixed values, signed ranges and per-axis ranges are supported for pure translation. (#1607)
    • A custom Affine config without translate_px on the Kornia augmentation backend that does not set rotate no longer rotates images. It used to sample a random ±15° rotation, which the Albumentations backend does not do, so an unset rotate now means no rotation on both. Set rotate=(-15, 15) to keep the old behavior. For configs with translate_px, use the Albumentations backend to combine pixel translation and rotation. The built-in presets are unaffected. (#1679)
  • In eval mode, a segmentation LWDETR no longer returns pred_masks in aux_outputs or enc_outputs unless it is called with all_layer_masks=True; the final pred_masks are unchanged. Code that runs SetCriterion on eval-mode outputs, such as an evaluation loop that computes a segmentation loss, got every layer's masks before and must now pass that keyword. Without it the criterion raises a ValueError that names it. See the matching entry under Changed. (#1628)
  • RFDETR.export() checks batch_size before any other work: batch_size=True, which used to export a batch of 1, now raises ValueError, as does any other value that is not a positive integer (2.0, "2" and None raised a TypeError from inside the export). A numpy integer is passed on as a plain int. Pass a positive integer, such as batch_size=1, to keep exporting. (#1556)

Changed

  • RFDETR.inference() picks its compile backend from the device when compile_backend is not given:
    • On MPS, RFDETR.inference() compiles with inductor when compile_backend is not given, and when the installed PyTorch has an MPS code generator for it; CUDA captures a CUDA graph (below) and every other device keeps TorchScript. The first predictor takes longer to build (several seconds, and again for each new input shape); pass compile_backend="torchscript" for the previous default. (#1668) (#1672)
    • On CUDA, RFDETR.inference() captures a CUDA graph (compile_backend="cudagraph") when compile_backend is not given, instead of tracing with TorchScript alone, which removes the per-kernel launch cost of each forward pass. The capture runs when the predictor is built: it synchronizes the device, empties the process-wide CUDA and pinned-host caches and keeps a graph-private memory pool, so build predictors before serving threads start. A failed capture can leave CUDA random-number state unusable until the process restarts. Pass compile_backend="torchscript" for the previous default. (#1672)
  • Host copies and post-processing:
    • On non-CUDA accelerators such as MPS, prediction no longer waits on the device per image: a batch's detection boxes, scores and labels reach the host in one copy before the per-image threshold filter, and image normalization reuses its mean and std tensors instead of rebuilding and checking them on the device every batch. Results are unchanged. (#1668)
    • On CPU, the box-only PostProcess path given score_threshold returns only the rows above it and sorts only those rows instead of every query/class pair, which cuts post-processing of exported models from about 1.6 ms to 0.1 ms per image; predicted detections are unchanged. (#1664)
  • On MPS, RFDETR.inference() and RFDETRInference build a faster predictor:
    • On MPS, RFDETR.inference() builds a faster predictor: its copy of the model samples deformable attention with PyTorch's fused F.grid_sample instead of the gather-based sampler kept for training, and, for TorchScript and compile=False predictors, folds each DINOv2 LayerScale into the Linear layer before it. At float32, predictions match the source model to float rounding, not bit for bit, and half precision can differ by more; the source model, other devices, the inductor backend's LayerScale and exports are unchanged. (#1666)
    • On MPS, RFDETRInference("weights.pth") (or a .pt or .ckpt path) applies the same model rewrites as RFDETR.inference() to the model it loads: fused deformable sampling and LayerScale folded into the Linear before it. Predictions match the unmodified checkpoint to float rounding, not bit for bit. A live model passed to RFDETRInference is never rewritten. (#1668)
  • RFDETRInference.predict() overlaps decoding and input preparation with the running batch:
    • On an accelerator such as CUDA or MPS, RFDETRInference.predict() decodes the next batch of image files on worker threads while the current batch runs, for directories, globs, image files and lists of them predicted in batches; results are unchanged. Cameras, live URLs, screens, videos named directly, manifests and models predicting on the CPU keep reading one batch at a time. (#1664)
    • Exported runtimes, such as Core ML or ONNX Runtime, now decode the next batch of image files on worker threads while the current batch runs, like native models on an accelerator; a native model on the CPU still decodes inline. Results are unchanged. (#1668)
    • Owned predictors (a path source or an RFDETR.inference() snapshot) build the next CUDA batch's input while the current batch runs; a borrowed live model keeps running batches one at a time. Results are unchanged. (#1670)
    • With ONNX Runtime's CUDA execution provider, RFDETRInference preprocesses each batch on the GPU and hands it, and freshly allocated outputs, to ONNX Runtime through IO binding instead of copying both through host memory. It keeps the host path when PyTorch cannot see the GPU, an output is a scalar or does not lead with the input's batch axis, or an output has a dynamic axis other than the batch. The CPU provider is unchanged. (#1672)
    • On a CUDA device and with include_source_image=False, RFDETRInference.predict() decodes the baseline color JPEG files of a directory, glob, manifest or batched list on the GPU with nvjpeg, one call per batch, instead of on CPU worker threads. The first file of each chroma subsampling is compared with the CPU decode, and a difference, or an nvjpeg error on every file of a batch, sends that kind of file back to the CPU decoder for the rest of the process; a single file nvjpeg refuses decodes on the CPU alone, so the model input stays byte-identical. Grayscale, CMYK, progressive and PNG files, and every other device, decode on the CPU as before. (#1672)
  • RFDETRInference.predict() without batch predicts directories and globs in batches:
    • RFDETRInference.predict() with batch=None now predicts an eager (stream=False) directory or glob, or a list of them, in batches of eight on a model without a fixed batch size; it ran one image per batch before. Batched results can differ from single-image ones by floating-point rounding. Streamed calls, manifests, compiled snapshots and fixed-batch exports keep one image per batch (exports with a dynamic batch axis batch as described in the item below); pass batch=1 to keep the previous behaviour. (#1664)
    • RFDETRInference.predict() without batch runs a directory or glob in batches of eight, or of the export's max_batch_size when that is smaller, on an export with a dynamic batch axis (ONNX, OpenVINO, a dynamic-batch TensorRT engine); it ran one image per batch. Streams and fixed-batch exports keep one image per batch. Batching raises peak memory (about 35% more resident memory for a Nano ONNX export on the ONNX Runtime CPU provider, with no measured gain in throughput there); pass batch=1 on a memory-bound host. On CUDA, the outputs of ONNX Runtime and of a TensorRT fp16 engine change slightly with the batch size, so a detection close to the threshold can appear or disappear on some images; the CPU providers of ONNX Runtime and OpenVINO gave the same detections at both batch sizes. Pass batch=1 for the previous results. (#1672)
  • TrainConfig.eval_backend is faster:
    • eval_backend="vernier" (the default) requires vernier>=0.6.0, whose compute() on RF-DETR validation state is 2.6× faster than "ufcoco" for boxes and 5.3× for boxes and masks, with identical metrics. (#1661)
    • The hotcoco evaluation backend requires hotcoco 1.2 (hotcoco>=1.2.1,<1.3, previously pinned to 1.0.1). A bbox+segm evaluation now switches prediction areas between IoU types with COCO.update_anns instead of rebuilding the prediction dataset per IoU type; its faster compute() is the combined effect of the hotcoco bump and update_anns. Box-only evaluation does not use update_anns; its gain comes from the hotcoco bump. No absolute timings are given, because they depend on the core count, which was not recorded. (#1590) (#1614)
    • eval_backend="ufcoco" loads box-only detections into COCO's loadRes as one array, as "hotcoco" already did, instead of building one Python dict per detection. On COCO val2017 with 733,070 detections at eval_max_dets=500, compute() falls from 2.64 s to 0.81 s with unchanged metrics. rfdetr[train] now requires ultrafast-pycocotools>=0.1.13, the first release that keeps those rows in columns through evaluation. (#1584)
  • The live loss values on the progress bar (loss, loss_cls, loss_box, loss_giou, and the mask and keypoint terms) show the mean of every step since the epoch started, weighted by batch size, instead of the latest batch. Each rank computes its own mean. The mean restarts each epoch. The mean applies to the default train_log_on_step=False; with train_log_on_step=True every progress term stays per batch, as before. The logged train/ metrics and the logger output are unchanged, but trainer.callback_metrics now holds the mean for these keys, so a scheduler or callback that monitors one of them (for example lr_scheduler_monitor set to loss) sees the mean, not the latest batch. (#1620) (#1625)
  • The rfdetr[plus] extra now requires rfdetr_plus>=1.1.0 (was >=1.0.1). An older rfdetr_plus that is already installed still works for the XLarge models, because the PE models are imported in their own block. (#1568)
  • build_backbone now forwards ModelDefaults.force_no_pretrain to the encoder builder. A caller that already passed defaults=ModelDefaults(force_no_pretrain=True) used to get DINOv2's upstream encoder weights anyway and now gets randomly initialised encoder weights, with pretrain_weights handled separately as before. The default False leaves every other caller unchanged. (#1568)
  • Export quantization is checked instead of silently ignored:
    • quantization for format="onnx" and format="openvino" is now checked: a value other than None, "fp32" or "int8" raises ValueError instead of being ignored, and quantization="int8" without calibration_data raises instead of silently exporting FP32. ONNX quantization="int8" returns {stem}_int8.onnx, written beside the FP32 {stem}.onnx, rather than the FP32 path. OpenVINO quantization="int8" writes {stem}_int8.xml/.bin so it no longer overwrites the FP32 IR; an explicit output_name is still used verbatim. (#1560)
    • format="tensorrt" no longer drops quantization and calibration_data silently. It used to build an FP16 or FP32 engine whatever they said; it now raises ValueError for a quantization other than None or "int8" ("fp16" and "fp32" included: choose the precision with fp16), and for calibration_data without quantization="int8". (#1024) (#1595)
  • Eager training (compile=False, the default) spends less of every step on layout and cast kernels in the decoder. Five rewrites in models/transformer.py, taken only on NVIDIA CUDA in eager execution (ROCm builds, torch.compile, tracing/export, torch.func transforms, CPU, MPS and XLA keep the previous ops): gen_sineembed_for_position divides by the dim // 2 distinct frequencies once and writes sin/cos interleaved straight into the output (in the autocast dtype when ref_point_head and its first linear have plain calls) instead of two strided slices, a stack and a cat; with_pos_embed sums that only feed autocast-cast matmuls are emitted in the compute dtype in one kernel; the grouped self-attention runs nn.MultiheadAttention's own projections on the ungrouped layout with the group regrouping as a view, removing three cats, their backward copies and the module's batch_first transposes; the FFN's relu(linear1(x)) applies bias and ReLU in the cuBLASLt GEMM epilogue (torch._addmm_activation, with the two-op backward spelled out, itself differentiable); and the two-stage selection casts the shared encoder memory once before its group_detr-fold expansion. The sine embedding and the FFN epilogue also run in eval(). Rewrites are limited to the plain modules the layer builds and to calls nothing observes or overrides: a subclass, a forward or backward hook (module or global), a compile() wrapper, an instance-level forward, an overridden with_pos_embed, a tensor-subclass weight, a non-fp32 source under autocast and an odd embedding width keep the previous ops, as do masks, nonzero attention dropout and eval mode for the self-attention. On the tested stack (NVIDIA L4, real COCO images) forward outputs, every loss term and the total loss are bitwise identical to the previous code; the GPU parity tests pin fp32 matmul precision to "highest", so the trainer's TF32 default ("high") is not covered, and the FFN's equality relies on cuBLASLt picking the same GEMM for the ReLU epilogue as for F.linear, which other GPUs or torch versions need not do. Gradients are not bitwise: the self-attention weight gradients reduce over the tokens in a different row order (at most one bf16 ulp under bf16 autocast), the folded positional adds sum the gradients of their two consumers in the compute dtype before the upcast (one extra rounding of at most half an ulp), and the module path's deformable-attention backward is not deterministic either. FlopCounterMode has no formula for torch._addmm_activation, so it undercounts the FFN on a CUDA model in eager execution. On an NVIDIA L4 (RF-DETR Nano, 384 px, bf16 autocast, real COCO val2017 images, five alternating fresh-process pairs), the eager training step at batch 56 — the largest that fits — went from 1245.1 ms to 1123.5 ms (−9.8%, ranges disjoint; −9.6% on a second VM) and peak allocated memory from 19.46 to 18.33 GiB; a public RFDETRNano().train() call at batch 32 with the default multi-scale recipe (7 workers, one epoch over 5000 images) went from 181.3 s to 170.7 s (−5.8%). The default batch-4 recipe, loader-bound on that GPU, stays within its run-to-run spread (139.5 s vs 138.9 s for a two-epoch train() call). (#1554)
  • Local .jpg, .jpeg and .png paths passed to predict() now use torchvision's native decoder instead of materializing an intermediate Pillow image. A natively decoded file is read once, and Pillow validates its header and decompression-bomb limit on those same bytes. Animated PNGs, files that are not JPEG or PNG despite their name, 16-bit PNGs (and any decode that is not uint8) and files the native decoder rejects fall back to Pillow, which opens the path again. Local files also fall back to Pillow when torchvision's image codecs, deprecated since torchvision 0.29, are unavailable or their deprecation warning is raised as an error, so import rfdetr and predict() keep working once torchvision removes them. On an NVIDIA L4, pretrained Nano FP16 TorchScript inference with include_source_image=False over a batch of eight real COCO images fell from a median of 32.76 ms to 28.36 ms end to end across seven independent runs (13.41% median paired reduction, 12.61-13.77% range); detections were identical. (#1550)
  • Pretrained weights download from https://repo.roboflow.com/rfdetr, a CDN in front of the same files, instead of https://storage.googleapis.com/rfdetr. Object paths and MD5 hashes are unchanged, so weights already in the cache are not downloaded again. A network that only allows listed hosts needs to allow repo.roboflow.com; the old URLs keep working for earlier releases. (#1600)
  • format="litert" exports sample the deformable attention from zero-padded pixel rows. litert-torch lowered grid_sample to coordinate and GATHER_ND index tensors expanded over every channel, all on LiteRT's single-threaded default kernels outside the XNNPACK delegate. The export now computes one float32 index per sample point and bilinear corner, which XNNPACK runs, and reads each corner with one EMBEDDING_LOOKUP. The values are the same as F.grid_sample up to float32 rounding, and only the LiteRT capture changes: eager inference, training and the other export formats are unaffected. Median latency with ai-edge-litert 2.2.0 on CPU at its default thread count, over the first 100 COCO val2017 images, before → after: Intel Xeon (Haswell, 8 cores) Nano 168.6 → 78.1 ms, Small 259.9 → 129.9 ms, Medium 340.1 → 166.0 ms, Large 431.1 → 256.8 ms, XLarge 1432.5 → 572.9 ms, 2XLarge 1676.3 → 820.2 ms; Apple M4 Max Nano 78.4 → 38.5 ms, Small 112.8 → 54.8 ms, Medium 141.4 → 68.9 ms, Large 162.9 → 94.1 ms, XLarge 519.4 → 183.2 ms, 2XLarge 565.0 → 223.2 ms. mAP@50:95 on those images is unchanged to within 0.01. (#1587)
  • format="executorch", backend="xnnpack" keeps the attention of the encoder and decoder of the detection and segmentation models inside the XNNPACK delegate. The export rewrites float32 attention without a mask as two batched matrix multiplications and a softmax, without the guards for fully masked rows, and computes the query, key and value slices of the decoder's packed in_proj_weight at export time. Attention with a mask (the keypoint decoder's instance self-attention can set one), broadcast leading dimensions, empty tensors or another dtype keeps the default path. On RFDETRNano the XNNPACK partitions decrease from 87 to 57. The figures below were measured with ExecuTorch 1.5.1, which is outside the rfdetr[executorch] pin (executorch>=1.3,<1.4). On the pin (ExecuTorch 1.3.1, torch 2.12.1, Apple M4 Max CPU, default threads, random weights) the median latency of RFDETRNano decreases from 49.3 to 31.8 ms and of RFDETRLarge from 201.7 to 91.5 ms; applying only the attention rewrite gives 34.7 and 94.5 ms, only the constant folding 46.5 and 198.6 ms. With ExecuTorch 1.5.1 at 16 threads (first 100 COCO val2017 images), the median latency for N/S/M/L/XL/2XL decreases on a Xeon (Haswell) CPU from 166.6/316.8/425.1/714.2/1480.1/1893.3 ms to 107.4/174.9/227.1/329.5/719.4/1012.2 ms, and on an Apple M4 Max CPU from 50.9/83.0/119.6/188.2/334.3/432.4 ms to 34.0/49.8/60.6/91.3/157.7/209.4 ms. COCO mAP@50:95 changes by at most 0.01 points. (#1601)
  • RFDETR.export() option handling:
    • format in RFDETR.export() is case-insensitive, as backend already was: format="ONNX" exports ONNX instead of raising Unsupported export format 'ONNX'. (#1556)
    • format="litert" refuses keypoint models with NotImplementedError naming the limitation. The refusal still comes after the forward pass, but before the conversion: litert-torch cannot lower the keypoint head, and the export used to fail inside it, after a full torch.export capture, with RuntimeError: Failed to export model to LiteRT. A backbone_only=True export of a keypoint model still converts. (#1556)
    • format="coreai", coreai_precision="float16" warns for keypoint models: that .aimodel terminates the process when Core AI runs it on the Neural Engine, which iOS and iPadOS pick for float16 by default. (#1556)
    • RFDETR.export(format="tensorrt", dynamic_batch=True) accepts a numpy integer batch_size. TensorRT's optimization-profile check refused one because it is not a Python int. (#1556)
    • format="tflite" warns with a UserWarning when calibration_data is passed, replacing the INFO log: the argument is not consumed when building the generated .tflite models. (#1585)
    • format="openvino", dynamic_batch=True is still refused, but the message now explains that the converted IR already has a dynamic input shape, so one export runs at any batch size. (#1585)
  • Single-GPU compiled BF16 detection training now combines global-norm clipping, AdamW, and the per-step EMA update in one multi-tensor Triton path. The existing public configuration selects it automatically only for the built-in AdamW optimizer with compile=True, fused_optimizer=True, use_ema=True, and ema_update_interval=1 on one device with the default devices=1 and strategy="auto"; other devices or strategy values and older Triton versions retain the standard PyTorch path, and a step whose tensor layout the kernels do not support applies clipped, non-fused AdamW instead. On an NVIDIA L4, a public RFDETRNano.train() run over COCO128 (default batch 4, 384 px) improved the median settled training epoch from 3.593 s to 3.135 s across five independent pairs (12.66%, range 11.55-16.86%) against an explicitly optimized PyTorch baseline using foreach gradient clipping, fused AdamW, one-call foreach EMA, and a host-side EMA counter. The paired reduction for the complete six-epoch call with populated compile caches had a 5.56% median (range 3.33-10.99%); one-time setup and final validation dilute the recurring-epoch saving. Full-dataset accuracy and larger variants were not measured. (#1529)
  • RFDETR.export() refuses a request it cannot complete before it copies or runs the model:
    • an unknown quantization for format="tflite", and an unknown openvino_precision, coreai_precision or coreml_precision name, which failed after the forward pass (TFLite's after the whole ONNX export);
    • notes that JSON cannot encode (float("nan"), an arbitrary object), which failed after the forward pass and left a .onnx without the notes behind;
    • a batch_size below 1, which failed while the example batch was built with RuntimeError: stack expects a non-empty TensorList and now raises a ValueError naming batch_size;
    • a missing package for the chosen format: onnx, which TFLite and TensorRT need too; TensorFlow, tf-keras and onnx2tf for TFLite; executorch, coremltools, coreai-torch, openvino or litert-torch. The error names the rfdetr[...] extra that installs it. format="onnx" without onnx used to fail with torch's Module onnx is not installed! after the trace, and format="tflite" without onnx2tf, TensorFlow or tf-keras only after the whole ONNX export. (#1556)
  • OpenVINOInference now defaults inference_precision to "f32", so latency and accuracy no longer depend on the device's default compute precision. Pass inference_precision=None to restore the device-default speed. It also takes a new config argument, a mapping of further compile_model properties (e.g. {"INFERENCE_NUM_THREADS": 4}) applied after inference_precision. (#1585)
  • Segmentation evaluation and training use less time and memory:
    • Segmentation validation and test runs encode masks to COCO RLE on the GPU. The COCO metric used TorchMetrics' update, which copies every predicted and ground-truth mask to host memory at full image resolution and encodes it there one mask at a time; on 2000x2000 px images that was 92% of a model.evaluate() run. The metric now encodes masks on CUDA when they are there and copies only the encoded runs to the host; masks on any other device go through the same code on the CPU. The stored RLE bytes are identical, so every metric is unchanged on every eval_backend. With RFDETRSegMedium on one RTX 4090, validating 500 COCO val2017 images takes 8.8 s instead of 59.1 s at their own size, and 24.7 s instead of 508.5 s upscaled 3x to about 1920x1440. (#1615)
    • Segmentation models in eval mode compute masks for the last decoder layer only, unless a loss is computed. LWDETR built a dense mask map, at a quarter of the input resolution by default, for every decoder layer and for the encoder proposals, although postprocessing reads the last layer alone and only the criterion reads the rest. predict(), validation without a loss (the default compute_val_loss="auto" when nothing monitors val/loss), evaluate(split="val") and evaluate(compute_test_loss=False) now skip those masks. A validation or test step that computes a loss calls the model with the new keyword all_layer_masks=True and gets every layer's masks as before; see the matching entry under Breaking Changes. The last layer's masks are bit-identical, so predictions and metrics do not change. With RFDETRSeg2XLarge on one RTX 4090, a predict() call on 8 images peaks at 1583 MiB of CUDA memory instead of 3612 MiB, and validating 500 COCO val2017 images at batch size 4 peaks at 1085 MiB instead of 2031 MiB. (#1628)
    • Segmentation training at high resolution is faster and uses less CUDA memory. The segmentation head's depthwise convolution now runs through cuDNN on GPUs with compute capability 8.0 or newer; the #731 workaround, which bypasses cuDNN and upcasts the backward to float32, is kept on CPU and on older GPUs such as the T4 and P100, where the failure was reported. Set RFDETR_DEPTHWISE_CUDNN=0 to force the workaround on every GPU, 1 to force cuDNN, or leave it at auto. The mask loss reads boolean CUDA target masks with a direct gather at any batch size instead of copying every matched mask as float at input resolution, and the points that land on an exact pixel-centre tie are resampled with one float copy per distinct mask instead of one per point. Mask-loss labels are identical to the previous sampling. On Ampere and newer GPUs, predict() masks can differ from the previous release's by a few pixels at instance boundaries: inference feeds the convolution a channels-last tensor, and cuDNN's kernel for it sums in a different order, which moves the output by about 1e-6 and flips pixels that sit exactly at the mask threshold. Boxes, scores and classes are unchanged. With RFDETRSegSmall at 1104 px, batch size 1, gradient checkpointing and bf16 on an RTX 5060, a training step takes 276 ms instead of 315 ms and peaks at 2.88 GiB of CUDA memory instead of 3.85 GiB. (#1513)
  • Deprecated-model warnings now name the replacement class (RFDETRSmall or RFDETRSegSmall). (#1618)
  • RFDETR.predict() raises TypeError for stream, batch, vid_stride and stream_buffer and points to RFDETRInference; they used to fall into the ignored **kwargs. (#1577)

Deprecated

  • Passing unknown keyword arguments to RFDETR.predict() is deprecated since v1.12.0: they are ignored with a FutureWarning and will raise TypeError in v2.0.0. (#1563)
  • rfdetr.export.inference.TRTInference and OpenVINOInference are deprecated in v1.12.0 and scheduled for removal in v1.15.0. Both remain available and emit a FutureWarning when constructed. Use RFDETRInference(...).predict(image) for image inputs and decoded results, passing legacy runtime settings through runtime_options; see the migration guide. (#1563)

Removed

  • Removed the rfdetr.datasets.aug_config compatibility module, deprecated since v1.9.0. Import the presets from rfdetr.datasets.aug_configs (plural) instead; the constants are unchanged. (#1662)
  • The [onnx] extra and the private ONNX exporter lose unused packages and helpers:
    • rfdetr[onnx] no longer installs onnxsim, onnx_graphsurgeon or polygraphy, and rfdetr[tflite] no longer installs polygraphy. ONNX export and ONNX Runtime inference have not used them since graph simplification was removed (#861, #1102); the ONNX exporter imported them only for the graph optimizer left behind in rfdetr.export._onnx, which nothing called, and that optimizer is removed too. If you exported to TensorRT with rfdetr[onnx] or rfdetr[tflite] and a separately installed tensorrt, install polygraphy yourself (it has no dependencies) or use rfdetr[tensorrt]. If you exported to TFLite with rfdetr[onnx] and a separately installed onnx2tf, install onnx_graphsurgeon yourself, or use rfdetr[tflite] on Python 3.12, the only version it installs on: without onnx_graphsurgeon the export is now refused before the forward pass, with an ImportError naming pip install onnx_graphsurgeon, because it would skip the GridSample rewrite and write a .tflite that produces incorrect scores. The warning that _rewrite_gridsample still logs when the rewrite's import fails names the same install command rather than rfdetr[tflite] alone. rfdetr[onnx] also no longer installs onnxsim>=0.7.0, so onnx2tf's own simplification pass now runs on the onnxsim that your onnx2tf release depends on (onnxsim-prebuilt for onnx2tf 2.4.x, onnxsim 0.6.5 for 2.6.9), and the .tflite it writes can differ from one built with rfdetr[tflite]. (#1588) (#1589) (#1563)
    • The private, unused rfdetr.export._onnx.OnnxOptimizer, rfdetr.export._onnx.exporter.onnx_simplify and the rfdetr.export._onnx.symbolic module (CustomOpSymbolicRegistry, register_optimizer). Importing rfdetr.export._onnx no longer imports its submodules; import rfdetr.export._onnx.exporter directly. (#1588) (#1589)

Fixed

  • predict() on keypoint models:
    • predict() on a keypoint model now includes key_points.data["covariance"], with shape (0, num_keypoints, 2, 2), on a frame with no detections. The key was omitted there, so code reading it raised KeyError only on empty frames. (#1681)
    • Keypoint log-mean-trace scores in float16 and bfloat16 are computed in float32 and cast back to the output dtype. (#1632)
  • Class counts and pretrained-weight loading:
    • A model built with an implicit class count from a checkpoint with fewer classes than its config (RFDETRSegXLarge(pretrain_weights="own.pth")) recorded the checkpoint's class count as if the user had set it, so train() on a dataset with more classes kept the old count with a warning. The head and loss were then too small for the new labels, and training could fail when it encountered an out-of-range label with a CUDA index-out-of-bounds assert (on Windows a display-driver reset with no traceback). load_pretrain_weights now clears the checkpoint-derived num_classes from model_fields_set, as from_checkpoint does, so the dataset count raises it and the head is expanded. (#1674)
    • The class-count warning claimed the checkpoint's class count was used when a set num_classes larger than the checkpoint was kept. load_pretrain_weights keeps the set count and resizes the loaded classification head to it; the warning now says so instead of "Using checkpoint class count". (#1654)
    • Loading a pretrained model no longer warns that DINOv2 backbone weights are skipped. Most variants differ from DINOv2 in patch size or positional encodings, so RFDETRNano() and similar default loads printed two such warnings even though RF-DETR pretrained weights replace the backbone. The warnings still appear when DINOv2 weights are requested, as with pretrain_weights=None. (#1653)
    • RFDETR.evaluate() no longer downloads DINOv2 pretrained weights when it rebuilds RFDETRBase or the deprecated Large, nor a registered encoder's upstream weights, so offline evaluation of those models works. (#1568)
  • predict() no longer raises TypeError: Got unsupported ScalarType BFloat16 on a bfloat16 tensor with the default include_source_image=True. (#1682)
  • inference() on a CUDA segmentation model no longer fails with "Expected all tensors to be on the same device, but got max is on cpu" after torch.use_deterministic_algorithms(True), for example through seed_all(). In that mode CUDA F.interpolate takes a backward-determinism decomposition that torch.jit.trace cannot record, so the prediction snapshot uses native upsampling without changing process-wide deterministic settings. (#1673)
  • import rfdetr no longer fails in Databricks notebooks, or on any host that replaces warnings.warn, with RuntimeError: Python builtin <built-in function warn> is currently not supported in Torchscript. TorchScript recognizes warnings.warn by object identity, so a replaced function broke compilation of the segmentation losses that rfdetr scripts at import. Those losses now fall back to their eager functions when TorchScript compilation fails, which compute the same result; on Python 3.14+ they already did. (#1649)
  • FP16 logits decoded by the shared export decoder, used by RFDETRInference and the ONNX and TFLite reference helpers, now compute the sigmoid with float32 intermediates and round once to FP16 before top-k and the strict > threshold, as PyTorch's FP16 sigmoid does. The decoder rounded every step in FP16 before: logit 1.1005859375 scored 0.75 instead of 0.75048828125 and was dropped at threshold 0.75, logits 8 and 9 tied at 1.0 so top-k could pick the lower score, and negative logits such as -12 overflowed the FP16 exponential. Some FP16 confidences and threshold or top-k outcomes therefore change; FP32 and FP64 scores keep their precision and the caller's logits are not modified. Integer and boolean logits, outside the annotated contract, keep floating-point confidences. (#1647) (#1648)
  • Segmentation and keypoint augmentation geometry:
    • In segmentation data pipelines, every geometric Albumentations wrapper now derives each instance's target box from its transformed mask (a pixel-snapped extent, up to 1 px from the corner-transformed box) and drops instances whose mask had foreground before the transform but is empty afterwards, including thin or tiny objects that a downscale erases. This removes the mask/box misalignment of rotations and non-rigid warps and the ghost boxes with empty masks seen during segmentation training. Box-only annotations (no mask foreground) and copies repeated by reflecting border modes keep their Albumentations boxes. The Kornia GPU path (augmentation_backend="gpu") and torchvision crops apply the same rule to instances that have mask foreground: boxes follow the augmented masks on the Kornia path, and a crop drops an instance whose mask lies outside it. (#1658)
    • Keypoints flipped by the torchvision augmentation backend now land where the flipped image content is. RandomHorizontalFlip mirrored keypoints with width - 1 - x, which assumes integer pixel centres, while RF-DETR keypoints are continuous and boxes already flip with width - x, so every flipped keypoint sat 1 px left of the image content. Visible keypoints of flipped training samples move 1 px to the right. A keypoint exactly on the right or bottom image edge (x == width or y == height) now stays visible through Resize, crops and the flip, so both edges survive a flip (x = 0 maps to x = width); before, such a point was cleared. Resize scales keypoints as x * new_size / old_size, which keeps an edge point exact in float32. Boxes, masks, inference and export are unchanged. The Albumentations path, which is the default CPU backend whenever rfdetr[augment] is installed, is not changed and still differs in two ways: it flips keypoints with width - 1 - x, and it clears a keypoint at x == width or y == height, so flipped keypoint labels differ by 1 px between the two CPU backends until a follow-up aligns them. The GPU (Kornia) path leaves keypoint coordinates unchanged. (#1597)
    • The Kornia augmentation path shifts detection and segmentation boxes into the pixel-index frame (-0.5 before, +0.5 after augmentation), as it already did for keypoints, so box edges stay continuous. (#1621)
  • format="tflite" no longer fails to convert every model with onnx2tf 2.4.3 on a fresh rfdetr[tflite] install. The extra listed onnxsim>=0.7.0 next to onnx2tf's own onnxsim-prebuilt; both wheels install the same onnxsim package and console script, so the one installed last replaced the other, and onnx2tf's simplification pass ran onnxsim 0.8.1, after which onnx2tf misplaced the backbone's token axes and stopped at embeddings/Reshape_1. The extra now leaves onnxsim to onnx2tf and caps onnx2tf below 2.5, whose releases move to numpy 2.2.6 and onnxsim 0.6.5. An environment that still has both packages, for example one upgraded in place, is refused before the forward pass with the commands that repair it: pip uninstall -y onnxsim onnxsim-prebuilt, then pip install "rfdetr[tflite]". (#1563)
  • COCO segmentation datasets load when an annotation's segmentation holds a polygon part with fewer than three points, such as [[]] or a two-point [x1, y1, x2, y2] line. pycocotools reads a four-value first part as a box and rejects it, and rejects an empty first part, so training or evaluation stopped with TypeError: Argument 'bb' has incorrect type or Exception: input type is not supported.. These parts enclose no area and are now skipped; an instance with no other part gets an empty mask, and a crowd region's segmentation becomes None while it keeps its box. (#1644)
  • rfdetr fit --config configs/<name>.yaml runs as the configs and the training reference document it. Parsing a shipped config failed on every jsonargparse release from 4.27.7 to 4.52; recent releases treat the pydantic ModelConfig and TrainConfig as final and reject the subclass each config names under class_path. RFDETRCli now accepts subclasses of both, and the cli extra requires jsonargparse>=4.46.0, the first release with that setting; an older or missing jsonargparse gets the install command. The CLI also built a plain Lightning Trainer that ignored train_config: past the parse error, configs/rfdetr_large.yaml would have trained for Lightning's default 1000 epochs in fp32, without its grad_accum_steps: 8 or gradient clipping, and without EMA, mAP or a best checkpoint. Every subcommand now builds its trainer with build_trainer, as RFDETR.train() and RFDETR.evaluate() do. A --trainer.* argument overrides the value built from train_config, and so does a config file's trainer: value that differs from Lightning's default; --trainer.callbacks adds to the RF-DETR callbacks. fit resumes from --ckpt_path or from train_config.resume, and keeps the rows already in metrics.csv. (#1629)
  • Resuming from last.ckpt or checkpoint_<epoch>.ckpt with a changed lr, lr_encoder, lr_vit_layer_decay, lr_component_decay or weight_decay now trains with the new value. The checkpoint's optimizer state used to overwrite it without a word, while training_config.json and the resumed run's checkpoints recorded values that were not in use. The order is now defaults, then the checkpoint, then what you pass: a setting you change wins while the optimizer's moments and the schedule's progress carry over (a new lr becomes the base the schedule scales), a setting you leave out keeps the checkpoint's value, and only the parameter groups a setting you pass feeds change, the rest keeping what their saved optimizer holds. The floor of a CosineAnnealingLR, alone or after the warmup ramp (eta_min), or of a ReduceLROnPlateau (min_lr) stays put when the base moves. A group a setting you pass feeds is rebuilt from all the settings that feed it, so one you leave out comes from the checkpoint's record there. The checkpoints record the resolved values, as does training_config.json when resume names the checkpoint file. Resuming logs which values came from the checkpoint and which you set, and trainer.fit(module, ckpt_path=...) follows the same order. A last.ckpt written before 1.11.1 records no settings: resumed unchanged it keeps its values, and with a setting passed the others take their defaults in the groups it feeds, with a warning. A ReduceLROnPlateau checkpoint from an earlier release restarts at a changed learning rate instead of keeping its reductions. (#1613) (#1619)
  • format="coreml" segmentation masks no longer land half a mask cell below and right of their objects when Core ML runs the model on the Apple Neural Engine. On the Neural Engine, upsample_bilinear by a factor of 3 samples half an output cell off, and the segmentation head upsamples its mask features 3x. The export now replaces each bilinear resize with two matmuls by constant interpolation matrices, which compute the same resize on every compute unit and stay on the Neural Engine. Bundles exported before this change keep the offset until they are exported again. (#1657)
  • Class names in the per-class metrics table printed after validation and test are no longer interpreted as Rich markup or emoji codes. Each name was passed to Rich as a plain string, which Rich parses as console markup and emoji codes, so helmet[red] printed as helmet, car [parked] as car, price:dollar: with an emoji in place of :dollar:, and a name containing a stray closing tag such as sign[/] raised rich.errors.MarkupError out of COCOEvalCallback, ending training at the first evaluated validation epoch. A non-string class name, such as null in an annotation file, is printed as text instead of raising. (#1598)
  • With mixed keypoint counts, keypoint mAP and mAR no longer depend on which keypoint-count group holds a category with no scorable ground truth on the evaluated images, such as a class missing from the validation split. The groups' accumulated arrays are now summarized as one evaluation, so each stat is the mean over the categories it scores. On affected datasets the keypoint_map_* and keypoint_mAR metrics can change, so best-checkpoint selection and, with early_stopping=True, early stopping can change and metrics are not comparable with earlier runs. Medium and large stats from direct CocoEvaluator use can also change when a category has no scorable ground truth in that area range. Datasets with one keypoint count are unaffected. (#1686)
  • BestMetricSingle and BestMetricHolder now raise ValueError for a better other than "large" or "small", where an invalid value used to raise a bare AssertionError and, under python -O, which strips assert, was accepted at construction and failed only later inside isbetter(). The check runs before the instance stores any state, and better is annotated as Literal["large", "small"]. Code that caught AssertionError from these constructors must catch ValueError. (#1608)
  • inference(dtype="float16") and inference(dtype="bfloat16") with compile_backend="inductor" compile again on torch 2.14. gen_sineembed_for_position built its frequency table in the model's half-precision dtype, and torch 2.14.0 and 2.14.1 lower that dim_t // 2 to a libdevice.isinf call on a float16 or bfloat16 operand that Triton cannot compile (pytorch/pytorch#197002), so the call in docs/learn/run/detection.md raised InductorError for detection, segmentation and keypoint models. The table is now computed in float32 for half-precision positions, as PositionEmbeddingSine does, and cast once to the positions' dtype; float32 and float64 positions keep their own dtype. float16, float32 and float64 outputs are unchanged. In bfloat16, 10 of the 128 frequencies are now rounded once from float32 instead of computed in bfloat16 on every route that runs the plain ops: CPU, MPS and XLA, torch.compile, tracing and export. Eager CUDA takes _sineembed_interleaved and is unchanged. (#1616)
  • export(output_name=...) keeps dots that are part of the name: output_name="detector-v1.2" writes detector-v1.2.onnx instead of detector-v1.onnx, so two versions of one name no longer overwrite each other, and a backbone_only=True TensorRT engine keeps its -backbone marker. Only an export file extension (.onnx, .trt, .engine, .tflite, .mlpackage, .pte, ...) is still stripped. (#1641)
  • Export metadata type checking no longer fails on Windows. (#1640)
  • RFDETR.from_checkpoint("run.pth") and RFDETRInference("run.pth") now load the weights from the file they were given. A bare filename used to read its metadata from the working directory and then reload from the model cache: the call raised FileNotFoundError when the cache had no such file, downloaded the published weights and loaded those when the name was a published one such as rf-detr-nano.pth, and otherwise returned the cached file's weights and class names with no error. A bare filename now means the file in the working directory and never downloads. With an architecture override, a bare published name outside the configured model cache no longer emits PretrainWeightsCompatibilityWarning, matching ./rf-detr-nano.pth. Bare names passed directly to a model constructor still resolve to the cache. If loading an existing checkpoint fails, the error now propagates instead of redownloading and possibly overwriting the file. (#1683)
  • BestModelCallback skips fast-dev-run and non-fitting runs (trainer.state.fn != FITTING) and the sanity check for EMA and smoothing; a dry run no longer promotes or tests a checkpoint. (#1678)

[1.11.2] — 2026-10-03

Added

  • find_yolo_data_file(dataset_dir, default=None) and find_yolo_train_images(dataset_dir) in rfdetr.datasets.yolo, re-exported from rfdetr.datasets. They locate a YOLO dataset's data.yaml/data.yml and resolve its training image directory from the YAML split paths, with the train/images/ fallback, which is the lookup format detection now uses. (#1570)

Breaking Changes

  • Removed rfdetr.datasets.yolo.REQUIRED_DATA_SUBDIRS. is_valid_yolo_dataset resolves split directories through the YAML-aware resolver, which derives the labels directory from the resolved images path, so the constant no longer described how any code finds a split's subdirectories. Its only reader was is_valid_yolo_dataset, rewritten here, so nothing in the package reads it any more; an external importer should inline ["images", "labels"]. (#1570)

Changed

  • YOLO split directories must stay inside the dataset root however they are reached. The containment rule applied only to paths declared in data.yaml, so a declared train: images/train pointing at mounted storage was refused while a train/images symlink to that same storage was read — the guard rejected by layout rather than by destination. Format detection and is_valid_yolo_dataset now refuse a conventional split directory resolving outside the root, and build_roboflow_from_yolo raises ValueError naming it. A dataset whose entire root is a symlink is unaffected, because containment resolves both sides; only an individual split escaping its own root is refused. (#1570)
  • ONNX exports no longer contain a single-input Concat or a Concat of an empty tensor where the two-stage selection assembles the decoder queries, and the segmentation head's export path emits MatMul instead of Einsum. ONNX Runtime's CoreML execution provider rejects all three, so each split the exported graph into another CoreML partition with a CPU round-trip in between. On an Apple M4 Max with onnxruntime 1.30 (ModelFormat=MLProgram, MLComputeUnits=CPUAndGPU), each graph now has two CoreML partitions, leaving only the CPU-resident top-k selection between them: RF-DETR Nano runs in 8.3 ms instead of 9.3 ms (4 partitions before), Large in 19.4 ms instead of 19.9 ms (4 before), and Seg Nano in 16.5 ms instead of 18.9 ms (5 before). All figures were measured with a static batch; dynamic_batch=True was not measured and is not expected to match, because forward_export's reshape then takes a runtime-computed shape instead of a constant. Exported outputs are bit-identical on ONNX Runtime's CPU provider, and eager training and predict() are unchanged; RFDETR.inference() switches the model into export mode, so it also takes the segmentation head's MatMul path — output identical on CPU (fp32/fp64/bf16 verified; CUDA/fp16 unverified). (#1566)
  • export(format="tflite", dynamic_batch=True) now raises NotImplementedError before conversion starts. The registry advertised TFLite as dynamic-batch capable, so the request was accepted and the export then failed inside onnx2tf with ValueError: Dimensions must be equal, but are 384 and 1025: a position-embedding Add broadcasts a batch-1 constant against the dynamic-shaped encoder input, which onnx2tf cannot lower. TFLite now joins ExecuTorch, CoreML, Core AI, OpenVINO and LiteRT as a fixed-batch format; export one .tflite per batch size instead. (#1591)
  • The [executorch] extra is capped at executorch>=1.3,<1.4 instead of <2.0, matching the ci-executorch-pin group CI already installs. 1.4.x needs torch>=2.13, and 1.5.x fails every Method.execute call with RuntimeError: tensor does not have a device, so an unpinned install could resolve to a release that exports a .pte but cannot run it. (#1591)
  • Export documentation reorganised and the cookbook benchmarks relabelled. docs/exports/index.md is now the overview; installation, basic export and format choice moved to a new Basics page, and the export() parameter table, format capability matrix and precision notes to a new Advanced page, both added to the nav. The overview's NVIDIA L4 row is split into the inference-models managed engine and the raw .trt file run by TRTInference, the Roboflow model page's latency is labelled "scope not stated" instead of "model only", and results headers name rfdetr develop rather than v1.11.0 because the cookbooks install develop. The CUDA, CPU, Mobile and Apple cookbooks share private display and scoring helpers from rfdetr.export._benchmark, benchmark batch sizes 1, 4 and 8 (CPU and Mobile: 1 and 4), and the Mobile cookbook installs executorch>=1.3.1,<1.4. Notebook tables of contents now keep headings that contain inline code. (#1562, #1567, #1591, #1594, 9143779)

Fixed

  • Downloaded pretrained weights, checkpoint_best_total.pth and training_config.json are readable by other users again. Each is written to a temporary file and renamed into place, and tempfile creates that file with mode 0600, which the rename kept, so all three ended up owner-only instead of following the umask like a new file written with open() (0644 under the usual 022). A weights cache shared through RF_HOME, or filled at image build time and read by another user at runtime, failed with PermissionError, and checkpoint_best_total.pth got a different mode from the checkpoint_best_regular.pth or checkpoint_best_ema.pth it is copied from. The temporary files are now created with mode 0666 minus the umask, and overwriting an existing file keeps its mode (e.g. strip_checkpoint on a file chmod-ed 0600 stays 0600). (#1582)
  • Keypoint pipelines composed from Sequential, OneOf, and SomeOf Albumentations containers now swap paired joint identities only after an odd number of applied horizontal flips, including repeated selection by SomeOf(replace=True). Trees containing any container type outside that recognised set -- programmatic containers such as OneOrOther, subclasses of the recognised container classes, and any other unknown container -- fall back to replay-based handling, which cannot distinguish repeated draws of the same child. (#1569)
  • The segmentation head's depthwise convolution no longer leaves cuDNN disabled for the whole process when several threads run it at once. It wraps F.conv2d in torch.backends.cudnn.flags(enabled=False), which saves the current value on entry and restores it on exit but is process-global and not thread-safe, so a second thread entering while the first was inside saved False and restored it last. After that every convolution in the process took the non-cuDNN path with no error or warning, and detections near the score threshold changed (25 instead of 24 on one real frame, with the flag as the only difference). The forward and backward scopes now share a lock, and a CPU input skips them entirely — ATen reads the flag only for a CUDA input, so the lock there only serialised independent convolutions. (#1565)
  • The deferred CPU-to-accelerator weight move on the first predict(), inference() or export() call is now serialised. Several threads sharing one model (for example one channel per thread) could reach that call together and run overlapping in-place nn.Module.to() calls on the same module. On a cold RF-DETR segmentation model, four threads calling predict() at once left 13 of 573 state_dict tensors corrupted, with no exception or warning, and the model then returned no detections on every frame. The device check and the move now both run under a lock, so a caller that arrives mid-move waits for it to finish instead of starting inference on a half-moved model. export()'s CPU staging move and its restore, and evaluate()'s move of the live weights to CPU for the eval transplant, take that same lock; each individual move is serialised, the span between a staging move and its restore is not. The lock is per model — held on the nn.Module being moved, so a cold move of one model does not stall a call on another, and two RFDETR instances sharing one module still serialise against each other. (#1564)
  • RFDETR.predict() now accepts NumPy views with negative strides, such as the BGR-to-RGB frame[:, :, ::-1] in the video file, webcam and RTSP examples of docs/learn/pretrained.md. Both the uint8 and the float conversion call torch.from_numpy, which rejects negative strides, so those examples raised ValueError: At least one stride in the given numpy array is negative on their first frame. Only arrays with a negative stride are copied now, and with include_source_image on, a uint8 view reuses the copy already kept as source_image; contiguous, positive-step and broadcast arrays still convert without an extra copy. A flipped frame gives the same detections as its contiguous copy. (#1573)
  • class_names that include a Roboflow export's top-level category now stop training with a ValueError instead of shifting every class name by one. Roboflow COCO exports list that category first, for example animals above cat, dog and horse. Since 1.9.2 it is not one of the model's classes, so a list read from every category gave each class its neighbour's name, in checkpoints and in predict() output, with no warning. RFDETRDataModule.setup("fit") now compares class_names with the dataset's names, and RFDETR.train() runs the same comparison up front, before it writes training_config.json and before trainer.fit() starts a wandb/MLflow run that a later rejection would leave behind as an orphan crashed run. Extra names above a class raise and say which list to pass, naming each one, whether they lead the list or sit between two real classes; the dataset's own names in another order warn, as does any other difference in length; a renamed list of the right length passes. The comparison applies to datasets whose labels are remapped category ids — a Roboflow COCO export, or a webdataset packed with category_ids="remap". A raw-id dataset such as dataset_file="coco" labels each object by its source category id, so the class_names that is correct there carries a placeholder for the unused id 0, and it is left alone rather than rejected as a shift. Every warning is a WARNING on the rf-detr log, emitted once per run rather than once per rank. The fine-tune detection and segmentation, custom optimizer and PyTorch Lightning cookbooks built their class names that way and now leave the category out. (#1572)
  • YOLO datasets whose data.yaml declares its split paths are now detected and loaded. Format detection required a train/images/ directory, so a layout such as train: images/train with labels under labels/train was not recognised as YOLO even though the YAML described it; it now resolves the training images from the declared paths, relative to the optional path base or the dataset root, and keeps the train/images/ fallback. COCO still takes precedence when both formats are present. (#1570)
  • YOLO data.yaml names mappings now resolve class IDs the same way in every entry point. RFDETR.train() read the class names of a dataset directory by sorting the mapping's raw keys, so string keys such as "10" and "2" came out in lexicographic order and a mix of integer and string keys raised TypeError; the YOLO dataset loader and the keypoint-schema reader used a different parser, which tested keys with str.isdigit() and so also took non-ASCII digit characters as IDs. Every entry point now shares one parser: keys must be ASCII-digit integers (quoted or not) forming a contiguous 0..N-1 range, no two distinct keys may collide on one ID (0 and "0"), names are ordered by ID, and a violation raises ValueError naming the file. A kpt_names mapping keyed "00" is recognised as key 0 as well. A YOLO pose data.yaml that fails validation now raises Invalid YOLO pose data file '<path>': <cause> instead of the generic "requires kpt_shape metadata" message, so the actual cause is visible. Unquoted zero-padded keys of three or more digits such as 010 still collide with 008, because YAML 1.1 reads them as octal; quote them or drop the padding, as the dataset-formats guide now says. (#1571)

[1.11.1] — 2026-09-30

Breaking Changes

  • RFDETRModelModule.clip_gradients renamed to configure_gradient_clipping (moves the keypoint fp16 clip fix onto the correct Lightning hook); clip_gradients reverts to LightningModule's base impl (Lightning: "do not override this method"). Only affects direct callers of clip_gradients, a super().clip_gradients(...) call inside an override, or an override on the fused-AdamW path — a self-contained override still runs on the non-fused path via self.clip_gradients(...) dispatch. See MIGRATION.md. (#1537, #1549)

Changed

  • Replaced 4 per-format export cookbooks (export-coreml, export-tensorrt, export-executorch, export-tflite) and inference-latency-benchmark with 4 per-hardware cookbooks — export-cuda, export-cpu, export-mobile, export-apple — each exporting every format for that hardware, running inference on it, and benchmarking against a PyTorch baseline. Adds first cookbook coverage for ONNX, OpenVINO, LiteRT, Core AI. Per-format docs (docs/exports/*.md) unchanged. Added rfdetr.export._benchmark (private): measure_latency (CUDA-event timing on device="cuda", else time.perf_counter) and measure_memory (host RSS delta via psutil, or CUDA free-memory delta via torch.cuda.mem_get_info()), shared by all 4 cookbooks; replaces and removes rfdetr.export._onnx.inference._onnx_runtime. Each cookbook benchmarks default config + one precision variant per exporter — OpenVINO fp32/fp16 IR, TensorRT auto/fp32, CoreML/Core AI fp32/fp16, TFLite fp32/fp16/dynamic-range INT8; ONNX and LiteRT stay single-row (no precision setting). (#1530)
  • RFDETR.from_checkpoint() no longer restores device from model_config — loading host's default wins; pass device= explicitly. Every other model_config field except pretrain_weights still restored, caller kwargs take precedence. (#1534, #1561)
  • TrainConfig.clip_max_norm now rejects a negative value at construction (Field(default=0.1, ge=0.0)) — was silently unclipped for the whole run. (#1537, #1549)

Fixed

  • YOLO dataset under a dir literally named images (e.g. /data/images/coco8) now loads correctly. Labels dir was derived by swapping the first images segment for labels, so images/val resolved to nonexistent /data/labels/coco8/images/val, fell back to the Roboflow valid/images convention, and training failed. Resolver now swaps the rightmost images segment with an existing labels sibling (Ultralytics behavior), rejecting candidates outside the dataset root. (#1557)
  • last.ckpt/checkpoint_<epoch>.ckpt now record training config (dataset class names), model class, model config, rfdetr version — same as best .pth files. from_checkpoint("output/last.ckpt") was KeyError: 'args', now rebuilds at trained resolution/architecture with that epoch's non-EMA weights; pretrain_weights="output/last.ckpt" now keeps dataset class names (was COCO's). Pre-1.11.1 files can't be repaired in place — raises ValueError pointing at checkpoint_best_ema.pth + training_config.json; resume= unaffected. Config values a weights-only torch.load can't read (e.g. pathlib.Path in notes) stored as repr() + warning. (#1552)
  • COCO training on PNG/BMP/non-JPEG (or JPEG without simplejpeg) back to 1.10 decode speed — CocoDetection was round-tripping every Pillow-decoded image through NumPy and back to PIL. Measured (Windows 11, Ryzen 7 7800X3D, 2 workers, batch 8, RFDETRLarge @1120): up to 27% slower/batch on 4096px BMP, 11% on PNG; JPEG (via simplejpeg) unaffected. CocoDetection/WebDataset shard reader now use Pillow's decoded image as-is. (#1544)
  • format="coreai" works with coreai-torch 0.4.3 (moved optimization into TorchConverter.to_coreai(), removed AIProgram.optimize()) — 1.11.0 fresh installs failed with 'AIProgram' object has no attribute 'optimize'. [coreai] now pins coreai-torch==0.4.3, Python 3.11–3.14 (coreai-core 1.0.0b3 ships cp314 wheels), and declares a uv conflict with [tflite] (onnx2tf pins vs coreai-core's numpy>=2.3). (#1527)
  • from_checkpoint("checkpoint_best_total.pth") rebuilds as trained. strip_checkpoint dropped model_config → silent fallback to class defaults: Nano @224 came back @384 (boxes up to 193px off), keypoint @552 came back @576; num_select lost, dec_layers only warned. Same path as the Roboflow SDK upload. device no longer taken from checkpoint either — GPU-trained checkpoints load/predict on CPU-only host (checkpoint_best_ema.pth/checkpoint_best_regular.pth already did). Pre-1.11.1 best-total files can't be repaired in place; from_checkpoint warns, points at the unstripped checkpoint + training_config.json. (#1533)
  • Keypoint fp16 training clips the real gradient. Manual-optimization path clipped before Lightning unscaled, so optimizer got clip_max_norm / 65536 (1.5e-6 vs 0.1 default) — silently stalled. Now clips in on_before_optimizer_step, after unscale. Affects 16-mixed: Apple Silicon (default AMP), CUDA amp_dtype="fp16", T4/V100 via amp_dtype="auto" (#1535). Detection/segmentation unaffected. (#1536)
    • Multi-device XLA keypoint training now clips reduced gradients — no GradScaler-unscale dependency.
    • Callbacks now see unclipped gradients in on_before_optimizer_step, matching Lightning's automatic-optimization hook order.
  • amp_dtype="auto" (default) now trains fp16 on NVIDIA GPUs without native bf16 (T4, V100). torch.cuda.is_bf16_supported() counts emulated bf16, so torch ≥2.3 got bf16-mixed there: Colab T4 Nano step 443ms vs 226ms fp16 vs 386ms fp32. "auto" now picks bf16 only when every training GPU is Ampere+ (per docs). Older GPUs: fp16 + GradScaler, batch_size="auto" probes in fp16, cuda_graphs=True stays eager. Explicit amp_dtype="bf16" still honored, now warns it's emulated. (#1535)
    • Loss curves/checkpoints before vs after are not comparable on pre-Ampere GPUs (different autocast dtype, GradScaler now active, fused AdamW off).
    • The ~2x per-step timing (443ms vs 226ms) is one RF-DETR Nano step at default compile=False, use_ema=True — not an accuracy/throughput claim. Pre-Ampere GPUs also lose fused AdamW (_fused_adamw_env_eligible) and the Triton fused AdamW+EMA kernel (_use_fused_adamw_ema): both require BF16-variant precision, which fp16 (16-mixed) isn't — required for correctness, not a regression.
  • Val/test mAP ignores detections on COCO crowd regions (iscrowd=1), matching pycocotools — crowd annotations were dropped before the metric saw them, so a detection inside one counted as a false positive. (#1531)
    • Pretrained RF-DETR Nano COCO val2017: 0.4802 mAP → 0.4842, matching pycocotools on the same predictions.
    • Training unchanged; only crowd-labelled COCO data gets different numbers — not directly comparable before/after.
    • Not covered: WebDataset shards, a hand-built Trainer.validate(model, dataloaders=...) with no datamodule. val/F1 only ignores a detection with IoU ≥0.5 vs the crowd (box for detection, mask for segmentation).
  • TRTInference (rfdetr.export._tensorrt.inference, behind python -m rfdetr.export.benchmark) now refuses an input TensorRT would misread, instead of returning wrong detections. A tensor on another device, wrong dtype, or non-contiguous (channels_last, a slice) now raises ValueError naming the fix — reads each input straight off its pointer. On a COCO val2017 image, channels_last/float16 input took pretrained RF-DETR Nano from 13 detections (>0.5) to none; a CPU tensor caused cudaError 700: an illegal memory access (unusable CUDA context). Convert with .to(...)/.contiguous(). Batch-only mismatch now names working settings — dynamic_batch=True, max_batch_size=N covering both batches — instead of bare dynamic_batch=True; given only when batch alone exceeds a dynamic profile or is the sole static-engine mismatch (batch 0 / wrong image size no longer get this advice). (#1546)
  • TRTInference.build_engine no longer truncates an existing .trt on a failed build — it opened the file before checking the result, then failed with TypeError: a bytes-like object is required, not 'NoneType'; now raises RuntimeError first. A dynamic-batch ONNX (no optimization profile on this builder) is one way to hit it; RFDETR.export(format="tensorrt", dynamic_batch=True, max_batch_size=N) builds those. (#1546)
  • An undeserializable engine (wrong TensorRT version/GPU arch, truncated file) now reports RuntimeError naming the file + rebuild instruction, instead of AttributeError: 'NoneType' object has no attribute 'create_execution_context'; a failed execution-context creation raises RuntimeError too, instead of AttributeError on None. (#1546)
  • TRTInference(device=...) now loads and runs the engine on that device — before, only output buffers went there while TensorRT used cuda:0 (unless switched), so --device 1 mixed two GPUs in one context. synchronize()/the latency timer follow; non-CUDA device raises ValueError. Unit-tested only, no multi-GPU hardware. Feeding inputs on another device now gets the input-refusal ValueError above — move inputs with .to("cuda:1") too. (#1546)
  • backbone_lora=True checkpoints reload their trained backbone. from_checkpoint()/RFDETR<Size>(pretrain_weights=...)/train() loaded before wrapping the encoder with LoRA — checkpoint stores encoder under PEFT names, none matched, encoder stayed at random init: evaluate() ≈0, predict()/exports ran untrained, training restarted the encoder. Only sign was a partial-load warning (223 model parameter(s) not in checkpoint … 331 checkpoint key(s) not consumed). (#1543)
    • Encoder now wrapped before load; backbone_lora=True keeps trained adapters (pass explicitly, or restored from model_config — checkpoint_best_total.pth doesn't carry one yet, #1533). backbone_lora=False (default) merges adapters into encoder weights.
    • Without peft installed, loading now raises ImportError (was silent random-encoder load). Patch-size check from #965 now covers these too.
    • Measured: RF-DETR Large @640, LoRA fine-tuned on 262 COCO train2017 person images, evaluated on 300 val2017 person images — val mAP50:95 0.0001 → 0.6221 (same as in-memory model). (#1540)
  • RFDETR.export(format="tensorrt") without tensorrt installed now fails before the ONNX export with ImportError (pip install rfdetr[tensorrt]) — rfdetr[onnx]'s Polygraphy imports tensorrt lazily, so the exporter previously wrote the full .onnx first, then failed with a Polygraphy error suggesting POLYGRAPHY_AUTOINSTALL_DEPS=1. That env var no longer installs tensorrt mid-export; install the extra instead. (#1541)
  • TensorRTExporter.build_engine on a dynamic_batch=True ONNX now raises ValueError unless TensorRTConfig sets dynamic_batch=True + max_batch_size — Polygraphy previously pinned the batch axis to 1 with just a warning. RFDETR.export(format="tensorrt") unaffected (applies one dynamic_batch setting to both ONNX graph and engine). (#1541)
  • compile=True with default multi_scale no longer recompiles the transformer per resolution. torch.compile(dynamic=True) specialized spatial_shapes (torch.as_tensor of Python-int (H, W), added #1411) and the encoder-proposal grid (torch.linspace) to traced values, hitting Dynamo's recompile limit (8) — RF-DETR Nano/RTX 5070/torch 2.14: scales 9–11 ran eager, 54/200 steps. spatial_shapes now stacked from 0-d tensors (symbolic), grid uses torch.arange (same values) — stays compiled at every scale. Applies only under torch.compile; eager/TorchScript/ONNX/torch.export (torch 2.7+)/eager CUDA-graph capture unaffected. Backbone projector still recompiles per resolution on CUDA (Inductor conv-backward lowering, pytorch/pytorch#178945, pins strides). (#1410)
  • COCO segmentation dataset mixing annotated and box-only instances now keeps every mask. ConvertCoco only checked the first annotation — a box-only first annotation dropped all masks for the image (e.g. 2 boxes, 0 masks); reversing order gave 2 masks. Masks now built for every annotation unconditionally, so masks.shape[0] == boxes.shape[0] always; box-only → empty mask (a real training target, not a placeholder). (#1558)
  • Fixed JSON serialization in the COCO val-subset builder. (fd3bb991, no PR — direct commit)

Documentation

  • Documented TrainConfig.eval_backend in the advanced training guide: comparison table (vernier/hotcoco/ufcoco/faster_coco_eval), walkthrough for registering a new backend, runnable compute() timing skeleton. (#1526)
  • Scoped the static (full-integer) INT8 export limit to TFLite specifically — _tflite exporter docstring + docs/exports/tflite.md now name the mechanism (full-integer mode: per-tensor 8-bit scale, no float fallback) and note other toolchains (ONNX Runtime QDQ quantize_static, OpenVINO NNCF) can keep transformer-sensitive ops in float. Behavior unchanged. (#1559)

[1.11.0] — 2026-09-23

Added

  • RFDETR.export(format="coreai") writes an Apple Core AI .aimodel for iOS, iPadOS and macOS 27 through torch.export and coreai-torch (new [coreai] extra); coreai_precision="float16" gives a half-size asset. Decomposes aten.grid_sampler_2d (unsupported by coreai-torch) and runs float16 two-stage topk in float32, since the Neural Engine returns corrupt float16 topk indices. Detection, segmentation and keypoint models match eager PyTorch within 1e-4 on CPU (e2e_coreai suite, macOS 27 CI).
  • RFDETR.export(format="tensorrt", dynamic_batch=True, max_batch_size=N) now builds one engine that accepts any batch from 1 to N. The ONNX graph already carried a dynamic batch axis; the engine was refused only because no TensorRT optimization profile was passed, so it accepted the traced batch alone. The exporter now hands polygraphy a profile with min=1, opt=batch_size, max=max_batch_size; TRTInference allocates bindings at the max batch and trims outputs per call, and its async path moves off execute_async_v2 (removed in TensorRT 11) to execute_async_v3. Verified within 1e-2 of eager PyTorch on an RTX 4090. At the profile's opt batch the dynamic engine costs ~2% latency versus a static one (up to ~5% on a T4); other batch sizes pay 1-48% more — the trade for one file instead of one engine per batch size. Documented in docs/learn/export.md. (#376)
  • Documented the Apple Neural Engine fallback boundary for CoreML exports in docs/learn/export.md: how to choose compute_units, why fp32 never reaches the ANE, what fp16 costs in accuracy, which ops RF-DETR leaves on the CPU, and the measured latency per compute unit. New tests pin that boundary and the ExecuTorch CoreML delegate's whole-graph lowering. (#1024)
  • CocoDetection, YoloDetection and the WebDataset shard reader now decode JPEG files with simplejpeg (libjpeg-turbo straight into a NumPy buffer) when installed — now included in [train]. Same pixels as Pillow's decoder on tested wheels, though simplejpeg bundles its own libjpeg-turbo copy so small rounding differences are possible. Up to 1.8x faster at the decode stage for a reader that consumes the array directly (i7-8750H); consumers that wrap the array back into a PIL image (Image.fromarray) see a smaller or hardware-dependent net gain. Falls back to Pillow for non-JPEG or rejected files, with the same decompression-bomb limit enforced. (#1391, #1473)
  • Opt-in CUDA graph training replay (ModelConfig.cuda_graphs), landed across three PRs:
    • Base BF16 replay path for single-GPU detection training. The registered detector stays unchanged for optimizer, EMA, and checkpoint ownership; the variable-length criterion remains eager, while each static input signature captures the model forward and its backward once and replays it on later batches. The enabled path logs once at train start and once per captured input shape, so an active graph run is distinguishable from eager in the console. Unsupported devices, distributed runs, segmentation, keypoints, non-BF16/non-FP8 precision (FP16, FP32), and gradient checkpointing stay eager with a warning (FP8 gained its own Transformer Engine capture route, described below), and a failed capture stops training with a process-restart instruction. On an NVIDIA L4 with RF-DETR Nano, BF16, batch 4, deterministic synthetic detection batches, and a fixed 8-resolution multi-scale set (expanded_scales=False; the default expanded_scales=True resolves to 11 resolutions for this model and was not benchmarked), the median public Lightning training batch fell from 149.8 ms eager to 101.7 ms graphed, a 32.4% median of the five paired per-run reductions (range 31.4-33.5%); all eight signatures captured on every run and none fell back. The speed costs memory: peak allocated CUDA memory rose from 2,395 MiB to 3,466 MiB and peak reserved memory from 2,754 MiB to 14,550 MiB for those 8 graph pools, because each resolution retains a private graph pool. Final-parameter and loss differences stayed inside an eager-vs-eager control. On real data at the other end of the range — Nano on an RTX PRO 6000, BF16, batch 64, resolution 384, multi_scale=False, COCO train2017 — graph replay matched eager at 3.54 it/s while compile=True cut the epoch from about 8 min 40 s to 7 min 10 s: graphs remove launch gaps and pay at small batch, compilation fuses kernels and pays at large batch. The advanced training guide has a decision rule. Full-dataset accuracy and larger variants were not measured. (#1410, #1468)
    • cuda_graphs=True can now be combined with compile=True: the model is compiled with Inductor's triton.cudagraphs option, so cudagraph trees record and replay the compiled forward and backward kernels, training_step marks each step for the graph-tree allocator, and the eager graph runner stays off. Measured on an RTX PRO 6000 (Nano, BF16, resolution 384, synthetic batches) the combination is 1.20× faster than compile=True alone at batch 4 (1.47× vs eager) and matches it at batch 64 (1.32× vs eager, +0.9%, inside noise); an A100 gave 1.70× over compile at batch 4. Scope is single-GPU detection training without gradient accumulation; segmentation, keypoints, gradient checkpointing, grad_accum_steps > 1, and multi-device runs fall back to compile-only with a warning, because cudagraph trees allocate gradient outputs inside the graph pool and cannot accumulate across replays. Only BF16 was measured. When amp_dtype="fp8" is combined with cuda_graphs=True and compile=True, RF-DETR warns and keeps the run compile-only — it does not wrap the compiled module in the Transformer Engine graph helper, and the two capture runtimes are never nested; the Transformer Engine capture route below needs compile=False. (#1410, #1479)
    • Added a Transformer Engine FP8 CUDA-graph capture route (cuda_graphs=True, amp_dtype="fp8", compile=False): the fixed-shape training step is captured through Transformer Engine's make_graphed_callables under the active Lightning FP8 recipe, with returned gradients cloned before Transformer Engine reclaims its buffers. Scope is single-GPU detection training with grad_accum_steps=1, multi_scale=False and square_resize_div_64=True; unsupported combinations stay eager with a warning, and a capture failure stops training with the original exception attached, matching the process-restart guidance above. The capture call needs a Transformer Engine release providing the 2.19 make_graphed_callables API (clone_param_grads_on_return); an older release raises instead of silently falling back. Tested against Transformer Engine 2.19.0, installed through the existing cuda extra (transformer-engine[pytorch]>=2.19,<3 on Linux x86-64). On a synthetic RF-DETR Nano run (384 px, batch 4, RTX PRO 6000 Blackwell, 20 warm-up plus 50 measured steps), eager averaged 78.075 ms per step versus 40.066 ms with Transformer Engine-aware graphs (about 1.95x), one capture serving 70 calls; this is a single fixed-shape synthetic measurement, not COCO parity, accuracy, or large-batch evidence. See the advanced training guide's CUDA graph training section for the full compatibility matrix. (#1481)
  • Two DDP training-step-time reductions from #1489, composing to a median -10.56% full step on 2x L4 (RF-DETR Nano, BF16):
    • DDPStrategy now also sets static_graph=True and gradient_as_bucket_view=True, alongside the existing find_unused_parameters=True. static_graph=True skips DDP's per-iteration autograd-graph search, safe because an empirical probe found only one constant unused parameter across real forward/backward runs (a DINOv2 mask token untouched in supervised training), not the data-dependent set the old flag comment assumed. Scope: automatic-optimization models with grad_accum_steps<=1; keypoint (manual optimization) and grad_accum_steps>1 keep the previous behavior. Alone: median -8.70% full step on 2x L4; gradients verified bit-identical under real NCCL. world_size>2 and real COCO data not exercised. (#1489)
    • Transformer._two_stage_group_selection now returns enc_out_class_embed's already-computed per-position output at the selected positions, instead of LWDETR.forward re-running that same nn.Linear a second time per group. Only the fast batched path (every shipped default) changes; the rare fallback loop still recomputes. Alone: median -2.75% full step on 2x L4 (below this project's 3% bar); composes with the DDP-flags change above to the reliable -10.56% combined median. (#1489)
  • Added RFDETR.export(format="litert"), a direct PyTorch → LiteRT .tflite route through litert-torch (torch.export capture, no ONNX/TensorFlow step), installed with pip install "rfdetr[litert]". Float32 only; runs on LiteRT's CPU (XNNPACK) delegate. On pretrained Nano/Seg-Nano, tracks eager PyTorch to ~1e-7 (boxes) / 3e-5 (logits, mask probabilities) over confident queries; keypoint models unsupported on litert-torch 0.9.4. (#1024, #1459)
  • Added TrainConfig.eval_backend, selecting the COCO evaluator used for validation and test mAP, later extended with two more values:
    • Base selection: both options ship with rfdetr[train]; "faster_coco_eval" restores the previous evaluator. Keypoint OKS and the ONNX/TensorRT benchmark evaluator are unaffected.
    • Added "ufcoco", selecting ultrafast-pycocotools (Rust, BSD-2-Clause), reproducing pycocotools' arrays byte for byte; parity tests require exact equality against faster_coco_eval. Default stays "hotcoco". (#1449)
    • Added "vernier", selecting vernier (Rust, MIT OR Apache-2.0), run in its corrected parity mode matching hotcoco/faster_coco_eval. Default stays "hotcoco".
  • WebDataset streaming training input, across two PRs:
    • Added python -m rfdetr.cli.webdataset as the dedicated packing entry point.
    • Added dataset_file="webdataset", an opt-in path streaming from pre-packed .tar shards instead of one file per image; packing needs no extra dependency, reading needs pip install "rfdetr[data]". Verified tensor-for-tensor identical to the source directory on 500 real COCO images. Training refuses to start when world_size × num_workers exceeds the shard count, and now raises (not just warns) past 30% shard-split skew. Loader throughput matched loose files on an NVMe instance (0.92x-1.03x), though construction is far faster since the annotation file is never parsed — packing pays off where per-file access is the real constraint. Accuracy not established either way: streaming trailed map-style by ~0.011 mAP@50:95 at low shard count, matched it at high count, with more seed-to-seed spread in both. Detection/segmentation only — keypoint rejects it (needs a parsed COCO file for its label space). (#1392, #1396)
  • RFDETR.inference() now accepts compile_backend="inductor" as an opt-in backend for long-running inference at a fixed batch size and resolution; default stays the TorchScript path. On CUDA, Inductor's setup runs and synchronizes before the first predict() call.
  • Added TrainConfig.pad_targets_to, padding every image's targets to a fixed row count so the loss keeps one tensor shape across batches — XLA recompiles per shape, and detection loss shape follows ground-truth box count, so a TPU run recompiled on every new per-image count (issue #1433: ~47s per new count vs 0.18s repeated, never reaching steady state on realistic data). On a Cloud TPU v6e-1, this cut compiles/time from 78/1453.9s unpadded to 9/159.0s median padded. Default None keeps the variable-length path CUDA wants; requires augmentation_backend off Kornia/GPU; excess boxes past the cap are dropped with a warning. Training dataloader only — evaluation loaders keep real targets. Semantically transparent: padded rows carry no cost in matching or loss (masked); the unmasked classification branches, segmentation mask loss and keypoint loss raise rather than reporting a silently wrong loss. (#1433, #1450)
  • Added a live opt-in end-to-end CI job (roboflow-deploy-e2e, -m e2e_roboflow) that deploys a real model to a dedicated Roboflow test project and independently polls server-side trained-model status — catching silent server-side upload failures deploy_to_roboflow()'s return value cannot surface. (#1116)
  • RFDETR.export(backbone_only=True) now exports the encoder and feature projector instead of calling the full detector and failing with AttributeError. ONNX exports retain every configured feature-pyramid level and support dynamic batches.
  • pack_targets correctness for segmentation targets (masks field) now covered by a regression test through the real RFDETRDataModule collate path, closing a parity gap #1399 shipped without. (#1451)
  • Added RFDETR.export(format="openvino", openvino_precision=...), an OpenVINO IR export route (OpenVINOExporter), installed with pip install "rfdetr[openvino]". openvino_precision accepts "float32", "float16", or None; dynamic batch export unsupported. (#1238)
  • Added rfdetr/export/inference.py, a public path for session-tier inference wrappers, replacing the previous private rfdetr.export._openvino.inference import. Preprocessing/decoding logic previously duplicated per format now lives in rfdetr/export/_runtime/ (preprocess_to_nchw, decode_detections), shared by the ONNX and TFLite inference helpers. (#1446)

Changed

  • training_config.json is now written when training starts (before model/dataset/trainer build) and rewritten with final values when train() returns — previously written only after fit() returned, so a crashed or Ctrl-C'd run left checkpoints with no record of how it was configured. Resumed runs write their own copy too. An output_dir with an existing copy gets it overwritten at start, not end, so a new run that dies early doesn't leave a stale complete-looking record. Start-of-run write is skipped on non-rank-0 processes via a distributed helper that now also covers srun (previously missed, letting every process write). Serialization runs fully before the file opens, and either write failing now logs a warning instead of ending the run. (#1493)
  • The coreml extra no longer pins torch<2.12, so pip install "rfdetr[coreml]" resolves current torch. The pin was attributed to a coremltools MIL regression; investigation found the real cause: untrained-weight encoder query scores sit ~1e-6 apart (within legitimate fp32 eager-vs-CoreML rounding drift), so a near-tied pair swaps rank and self-attention spreads that into every logit — a property of the ranking, not the conversion. e2e_coreml parity exports now assert a query-gap-vs-drift margin before comparing raw tensors, instead of trusting it blindly. Lifting the pin pulls a two-minor torch jump (2.11 to 2.14) into rfdetr[coreml] installs and brings the macOS CPU-suite legs in line with Linux/Windows. (#1024)
  • On CUDA, compile=True now also compiles the matcher's L1 box cost with dynamic shapes, independently of model CUDA graphs, bypassing eager target chunking for a fused reduction. Encoder-stage matching stays on eager torch.cdist (mixed precision there). End-to-end GPU throughput/memory improvements remain unverified.
  • HungarianMatcher builds its L1 box cost with a broadcast reduction over the four box coordinates instead of torch.cdist(..., p=1) (rfdetr.utilities.box_ops.pairwise_box_l1_cost) — torch.cdist's general Minkowski kernel can't exploit a 4-wide feature dimension and was the matcher's single most expensive operator (15.3% of device time in a torch.profiler trace on RF-DETR Medium). Bit-identical results (torch.equal holds against torch.cdist on all tested shapes); target axis chunked under a fixed element budget to bound the broadcast intermediate. Microbenchmarked 15-29x faster on an RTX 5060 Ti; end-to-end training throughput on a real 3,000-image subset rose +15.0% (math-SDPA) and +5.4% (fused-attention).
  • The training-only group_detr ranking/selection block now batches each group's projections, normalization, class ranking, top-k, and box MLP. On an L4 (RF-DETR Nano, BF16, batch 4), median steady-state step time fell 15.76% (141.13ms to 119.09ms). A separate 3-seed COCO-subset screen showed mixed mAP50-95 deltas — no accuracy claim.
  • CUDA segmentation matching now computes class/bbox/GIoU through the compact per-image route once it can avoid at least 728,000 cross-image entries (BCE+Dice mask cost keeps the full-batch calculation). An imbalanced batch dominated by one image's targets stays on the full-cartesian path; CPU/MPS, keypoint and padded targets also retain it. On an L4 (RFDETRSegNano, BF16), batch 8 improved 7.98% and batch 20 improved 12.28%; batch 4 was neutral (below the gate). Peak CUDA memory unchanged.
  • XLA validation/test/train-split evaluation callbacks now materialize each model forward once before COCO metric code reads tensors on the host, avoiding repeated compilation of overlapping lazy-graph fragments. On a Cloud TPU v5e-1 (RFDETRNano), median fit time fell 25.2% and validation time fell 73.4%; metrics and checkpoint tensors unchanged. (#1058, #1467)
  • Multi-GPU keypoint training with grad_accum_steps > 1 now synchronizes gradients once per optimizer step instead of once per microbatch, avoiding redundant DDP reductions.
  • On Python 3.14+, dice_loss_jit, sigmoid_ce_loss_jit, batch_dice_loss_jit and batch_sigmoid_ce_loss_jit become plain aliases of their eager functions, since TorchScript is unsupported there; import rfdetr no longer warns about it on 3.14+. Python 3.14 joins the CPU CI matrix and package classifiers.
  • The ONNX Runtime CPU inference session built by RFDETR.export(format="onnx")'s inference helper no longer lets its intra-op thread pool busy-spin between calls, which previously contended for CPU with other work sharing the process for as long as the session was alive.
  • Fixed-resolution (multi_scale=False) compile=True detection training now statically specialises the model and batches the default IA-BCE classification, box, GIoU and cardinality losses across final/auxiliary/encoder outputs into one lazily compiled tensor function. Multi-scale, aspect-ratio resize, CUDA graphs, segmentation/keypoint and custom criteria keep their established paths. On an L4 (RF-DETR Nano, BF16, batch 4), median settled epoch fell 14.43% (3.225s to 2.759s). Multi-GPU, EMA-on training, convergence and mAP not measured.
  • predict() now queues each image's per-tensor GPU→CPU transfers as non-blocking copies and synchronizes once per image instead of once per tensor — up to six blocking round-trips collapse to one barrier. CUDA tensors only; non-CUDA accelerators (e.g. MPS) keep the previous blocking-copy behavior. Output values unchanged. (#1420)
  • Progress-bar max_mem and free_mem values now switch from MB to GB with one decimal above 1000 MB, shortening large readings on both the TQDM and Rich progress bars. (#1470)
  • Detection and segmentation COCO evaluation now runs on hotcoco by default — a Rust evaluator under MIT, added to the train extra. Reported metrics unchanged: parity tests require exact equality against the previous backend for box-only and box-plus-mask evaluation. Set TrainConfig.eval_backend="faster_coco_eval" to restore the previous evaluator (still installed/required). compute() itself got much faster on synthetic COCO-val-shaped state (6.2s to 1.1s, macOS CPU), mostly because predictions are handed over as one detection array instead of TorchMetrics' per-annotation dictionaries. One hotcoco quirk is handled defensively: its dataset getter returns a copy, so in-place mutation is silently discarded — which would otherwise leak one IoU type's annotation areas into the other's size buckets. (#1402)

Deprecated

  • ModelConfig.amp is deprecated, removal in v1.14, superseded by TrainConfig.amp_dtype, which now accepts None to disable autocast. The two settings previously split one decision across both configs: the boolean gated AMP on the model config while the dtype was chosen on the train config, and amp=False silently voided any amp_dtype. amp_dtype is now the single authority — an explicitly set value always wins, including "fp8", which reaches the Transformer Engine hardware checks instead of being turned off. The legacy toggle still applies while amp_dtype is left at its default "auto", emitting a FutureWarning; "left at its default" is evaluated by value rather than by which fields were passed, so a config reloaded from training_config.json keeps honoring it. Replace amp=False with amp_dtype=None. One behavior change is not covered by that fallback: amp=False combined with an explicit amp_dtype="fp8" previously raised amp_dtype='fp8' requires model_config.amp=True and now runs FP8.
  • TrainConfig.fp16_eval is deprecated, removal in v1.14. It has had no runtime consumer since the PyTorch Lightning migration — evaluation precision follows amp_dtype — so it is warned about rather than migrated. Setting it to True emits a FutureWarning; the default stays silent, including on a dumped-config reload. Use amp_dtype="fp16" to evaluate in FP16.

Fixed

  • TrainConfig.eval_backend="vernier" now converts evaluation state through vernier's own columnar coco_inputs_from_columns (vernier 0.5.2+) instead of ~210 lines of hand-rolled conversion — fixing a bucketing mismatch where, under a combined bbox+segm run, box AP was bucketed by box area while hotcoco/faster_coco_eval bucket by mask area, so results disagreed for objects straddling the 32**2/96**2 size thresholds. Ground-truth area now always derives from the mask when segm is requested, matching the other backends. Also ~2-10% faster compute(), since conversion is now columnar. (#1512)
  • CoreML exports of keypoint models no longer invert the keypoint self-attention mask, which caused garbled or missing keypoint detections. nn.MultiheadAttention turns a boolean attn_mask into a float one via zeros_like(mask, dtype=...), and coremltools (9.0, 9.1) drops that dtype keyword during torch.export conversion, so the mask applied with the opposite meaning — with multiple keypoint classes, cross-class logits diverged from eager by up to 1.19; with one class every logit got a -3e4 bias, and float16 found 0/11 detections on CPU_ONLY (now finds all 11). The decoder now passes the additive float mask directly; eager outputs unchanged. Fixed upstream in apple/coremltools#2865.
  • last.ckpt and checkpoint_<epoch>.ckpt are now written on every epoch they're due, regardless of eval_interval — previously both ModelCheckpoint callbacks saved only from on_validation_end, so with eval_interval=2 the resume checkpoint refreshed only every other epoch and a crash between validations lost up to eval_interval - 1 epochs of progress (e.g. checkpoint_interval=3 with eval_interval=2 never archived epoch 2 or 8). Both callbacks now save at train epoch end, still carrying that epoch's validation results.
  • RFDETR.export(format="tflite") now converts keypoint models — onnx2tf previously failed on the keypoint decode with Dimensions must be equal, but are 17 and 100 (a rank-4 tensor broadcast over the keypoint axis). The decode now runs on a flattened (..., K * 2) layout with the same result; FP32 .tflite matches ONNX within 3.2e-3. (#1514)
  • Multi-GPU validation and test metrics no longer count the images Lightning's DistributedSampler repeats to pad the split to a multiple of world_size — up to world_size - 1 images were scored twice (a 21-image split over 2 GPUs held 22 in the merged mAP state). COCOEvalCallback now reads the sampler's own rank/replica/total-size info and skips accumulation past the dataset length. Measured: 2-GPU val/mAR/val/mAP_50_95 now match the 1-GPU values exactly (were 0.1286/0.0286 vs 0.1299/0.0287).
  • run_test=True no longer hangs multi-GPU training at the end of fit — trainer.test() was called from the main process alone under strategy="ddp"'s collective test loop while other ranks had already left fit, so the run never finished (reproduced: rank 0 still waiting at 600s). Every rank now enters trainer.test(), with file work and the test-or-not decision still made and broadcast from the main process. Spawn-based launchers (ddp_spawn/ddp_notebook), which can't host a nested trainer.test(), now warn and skip the fit-end test instead of crashing with DistNetworkError: EADDRINUSE.
  • Detection and segmentation training with grad_accum_steps > 1 no longer divides the loss by the accumulation count twice — training_step divided by accumulate_grad_batches on top of Lightning's own division in ClosureResult.from_training_step_output, so the optimizer saw 1/N of the mean gradient instead of the mean (verified: accumulated gradient was 0.5 at N=2, 0.25 at N=4, where it should be 1.0). AdamW is largely scale-invariant, but clip_max_norm engaged N times less often and SGD-family optimizers trained at an effective lr / N; batch_size="auto" selects grad_accum_steps > 1 on its own, so unrequested runs were affected too. Introduced in 1.8.0 (#1117).
  • python -m rfdetr.export.benchmark no longer crashes with AttributeError: 'str' object has no attribute 'dataset' when COCO evaluation is enabled — main() passed the raw annotation path to CocoEvaluator instead of a loaded COCO object. --disable_eval was unaffected throughout.
  • fp16 native CoreML exports of an untrained model now compile for the Apple Neural Engine. The ANE compiler rejected the whole program when the pre-topk norm still carried its identity affine (untrained weights) and the encoder token count was a multiple of 32, so the model ran CPU-only under CPU_AND_NE and failed to load under the default ALL. Trained checkpoints were never affected. The selection now gathers pre-norm rows and normalizes only the selected tokens (same computation, LayerNorm acts per token); eager outputs unchanged, mAP unchanged within noise. (#1024)
  • Compiled training keeps positional-embedding interpolation eager, working around PyTorch's symbolic antialiased-bicubic backward assertion; interpolation outputs/gradients unchanged. RF-DETR no longer globally suppresses compiler errors, and a failed CUDA graph capture now stops with the original exception rather than retrying against invalid CUDA state.
  • Explicit amp_dtype="bf16", and the default amp_dtype="auto", now select XLA's bf16-true precision instead of silently falling back to FP32 when training on TPU.
  • Multi-device TPU/XLA training now disables EMA with a runtime warning, since per-step EMA weight reads can corrupt subsequent optimizer updates on four TPU cores under BF16. One-device XLA keeps EMA (verified over 100-350 updates on a Cloud TPU v6e-1 with finite tensors). CPU and CUDA EMA unaffected. (#1058, #1476)
  • Segmentation validation/test mAP no longer risks CUDA OOM in _compute_mask_iou's boolean-to-float32 mask conversion, which previously materialized every matched prediction and ground truth of a class at once (N x H x W / M x H x W at full resolution) — a densely annotated class could make both large. Both sides now convert 32 rows at a time; output is bit-identical for nonzero counts, and a zero count on either side now returns an empty result instead of raising RuntimeError. (#1460, #1463)
  • WebDataset fixes:
    • Validation/test-only runs retain the training index's class names even when evaluation categories are a subset, for both raw and remapped labels.
    • Training keeps shard permutations consistent across ranks with real DataLoader workers, aligns accumulation at rank level, and sizes raw-label heads from all declared categories.
  • Distributed (DDP) training now preserves the minimum five optimizer steps per epoch for short datasets instead of losing the replacement sample count when Lightning injects its distributed sampler.
  • Installing the [onnx] extra from a source checkout with uv on Python 3.10, 3.11 or 3.13 now brings in ml-dtypes again — an ml-dtypes==0.5.1 override meant for Python 3.12's TFLite stack was dropping the requirement on every other interpreter, leaving import onnx failing with ModuleNotFoundError.
  • Kornia Affine now applies scalar translate_percent to both axes and reads scalar scale as a fixed range, avoiding silent horizontal-translation loss and construction failures. Scalar translation now emits a warning, since Kornia samples signed offsets where Albumentations applies a fixed positive offset.
  • TFLite INT8 documentation and warnings now reflect that dynamic-range quantization needs no calibration data. (#1363, #1364)
  • RFDETR.export(format="tensorrt", fp16=True) now actually builds an FP16 engine on strongly typed TensorRT (11+) instead of silently falling back to FP32 — the ONNX graph is now cast to FP16 first (rfdetr[tensorrt] now also pulls onnx and onnxconverter-common, and raises ImportError if missing rather than silently building FP32). (#1453, #1454)
  • Albumentations OneOf and Sequential transform containers in aug_config now respect an explicit container-level p, instead of always forcing p=1.0. Omitting p still defaults to 1.0. (#1515)
  • MultiScaleProjector no longer crashes with IndexError on feat_fuse_list[0] for scale_factors=0.25 — an elif continue only broke the inner per-channel loop, leaving an empty pyramid stage built on top of nothing. The scale==0.25 check now skips the whole scale up front, matching its intent as an extra max-pool marker. (#1498)
  • The GT/pred visualization saver no longer crashes with IndexError on an empty gt_boxes or pred_boxes list — an empty list produced a 1-D array that xywh_to_xyxy indexed as 2-D; reshape(-1, 4) now keeps it a well-formed (0, 4) array so the existing no-op guards actually run. (#1497)
  • Compiled training with torch.compile(..., dynamic=True) no longer silently falls back whole forward frames to eager — triton.coalesce_tiling_analysis is unsupported on the dynamic-shape path but still ran, hitting a size-hint assert with no symbolic-shape escape. Now disabled before the compile call. (#1455)
  • The export-mode switch is now idempotent — calling it a second time in the same state no longer raises, and the ONNX export path routes through the same guarded helper as the other formats. (#1445)

Breaking Changes

  • Removed RFDETR.optimize_for_inference(), deprecated since v1.9.0. Call RFDETR.inference(); the signature is unchanged.
  • Removed TrainConfig.lr_drop and TrainConfig.lr_min_factor, deprecated since v1.9.0, together with the validator that folded them into lr_scheduler_kwargs. Pass lr_scheduler_kwargs={"lr_drop": ..., "min_factor": ...}; the managed "step" / "cosine" presets fall back to lr_drop=100 and min_factor=0.0 when a key is absent. TrainConfig rejects unknown fields, so a training_config.json written by v1.9 or v1.10 must have the two keys removed before it is passed back to TrainConfig(**...); the migrated values are already present in its lr_scheduler_kwargs.
  • Removed rfdetr.datasets.synthetic (generate_coco_dataset, generate_synthetic_sample, draw_synthetic_shape, calculate_boundary_overlap, DatasetSplitRatios, SYNTHETIC_SHAPES, SYNTHETIC_COLORS). The module only ever fed RF-DETR's own test fixtures and is replaced by the fuse-augmentations package, whose fuse_augmentations.data module generates the same shape datasets in COCO or YOLO layout for detection, segmentation, and OBB. Callers migrate to pip install fuse-augmentations plus from fuse_augmentations.data import generate_dataset; note that it writes dense COCO category ids where the removed generator wrote sparse ones, and its shape set adds rectangle.
  • Restructured the export internals into one Exporter class per format, each with its own config dataclass, reached through a registry mapping format name to module. RFDETR.export() is unchanged. Moved: rfdetr.export.main (main(), make_infer_image → rfdetr.export.prepare.make_infer_image) and rfdetr.export.protocols.ExporterProtocol (→ rfdetr.export.base.Exporter) are removed; export_onnx/export_openvino/export_coreml/export_executorch/export_tflite are replaced by OnnxExporter/OpenVINOExporter/CoreMLExporter/ExecuTorchExporter/TFLiteExporter; rfdetr.export._tensorrt.build_engine → TensorRTExporter.build_engine in rfdetr.export._tensorrt.exporter; rfdetr.export.benchmark.TRTInference → rfdetr.export._tensorrt.inference. Format-independent graph prep now runs once in rfdetr.export.prepare; an unsupported capability (e.g. dynamic_batch on CoreML/ExecuTorch/OpenVINO) is now refused from registry data before the optional dependency imports. See the new Exporter Blueprint doc page.
  • RFDETR.export(format="tensorrt", backbone_only=True, output_name=...) now writes {output_name}-backbone.trt instead of {output_name}.trt, matching every other format and what export()'s own documentation already described. Without the marker a backbone engine silently overwrote a full-detector engine exported under the same name; scripts that rebuilt the engine path from output_name need the suffix added.
  • TrainConfig.multi_scale is now a MultiScale enum (rfdetr.config.MultiScale), absorbing the removed do_random_resize_via_padding flag: "per-batch" (new default) draws one random scale per batch, "per-sample" draws per sample inside dataset transforms and pads at collate, "off" trains at a fixed resolution. Booleans still accepted as input (True→"per-batch", matching the old default; False→"off"), but the stored value is always the enum member. Old do_random_resize_via_padding=True becomes multi_scale="per-sample"; do_random_resize_via_padding itself is now rejected.

[1.10.1] — 2026-09-07

Fixed

  • Reduced peak CUDA memory in segmentation loss: matched boolean ground-truth masks are now sampled one image at a time on CUDA instead of concatenating a batch-wide float mask tensor. (#1437)
  • point_sample(mode="nearest") no longer falls back to a host op on MPS/XLA — routed through a backend-agnostic gather path instead of F.grid_sample. CUDA/CPU are unaffected. Measured on a Cloud TPU v6e-1 with RFDETRSegNano: aten::grid_sampler_2d host fallbacks went from 50 to 0 per 5-step fit. (#1432, issue #1058)
  • SetCriterion.loss_masks no longer reads its normalizing denominator back to the host on every call — dice_loss/sigmoid_ce_loss now accept Union[Tensor, float, int] and loss_masks passes the Tensor straight through. Side effect: dice_loss_jit/sigmoid_ce_loss_jit — reachable only through lwdetr.py's backward-compat re-exports, not part of the public API — now reject most NumPy scalar denominators (np.float64 still works, np.float32/np.int64 and similar now raise RuntimeError); the eager dice_loss/sigmoid_ce_loss functions are unaffected. (#1428, issue #1058)
  • build_trainer now selects XLAStrategy for multi-device XLA/TPU training when strategy="auto" — previously this crashed at Trainer construction (DDPStrategy built before Lightning's XLA-first auto selection could apply). Also routes single-device accelerator="auto" runs on an XLA-available host through Lightning's XLAPrecision plugin instead of a plain precision= kwarg. Keypoint models are excluded from the strategy promotion. (#1427, issue #1058)
  • XLA-marked tests now pass on real TPU hardware. (#1426, issue #1058)
  • compile=True now takes effect on CUDA with the default multi_scale=True, instead of logging a notice and training eagerly. (#1436; #1411 made compilation reachable in the first place)

[1.10.0] — 2026-09-04

Added

  • TrainConfig.pack_targets (default True) concatenates each batch's per-sample target dicts into one tensor per field before the DataLoader worker-to-main boundary, rebuilding them losslessly on the other side: a batch of 16 crosses as 9 objects, not 114, with bit-identical values. Loaders yield PackedTargets when packing is lossless, else the original tuple of dicts. (#1399)
  • TrainConfig.eval_batch_size decouples the validation/test/predict dataloaders from the training batch_size. Default None inherits batch_size; unlike batch_size it accepts no "auto". (#1378)
  • TrainConfig.best_model_metric ("map" or "mar", default "map") ranks checkpoints and early-stopping by mAR instead of mAP. (#1305)
  • Training progress bar restored/extended:
    • Restored peak GPU memory (max_mem), dropped during the PyTorch Lightning migration (#794). (#974)
    • Live free/total GPU memory (free_mem) alongside max_mem (trainer.fit() only). (#1314)
    • Restored train/lr, including per-group learning rates. (#1310)
  • deploy_to_roboflow():
    • version is now optional; when omitted, the highest existing dataset version resolves automatically via the Roboflow API. (#1116)
    • Accepts ROBOFLOW_HOME as an alias for the RF_HOME weights cache directory. (#1264)
  • Experimental, undocumented XLA/TPU training path, not announced in the release notes and not exercised by any 1.10.0 benchmark: build_trainer() routes accelerator="xla"/"tpu" through an XLAPrecision("bf16-true") plugin, with a new xla optional extra (torch_xla==2.9.*, Linux only, py3.10-3.13). (#1257, #1256, #1254)
  • Kornia GPU augmentation backend gains seven ops: ToGray, Blur, Sharpen, Equalize, CLAHE, Perspective, ShiftScaleRotate. Params Kornia cannot express are warned about, not silently dropped; HueSaturationValue remains unsupported. (#1249, #1277, #1330, #1370)
  • GPU batched linear-assignment solver (rfdetr.models._assignment) wraps torch_linear_assignment (Triton-backed), folding every decoder layer's assignment problem into one solve. SciPy's linear_sum_assignment remains the CPU/fallback path, and wherever the Triton backend cannot run (non-Linux, compute capability < 8.0, old torch) it falls back internally to that same SciPy solve. New [train]-extra dependency torch-hungarian, pinned to the 0.1.0rc0 pre-release on PyPI pending a stable 0.1.0, imported lazily so inference-only installs are unaffected. (#1368)

Changed

  • RFDETR.predict() performance work, none of it changing detections: every entry measured byte-identical or checksum-identical against the previous path.
    • Skips the recursive eval() reassignment when the module tree is already in eval mode, saving ~0.4-0.5 ms/call on RTX 4060/L4 in the common repeated-inference case. (#1419)
    • Transfers PIL/uint8 NumPy inputs to device in their original byte storage and widens to float on-device, not on host, cutting host-to-device transfer size 4x. (#1415)
    • Converts PIL/uint8 NumPy inputs to contiguous CHW float storage in one fused allocation, not a separate dtype/layout pass. (#1390)
    • include_source_image=True converts CUDA float images to uint8 source bytes on-device before the host transfer, not on CPU. CPU tensors and unsupported CUDA dtypes (e.g. bfloat16) keep the previous path. (#1388)
    • Skips the deferred [0, 1] pixel-range scan (from #1341) for PIL/uint8 NumPy inputs, since to_tensor already guarantees that range for them; tensor and non-uint8 NumPy inputs are unaffected. (#1387)
  • Single-feature-level fast paths reuse tensors instead of re-materializing them (current Nano/Small/Medium/Large models; legacy RFDETRLargeDeprecatedConfig unaffected where noted); outputs bit-identical:
    • Eager forward pass skips rebuilding the sine position embedding, padding masks, and padded batch tensor when a batch carries no padding, tracked via NestedTensor.no_padding; position embeddings are served from a small cache in eval mode. Batches with real padding are unaffected. (#1416)
    • Deformable attention reuses its sampled tensor directly for single-level inputs instead of stack+flatten over a one-element list, mainly benefiting keypoint cross-attention. (#1385)
    • Transformer.forward reuses flattened tensors instead of torch.cat over a one-element list. (#1377)
    • Decoder's grouped self-attention reuses the regrouped query tensor as the key, not materializing the same grouping twice. (#1371)
  • Evaluation:
    • New TrainConfig.eval_base_model (default False) restores base+EMA validation comparison when only one model is evaluated (see Breaking Changes). TrainConfig.eval_ema_only is deprecated, removal in v1.13. (#1380)
    • COCO mAP computation consolidated into a new rfdetr.training.coco_map.OnePassCocoMeanAveragePrecision adapter: base and EMA share one evaluation pass, and each image's detection scores convert once, not once per detection. Narrows the torchmetrics[detection] pin to >=1.8.2,<1.9.0, which validates a TorchMetrics-internal contract this adapter relies on. (#1375, #1379)
    • Shares bbox IoU per image with a unified tie-break contract, plus C=1/no-crowd fast paths. (#1373)
    • mAP metric state kept on CPU, restricted to consumed metrics only; the train hot path is gated on eval epochs. (#1356)
    • Detection validation converts each batch's ground-truth targets once and shares the result between base and EMA mAP accumulators. Segmentation still converts twice, because per-head mask grids can differ. (#1381)
  • Segmentation postprocessing, both bit-identical to the previous output:
    • Reads each image's mask resize target once per batch, not per image, cutting CUDA syncs; same fix applied to COCOEvalCallback._convert_targets. (#1369)
    • Writes thresholded interpolation chunks directly into a preallocated buffer instead of torch.cat-ing a list. Small CUDA selections keep the prior path, for lower peak memory at shipped num_select=100 defaults. (#1374)
  • SetCriterion.loss_masks samples matched ground-truth mask labels via direct tensor indexing instead of point_sample, under size/contiguity/dtype guards; CUDA keeps the previous path. Measured 6.7-7.1x faster on a single-thread CPU microbenchmark of the full loss_masks call, labels bit-identical either way. (#1367)
  • HungarianMatcher batches host transfers instead of issuing them per problem. (#1361)
  • Oversized JPEGs, including 1080p sources, are draft-decoded while preserving draft geometry. (#1389)
  • Torch-free NumPy export kernels: bilinear resize made separable, top-k selection partitioned. (#1394, #1393)
  • Kornia GaussianBlur.sigma default changed (0.1, 2.0) → (0.5, 3.0) and GaussNoise.std_range default changed (0.01, 0.05) → (0.2, 0.44), 4-9x stronger, matching Albumentations' defaults. Silently changes augmentation strength for any config that omits these params on the Kornia/GPU backend (e.g. AUG_INDUSTRIAL reaches the blur default); pin explicit values if you rely on the old strength. (#1395)
  • Training skips PyTorch Lightning's pre-training sanity validation batches by default; num_sanity_val_steps restores it. Per-microbatch training-loss metrics are compacted, 17 → 9 keys on default RFDETRSmall, and compact_train_metrics=False restores per-layer keys. LR metrics emit only on optimizer updates, not every microbatch: a no-op at the new grad_accum_steps=1 default, but ~75% fewer log calls at grad_accum_steps=4, the 1.9.x default. (#1360)

Deprecated

  • rfdetr.datasets.aug_config compatibility shim now has a concrete removal target: deprecated since 1.9.0, removal in v1.12.0. Use rfdetr.datasets.aug_configs (plural) instead; constants unchanged. (#1103, #1037)
  • TrainConfig.eval_ema_only is deprecated, removal in v1.13, superseded by eval_base_model. Legacy True/False still migrate to the equivalent eval_base_model value with a FutureWarning; it still requires use_ema=True and conflicts with eval_base_model=True. (#1380)

Fixed

  • Packed targets materialize directly into per-sample device tensors instead of clone-after-move, removing a transient CUDA allocation equal to the mask field's size. (#1405)
  • Empty COCO targets keep iscrowd/area dtypes matching populated targets, enabling lossless packed-target transport for mixed empty/populated batches. (#1404)
  • Fixed compile=True aborting training on supported PyTorch versions, including 2.2. spatial_shapes is now built from Python ints under compilation instead of torch._shape_as_tensor, which Dynamo could not trace. Eager, torch.jit.trace, and the ONNX/TensorRT export path (#1155) are unaffected. (#1411)
  • Kornia CLAHE reads a scalar clip_limit as a range, matching Albumentations, and rejects the same sequences Albumentations rejects. (#1350)
  • Corrupt COCO zip downloads are retried, size validated against Content-Length, up to 3 attempts with linear backoff, instead of failing the dataset build outright. (#1306)

Breaking Changes

  • TrainConfig.grad_accum_steps now defaults to 1 (was 4), changing the default effective batch size from 16 to 4 — a training-semantics change, not just throughput. Set grad_accum_steps=4 explicitly to restore prior behavior. batch_size="auto" runs are unaffected, since the auto-batch probe overwrites grad_accum_steps. Measured 27% faster/epoch on one L4 (batch_size=16, grad_accum_steps=1 vs. the old 4/4), mAP equal within noise. (#1378)
  • Validation now evaluates one model per epoch, EMA when use_ema=True (the default) and base otherwise, instead of both, removing a full validation pass worth ~5% epoch time in one measured L4 run. Metric keys move: val/mAP_*, val/mAR, per-class val/AP/<class>, and val/loss report whichever model was evaluated, the EMA model by default, instead of always the base model — changing what a ReduceLROnPlateau scheduler, ModelCheckpoint(monitor=...), or early stopping watching those keys tracks. val/ema_* remains available for explicit EMA consumers. checkpoint_best_regular.pth is no longer written when the base model is not evaluated. Set TrainConfig.eval_base_model=True to restore the previous base+EMA comparison; use_ema=False runs are unaffected. (#1380)
  • Optimizer parameter groups are now one per distinct learning-rate/weight-decay combination instead of one per parameter (rfdetr-nano: 465 → 28 groups), letting fused/foreach AdamW batch properly. AdamW steps are bit-identical and old checkpoints auto-regroup on load, but an explicit lr_scheduler_kwargs list sized to the old per-parameter group count, e.g. LambdaLR's per-group lr_lambda, must be resized to the new group count. (#1409)
  • Dataset builders (build_roboflow_from_coco, build_roboflow_from_yolo, build_o365_raw) now require seven image-pipeline options (square_resize_div_64, segmentation_head, multi_scale, expanded_scales, do_random_resize_via_padding, patch_size, num_windows; build_o365_raw takes no segmentation_head) instead of silently substituting contradictory defaults when called with an incomplete config namespace, which could previously train with multi-scale off and the wrong crop scales without warning. Callers passing a complete TrainConfig/ModelConfig are unaffected; callers assembling a partial namespace by hand must supply every field. (#1413)
  • TrainConfig.log_per_class_metrics now defaults False (was True), so per-class AP keys are no longer emitted by default. TrainConfig.compute_val_loss now defaults "auto" (was True), so val/loss is computed only when a scheduler/callback consumes it. Set either explicitly to restore the prior unconditional behavior. (#1372)

[1.9.4] — 2026-08-24

Fixed

  • ONNX and TFLite reference inference helpers accept an explicit background_class_id: -1 preserves the existing final-background default, None retains every exported logit slot for sparse-ID COCO checkpoints, and 0 supports legacy background-first keypoint checkpoints. (#1397)
  • Fixed the TFLite reference inference helper assuming a lone rank-4 output is a segmentation mask. ONNX output names rarely survive the conversion — RF-DETR's own TFLite files arrive as StatefulPartitionedCall:N — so a keypoint export's pred_keypoints tensor was indistinguishable from a mask by name and was silently upsampled into Detections.mask. rank4_output now defaults to None, decoding only named masks; pass "masks" explicitly for a name-stripped segmentation export. (#1397)
  • Fixed the torchvision-native non-square training pipeline resampling crop-branch outputs twice. _build_train_resize_transforms(square=False) resizes each crop directly to a randomly selected target scale, matching the square and Albumentations paths. This changes the augmented pixel distribution for non-square training by avoiding the fixed 384x384 intermediate and its extra resampling step. Square training, the released default for every shipped model config, is untouched, as are validation, prediction, and export preprocessing. (#1383)
  • Fixed custom Albumentations configs treating TimeReverse as a pixel-only transform, which flipped images while leaving boxes and keypoints unchanged. TimeReverse now shares the geometric-transform and replay-based keypoint handling used by HorizontalFlip. The keypoint safety filter disables both TimeReverse and SquareSymmetry when keypoint_flip_pairs=[]; detection-only pipelines (keypoint_flip_pairs=None) retain them, and configured pairs enable their keypoint-slot swapping. SquareSymmetry already had geometric and replay handling as the alias of D4; this fix extends the no-pairs safety filter to it. The default torchvision pipeline is unchanged, as are configs already using the canonical HorizontalFlip/D4 names.
  • Fixed TFLite export failing when onnx2tf could not resolve the installed onnxsim console script from a non-activated virtual environment. onnx2tf invokes the bare onnxsim name; when that lookup raises FileNotFoundError it logs Failed to optimize the onnx file, a warning that also appears in working runs, and a stock RFDETRSmall() export then failed with RuntimeError: onnx2tf conversion failed: Output tensors of a Functional model must be the output of a TensorFlow Layer. RF-DETR now temporarily adds the running interpreter's script directory to PATH during conversion. (#1365)
  • Fixed the default torchvision-native training pipeline silently corrupting keypoint annotations when keypoint_flip_pairs is empty on a schema with genuine left/right pairs. RandomHorizontalFlip on this backend always mirrored keypoint x-coordinates when a flip was drawn, but relabeled left/right joints only if self.keypoint_flip_pairs: — with an empty list, the pydantic default and one possible outcome when automatic flip-pair inference from dataset metadata misses an asymmetric schema, affected training samples got their keypoints mirrored in position while keeping their original left/right label, with no warning. _build_torchvision_pipeline now drops the flip entirely for an empty-but-not-None keypoint_flip_pairs, logging the warning the Albumentations backend already emits via filter_keypoint_hflip_augmentations, worded for this backend's lack of an editable aug_config, matching the annotation-safety behavior that backend has had since #1122. An empty list can also legitimately mean the schema has no left/right pairs at all, e.g. a single midline keypoint; the unpatched flip was harmless there since nothing needed relabeling, but this fix disables it there too, for consistency with the Albumentations backend's contract, at the cost of a now-unavailable-by-default augmentation for that narrower case. Detection-only pipelines (keypoint_flip_pairs=None) and keypoint pipelines with real pairs are unaffected.
  • Fixed BestModelCallback treating PyTorch Lightning's pre-training sanity-check validation pass as a real epoch's result. Its EMA-checkpoint tracking and the smooth_alpha smoothing accumulator are custom bookkeeping sitting outside ModelCheckpoint's own trainer.sanity_checking guard, which the regular-checkpoint path already inherits, so a positive sanity-check score — common when starting a new run initialized with pretrain_weights from a checkpoint pretrained on a different dataset — could be written out as the permanent "best" checkpoint_best_ema.pth before a single real epoch ran, and real training could then never surpass it. This is distinct from PTL's own resume/ckpt_path restart, which PTL itself skips the sanity check for (not val_loop.restarting). (#1357, fixes #1348)

[1.9.3] — 2026-08-17

Changed

  • HungarianMatcher's compact-path safety gate now computes its target-side finiteness checks once per training step, not once per matcher() call — SetCriterion.forward invokes matcher() separately for the final, auxiliary and encoder layers with the same targets, so the precheck is cached (keyed on targets identity plus dtype/device/num_classes) and reused. Matching results unchanged; callers must not mutate targets in place between precompute and reuse. (#1340)
  • Per-class confidence-threshold sweeps in evaluation are O(N log N), not O(T·N): one stable ascending sort per class plus np.searchsorted into precomputed suffix sums replaces a full rescan per threshold. NaN scores are explicitly masked so they never count as "above threshold". Results are unchanged. (#1339)
  • RFDETR.predict() no longer blocks the host on a per-image CUDA sync for its [0, 1] pixel-range validation — range-check tensors are collected unsynced across all images and resolved once after every image is queued, letting later images' GPU work overlap the sync. A malformed-rank input with include_source_image=True now raises a public ValueError instead of an internal RuntimeError from permute(). (#1341)
  • Transformer.forward's two-stage query selection gathers the torch.topk-selected rows before running the bbox-delta MLP (enc_out_bbox_embed), not after: the MLP is pointwise with no cross-token mixing, so it needs at most the num_queries rows that survive selection, not every one of the sum(H*W) encoder positions. (#1334)
  • PostProcess box/mask/keypoint selection is now deterministically tie-broken (torch.argsort(..., stable=True) + slice, replacing torch.topk) — ties resolve by descending score then ascending flattened query/class index, shared with the torch-free export decoders. Output ordering among tied scores may differ from 1.9.2 (same detections, different order); it was never contractual. PostProcess(num_select=<negative>) now raises ValueError at construction. (#1320)

Fixed

  • Fixed evaluate(split="test") on YOLO-format datasets silently evaluating valid/ instead of the real test/ split. When no resolvable test split exists, evaluation now falls back to valid/ with a logged warning instead of silently mislabeling it; a new YoloSplitUnavailableError (a FileNotFoundError subclass) drives that fallback. A declared-but-unresolvable test path, an empty images directory, or a missing labels directory still raises. COCO-format datasets have no such fallback and still raise FileNotFoundError. (#1329, #1343)
  • Fixed metrics.csv training history being wiped by a resumed run — PyTorch Lightning's _ExperimentWriter deletes any pre-existing metrics.csv the first time .experiment is accessed, removing every pre-resume row. The file is now snapshotted before that access and restored after, gated on resume being set — a fresh non-resumed run into a reused output_dir still resets the file. (#1325, closes #1321)
  • Fixed SegmentationHead's skip_blocks branch skipping the learned spatial_features_proj 1×1 convolution before computing mask logits, matching the non-skip branch. Affects encoder-branch aux mask supervision during training only (sparse_forward, skip_blocks=True) — the export path and main decoder path were already projected, so predict() and exported models are unchanged. (#1331)
  • Fixed non-finite keypoint predictions poisoning the shared box head's gradients, in both decoder and encoder branches — a NaN delta could propagate through 0.0 * nan == nan into ref_wh, shared with the box head. Deltas are now sanitized at the source with torch.nan_to_num(..., 0.0); the keypoint loss also masks non-finite predicted keypoints and target areas. The matcher's own keypoint cost still lacks the equivalent guard. (#1336)
  • Fixed batch_size="auto" probing ignoring AdamW's optimizer-state memory (exp_avg/exp_avg_sq), causing OOM on the first optimizer step when the probed batch size overshot what real training could fit — it now accounts for that via a shadow optimizer, and warns when a non-AdamW optimizer is configured. (#1342)
  • Fixed the ONNX Runtime export benchmark ignoring the requested device: the inference session is built with providers=[("CUDAExecutionProvider", {"device_id": device})] instead of the bare provider name, which previously always bound to GPU 0 regardless of --device N. (#1346)
  • Fixed training metric plots drawing a legend only on the subplot titled "Loss"; every subplot now gets one. (#1335)
  • Fixed the ONNX and TFLite reference decoders taking a per-query argmax, which silently dropped legitimate detections whenever a query scored above threshold on more than one class — both now mirror PostProcess's multi-label selection via a shared _select_topk_multiclass helper using the same deterministic tie rule. (#1320)
  • Fixed EMA training performing an extra averaged-model update at epoch boundaries after the final optimizer step, which let one update per epoch bypass ema_update_interval and change the EMA trajectory. (#1319)
  • Fixed model.export(format="tflite") hanging forever at the ONNX → TFLite conversion step, with no traceback, no error, 0% CPU. onnx and TensorFlow both statically link Abseil and export its symbols weakly, so whichever loads first supplies them to both; the TFLite route runs a full ONNX export first, so TensorFlow's executor blocked forever in absl::Notification::WaitForNotification() restoring the SavedModel bundle. TensorFlow is now imported before the ONNX export; a warning logs when the calling process already imported onnx first (that order can't be repaired in-process). (#1322, #1323)

Breaking Changes

  • Exported artifact filenames encode precision or backend for variant-derived/default names: TFLite {stem}_float32.tflite / {stem}_float16.tflite → {stem}_fp32.tflite / {stem}_fp16.tflite; ExecuTorch {variant}.pte → {variant}_{backend}.pte (or {variant}_qnn_{soc}.pte); CoreML {variant}.mlpackage → {variant}_fp32.mlpackage / {variant}_fp16.mlpackage; TensorRT {stem}.trt → {stem}_fp16.trt / {stem}_fp32.trt. ONNX filenames are unchanged. Update scripts that hardcode or glob these artifact filenames; explicit output_name overrides are unchanged.

[1.9.2] — 2026-08-11

Changed

  • HungarianMatcher's detection-only cost matrix is now built padded to each batch's max(T_i) target count and diagonal-extracted, not padded to the cross-image sum(T_i), whenever a fast eligibility check passes (ineligible batches fall back to the previous full-cartesian path with identical results). Training-time only (under torch.no_grad()). On real COCO batches, matcher time drops ~51% and peak CUDA memory ~73-76%; measured end-to-end training step 288.364ms to 232.457ms on an A100. Saving scales with target-count evenness — a batch where one image holds nearly all targets sees little improvement. (#1297, #1281, #1312)
  • seed_all() now escalates to torch.use_deterministic_algorithms(True, warn_only=True) after setting the cuDNN flags, so every op with a deterministic kernel uses it; ops without one (some scatter / grid_sample CUDA kernels) warn at execution time instead of raising. User-visible as new runtime warnings and a possible slight performance cost. (#1307)
  • RFDETR.predict() pins CPU image tensors before the CUDA transfer. (#1313)
  • Two-stage query selection avoids materialising repeated top-k gather indices. (#1278)
  • Evaluation matching counts labels on the host, not the device. (#1276)
  • Keypoint decode skips redundant CUDA presence checks in postprocessing. (#1282)

Fixed

  • Fixed loading a detection checkpoint published before keypoint support warning that _kp_active_mask is a "model parameter not in checkpoint (left at random init)" — it's a deterministic schema buffer the model always rebuilds, empty for detection-only variants, not a learned parameter, so its absence never affected loaded weights. Affects Nano, Small, Large (2026) and SegSmall. An unexpected _kp_active_mask in a checkpoint still warns. (#1302)
  • Fixed resuming training from one of BestModelCallback's four lightweight checkpoints (checkpoint_best_regular.pth, checkpoint_best_ema.pth, checkpoint_best_total.pth, last_ema.pth) silently restarting per-callback state cold — it now restores (those files intentionally omit optimizer/LR-scheduler state, with a warning saying so). Best-score restore additionally requires the original output_dir to match. (#1318)
  • Fixed training-time log calls corrupting or duplicating the completed Rich epoch progress bar when RichProgressBar(leave=True) is active — a new stream handler now re-resolves stdout/stderr on every emit, following Rich's redirect proxies instead of capturing the pre-redirect stream once at import time. (#1316)
  • Fixed an index-less torch.device("cuda") never matching an indexed device like cuda:0 in the deferred-move guard, which re-moved every parameter on every call — it's now normalised to the current device index before comparison. (#1311)
  • Fixed the legacy query-embedding fallback warning on every load; it now warns only when it actually truncates weights. (#1301)
  • Fixed eval_ema_only runs logging no validation output at all when the base metric was empty — EMA metrics are now computed and logged in that case (val/ema_mAP_50_95, val/ema_mAP_50, val/ema_mAR, per-class AP, val (ema) summary table), and val/F1 is no longer silently dropped. Point monitor_ema at val/ema_mAP_50_95, since val/mAP_50_95 stays unpopulated under eval_ema_only. (#1289)
  • Fixed ModelContext.reinitialize_detection_head() raising AttributeError: 'NoneType' after RFDETR.inference(inplace=True) cleared the weights — it now raises a clear RuntimeError, before args.num_classes is mutated so a rejected call can't leave the context half-updated. (#1283)
  • Fixed evaluate() not building its datamodule from the resolution-override config. (#1280)

Breaking Changes

  • COCO datasets containing an unannotated grouping category no longer spend a model output slot on it. Roboflow COCO exports prepend a synthetic root category (id 0, supercategory: "none", named after the project) that every real class then lists as its own supercategory; it carries no annotations, but previously took label index 0 and an extra class channel. CocoDetection.cat2label, the auto-detected num_classes and RFDETR._load_classes() now share one filter (rfdetr.datasets.coco.filter_parent_categories), so training such a dataset builds an N-class head instead of N+1 and every real class shifts down one label index. A parent category that owns annotations keeps its slot, and flat datasets are unaffected. Checkpoints trained before this change keep their N+1-class head — evaluating one against the same dataset now misaligns per-class metrics, firing the existing class-count UserWarning; retrain. Passing num_classes explicitly preserves the checkpoint's N+1-class head width so the weights still load, but does not restore the old label indices: CocoDetection drops the grouping category whenever remap_category_ids=True, so every real class still shifts down one slot and the pretrained head is misaligned against the new labels. The keypoint remapping path (_build_keypoint_cat2label) is unchanged, so keypoint datasets still include the grouping category. For hierarchical datasets, the train/valid/test splits now share one label mapping, always derived from the train split, so a grouping category annotated in only some splits no longer shifts that split's label indices out from under the others. (#1303)

[1.9.1] — 2026-08-03

Changed

  • PostProcess selects boxes, masks, and keypoints with index_select/expand instead of materialising a repeated int64 gather index, an allocation reaching 21–84 MiB per image for the segmentation mask head. Mask post-processing at head resolution is 2.6–3.0× faster; the output is bit-for-bit identical. (#1268)
  • RFDETR.predict() no longer upsamples segmentation masks whose scores fall below the caller's threshold before discarding them; on typical COCO images only a few of the num_select masks survive threshold=0.5. End-to-end predict() is ~20% faster at 1080p, the saving scaling with image area and neutral at 640 px; the output is unchanged. (#1265)
  • ExecuTorch export lowers the addmm operations the XNNPACK partitioner leaves undelegated back into aten.linear via AddmmToLinearTransform, which runs ~100× faster for those shapes. RFDETRNano on XNNPACK / Apple silicon is ~2.5× faster (119.9 → 48.3 ms median); outputs match the previous lowering to ~1e-4. (#1262)

Fixed

  • Fixed keypoint_flip_pairs silently disabling horizontal-flip augmentations (HorizontalFlip, Flip, D4) on detection-only datasets when a custom aug_config is supplied. AlbumentationsWrapper.from_config treats an empty keypoint_flip_pairs as "keypoint pipeline with no flip pairs defined" and drops flip transforms for annotation safety; detection pipelines must pass None instead of [] to keep flips enabled. (#1248)
  • Fixed export inference and INT8 calibration resizing through PIL's antialiased BILINEAR/BICUBIC filters, which diverge from predict() on downscale and shift exported-model confidence scores and INT8 calibration ranges. The ONNX inference, TFLite inference, INT8 TFLite calibration, and benchmark/traced-example paths now resize with RFDETR.predict()'s exact convention: bilinear, half-pixel centers, antialias=False. A shared torch-free _bilinear_resize_half_pixel NumPy kernel (rfdetr/export/_resize.py) mirrors the convention wherever torchvision is unavailable. Re-export any INT8 TFLite model to recalibrate against the corrected pixel distribution. (#1269)
  • Fixed pip install 'rfdetr[onnx]' on Python 3.10 and pip install 'rfdetr[executorch]' on Python 3.14 failing during install. Each extra previously resolved to a version (onnxruntime, executorch) shipping no wheel for that interpreter and with no source distribution to fall back on; the extras are now gated to interpreters that publish wheels. (#1267)
  • Fixed the Kornia augmentation builders (GaussianBlur, GaussNoise) rejecting scalars for range parameters; they now accept either a scalar or a (min, max) pair, matching the Albumentations path. A custom aug_config valid under Albumentations no longer raises a bare TypeError when augmentation_backend="cpu"/"auto" resolves to Kornia, i.e. Kornia installed and CUDA available. (#1255)
  • Fixed uv sync failing to create .venv; an executorch/tflite extra conflict previously blocked resolution of the development environment. (#1253)

Documentation

  • Corrected RF-DETR Keypoint Preview's parameter count (126.4 M → 40.7 M), added deployment parameter-count columns to the keypoint benchmark tables, and clarified that the new SAM 3 RF100-VL result is author-reported rather than measured in SAB. (#1258, #1261)
  • Documented ONNX Runtime raw-output decoding and expanded the LLM keypoint task/model/benchmark/API reference. (#1251, #1260)

[1.9.0] — 2026-07-27

  • Default dataset augmentations use torchvision-native transforms unless Albumentations is installed, in which case augmentation_backend="auto"/"cpu", the default, auto-selects Albumentations instead — identical user code can therefore resolve to a different resize backend, and slightly different pixel values / mAP, purely based on whether rfdetr[augment] is installed. Pass augmentation_backend="torchvision" to pin torchvision regardless of what is installed. Non-empty custom aug_config dictionaries use the optional Albumentations integration and Kornia GPU backend, both via pip install 'rfdetr[augment]'. The [train] extra no longer installs Albumentations or Kornia. See the migration guide's "Upgrade 1.8 → 1.9" section for remediation steps. (#1112)

Added

  • Native CoreML export: format="coreml" on RFDETR.export() produces a .mlpackage (mlprogram, iOS 16+) directly from torch.export, with no ONNX intermediary — distinct from ExecuTorch's format="executorch", backend="coreml" .pte path. Install with pip install 'rfdetr[coreml]' (macOS only; coremltools>=8.0,<10.0). (#1235)
  • Multi-GPU / multi-node keypoint (pose) training under DistributedDataParallel. Keypoint models (RFDETRKeypointPreview) previously raised NotImplementedError for any distributed strategy, num_nodes > 1, or devices > 1; they now train with strategy="ddp" / strategy="auto" on multiple GPUs and nodes, launched with torchrun exactly like detection models. Because keypoint models use manual optimization, gradients synchronize on every microbatch — keep grad_accum_steps=1 on multi-GPU for best throughput (grad_accum_steps > 1 is correct but performs redundant all-reduces). Sharded strategies (FSDP / DeepSpeed) remain unsupported for keypoint models and raise a clear error. See the "Keypoint / Pose models" note in docs/learn/train/advanced.md. (#1232)
  • scale_jitter: bool = True on TrainConfig — independent control for the resize → crop → resize branch (Option B) in the training resize pipeline. Disabling this branch previously required passing aug_config={}, which also disabled the entire Albumentations augmentation stack; aug_config now controls only that stack. Set scale_jitter=False to use direct resize only, with annotations near image borders never clipped.
  • AugmentationBackend.TV (augmentation_backend="torchvision") — forces the torchvision-native default pipeline. Unlike "cpu"/"auto", which auto-select the best installed backend (Albumentations > Kornia > torchvision) and can therefore resolve differently across environments, "torchvision" always resolves to torchvision regardless of what optional packages are installed. AugmentationBackend now holds only concrete, directly-usable backends (TV, ALBU, KORNIA); "cpu"/"auto" remain accepted augmentation_backend input strings, resolved lazily at dataset-build time to keep saved configs portable across environments, but are no longer enum members. AugmentationBackend.TV/.ALBU values changed from "tv"/"albu" to "torchvision"/"albumentations"; the old "tv"/"albu"/"gpu" strings are still accepted as legacy input aliases.
  • TrainConfig.optimizer (str | Callable) and optimizer_kwargs — configurable training optimizer. optimizer="adamw", the default, keeps RF-DETR's built-in fused torch.optim.AdamW path unchanged. A bare short name selects a native torch.optim optimizer only (e.g. "sgd", "adam"); any other optimizer, including third-party ones such as pytorch-optimizer (install separately), is selected by a full dotted import path ("pytorch_optimizer.Lion") or a callable / functools.partial called with the RF-DETR parameter groups. optimizer_kwargs forwards constructor arguments, ignored for callables, which bake their own arguments in. (#1006)
  • TrainConfig.lr_scheduler (str | Callable) plus lr_scheduler_kwargs, lr_scheduler_interval, and lr_scheduler_monitor — configurable LR scheduler, mirroring optimizer. lr_scheduler="step"/"cosine", the managed presets, keep RF-DETR's built-in warmup-aware schedules unchanged; any other scheduler is selected by a full dotted import path ("torch.optim.lr_scheduler.OneCycleLR") or a callable / functools.partial called with the optimizer. Explicit schedulers are built from lr_scheduler_kwargs only, with no total_steps/T_max injected, are auto-wrapped in a SequentialLR linear warmup when warmup_epochs>0, and step at lr_scheduler_interval ("step"/"epoch"). ReduceLROnPlateau is supported end-to-end: it steps once per epoch on the metric named by lr_scheduler_monitor (default "val/loss"), in both the automatic and manual (keypoint) optimization paths.

Deprecated

  • TrainConfig.lr_drop and lr_min_factor — pass them through lr_scheduler_kwargs instead ({"lr_drop": ...} / {"min_factor": ...}). Deprecated since v1.9.0, removal in v1.11.0. The fields still work meanwhile and are folded into lr_scheduler_kwargs for the managed presets with a FutureWarning; default values, e.g. on config reload, do not warn. Set with an explicit, non-managed scheduler they are inert and emit a FutureWarning.

Fixed

  • Fixed the keypoint L1-loss helper (compute_l1_keypoint_loss) returning detached new_zeros on its out-of-schema class-index guard; it now returns graph-connected zeros. A detached zero left the keypoint-head parameters without a gradient path on that batch, which desyncs DistributedDataParallel's gradient reducer across ranks (hang or "parameter did not receive grad") when the guard fires on some ranks but not others. This is a prerequisite for the multi-GPU keypoint training above.
  • Fixed non-square Albumentations training resize (aug_config set, augmentation_backend resolving to "albumentations") silently inflating every image's longest side to max_size, 1333 by default. SmallestMaxSize → LongestMaxSize always forces an exact resize in Albumentations, not a conditional cap; a new CappedLongestMaxSize internal transform only shrinks, never upscales, matching torchvision's RandomResize semantics.
  • Fixed explicit augmentation_backend="albumentations" resolving successfully without Albumentations installed and failing later, deep in dataset construction; it now raises a clear ImportError immediately.
  • Fixed RFDETR.from_checkpoint(..., trust_checkpoint=True) having no effect. It previously bypassed the safe-load check only for the checkpoint's own metadata read; model construction then silently reloaded the same file through load_pretrain_weights() with the unsafe-load default, so the flag did nothing for checkpoints that genuinely needed it and raised the same RuntimeError it was supposed to bypass. (#1239)
  • Fixed segmentation evaluation resizing ground-truth masks to each image's original resolution before comparison, a lossy round trip vs. the mask head's native grid; GT masks now resize directly to each prediction's own pixel grid, so segm mAP is computed on consistent pixel grids. (#1241)
  • Fixed pip install 'rfdetr[onnx]' (and [tflite]) hanging while building onnxsim from source on CPython 3.11/3.13 and Linux aarch64. The previous onnxsim<0.6.0 pin resolved to 0.5.0, which ships no wheels for those targets, so pip compiled onnxsim's bundled onnxruntime/onnx from source. The constraint is now onnxsim>=0.7.0, which publishes prebuilt wheels across CPython 3.10–3.13 on Linux x86_64/aarch64, Windows x86_64, and macOS arm64. (#1242)

Deprecated

  • RFDETR.optimize_for_inference() renamed to RFDETR.inference(), same signature. The old name is kept as a deprecated alias that forwards to inference() and emits a FutureWarning. Deprecated since v1.9.0, removal in v1.11.0.

Changed

  • Matched-pair IoU targets in the classification/matching losses compute via elementwise_box_iou/elementwise_generalized_box_iou, new public helpers in rfdetr.utilities.box_ops, instead of torch.diag(box_iou(...)). The old path built the full NxN pairwise IoU matrix just to read its diagonal; the new one computes only the N matched pairs directly, reducing peak GPU memory during loss calculation. Both new helpers raise ValueError on mismatched-length inputs instead of silently broadcasting. (#1245)
  • The [tensorrt] extra no longer installs pycuda, needed only for TRTInference's async benchmarking mode, which now requires the separate [tensorrt-bench] extra (pip install 'rfdetr[tensorrt-bench]'); the standard export→engine path (polygraphy, no pycuda) is unaffected. (#1246)

Security

  • RFDETR.from_checkpoint() uses safe deserialization by default (weights_only=True) instead of always running full pickle deserialization. Checkpoints containing custom Python objects beyond argparse.Namespace or types.SimpleNamespace need the new keyword-only trust_checkpoint: bool = False parameter set to True to opt into the old, unsafe behavior; resume-from-checkpoint during training honors the same flag. (#1179)
  • TensorRT export no longer shells out to the trtexec CLI — engines are built in-process through the polygraphy Python API, removing the subprocess/shell-injection surface entirely. (#853)

Removed

  • [kornia] extra removed — GPU-side augmentation installs via [augment] (pip install 'rfdetr[augment]') instead. There is no [kornia] alias extra; pip install 'rfdetr[kornia]' will fail.
  • rfdetr.util.* and rfdetr.deploy import paths, deprecated since v1.6.0 with remove_in="1.9.0". Use rfdetr.utilities.*, rfdetr.assets.coco_classes, rfdetr.training.drop_schedule, rfdetr.training.param_groups, rfdetr.visualize.data, rfdetr.models.heads.segmentation, and rfdetr.export instead.
  • rfdetr._namespace.build_namespace(model_config, train_config), deprecated since v1.7.0 with remove_in="1.9.0". Use rfdetr.models.build_model_from_config and build_criterion_from_config instead.
  • The train_config argument to load_pretrain_weights(nn_model, model_config, train_config), deprecated since v1.7.0 with remove_in="1.9.0". Call it with just (nn_model, model_config).
  • The start_epoch, do_benchmark, and callbacks keyword arguments to .train()/.evaluate(), deprecated since v1.7.0 with remove_in="1.9.0". PTL resumes automatically via resume=; use the rfdetr.export.benchmark module for benchmarking; pass PTL Callback objects directly instead of a callbacks dict.
  • TrainConfig.group_detr, TrainConfig.ia_bce_loss, TrainConfig.segmentation_head, TrainConfig.num_select, and ModelConfig.cls_loss_coef, deprecated since v1.7.0 with remove_in="1.9.0". group_detr, ia_bce_loss, segmentation_head, and num_select now live only on ModelConfig; cls_loss_coef now lives only on TrainConfig.
  • RFDETRLarge's automatic silent fallback to RFDETRLargeDeprecatedConfig on checkpoint/config incompatibility errors. Loading legacy deprecated-Large weights through RFDETRLarge now raises the original error instead of retrying; use RFDETRLargeDeprecated directly to load those checkpoints.

[1.8.3] — 2026-06-27

Added

  • optimize_for_inference(inplace=True) — new keyword-only argument on RFDETR.optimize_for_inference(); skips the deep-copy of the base model for memory-constrained inference-only deployments, ~0.5× model-weight peak memory reduction. Requires compile=False. After inplace optimization, export() raises RuntimeError and remove_optimized_model() issues a UserWarning and returns cleanly instead of silently clearing state. New RFDETR.is_optimized_inplace property returns True after a successful inplace optimization. (#1089)
  • CocoKeypointSchema.keypoint_flip_pairs and YoloKeypointSchema.keypoint_flip_pairs fields — horizontal-flip swap pairs inferred automatically from keypoint names (left/right naming convention) for COCO schemas, and from flip_idx permutation for YOLO schemas. Auto-populated by infer_coco_keypoint_schema and infer_yolo_keypoint_schema respectively. (#1164)
  • infer_coco_keypoint_schema and infer_yolo_keypoint_schema re-exported from rfdetr.datasets, previously only accessible from rfdetr.datasets._keypoint_schema. (#1164)

Changed

  • Horizontal flip detection in AlbumentationsWrapper uses Albumentations ReplayCompose replay metadata instead of heuristic bbox-center mirroring, eliminating false positives on non-flip transforms that shift box centers. Falls back to alb.Compose with a UserWarning when albumentations <1.3 is detected. (#1164)
  • Keypoint schema inference supports native COCO format (dataset_file="coco") in addition to "roboflow" and "yolo". (#1164)
  • _keypoint_schema_cache key changed from dataset_dir (string) to (dataset_file, dataset_dir) tuple, preventing cross-format cache collisions when the same directory is used with different dataset formats. (#1164)

Fixed

  • Fixed unbounded box regression producing negative or out-of-frame coordinates: predicted bounding boxes are clamped to image bounds [0, width] × [0, height] in PostProcess._postprocess_boxes(). scale_fct is also cast to boxes.dtype before multiplication, preventing dtype mismatch when boxes are float16. (#1168)
  • Fixed SegmentationTrainConfig.cls_loss_coef default of 5.0, corrected to 1.0 to restore the pre-v1.7 effective classification loss weight. The 5.0 value was present since v1.6 but dead code until the v1.7 TrainConfig ownership migration activated it, silently over-penalising classification relative to mask losses during segmentation fine-tuning. To reproduce pre-fix behaviour, pass cls_loss_coef=5.0 explicitly. (#1165)
  • Fixed KeypointTrainConfig.keypoint_nll_loss_coef, restored to 1.0 to align with the other keypoint loss terms (keypoint_l1_loss_coef, keypoint_findable_loss_coef, keypoint_visible_loss_coef). The previous default of 0.5 was set to dampen OKS@75 oscillation but under-weighted the NLL loss relative to other terms in practice. (#1165)

[1.8.2] — 2026-06-25

Added

  • YOLO pose keypoint dataset support: load Ultralytics YOLO pose datasets (.yaml with kpt_shape) directly for keypoint fine-tuning. Schema is inferred automatically via infer_yolo_keypoint_schema. (#1156)
  • is_bg_first_schema, to_active_first, to_bg_first, schemas_semantically_equal utilities in rfdetr.utilities.keypoints, re-exported from rfdetr.utilities, for schema-aware keypoint processing. (#1160)
  • amp_dtype field on TrainConfig ("auto" / "bf16" / "fp16"): pin the mixed-precision autocast dtype instead of relying on device-capability auto-detection. "auto", the default, preserves the historical behaviour — bf16-mixed on Ampere+ CUDA, 16-mixed otherwise. Invalid values degrade gracefully to "auto" with a UserWarning. (#1143)
  • Instance segmentation fine-tuning cookbook (docs/cookbooks/fine-tune_segmentation.ipynb) — end-to-end walkthrough using RFDETRSegSmall across seven diverse segmentation datasets. (#1159)
  • Inference latency benchmark cookbook (docs/cookbooks/inference-latency-benchmark.ipynb) — benchmarks CPU/GPU throughput across model sizes with reproducible measurement methodology. (#1152)

Changed

  • Default num_keypoints_per_class in RFDETRKeypointPreviewConfig changed from [0, 17] (background-first) to [17] (active-first). Legacy bg-first checkpoints auto-align on load via _kp_active_mask. (#1160)

Fixed

  • Fixed RFDETR.from_checkpoint() misreading num_classes as shape[0], i.e. num_classes + 1 including the background class, causing load_state_dict shape mismatches or a silent extra output class on every load. It now infers num_classes and num_keypoints_per_class from checkpoint weights, class_embed.weight.shape[0] - 1 and _kp_active_mask respectively. BestModelCallback._serialize_model_config is also fixed to persist the correct foreground-only num_classes. (#1158)
  • Fixed HungarianMatcher.forward() hardcoding 0.25 in the focal classification matching cost, silently ignoring any non-default focal_alpha passed to the constructor or build_matcher; it now uses the configured value. This had misaligned the bipartite matching cost with the focal classification loss in criterion.py, which correctly used self.focal_alpha. (#1147)
  • Fixed spatial_shapes in Transformer.forward() being built by torch.empty + in-place index assignment, which emitted a ScatterND feeding a shape tensor (level_start_index) that TensorRT rejected with "IScatterLayer cannot be used to compute a shape tensor". It now uses symbolic Shape ops, torch.stack of per-level torch._shape_as_tensor slices. Required to export any RF-DETR model to a TensorRT engine. (#1155)
  • Fixed keypoint model inference returning the wrong class_name field in predictions. (#1151)
  • Fixed silent train-mode inference after the first prediction: predict() re-asserts eval mode before each call for unoptimized models. (#1146)
  • Fixed TFLite inference preprocessing and mask decoder diverging from PyTorch predict() behaviour. (#1131)
  • Fixed a Python version mismatch in optional-dependency version overrides. (#1137)

[1.8.1] — 2026-06-19

Changed

  • Config path parameters, e.g. dataset_dir, output_dir, pretrain_weights, accept pathlib.Path objects in addition to strings. Paths are coerced to str automatically via the expand_paths validator. No API changes required; existing string usage unaffected. (#1124)
  • Keypoint training disables horizontal flip augmentation until keypoint flip-pair swapping is implemented. Flipping was previously applied without reordering keypoint pairs, producing incorrect labels. (#1122)
  • Training metric plots improved with optional seaborn error bands, AP@0.75 metric grouping, and custom AP metric group configuration. (#1122)

Fixed

  • Fixed the keypoint encoder in eval mode splitting num_queries queries across all group heads, because group_detr = len(self.enc_out_keypoint_embed); an if self.training else 1 guard now routes all queries through head 0. (#1135)
  • Fixed config.use_return_dict, deprecated in transformers, replaced with config.return_dict in the DINOv2 windowed attention backbone. (#1135)
  • Fixed epoch metric tables rendering incorrectly when a Rich progress bar callback is active. Tables print through the progress bar's owned Rich console, preventing cursor conflicts with active live displays. (#1128)
  • Fixed spurious keypoint fine-tuning checkpoint switches on noisy OKS metrics: selection is stabilised with smoothed (EMA) best-metric comparison, and smoothing state is correctly restored on training resume. (#1122)
  • Fixed Group DETR train-time metric evaluation crashing on non-tensor mask outputs from auxiliary decoder layers; it now evaluates only the primary query group. (#1122)
  • Fixed _detect_horizontal_flip in the Albumentations transform pipeline using not bboxes, which mishandles Albumentations 2.x where bboxes is a NumPy array, falsy even when non-empty; it now uses len(bboxes) == 0. (#1126)
  • Fixed a crash inside _log_hyperparams when tensorboard is installed alongside a NumPy-2.0-incompatible tensorflow; the TensorBoard logger is now disabled gracefully and training degrades to CSV-only logging with a clear warning. (#1123)

[1.8.0] — 2026-06-13

Added

  • RFDETRKeypointPreview — keypoint detection model variant with GroupPose-style head, covariance-based uncertainty (precision-Cholesky parameterization), and COCO keypoint AP evaluation. Public config classes: KeypointTrainConfig, RFDETRKeypointPreviewConfig (from rfdetr.config). Utility: precision_cholesky_to_pixel_covariance (from rfdetr.utilities). Schema helpers infer_coco_keypoint_schema, CocoKeypointSchema, active_keypoint_counts accessible via rfdetr.datasets._keypoint_schema. (#1099)
  • RFDETR.export_for_roboflow(output_dir) — writes a Roboflow upload bundle (weights.pt + class_names.txt) without a network call; extracted from deploy_to_roboflow, which now delegates to it. (#1086)
  • Keypoint fine-tuning cookbook (docs/cookbooks/fine-tune_keypoints.ipynb) — end-to-end walkthrough: dataset download, schema inference, KeypointTrainConfig, training metrics, and inference with covariance uncertainty. (#1104)
  • MetricKeypointOKS — reusable OKS metric facade over CocoEvaluator, exported from rfdetr.evaluation. Supports arbitrary keypoint counts, per-category OKS sigma values, DDP-safe evaluation with first-rank-wins deduplication, and an OKSKey enum (mAP, mAP@50, mAP@75, mAR) for standardised metric keys. (#1107)

Changed

  • DDP strategy enables find_unused_parameters=True for all detection, keypoint, and segmentation models under strategy='ddp' or strategy='auto' with a distributed launcher, previously segmentation only. Opt out via trainer_kwargs={"strategy": DDPStrategy(find_unused_parameters=False)}. (#1094)
  • rfdetr.datasets.aug_config module renamed to rfdetr.datasets.aug_configs (plural). Direct imports from rfdetr.datasets.aug_config must be updated; the augmentation preset constants (AUG_AGGRESSIVE, etc.) are unchanged. (#1103)

Removed

  • RFDETR.export(simplify=..., force=...) — both kwargs removed from the signature. Deprecated since v1.6.0 with remove_in="1.8.0"; both were no-ops during the deprecation window. Callers passing these args must remove them before upgrading. (#1102)

Fixed

  • Fixed RFDETR.from_checkpoint() treating num_classes loaded from the checkpoint as a user-supplied override, which silently refused fine-tuning on a dataset with a different class count — the head refused to re-initialise and trained against the stale class count. An explicit num_classes kwarg from the caller still wins over both the checkpoint value and the dataset. (#1106)
  • Fixed scale jitter missing from the non-square training crop: RandomCrop in the option_b branch replaced with RandomSizedCrop, restoring the scale-augmentation behaviour lost during the Albumentations migration. (#1088)
  • Fixed a multi-GPU validation deadlock in COCO mAP synchronization; _merge_metric_state_across_ranks is now safe across zero-batch ranks. (#1085)
  • Fixed import rfdetr failing on NumPy 2.x when a transitive dependency references the removed np.complex_ alias. (#1064)
  • Fixed the rfdetr_plus module availability check giving a false-positive hit when the package was partially installed. (#1083)
  • Fixed a spurious "Keypoint class-logit boost has N classes but detection head has M" warning on custom, non-Roboflow keypoint datasets: _align_num_classes_from_dataset now zero-pads num_keypoints_per_class when auto-adjusting num_classes beyond the schema length. (#1113)
  • Fixed loss scaling for keypoint training under gradient accumulation (accumulate_grad_batches > 1). Keypoint models use manual optimization to normalize losses by the accumulated box count across the effective batch; detection and segmentation remain on Lightning's automatic-optimization path. Optimizer-step scheduling, LR warmup/decay, and epoch-boundary flushing are correctly handled in both paths. (#1117)
  • Fixed device auto-detection assigning a CUDA device on a machine with CUDA headers but no GPU driver, which then failed at first use; it now verifies accelerator runtime availability first (PyTorch ≥ 2.4: torch.accelerator.current_accelerator; older builds: torch.cuda.is_available()). (#1111)
  • Fixed RFDETR.from_checkpoint() and related APIs silently treating an explicit num_classes as unset when its value equals the model default, e.g. 80 for COCO, which refused fine-tuning on a different class count. (#1109)
  • Fixed RFDETR.from_checkpoint() raising an error or silently loading the wrong model class for starter-like checkpoints without an explicit pretrain_weights entry; it now infers the model variant from the checkpoint filename when pretrain_weights is absent or unset-like — empty string, None, whitespace. (#1065)

[1.7.0] — 2026-04-29

Added

  • augmentation_backend field on TrainConfig ("cpu" / "auto" / "gpu"): opt-in GPU-side augmentation via Kornia, applied in RFDETRDataModule.on_after_batch_transfer once the batch is on the GPU. The CPU path is unchanged and remains the default. Install with pip install 'rfdetr[augment]'. (#1003)
  • Kornia GPU augmentation supports instance segmentation: images, boxes, and per-instance masks augmented in sync on the GPU, where augmentation_backend="gpu"/"auto" was previously ignored silently. New public helper collate_masks; build_kornia_pipeline gains with_masks: bool = False; unpack_boxes gains an optional masks_aug tensor. Note: the mask buffer is [B, N_max, H, W] float32, roughly 500 MB at B=8, N_max=50, H=W=560; use augmentation_backend="cpu" on cards with limited VRAM. (#1003, closes #997)
  • BuilderArgs — a @runtime_checkable typing.Protocol documenting the minimum attribute set consumed by build_model(), build_backbone(), build_transformer(), and build_criterion_and_postprocessors(). Enables static type-checker support for custom builder integrations. Exported from rfdetr.models. (#841)
  • build_model_from_config(model_config, train_config=None, defaults=MODEL_DEFAULTS) — config-native alternative to build_model(build_namespace(mc, tc)); accepts Pydantic config objects directly and constructs the internal namespace automatically. Exported from rfdetr.models. (#845)
  • build_criterion_from_config(model_config, train_config, defaults=MODEL_DEFAULTS) — config-native alternative to build_criterion_and_postprocessors(build_namespace(mc, tc)); returns a (SetCriterion, PostProcess) tuple. Exported from rfdetr.models. (#845)
  • ModelDefaults dataclass — exposes the 35 hardcoded architectural constants previously buried inside build_namespace(). Pass a dataclasses.replace(MODEL_DEFAULTS, ...) override to the new config-native builders to customise individual constants. Note: fields may be promoted to ModelConfig/TrainConfig in future phases. Exported from rfdetr.models. (#845)
  • MODEL_DEFAULTS — the canonical ModelDefaults singleton with production defaults. Exported from rfdetr.models. (#845)
  • RFDETR.predict(include_source_image=...) — opt-out flag, default True, to skip storing the source image in detections.metadata["source_image"]; set False to reduce memory use when the image is not needed for annotation. (#912)
  • model_name is stored in checkpoint files during training, so RFDETR.from_checkpoint() resolves the model class from the checkpoint without a caller-supplied hint. strip_checkpoint() preserves it; checkpoints without it still resolve via pretrain_weights filename matching. (#895)
  • rfdetr_version is stored in checkpoint files during training for provenance and compatibility hints. strip_checkpoint() preserves it; the key is omitted gracefully when the package version cannot be resolved, and checkpoints without it load normally. (#918)
  • notes parameter on RFDETR.train() and RFDETR.export() — embed arbitrary JSON-serialisable provenance metadata (labeller, date, class names, etc.) into best-model .pth checkpoints, under checkpoint["args"]["notes"], and ONNX files, under the "rfdetr_notes" metadata property. String values are stored verbatim; all other types are JSON-encoded. (#1025, closes #1021)
  • RF_HOME environment variable controls where pretrained weights are cached, default ~/.roboflow/models. Bare filenames passed as pretrain_weights, e.g. "rf-detr-base.pth", resolve relative to it; paths with a directory component are used as-is, parent directories created automatically. (#130)
  • Grayscale and multispectral imagery support: models accept any channel count, not just 3, with pretrained DINOv2 patch-embedding weights adapted to it at construction time and no extra dependencies. (#180, closes #75)
  • Training configuration is saved to training_config.json in the output directory after training, capturing the full TrainConfig, ModelConfig, effective training parameters, class names, and class count. (#194)
  • dinov2_registers_windowed_small backbone is available as a config option in ModelConfig.encoder. (#236)
  • rfdetr.from_checkpoint(path) — new top-level convenience function that loads a checkpoint and infers the correct model subclass automatically, without the caller specifying a class. Equivalent to RFDETR.from_checkpoint(path) but importable directly from the rfdetr package. (#664)
  • ONNX export filenames include the model variant name, e.g. rfdetr-medium.onnx, instead of the generic inference_model.onnx. Exporting multiple variants to the same directory no longer overwrites previous exports. (#910)
  • Background images, those without a matching label file, are included in YOLO detection datasets as empty-detection samples instead of being dropped; detection and segmentation both use _LazyYoloDetectionDataset. (#915)
  • TFLite export via model.export(format="tflite"). Converts through ONNX using onnx2tf; FP32 and FP16 outputs are always produced, INT8 quantization is available with a calibration image directory: model.export(format="tflite", quantization="int8", calibration_data="path/to/images/"). Requires pip install 'rfdetr[onnx,tflite]'. (#920)
  • PyTorch Lightning .ckpt files are accepted as pretrain_weights; keys are normalized from PTL format automatically (state_dict with model.-prefixed keys, hyper_parameters → args), so weight loading, class-name extraction, and compatibility checks need no manual conversion. (#951)
  • skip_best_epochs parameter for RFDETR.train() and TrainConfig: the first N epochs are excluded from best-checkpoint selection and early-stopping comparison, preventing strong pretrained weights or resumed checkpoints from locking in a suboptimal early score. (#1000, closes #789)
  • TFLite inference decodes segmentation mask outputs into sv.Detections.mask, upsampled to source size with Pillow bilinear resampling and thresholded at zero, matching PostProcess.forward. The mask tensor is detected by output name, "masks" substring, with a rank-4 shape fallback. (#1053)
  • PretrainWeightsCompatibilityWarning — new warning class emitted when a ModelConfig override, e.g. custom encoder, num_queries, or num_feature_levels, risks breaking pretrained weight loading. Importable as from rfdetr.config import PretrainWeightsCompatibilityWarning for targeted filtering. (#1017)

Changed

  • peft is no longer installed as part of the default rfdetr package; it moved to the [lora] and [train] optional extras. For LoRA fine-tuning, install with pip install 'rfdetr[lora]'. (#838)
  • Native RLE annotation support in the COCO segmentation pipeline: convert_coco_poly_to_mask explicitly detects and decodes both compressed (string counts) and uncompressed (int-list counts) RLE formats alongside existing polygon support. Malformed annotations now raise instead of being silently swallowed. (#897)
  • Pinned PyTorch Lightning to exclude known-compromised versions. (#1020)

Deprecated

  • build_namespace(model_config, train_config) — no longer used internally and deprecated in this release; use build_model_from_config, build_criterion_from_config, or _namespace_from_configs directly. Removal in v1.9; emits a DeprecationWarning on use. (#845)
  • load_pretrain_weights(nn_model, model_config, train_config) — the train_config positional argument is deprecated, removal in v1.9, and is no longer used internally. Omit it: load_pretrain_weights(nn_model, model_config). Passing a non-None value emits a DeprecationWarning. (#845)
  • TrainConfig.group_detr, TrainConfig.ia_bce_loss, TrainConfig.segmentation_head, TrainConfig.num_select → ModelConfig; ModelConfig.cls_loss_coef → TrainConfig. Each emits DeprecationWarning when set on the wrong config object and will be removed in v1.9. SegmentationTrainConfig users: remove the num_select override, the model config value is always used. (#841)
  • RFDETRBase — use RFDETRNano, RFDETRSmall, RFDETRMedium, or RFDETRLarge instead. Emits FutureWarning on instantiation; scheduled for removal in v2.0. (#900)
  • RFDETRSegPreview — use RFDETRSegNano, RFDETRSegSmall, RFDETRSegMedium, or RFDETRSegLarge instead. Emits FutureWarning on instantiation; scheduled for removal in v2.0. (#900)
  • rfdetr.util and rfdetr.deploy sub-modules are deprecated, removal in v1.9. A __getattr__ hook on the rfdetr package emits a clear ImportError with migration guidance when these legacy paths are accessed. (#839)

Fixed

  • TFLite GridSample lowering bug fixed in two steps:
    • Fixed TFLite export (format="tflite") producing detection scores that collapse to ~0.02, vs ~0.62 from ONNX; cause was an onnx2tf GridSample lowering bug (PINTO0309/onnx2tf#274) compounding through RF-DETR's per-decoder-layer F.grid_sample. The converter now passes onnx2tf's pseudo-GridSample replacement kwarg, logging a warning when it is absent. (#1041)
    • Fixed TFLite detection scores collapsing for all queries when GridSample was used as an onnx2tf pseudo-operator; the node is rewritten to Gather-based integer-index arithmetic before conversion. Supersedes the runtime-kwarg approach above. (#1054)
  • Fixed WindowedDinov2WithRegistersEmbeddings.forward() failing silently under -O when input spatial dimensions are not divisible by patch_size * num_windows; it now raises ValueError with a clear message identifying the divisor and actual shape. (#167)
  • Fixed _namespace.py: num_select in the builder namespace always reads from ModelConfig, where TrainConfig.num_select (default 300) silently overrode model-specific values of 100–200 for segmentation variants. (#841)
  • Fixed models/weights.py: load_pretrain_weights auto-aligns the model head when the checkpoint has fewer classes than the configured default, preventing a silent mismatch when the caller did not set num_classes. (#845)
  • Fixed models/weights.py: load_pretrain_weights slices refpoint_embed.weight and query_feat.weight per-group when reshaping checkpoint queries; the previous flat slice scrambled groups 1+ when num_queries decreased with group_detr > 1, corrupting training-resume. Inference, which reads group 0 only, was unaffected. (#1019)
  • Fixed YOLO segmentation training on large datasets hitting OS out-of-memory, caused by supervision.DetectionDataset.from_yolo(force_masks=True) eager-rasterising every image's masks at construction time. A new _LazyYoloDetectionDataset stores polygons and defers rasterisation to __getitem__, keeping RAM proportional to annotation count. (#851)
  • Fixed ONNX/TRT dynamic batch inference: the tracer baked the training batch size as a compile-time constant, so TRT engines built with smaller --minShapes failed with Reshape: reshaping failed. Six call sites in gen_encoder_output_proposals and Transformer.forward now use ONNX-symbolic equivalents, keeping the batch dimension dynamic. (#950, closes #949)
  • Fixed training failure when square_resize_div_64=False: the non-square resize pipeline did not guarantee dimensions divisible by patch_size * num_windows, raising ValueError. A PadIfNeeded step is appended after the resize pair in the train and val/test pipelines. (#991, closes #983)
  • Fixed non-square batch padding: block_size rounding is applied in the DataLoader collator as well as the transform-level PadIfNeeded, so divisibility by patch_size * num_windows survives Compose reordering and applies to custom evaluation harnesses. (#992)
  • Fixed RFDETRModelModule.on_load_checkpoint crashing with RuntimeError when resuming from a checkpoint saved at a different image resolution; DINOv2 positional embeddings are bicubic-interpolated to model_config.positional_encoding_size first. (#1002, closes #998)
  • Fixed RFDETRLarge initialization showing two conflicting ValueErrors, for patch_size=14 and patch_size=16, when the deprecated-config fallback retry also fails; the fallback re-raises the original error without chained context. (#975)
  • Fixed RFDETRModelModule.__init__ crashing with RuntimeError: size mismatch for backbone.0.encoder.encoder.embeddings.position_embeddings when training segmentation models at a custom resolution, e.g. RFDETRSegLarge(resolution=1008); the training entry path delegates to load_pretrain_weights, which interpolates the positional embeddings. (#1040, closes #1038, #1023)
  • Fixed class_name lookup for pretrained COCO models: sparse COCO category IDs, 1–90 for 80 classes, made flat 0-based indexing return the wrong name. Detection uses a coco_id → class_name mapping built from COCO_CLASSES; fine-tuned models keep direct 0-based indexing. (#1051)

[1.6.5] — 2026-04-22

Breaking Changes

  • predict() stores the source image in detections.metadata["source_image"], not detections.data["source_image"], which supervision indexed per-detection and raised IndexError on. Update any code that reads detections.data["source_image"]. (#972, #968)

Fixed

  • Fixed segmentation training crash on T4 and P100 GPUs, caused by cuDNN engine selection for depthwise convolution backward on some CUDA stacks. A custom autograd.Function disables cuDNN in forward and backward. (#967)
  • Fixed ema_segm_mAP_50_95 and ema_segm_mAP_50 being computed from the base, non-EMA, metric accumulator instead of the EMA accumulator, producing misleading validation scores for segmentation models. (#980)
  • Fixed BestModelCallback losing the best EMA score on training resume, because _best_ema was not persisted in state_dict(). (#973)
  • Fixed positional_encoding_size not updating when resolution is set at construction time, e.g. RFDETRLarge(resolution=640), causing shape mismatches during forward. A model validator now auto-syncs PE size. (#956)
  • Fixed a pretrained weight loading crash with custom resolution: DINOv2 positional embeddings are bicubic-interpolated to match the target grid before load_state_dict. (#964)
  • Fixed validate_checkpoint_compatibility producing a cryptic RuntimeError on patch_size mismatch when the checkpoint lacks explicit args.patch_size; it now infers patch_size from the DINOv2 projection weight shape and raises a descriptive ValueError. (#971)
  • Fixed predict() storing detections.data["source_shape"] as a Python tuple, which raised TypeError whenever sv.Detections was iterated. The value is now an np.ndarray of shape (N, 2) and dtype int64. (#966, #963)
  • Fixed predict() emitting a misleading "class_id out of range" warning for the background/no-object class, class index num_classes. Background-class detections map data["class_name"] to "__background__" without any warning. (#970)

[1.6.4] — 2026-04-10

Changed

  • predict() includes class_name in detections.data, mapping each detection's 0-indexed class ID to its human-readable name. (#914)

Fixed

  • Fixed segmentation multi-GPU DDP training crashing with RuntimeError: It looks like your LightningModule has parameters that were not used in producing the loss, because the segmentation head's sparse_forward() leaves parameters unused on some steps: build_trainer() wraps strategy="ddp" with DDPStrategy(find_unused_parameters=True) when segmentation_head=True. Non-segmentation DDP and other strategies are unchanged. (#942, #947)
  • Fixed fused AdamW crashing under FP32 multi-GPU training with RuntimeError: params, grads, exp_avgs, and exp_avg_sqs must have same dtype, device, and layout: configure_optimizers() and clip_gradients() gate fused AdamW on the trainer's actual precision, not GPU capability, which reports BF16 support on Ampere+ even at precision="32-true". (#942, #947)
  • Fixed multi-GPU DDP training crashing in Jupyter notebooks and Kaggle: the fork-based ddp_notebook strategy is replaced with a spawn-based one, avoiding OpenMP thread pool corruption after fork(). (#928)
  • Fixed RFDETR.train(resolution=...) being silently ignored; the kwarg is applied to model_config before training begins, with validation that the value is divisible by patch_size * num_windows. (#933)
  • Fixed save_dataset_grids being silently a no-op; DatasetGridSaver is wired into the training loop, saving sample grids to {output_dir}/dataset_grids/ when enabled. Grid save failures are caught without interrupting training. (#946)
  • Fixed partial gradient-accumulation windows at the tail of training epochs: the training dataset is padded to an exact multiple of effective_batch_size * world_size, so every optimizer step uses a full gradient window. Workaround for pytorch-lightning#19987. (#937)
  • Fixed torch.export.export failing on the transformer decoder, by threading spatial_shapes_hw through all decoder layers. (#936)
  • Fixed download_pretrain_weights() overwriting fine-tuned checkpoints that share a filename with a registry model, e.g. rf-detr-nano.pth, where an MD5 mismatch silently restored the original COCO checkpoint. It now returns early whenever the file exists and redownload=False, warning when the hash differs; pass redownload=True to force a fresh download. (#935)

[1.6.3] — 2026-04-02

Changed

  • predict() stores the original image and its shape on returned sv.Detections objects — detections.data["source_image"] (NumPy array) and detections.data["source_shape"] (NumPy array of shape (N, 2), each row [height, width]) let you annotate results without loading the image separately. (#892)
  • RFDETR.train() auto-detects num_classes from the dataset directory when not explicitly set, reinitializing the detection head to the correct class count automatically. A warning is emitted when the configured value differs from the dataset count. (#893)
  • optimize_for_inference() accepts dtype as a string name, e.g. "float16", in addition to a torch.dtype object; invalid dtype inputs uniformly raise TypeError. (#899)

Fixed

  • Fixed models/lwdetr.py: reinitialize_detection_head replaces nn.Linear modules instead of mutating .data in place, keeping out_features consistent with the weight shape, so ONNX export and torch.jit.trace no longer emit stale class counts for fine-tuned models. (#904)
  • Fixed RFDETR.optimize_for_inference() leaking a CUDA context on multi-GPU setups: the deep-copy, export, and JIT-trace steps run inside torch.cuda.device(device) to pin the context to the correct device. (#899)
  • Fixed optimize_for_inference() leaving inconsistent state on failure: prior optimized state is reset and flags are committed only after a successful build/trace; temp download files use unique per-process paths to avoid parallel worker collisions.
  • Fixed deploy_to_roboflow failing with FileNotFoundError after the PyTorch Lightning migration: class_names.txt is written to the upload directory and args.class_names is populated before saving the checkpoint. (#890)

[1.6.2] — 2026-03-27

Added

  • RFDETR.predict(shape=...) — optional (height, width) tuple overrides the default square inference resolution; useful when matching a non-square ONNX export. Both dimensions must be positive integers divisible by patch_size × num_windows as determined by the model configuration. (#866)

Changed

  • ModelConfig.device and RFDETR.train(device=...) accept torch.device objects and indexed device strings such as "cuda:0". Values are normalized to canonical torch-style strings. RFDETR.train() warns when an unmapped device type is passed to PyTorch Lightning auto-detection. (#872)

Fixed

  • Fixed ONNX export ignoring an explicit patch_size argument: export() and predict() resolve patch_size from model_config by default, validate it strictly (positive integer, not bool), and enforce that (H, W) dimensions are divisible by patch_size × num_windows. (#876)
  • Fixed ONNX export for models with dynamic batch dimensions: H_.expand(N_) replaced with torch.full for Python-int spatial dims, eliminating tracer failures. (#871)

[1.6.1] — 2026-03-25

Deprecated

  • RFDETR.export(..., simplify=..., force=...) — both arguments are now no-ops and emit a DeprecationWarning. RF-DETR no longer runs ONNX simplification automatically; remove these arguments from your calls. Removal in v1.8. (#861)

Fixed

  • Fixed RFDETR.train() raising a bare ModuleNotFoundError on a missing rfdetr[train] install; it now raises an ImportError naming the fix, pip install "rfdetr[train,loggers]". (#858)
  • Fixed AUG_AGGRESSIVE preset: translate_percent (0.1, 0.1) was a degenerate range forcing Affine to always translate right/down by exactly 10%, corrected to (-0.1, 0.1). (#863)
  • Fixed the PTL training path: latest.ckpt and per-interval checkpoints (checkpoint_interval_N.ckpt) are written and restored on resume. (#847)
  • Fixed BestModelCallback and checkpoint monitor raising MisconfigurationException on non-eval epochs when eval_interval > 1; monitor key absence is handled gracefully. (#848)
  • Fixed the protobuf version constraint in the loggers extra, guarding against the TensorBoard descriptor crash (TypeError: Descriptors cannot be created directly) with protobuf ≥ 4. (#846)
  • Fixed duplicate ModelCheckpoint state keys when checkpoint_interval=1; last.ckpt is omitted in that configuration to avoid collision. (#859)

[1.6.0] — 2026-03-20

Added

  • PyTorch Lightning training building blocks: RFDETRModelModule, RFDETRDataModule, build_trainer(), and callbacks (RFDETREMACallback, COCOEvalCallback, BestModelCallback, DropPathCallback, MetricsPlotCallback) — standard PTL components, swap/subclass/extend any piece. Level 3: rfdetr fit --config CLI, zero Python required. (#757, #794)
  • Multi-GPU DDP via model.train(): strategy, devices, and num_nodes added to TrainConfig; single-GPU behaviour unchanged when omitted. (#808)
  • batch_size='auto': CUDA memory probe finds the largest safe micro-batch size, then recommends grad_accum_steps to reach a configurable effective batch target, default 16 via auto_batch_target_effective. (#814)
  • ModelContext promoted from _ModelContext to a public, exported API — inspect class_names, num_classes, and related metadata via model.context after training. (#835)
  • backbone_lora and freeze_encoder added as first-class fields in ModelConfig. (#829)
  • generate_coco_dataset(with_segmentation=True) produces COCO polygon annotations alongside bounding boxes for segmentation fine-tuning with synthetic data. (#781)
  • set_attn_implementation("eager" | "sdpa") on the DINOv2 backbone — switch attention implementation at runtime. (#760)
  • eval_max_dets, eval_interval, and log_per_class_metrics added to TrainConfig.
  • python -m rfdetr entry point alongside the rfdetr console script.
  • py.typed marker — RF-DETR is now PEP 561–compliant.

Changed

  • Breaking: Minimum transformers version bumped to >=5.1.0,<6.0.0. The DINOv2 windowed-attention backbone uses the transformers v5 API (BackboneMixin._init_transformers_backbone(), removed head_mask plumbing). Projects still on transformers v4 must pin rfdetr<1.6.0. (#760)
  • Breaking: PyPI install extras renamed — rfdetr[metrics] → rfdetr[loggers], rfdetr[onnxexport] → rfdetr[onnx].
  • draw_synthetic_shape returns Tuple[np.ndarray, List[float]], not np.ndarray. The second element is a flat COCO-style polygon list [x1, y1, x2, y2, …]. Any caller that did img = draw_synthetic_shape(...) must be updated to img, polygon = draw_synthetic_shape(...). (#781)
  • Albumentations version constraint broadened to >=1.4.24,<3.0.0; RandomSizedCrop configs using height/width kwargs are adapted automatically to the 2.x size=(height, width) API. (#786)
  • Current learning rate is shown in the training progress bar alongside loss. (#809)
  • supervision, pytorch_lightning, and other heavy dependencies are imported lazily, on first use, rather than at module load, reducing cold-import time in inference-only environments. (#801)

Deprecated

  • rfdetr.deploy.* — redirects to rfdetr.export.* with a DeprecationWarning. Migrate before v1.7.
  • rfdetr.util.* — redirects to rfdetr.utilities.* with a DeprecationWarning. Migrate before v1.7.

Fixed

  • Fixed a cryptic RuntimeError / tensor-size mismatch when a checkpoint is incompatible with the current model architecture; a descriptive ValueError is raised instead, covering segmentation_head mismatch and patch_size mismatch. (#810)
  • Fixed class_names not reflecting dataset labels on model.predict() after training; class names are synced from the dataset so inference always uses the correct label list. (#816)
  • Fixed detection head reinitialization overwriting fine-tuned weights when loading a checkpoint with fewer classes than the model default. The second reinitialize_detection_head call fires only in the backbone-pretrain scenario. (#815, #509)
  • Fixed grid_sample and bicubic interpolation silently falling back to CPU on MPS (Apple Silicon); both run natively on the MPS device. (#821)
  • Fixed early_stopping=False in TrainConfig being silently ignored; the setting propagates correctly. (#835)
  • Fixed an AttributeError crash in update_drop_path when the DINOv2 backbone layer structure does not match any known pattern.
  • Added warning when drop_path_rate > 0.0 is configured with a non-windowed DINOv2 backbone, where drop-path is silently ignored.
  • Fixed ValueError: matrix entries are not finite in HungarianMatcher when the cost matrix contains NaN or Inf; non-finite entries are replaced with a finite sentinel before linear_sum_assignment, warning emitted at most once per matcher instance. (#787)
  • Fixed YOLO dataset validation rejecting data.yml; both .yaml and .yml are accepted. (#777)
  • Silently dropped degenerate bounding boxes, zero width or height, before Albumentations validation instead of raising ValueError. (#825)

[1.5.2] — 2026-03-04

Added

  • Added peak GPU memory (max_mem in MB) to training and evaluation progress bars on CUDA; omitted on CPU and MPS. (#773)

Fixed

  • Fixed aug_config being silently ignored when training on YOLO-format datasets; build_roboflow_from_yolo never forwarded the value, so transforms always fell back to the default. (#774)
  • Fixed segmentation evaluation metrics not being written to results_mask.json during validation and test runs. (#772)
  • Fixed an AttributeError crash in update_drop_path when the DINOv2 backbone layer structure does not match any known pattern; _get_backbone_encoder_layers returns None for unrecognised architectures. (#762)
  • Fixed drop_path_rate not being forwarded to the DINOv2 model configuration, so stochastic depth was never applied even when explicitly set. Added a warning when drop_path_rate > 0.0 is used with a non-windowed backbone. (#762)
  • Fixed incorrect COCO hierarchy filtering that excluded parent categories from the class list. (#759)
  • Fixed evaluation metric corruption on 1-indexed Roboflow datasets, caused by a flawed contiguity check in _should_use_raw_category_ids. (#755)

[1.5.1] — 2026-02-27

Added

  • Added support for nested Albumentations containers (OneOf, Sequential) inside aug_config. (#752)

Changed

  • Migrated dataset transform pipeline to torchvision-native Compose, ToImage, and ToDtype; Normalize defaults to ImageNet mean/std. (#745)

Fixed

  • Fixed RFDETRMedium missing from the public API; __all__ contained a duplicate RFDETRSmall entry. (#748)
  • Fixed AR50_90 reporting an incorrect value in MetricsMLFlowSink, due to a wrong COCO evaluation index. (#735)
  • Fixed supercategory filtering in _load_classes for COCO datasets with flat or mixed supercategory structures. (#744)
  • Fixed a crash in geometric transforms when a sample contained zero-area or empty masks. (#727)
  • Fixed segmentation training on Colab; DepthwiseConvBlock disables cuDNN for depthwise separable convolutions. (#728)
  • Pinned onnxsim<0.6.0 to prevent pip install from hanging indefinitely. (#749)

[1.5.0] — 2026-02-23

Added

  • Added custom training augmentations via aug_config in model.train() — accepts a dict of Albumentations transforms, a built-in preset (AUG_CONSERVATIVE, AUG_AGGRESSIVE, AUG_AERIAL, AUG_INDUSTRIAL), or {} to disable. Bounding boxes and segmentation masks are transformed automatically. (#263, #702)
  • Added save_dataset_grids=True in TrainConfig to write 3×3 JPEG grids of augmented samples to output_dir before training begins. (#153)
  • Added ClearML logger: set clearml=True in TrainConfig to stream per-epoch metrics to ClearML. (#520)
  • Added MLflow logger: set mlflow=True in TrainConfig to log runs and metrics to MLflow with custom tracking URI support. (#109)
  • Added live progress bar for training and validation with structured per-epoch logs. (#204)
  • Added device field to TrainConfig for explicit device selection. (#687)
  • ModelConfig raises an error on unknown parameters, preventing silent misconfiguration. (#196)

Changed

  • Deprecated OPEN_SOURCE_MODELS constant in favour of ModelWeights enum. (#696)
  • Added MD5 checksum validation for pretrained weight downloads. (#679)

Fixed

  • Fixed Albumentations bool-mask crash during segmentation training. (#706)
  • Fixed UnboundLocalError when resuming training from a completed checkpoint. (#707)
  • Prevented corruption of checkpoint_best_total.pth via atomic checkpoint stripping. (#708)
  • Fixed PyTorch 2.9+ compatibility issue with CUDA capability detection. (#686)
  • Fixed dtype mismatch error when use_position_supervised_loss=True. (#447)
  • Fixed inconsistent return values from build_model. (#519)
  • Fixed positional_encoding_size type annotation (bool → int). (#524)
  • Fixed ONNX export output_names to include masks when exporting segmentation models. (#402)
  • Fixed num_select not being updated correctly during segmentation model fine-tuning. (#399)
  • Fixed np.argwhere → np.argmax misuse. (#536)
  • Fixed COCO sparse category ID remapping for non-contiguous or offset category IDs. (#712)
  • Fixed segmentation mask filtering when using aggressive augmentations. (#717)

[1.4.3] — 2026-02-16

Changed

  • Pretrained weight downloads validate against an MD5 checksum to detect corrupted files. (#679)

Fixed

  • Fixed deploy_to_roboflow failing for segmentation model exports. (#578)
  • Fixed missing info key in COCO export format. (#681)

[1.4.2] — 2026-02-12

Added

  • Added generate_coco_dataset() utility for generating synthetic COCO-format datasets with configurable class counts, split ratios, and bounding box annotations. (#617)
  • Added run_test=False to TrainConfig — skip test-split evaluation when your dataset has no test set. (#628)

Changed

  • model.predict() accepts image URLs directly, with no need to download images before inference. (#629)
  • Plus models (RFDETRXLarge, RFDETR2XLarge) are distributed as a separate rfdetr_plus package under the Roboflow Model License. (#645)

Fixed

  • Fixed segmentation ONNX export failure. (#626)

[1.4.1] — 2026-01-30

Added

  • Added native YOLO dataset format support alongside COCO. (#74)
  • Added --print-freq CLI argument to control training log frequency. (#603)

Changed

  • Pinned transformers to <5.0.0 to prevent incompatibility with the transformers v5 API. (#599)

Fixed

  • Fixed class count mismatch in train_from_config for Roboflow-uploaded datasets. (#588)
  • Improved num_classes mismatch warning messages to be actionable rather than misleading. (#261)
  • Fixed CLI crash when specifying the device argument. (#246)

[1.4.0] — 2026-01-22

Headline release introducing new pre-trained model sizes — L, XL, and 2XL for object detection, and the full N/S/M/L/XL/2XL range for instance segmentation. Also added YOLO format training support, simplified the dependency footprint by removing several heavy packages (cython, fairscale, timm, einops, and others), and fixed per-class precision/recall/F1 computation. Drops Python 3.9 support.