RF-DETR → ExecuTorch Export & Inference¶

Export an RF-DETR detector to a portable ExecuTorch program (.pte) for on-device deployment on mobile and edge hardware, then run it on the CPU and visualize the detections. Unlike ONNX or TFLite, the model is captured directly via torch.export (no intermediate conversion) and lowered to a hardware backend:

Backend	Target	Precision
xnnpack (this notebook)	portable CPU (Android / iOS / Linux / macOS)	fp32
coreml	Apple Neural Engine	fp16
qnn	Qualcomm Snapdragon HTP	fp16

!!! warning "The input tensor must be contiguous"

The ExecuTorch runtime reads the input buffer as contiguous NCHW and ignores tensor strides. Any
preprocessing that permutes axes — `np.transpose`, `Tensor.permute`, torchvision's `ToImage` —
returns a strided view rather than a copy, and the runtime misreads such a view as a scrambled
image. Nothing errors: the model runs without error and returns plausible-shaped output, but every
detection's score collapses below threshold. This cookbook is safe because `infer_transforms`
already materializes a contiguous tensor (its final `_ensure_contiguous` step); Step 6 also calls
`.contiguous()` defensively so the requirement stays visible for anyone adapting the preprocessing.

1. Install¶

The [executorch] extra provides the exporter and the XNNPACK backend.

Colab ships mutually inconsistent preinstalled packages that otherwise crash import rfdetr, so two are aligned: torchaudio is uninstalled (RF-DETR never uses it, but transformers imports it when present and a torch/torchaudio CUDA-version mismatch then errors), and pillow is force-reinstalled to a clean version (a half-upgraded PIL breaks torchvision's import with cannot import name '_Ink').

flatc — ExecuTorch serializes the .pte with the FlatBuffers compiler. It ships in the Linux executorch wheel (so Colab works out of the box); on macOS install it separately with brew install flatbuffers.

Colab: after this cell, Runtime → Restart session, then run from the next cell — Colab keeps the old package versions loaded until a restart.

In [ ]:

!pip install -q "rfdetr[executorch]>=1.9.0" "torch<2.13" supervision
!pip install -q --force-reinstall --no-deps "pillow==11.3.0"
!pip uninstall -q -y torchaudio

!pip install -q "rfdetr[executorch]>=1.9.0" "torch<2.13" supervision !pip install -q --force-reinstall --no-deps "pillow==11.3.0" !pip uninstall -q -y torchaudio

2. Setup¶

ExecuTorch export is CPU-only — no GPU is needed.

In [ ]:

from pathlib import Path

from rfdetr import RFDETRSmall

EXPORT_DIR = Path("export_executorch")
EXPORT_DIR.mkdir(exist_ok=True)

from pathlib import Path from rfdetr import RFDETRSmall EXPORT_DIR = Path("export_executorch") EXPORT_DIR.mkdir(exist_ok=True)

3. Export to a .pte program¶

format="executorch" requires an explicit backend. The COCO-pretrained RFDETRSmall is exported directly — pass pretrain_weights="<path/to/checkpoint.pth>" to export a fine-tuned model instead.

The file is named after the model variant (rfdetr-small.pte). torch.export bakes a fixed input shape into the graph (square at the model's resolution), so dynamic_batch is not supported — export one .pte per batch size.

In [ ]:

model = RFDETRSmall()

pte_path = model.export(format="executorch", backend="xnnpack", output_dir=str(EXPORT_DIR))
print(f"ExecuTorch program: {pte_path}  ({pte_path.stat().st_size / 1e6:.1f} MB)")

model = RFDETRSmall() pte_path = model.export(format="executorch", backend="xnnpack", output_dir=str(EXPORT_DIR)) print(f"ExecuTorch program: {pte_path} ({pte_path.stat().st_size / 1e6:.1f} MB)")

4. Other backends¶

Swap the backend argument to target a different on-device runtime.

Apple (CoreML, fp16)¶

Install coremltools, then rerun the export cell with:

model.export(format="executorch", backend="coreml", output_dir="export_executorch")

Qualcomm Snapdragon (QNN, fp16)¶

Build ExecuTorch from source against the QAIRT SDK (the QNN backend is not pip-installable), then rerun the export cell with:

model.export(format="executorch", backend="qnn", soc="SM8650", output_dir="export_executorch")

CoreML runs fp16 on the Apple Neural Engine; QNN targets the Snapdragon HTP and bakes in the target SoC.

5. Sample image¶

A single street scene with several COCO classes (dog, person, backpack, car) is enough to verify detections. The image is downloaded once and reused.

In [ ]:

import urllib.request

from PIL import Image

IMAGE_URL = "https://media.roboflow.com/notebooks/examples/dog.jpeg"
IMAGE_PATH = EXPORT_DIR / "sample.jpg"
if not IMAGE_PATH.exists():
    urllib.request.urlretrieve(IMAGE_URL, IMAGE_PATH)

image = Image.open(IMAGE_PATH).convert("RGB")
print(f"Sample image: {image.size[0]}×{image.size[1]}")

import urllib.request from PIL import Image IMAGE_URL = "https://media.roboflow.com/notebooks/examples/dog.jpeg" IMAGE_PATH = EXPORT_DIR / "sample.jpg" if not IMAGE_PATH.exists(): urllib.request.urlretrieve(IMAGE_URL, IMAGE_PATH) image = Image.open(IMAGE_PATH).convert("RGB") print(f"Sample image: {image.size[0]}×{image.size[1]}")

6. Run the .pte on the CPU¶

Runtime.load_program(...).load_method("forward") gives a callable graph whose two outputs are dets (boxes, cxcywh, normalized) and labels (class logits). The .pte expects the same input the model was trained on: NCHW, ImageNet-normalized, at the traced resolution — infer_transforms builds exactly that pipeline.

The trailing .contiguous() is defensive. infer_transforms already returns a contiguous tensor (its final _ensure_contiguous step), so the call is a no-op here — it documents the runtime's requirement explicitly. If you swap in your own preprocessing that permutes axes without copying, the ExecuTorch runtime would ignore the strides, misread the buffer as a scrambled image, and every detection would collapse below threshold.

In [ ]:

import torch
from executorch.runtime import Runtime

from rfdetr.export.benchmark import infer_transforms, post_process

CONFIDENCE_THRESHOLD = 0.5

resolution = model.model_config.resolution
image_tensor, _ = infer_transforms((resolution, resolution))(image, None)
pixel_values = image_tensor[None].float().contiguous()

method = Runtime.get().load_program(str(pte_path)).load_method("forward")
dets, labels = method.execute([pixel_values])

target_sizes = torch.tensor([[image.height, image.width]])
result = post_process({"dets": dets, "labels": labels}, target_sizes)[0]
print(f"Top score: {result['scores'][0]:.3f}")

import torch from executorch.runtime import Runtime from rfdetr.export.benchmark import infer_transforms, post_process CONFIDENCE_THRESHOLD = 0.5 resolution = model.model_config.resolution image_tensor, _ = infer_transforms((resolution, resolution))(image, None) pixel_values = image_tensor[None].float().contiguous() method = Runtime.get().load_program(str(pte_path)).load_method("forward") dets, labels = method.execute([pixel_values]) target_sizes = torch.tensor([[image.height, image.width]]) result = post_process({"dets": dets, "labels": labels}, target_sizes)[0] print(f"Top score: {result['scores'][0]:.3f}")

7. Visualize with supervision¶

post_process returns boxes in absolute xyxy pixel coordinates along with scores and COCO class ids. Filter by confidence, wrap the result in a supervision.Detections, and annotate the original image.

Annotate the PIL image rather than np.array(image). supervision's annotators accept either, but a numpy scene is assumed to be OpenCV-style BGR — handing them an RGB array swaps the red and blue channels of the annotations, and sv.plot_image then swaps the photo itself. Passing PIL lets supervision convert in both directions and keeps the colours right.

In [ ]:

import supervision as sv

from rfdetr.assets.coco_classes import COCO_CLASSES

keep = result["scores"] > CONFIDENCE_THRESHOLD
detections = sv.Detections(
    xyxy=result["boxes"][keep].numpy(),
    confidence=result["scores"][keep].numpy(),
    class_id=result["labels"][keep].numpy().astype(int),
)
print(f"Kept {len(detections)} detections above {CONFIDENCE_THRESHOLD}")

names = [COCO_CLASSES.get(int(c), str(c)) for c in detections.class_id]
annotation_labels = [f"{name} {conf:.2f}" for name, conf in zip(names, detections.confidence)]
print(f"Detections: {annotation_labels}")

annotated = sv.BoxAnnotator(thickness=3).annotate(scene=image.copy(), detections=detections)
annotated = sv.LabelAnnotator(text_scale=0.6, text_thickness=1, text_padding=4).annotate(
    scene=annotated, detections=detections, labels=annotation_labels
)

OUTPUT_PATH = EXPORT_DIR / "annotated_executorch.jpg"
annotated.save(OUTPUT_PATH)
print(f"Saved annotated image: {OUTPUT_PATH}")
sv.plot_image(annotated)

import supervision as sv from rfdetr.assets.coco_classes import COCO_CLASSES keep = result["scores"] > CONFIDENCE_THRESHOLD detections = sv.Detections( xyxy=result["boxes"][keep].numpy(), confidence=result["scores"][keep].numpy(), class_id=result["labels"][keep].numpy().astype(int), ) print(f"Kept {len(detections)} detections above {CONFIDENCE_THRESHOLD}") names = [COCO_CLASSES.get(int(c), str(c)) for c in detections.class_id] annotation_labels = [f"{name} {conf:.2f}" for name, conf in zip(names, detections.confidence)] print(f"Detections: {annotation_labels}") annotated = sv.BoxAnnotator(thickness=3).annotate(scene=image.copy(), detections=detections) annotated = sv.LabelAnnotator(text_scale=0.6, text_thickness=1, text_padding=4).annotate( scene=annotated, detections=detections, labels=annotation_labels ) OUTPUT_PATH = EXPORT_DIR / "annotated_executorch.jpg" annotated.save(OUTPUT_PATH) print(f"Saved annotated image: {OUTPUT_PATH}") sv.plot_image(annotated)

Next steps¶

• Deploy on-device — copy the .pte to your Android / iOS / edge app and run it with the ExecuTorch runtime for that platform. See the ExecuTorch docs.
• Fine-tuned weights — pass pretrain_weights="<path/to/checkpoint.pth>" when constructing the model.
• Other runtimes — see the TensorRT cookbook (NVIDIA GPU) or inference-models (PyTorch / ONNX / TensorRT).
• See the Export documentation for all formats and options.