RF-DETR → Native CoreML Export & Inference¶
Export an RF-DETR detector directly to a native CoreML .mlpackage you can drag straight into Xcode —
no ONNX intermediary and no ExecuTorch runtime involved.
Not the same as the ExecuTorch CoreML backend.
format="coreml"(this notebook) exports viatorch.export+coremltoolsstraight to a native.mlpackage.format="executorch", backend="coreml"is a different path — it produces a.ptefile for the ExecuTorch runtime instead. See the ExecuTorch cookbook for that path.
macOS only. Running (not just building) a CoreML model requires the Core ML runtime, which only exists on macOS / iOS. This notebook must run on a local macOS machine — it will not work on Colab or any Linux runner.
1. Install¶
The [coreml] extra provides coremltools, the exporter's only dependency beyond torch.
!pip install -q "rfdetr[coreml]>=1.9.0" supervision
2. Setup¶
CoreML export itself runs on CPU — no GPU is needed.
from pathlib import Path
from rfdetr import RFDETRSmall
EXPORT_DIR = Path("export_coreml")
EXPORT_DIR.mkdir(exist_ok=True)
3. Export to a .mlpackage¶
The COCO-pretrained RFDETRSmall is exported directly — pass pretrain_weights="<path/to/checkpoint.pth>"
to export a fine-tuned model instead.
The file is named after the model variant (rfdetr-small.mlpackage). dynamic_batch=True is not
supported — fixed shapes are required for reliable ANE / GPU scheduling, so export one .mlpackage per
batch size.
Export leaves coreml_precision at its default None, which selects FP32 for tight CPU parity with eager PyTorch. Pass
coreml_precision="float16" instead for a smaller, ANE-oriented bundle (expect larger numeric drift).
model = RFDETRSmall()
mlpackage_path = model.export(format="coreml", output_dir=str(EXPORT_DIR))
print(f"CoreML package: {mlpackage_path}")
4. Sample image¶
A single street scene with several COCO classes (dog, person, backpack, car) is enough to verify detections. The image is downloaded once and reused.
import urllib.request
from PIL import Image
IMAGE_URL = "https://media.roboflow.com/notebooks/examples/dog.jpeg"
IMAGE_PATH = EXPORT_DIR / "sample.jpg"
if not IMAGE_PATH.exists():
urllib.request.urlretrieve(IMAGE_URL, IMAGE_PATH)
image = Image.open(IMAGE_PATH).convert("RGB")
print(f"Sample image: {image.size[0]}×{image.size[1]}")
5. Run the .mlpackage with coremltools¶
coremltools infers its own input/output names for the .mlpackage spec — they are not renamed to
input / dets / labels. Read the input name and output order from mlmodel.get_spec().description
instead of hardcoding them, then match outputs by position in the same order as the ONNX
output_names contract (dets, labels for detection; dets, labels, masks for segmentation).
Preprocessing is the same NCHW, ImageNet-normalized pipeline the ONNX and ExecuTorch exports expect —
infer_transforms builds it directly.
import coremltools as ct
import numpy as np
import torch
from rfdetr.export.benchmark import infer_transforms, post_process
CONFIDENCE_THRESHOLD = 0.5
resolution = model.model_config.resolution
image_tensor, _ = infer_transforms((resolution, resolution))(image, None)
pixel_values = image_tensor[None].numpy()
mlmodel = ct.models.MLModel(str(mlpackage_path))
spec = mlmodel.get_spec()
input_name = spec.description.input[0].name
output_names = [o.name for o in spec.description.output]
prediction = mlmodel.predict({input_name: pixel_values})
dets, labels = (torch.from_numpy(np.asarray(prediction[name], dtype=np.float32)) for name in output_names[:2])
target_sizes = torch.tensor([[image.height, image.width]])
result = post_process({"dets": dets, "labels": labels}, target_sizes)[0]
print(f"Top score: {result['scores'][0]:.3f}")
6. Visualize with supervision¶
post_process returns boxes in absolute xyxy pixel coordinates along with scores and COCO class ids.
Filter by confidence, wrap the result in a supervision.Detections, and annotate the original image.
Annotate the PIL image rather than np.array(image). supervision's annotators accept either, but a
numpy scene is assumed to be OpenCV-style BGR — handing them an RGB array swaps the red and blue channels
of the annotations, and sv.plot_image then swaps the photo itself. Passing PIL lets supervision convert
in both directions and keeps the colours right.
import supervision as sv
from rfdetr.assets.coco_classes import COCO_CLASSES
keep = result["scores"] > CONFIDENCE_THRESHOLD
detections = sv.Detections(
xyxy=result["boxes"][keep].numpy(),
confidence=result["scores"][keep].numpy(),
class_id=result["labels"][keep].numpy().astype(int),
)
print(f"Kept {len(detections)} detections above {CONFIDENCE_THRESHOLD}")
names = [COCO_CLASSES.get(int(c), str(c)) for c in detections.class_id]
annotation_labels = [f"{name} {conf:.2f}" for name, conf in zip(names, detections.confidence)]
print(f"Detections: {annotation_labels}")
annotated = sv.BoxAnnotator(thickness=3).annotate(scene=image.copy(), detections=detections)
annotated = sv.LabelAnnotator(text_scale=0.6, text_thickness=1, text_padding=4).annotate(
scene=annotated, detections=detections, labels=annotation_labels
)
OUTPUT_PATH = EXPORT_DIR / "annotated_coreml.jpg"
annotated.save(OUTPUT_PATH)
print(f"Saved annotated image: {OUTPUT_PATH}")
sv.plot_image(annotated)
Next steps¶
- Deploy in Xcode — drag
rfdetr-small.mlpackageinto your Xcode project and load it with theVisionorCoreMLframework for on-device iOS / macOS inference. - Fine-tuned weights — pass
pretrain_weights="<path/to/checkpoint.pth>"when constructing the model. - Smaller bundle — pass
coreml_precision="float16"toexport()for an ANE-oriented build (expect larger numeric drift from the fp32 PyTorch baseline). - On-device via ExecuTorch instead — see the ExecuTorch cookbook for the
format="executorch", backend="coreml"path, which produces a.ptefor the ExecuTorch runtime rather than a native.mlpackage. - See the Export documentation for all formats and options.