Inference API Reference

For a narrative guide with runnable examples, see Inference.

Predict with a live native model, checkpoint, or exported artifact.

This facade shares prediction behavior across native models and exported runtimes. model.inference() returns an owned, optimized snapshot of a live model. Passing a live model directly to this constructor borrows it instead: predictions follow later changes to that model, and its batches run one at a time, so each batch reads the model's state when it starts. A predictor that owns its model or runtime may build the next CUDA batch while one runs.

Examples:

Skipped when run as a doctest: requires model weights and an image file.

>>> from rfdetr import RFDETRInference, RFDETRSmall
>>> predictor = RFDETRSmall().inference(compile=False)
>>> detections = predictor.predict("image.jpg")

Attributes

class_names property

Return a copy of the source class names.

runtime_info property

Return runtime policy and current input constraints without preparing the model.

input_shape uses canonical NCHW order with None for dynamic axes. default_shape is the default image height and width. batch_size is an exact requirement; max_batch_size is an upper bound. Neither is a batching request. The returned dictionary is independent of the source.

Functions

__init__(source, *, device='auto', metadata=None, trust_checkpoint=False, runtime_options=None)

Create a prediction facade from native weights or an exported artifact.

Each source accepts its own device vocabulary; a value valid for one format can be refused by another:

Source	Accepted device values
Live RFDETR model	auto only; configure the model's own device instead
Checkpoint path	Any device RFDETR.from_checkpoint accepts; auto defers to it
ONNX	auto (CUDA provider when installed, else CPU), cpu, cuda, cuda:N
TensorRT	auto (cuda:0), cuda (the current CUDA device), cuda:N
OpenVINO	auto (AUTO plugin), cpu, gpu, npu, gpu.N, npu.N, any case
TFLite, LiteRT	auto, cpu
CoreML	auto (all compute units), cpu (CPU only), cpu_and_gpu, cpu_and_ne (preferences)
Core AI	auto, cpu; gpu and ane are refused as mere preferences
ExecuTorch	By delegate: XNNPACK auto/cpu, CoreML auto, QNN auto/qnn

The CoreML cpu_and_gpu and cpu_and_ne values are preferences; Core ML may still run on the CPU.

A live model owns its device policy. Omit device for that source; configure the native model itself to select its device.

A checkpoint loaded on MPS takes the MPS rewrites of :meth:RFDETR.inference: fused deformable sampling and LayerScale folded into the Linear before it. Its predictions match the unmodified model's to float rounding, not bit for bit. A live model is never rewritten.

Parameters:

Name	Type	Description	Default
source	RFDETR | str | PathLike[str]	A live RFDETR model or a checkpoint or exported artifact path.	required
device	str | device	Runtime device, as a string or torch.device (converted with str()), or "auto" to inherit the source device.	'auto'
metadata	Mapping[str, Any] | str | PathLike[str] | None	Missing inference semantics for an older exported artifact, supplied as a JSON file path or JSON-serializable mapping. Accepts a plain metadata object or an artifact_sha256 / metadata envelope whose digest matches the artifact. Required fields describe the format, task, input shape, outputs, normalization, classes and postprocessing; see "Metadata for older exports" in the export basics guide for the complete schema and example. Supplied values must agree with stored metadata; unknown fields and unsupported schema versions are rejected. Invalid for native models or checkpoints.	None
trust_checkpoint	bool	Allow loading a checkpoint that contains custom Python objects.	False
runtime_options	Mapping[str, Any] | None	Settings for an exported artifact's runtime. Each format reads only its own keys and raises ValueError for any other: TensorRT takes cuda_graph, engine_host_code_allowed and verbose (each a bool); OpenVINO takes cache_dir, inference_precision and config; TFLite and LiteRT take num_threads (a positive int); the other formats take none.	None

Raises:

Type	Description
FileNotFoundError	If the artifact or metadata file is missing.
ImportError	If a required format extra or runtime package is not installed.
RuntimeError	If the host or requested device cannot run the runtime.
TypeError	If runtime_options is not a mapping.
ValueError	If metadata, runtime_options or device conflicts with a native source; metadata is missing, stale or conflicting; the format is unsupported; the artifact is backbone-only; or the loader refuses a device or runtime option.

predict(images, threshold=0.5, shape=None, patch_size=None, include_source_image=True, *, antialias=None, stream=False, batch=None, vid_stride=1, stream_buffer=False)

Predict images, video, and live sources through the shared prediction pipeline.

Parameters:

Name	Type	Description	Default
images	_PredictionInput	An image, directory, glob, video, webcam index, stream URL, manifest, or an iterable of these. Images can be paths, HTTP URLs, PIL images, NumPy arrays, or tensors. Iterables other than lists and tuples are read into a list first. TXT/CSV manifests list sources; .streams files capture streams concurrently. YouTube URLs and screen capture require the optional rfdetr[stream] dependencies. String sources are downloaded, opened or expanded without size or host limits, so pass only trusted strings.	required
threshold	_ThresholdInput	The confidence score a detection must exceed to be kept. A single float applies to every class. A :class:~rfdetr.ClassThresholds, such as ClassThresholds({"person": 0.35}, default=0.5), applies each class's own threshold to its detections, so that a class the model finds less reliably can be held to a different bar. It leaves out every detection that belongs to no entry of class_names: the no-object slot of a fine-tuned model, the background slot of a legacy keypoint checkpoint, and the unused category IDs of a pretrained COCO model. A threshold of float("inf") drops every detection of that class.	0.5
shape	tuple[int, int] | None	Input height and width; exports require their recorded shape.	None
patch_size	int | None	Backbone patch size for shape validation.	None
include_source_image	bool	Store RGB source pixels in each result.	True
antialias	bool | None	Override resize antialiasing. None uses artifact metadata for exports and False for native models.	None
stream	bool	Yield results lazily. Close the generator on early exit.	False
batch	int | None	Maximum finite-source batch size. None runs a bare BCHW tensor or an eager image-only list as one batch, finite expanded or streamed sources in batches of the fixed batch size of a compiled snapshot or fixed-batch export, an eager directory or glob in batches of eight (or of a smaller export max_batch_size) when the model or export has no fixed batch size, and other finite sources one image or frame at a time. An integer bounds these batches, including eager image lists. Live captures and .streams keep one frame per source in each simultaneous batch, regardless of this setting; the source count must match a fixed runtime batch size. Each batch must satisfy the runtime limits. Fixed-batch runtimes also reject a short final finite batch. Use batch-one or dynamic-batch runtimes for individual live sources.	None
vid_stride	int	Read every Nth video frame.	1
stream_buffer	bool	Queue live frames instead of keeping the latest frame.	False

Returns:

Type	Description
_PredictionOutput | Generator[_PredictionResult, None, None]	A generator with stream enabled, otherwise a scalar image result or a list.

Raises:

Type	Description
TypeError	If threshold is neither a number nor a sequence of numbers.
ValueError	If shape, batch, stride, buffer, capture, or runtime limits are violated, or if a threshold sequence is not one-dimensional, differs in length from class_names or holds a NaN.
FileNotFoundError	If a local file is missing or a directory/glob contains no supported media.
ImportError	If a source needs the optional rfdetr[stream] dependencies and they are not installed.

Examples:

Skipped when run as a doctest: requires model weights and image files.

>>> from rfdetr import RFDETRInference, RFDETRSmall
>>> predictor = RFDETRSmall().inference(compile=False)
>>> results = predictor.predict(["a.jpg", "b.jpg"], threshold=0.4)
>>> len(results)
2