Pipeline¶
Vizor is the front door. It owns a primary model and a Refiner, and drives
them over a video. Refiner is the part that decides what to send to the
secondary and what to do with the answer, and you can use it on its own if you
already have detections.
Vizor
¶
Run a fast tracker on every frame and refine it with a slower model.
The primary gives boxes and track ids at frame rate. The secondary is only asked about tracks the primary is unsure of, and its answers are cached against the track id, so the cost is paid once per object rather than once per frame.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
primary
|
Model
|
model with a |
required |
secondary
|
Model | None
|
model with |
None
|
conf
|
float
|
tracks at or below this confidence go to the secondary. |
0.5
|
mode
|
str
|
|
'full'
|
names
|
dict[int, str] | list[str] | None
|
class id to name mapping. Defaults to whatever the primary reports. |
None
|
Remaining keyword arguments go to Refiner. Pass
workers there to stop the frame loop waiting on the secondary.
Source code in vizor/core.py
reset
¶
Clear the vote cache and any state the primary keeps between frames.
wait
¶
close
¶
step
¶
Process one frame and return the refined Tracks.
Source code in vizor/core.py
run
¶
Yield refined tracks for every frame of src.
src is anything Video opens: a file path,
a camera index, or a stream url. Pass save to also write an annotated
video, and show to display it in a window.
Each yielded Tracks carries its frame, so out.draw() needs no
argument. Drawing happens on the frame itself, so copy it first if you
want the original.
Source code in vizor/core.py
Refiner
¶
Refiner(model=None, conf=0.5, mode='full', iou=0.5, names=None, votes=1, best=True, size=256, hist=25, workers=0, samples=4, every=5, cell=128, cols=None)
Correct low confidence tracks with a second model and cache the answers.
Two modes:
full
Run the secondary on the whole frame, match its boxes to the tracks by
IoU, and take its box, confidence and class. Best when the secondary
is a grounding model or a heavier detector.
crop
Cut each low confidence track out of the frame and ask the secondary
what it is. Best when the secondary is a chat VLM that classifies but
does not localise.
collage
Gather samples crops of the same track, every frames apart, tile
them into one image and ask about that. Best when one frame is not
enough to tell, so an attribute like which way someone is facing or what
they are carrying.
Every answer is a vote against the track id, so a track keeps its corrected class on later frames without the secondary running again.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model
|
Model | None
|
the secondary. Needs |
None
|
conf
|
float
|
tracks at or below this confidence are sent to the secondary. Set it to 1.0 to send every track, which is what an attribute question usually wants. |
0.5
|
mode
|
str
|
|
'full'
|
iou
|
float
|
minimum IoU to match a secondary box to a track, full mode only. |
0.5
|
names
|
dict[int, str] | list[str] | None
|
class id to name mapping. Falls back to whatever the primary reports. |
None
|
votes
|
int
|
stop asking about a track once it has this many votes. Crop and collage modes. |
1
|
best
|
bool
|
keep only the best IoU match per track instead of every match above |
True
|
size
|
int
|
how many track ids to keep in the vote cache, and how many may be part way through a collage at once. |
256
|
hist
|
int
|
how many votes to keep per track id. |
25
|
workers
|
int
|
run the secondary on this many background threads instead of blocking the frame loop. 0, the default, blocks. Crop and collage modes. |
0
|
samples
|
int
|
crops per collage, collage mode only. |
4
|
every
|
int
|
frames between one track's crops, collage mode only. 0 takes one every frame. |
5
|
cell
|
int | tuple[int, int]
|
collage cell size, one number for a square or a |
128
|
cols
|
int | None
|
columns in the collage. Defaults to a square-ish grid. |
None
|
Source code in vizor/refine.py
wait
¶
Block until every request in flight has come back, then bank the votes.
Only useful with workers. The votes land too late for the frames that
triggered them, but they are there for whatever you refine next.
Source code in vizor/refine.py
close
¶
Stop the workers. Anything still in flight is dropped.
reset
¶
Forget every vote and drop anything in flight. Call this between videos.
run
¶
Refine tracks in place and return them.
img is the current frame. preds lets you supply the secondary's
output yourself, which skips the model call entirely.
Source code in vizor/refine.py
Vote
¶
Majority vote per track id, capped at size ids and hist votes each.
Ids below zero are untracked detections that share the same placeholder id, so they are ignored instead of being pooled together.