Quick Answer: CoTracker3 remains a strong open research model for tracking points across video, especially when you need joint tracking, occlusion handling, dense or user-selected points, and both online and offline inference. Its biggest limitations in 2026 are its non-commercial CC-BY-NC licensing, GPU-oriented setup, failures on low-texture surfaces, and the emergence of newer point trackers that now outperform it on some benchmarks.
Last Updated: September 18, 2026
CoTracker3 is a transformer-based point-tracking model developed by researchers from Meta AI and the Visual Geometry Group at the University of Oxford. It tracks selected pixels across video frames and predicts whether those points remain visible, even when they temporarily disappear behind objects or leave the field of view.
The original CoTracker3 paper was submitted in October 2024 and subsequently published at ICCV 2025. The model introduced both a simplified tracking architecture and a semi-supervised training strategy that uses pseudo-labels on real video.
Developers who want a shorter product summary can first explore the CoTracker3 tool profile, while teams evaluating a wider technical stack can browse CompareBestAI's Developer AI tools category.
CoTracker3 Review: Key Facts
| Specification | CoTracker3 |
|---|---|
| Developer | Meta AI + University of Oxford VGG |
| Primary task | Any-point video tracking |
| Original release | October 15, 2024 |
| Peer-reviewed publication | ICCV 2025 |
| Current repository version | 3.0.0 |
| Inference modes | Online and offline |
| Output | 2D point trajectories + visibility |
| Point selection | Manual points or regular grid |
| Pretrained checkpoints | Available |
| PyTorch Hub | Supported |
| Recommended hardware | CUDA-capable GPU |
| CPU use | Possible for small tasks |
| Main repository license | CC-BY-NC |
| Commercial use | License requires careful review |
The repository currently identifies the package version as 3.0.0, provides pretrained online and offline checkpoints, supports PyTorch Hub loading, and strongly recommends CUDA even though small workloads can run on CPU.
What Is CoTracker3?
CoTracker3 is an any-point tracker.
Instead of detecting predefined object categories such as “person,” “car,” or “dog,” you give the model one or more spatial points in a video. It then estimates where those exact points move over subsequent frames.
The official implementation supports three useful tracking patterns:
- manually selected points,
- points sampled over a regular grid,
- and large sets of points tracked jointly.
This makes the model fundamentally different from object detection or segmentation systems.
For example, a researcher could select several points on a moving hand and follow their motion through a video. A robotics workflow could track surface features on manipulated objects. A reconstruction pipeline could use point trajectories to help understand motion and geometry.
If you need segmentation masks rather than individual trajectories, SAM 2 by Meta addresses a different but complementary problem.
How CoTracker3 Works
The central idea behind CoTracker3 is not merely a larger neural network.
The authors deliberately simplified components from previous trackers and combined the new architecture with a different training strategy.
Joint Point Tracking
CoTracker models multiple points together instead of treating every tracked point as completely independent.
That allows trajectories to share information about the motion and structure of the scene.
This approach is useful when several points belong to the same object or when scene motion provides context for uncertain trajectories.
Pseudo-Label Training on Real Videos
Point tracking has a difficult data problem: real videos are easy to collect, but accurate trajectory annotations are expensive.
Synthetic video can provide perfect labels, but synthetic scenes do not fully reproduce the appearance and motion statistics of real footage.
CoTracker3 addresses this gap by using existing trackers as teachers. Those teacher models generate pseudo-labels for unlabeled real videos, allowing CoTracker3 to learn from real-world visual data without manually annotating every trajectory.
The authors report that this training strategy achieved stronger results while using 1,000 times less additional real video data than BootsTAPIR's scaled training setup.
CoTracker3 Online vs Offline
CoTracker3 is available in online and offline variants.
That distinction is important because the two modes solve different deployment problems.
CoTracker3 Offline
The offline model can use information from the broader video sequence while estimating trajectories.
This is useful when:
- the entire video already exists,
- real-time output is unnecessary,
- and maximum long-range consistency is more important than causal inference.
Access to future frames can help an offline tracker reason about points that temporarily disappear and later reappear.
CoTracker3 Online
The online version is intended for causal or streaming-style processing.
It processes video progressively instead of depending on future frames.
This makes it more appropriate for robotics, live systems, streaming pipelines, and other scenarios where future video has not yet been captured.
The official repository exposes separate cotracker3_online and cotracker3_offline pretrained interfaces.
CoTracker3 Benchmark Results
The current official repository provides reproducible results for the released CoTracker3 checkpoints.
| Model | Kinetics δavg-vis | DAVIS δavg-vis | RoboTAP δavg-vis | RGB-Stacking δavg-vis |
|---|---|---|---|---|
| CoTracker2 | 61.8 | 74.6 | 69.6 | 73.4 |
| CoTracker2.1 | 63.0 | 76.1 | 70.6 | 79.6 |
| CoTracker3 Offline | 67.8 | 76.9 | 78.0 | 85.0 |
| CoTracker3 Online | 68.3 | 76.7 | 78.8 | 82.7 |
These values come from the evaluation instructions associated with the released repository and checkpoints. The maintainers note that exact paper reproduction requires the evaluation configuration described in the repository.
The results show clear gains over earlier CoTracker versions across the listed benchmark measurements.
However, benchmark context matters.
CoTracker3 was released as a leading tracker, but the point-tracking field has continued moving. Track-On2 and the newer Track-On-R work report stronger performance in several 2026 evaluations, so a current review should describe CoTracker3 as a strong established baseline and practical open implementation, not automatically the best tracker available today.
Tracking Through Occlusion
Occlusion handling is one of CoTracker3's strongest design areas.
A point can disappear because:
- another object passes in front,
- the tracked object rotates,
- it leaves the visible camera region,
- or the camera itself moves.
The official project demonstrates trajectories continuing across temporary occlusions and describes both online and offline variants as capable of tracking visible and occluded points.
This is particularly relevant for:
- robotics,
- human motion,
- animal movement,
- dynamic scene reconstruction,
- object manipulation,
- and long video sequences.
CoTracker3's project page also demonstrates tracking 10,000 grid-sampled points in its object-centric examples, illustrating why point tracking can behave more like a structured motion representation than a traditional single-object tracker.
CoTracker3 Failure Cases
A strong technical review should explain where the model fails, not only where demonstrations look impressive.
The official CoTracker3 project specifically identifies featureless surfaces as a common failure mode.
Points placed in areas such as open sky or water may lack enough distinctive visual texture for reliable correspondence across frames.
Other practical limitations include long-term drift, extreme appearance change, heavy motion, repeated visual patterns, and difficult reappearance scenarios.
More recent tracking benchmarks also show that point-tracking performance can fall considerably as real-world motion complexity increases.
That means CoTracker3 should not be treated as a guaranteed “track anything forever” system.
CoTracker3 Installation and Hardware Requirements
CoTracker3 is primarily a developer and research tool rather than a polished consumer application.
The official repository supports loading pretrained models through PyTorch Hub.
A minimal offline workflow uses:
import torch cotracker = torch.hub.load( "facebookresearch/co-tracker", "cotracker3_offline" ).cuda()
The online equivalent uses:
cotracker = torch.hub.load( "facebookresearch/co-tracker", "cotracker3_online" ).cuda()
The repository recommends installing PyTorch and TorchVision with CUDA support. Small tasks can run on CPU, but serious video workloads are much more practical with GPU acceleration.
Teams building larger computer-vision pipelines may also want to compare deployment and dataset tooling such as Roboflow, especially when point tracking is only one stage of a broader vision workflow.
CoTracker3 License: Is It Free?
The code and pretrained checkpoints are publicly accessible, but publicly accessible does not mean unrestricted commercial software.
The official repository states that the majority of CoTracker is licensed under CC-BY-NC. Portions inherited from other projects use MIT or Apache 2.0 licenses.
The NC component means non-commercial.
Therefore:
- academic experimentation is a natural fit,
- research prototypes are a natural fit,
- educational use is generally consistent with the project's positioning,
- but companies should review the exact license before using CoTracker3 in a commercial product or service.
This is one of the most important points to include in a 2026 CoTracker3 review because it materially affects adoption decisions.
Where CoTracker3 Fits in a Computer Vision Stack
CoTracker3 solves trajectory estimation.
Other AI vision systems solve different problems.
CoTracker3 vs SAM 2
Meta SAM 2 focuses on segmenting and tracking object masks.
CoTracker3 tracks spatial points.
Use SAM-style segmentation when the boundary or full mask of an object matters.
Use CoTracker3 when individual point trajectories and fine motion are the important representation.
Some advanced pipelines can benefit from both.
CoTracker3 vs V-JEPA
V-JEPA by Meta is aimed at learning video representations and understanding temporal visual information rather than outputting explicit per-pixel trajectories.
CoTracker3 answers:
“Where did this point move?”
A video representation model is more likely to support questions such as:
“What is happening in this video?”
They belong to the same broader video-AI ecosystem but solve different tasks.
CoTracker3 vs Motion Capture Tools
A dedicated webcam motion-capture tool may be easier for creators who want finished human-motion output without building a computer-vision pipeline.
CoTracker3 is lower-level infrastructure.
That flexibility is useful for researchers and engineers, but it also means more development work.
CoTracker3 for 3D Reconstruction
Point trajectories are useful beyond pure 2D tracking.
Structure-from-motion and dynamic reconstruction systems need correspondences across frames to infer how camera viewpoints and scene geometry change.
The CoTracker repository highlights VGGSfM as an example of a reconstruction pipeline where point tracking is a core component.
For teams working with geometry, video understanding, or visual reconstruction, CoTracker3 can therefore function as an upstream motion-estimation component rather than the final application.
Researchers exploring adjacent perception and analysis systems can also browse CompareBestAI's Research & Data tools.
CoTracker3 for Robotics
Robotic manipulation creates several point-tracking challenges:
- moving cameras,
- object rotation,
- temporary occlusion,
- small objects,
- and fast interactions.
RoboTAP is therefore one of the benchmark families used to evaluate CoTracker3.
The current released online model reports a δavg-vis score of 78.8 on the repository's RoboTAP evaluation, versus 70.6 for CoTracker2.1.
That does not mean CoTracker3 alone solves robotic perception.
A production robot still needs detection, segmentation, depth or geometry, control, uncertainty handling, and system-specific safety logic.
CoTracker3 for Video Editing and Effects
Point tracking can also support visual-effects workflows.
Tracked trajectories can provide motion information for:
- attaching graphics,
- stabilizing regions,
- estimating motion,
- propagating edits,
- and analyzing local deformation.
However, CoTracker3 is not an Adobe-style finished editor.
It provides tracking outputs that a developer can integrate into another workflow.
If the end goal is video generation rather than motion analysis, tools in CompareBestAI's broader Video Generation category address a different part of the workflow.
CoTracker3 Pros and Cons
| Pros | Cons |
|---|---|
| Tracks arbitrary points | Requires technical setup |
| Online and offline variants | CUDA/GPU preferred |
| Strong occlusion handling | Featureless areas can fail |
| Joint multi-point reasoning | Long-term drift remains possible |
| Pretrained checkpoints available | Not a consumer application |
| Reproducible benchmark code | CC-BY-NC limits commercial use |
| PyTorch Hub support | Newer trackers now outperform it on some benchmarks |
| Useful for research pipelines | Requires surrounding pipeline for end-user applications |
Is CoTracker3 Still Worth Using in 2026?
Yes—for the right user.
CoTracker3 remains particularly attractive when you need:
- an established research implementation,
- pretrained point-tracking checkpoints,
- both causal and offline inference,
- arbitrary-point or dense grid tracking,
- reproducible benchmark tooling,
- or an open research baseline for computer vision experiments.
Its value is lower when you need:
- a no-code application,
- guaranteed commercial licensing,
- an all-in-one object-tracking product,
- the newest benchmark-leading online tracker,
- or finished segmentation masks rather than point trajectories.
This distinction is important.
CoTracker3 is best evaluated as research/developer infrastructure, not as SaaS software.
Who Should Use CoTracker3?
CoTracker3 is well suited to:
Computer vision researchers who need point tracking for experiments or comparisons.
Robotics researchers who need trajectories through object manipulation and occlusion.
3D vision teams using point correspondence in reconstruction pipelines.
AI engineers prototyping video-analysis systems.
Academic labs and students studying tracking, motion, temporal correspondence, or video understanding.
It is less appropriate for nontechnical users who expect a browser-based upload-and-export workflow.
How We Evaluated CoTracker3
CompareBestAI evaluates technical AI tools using product documentation, original research, released code, reproducible benchmark information, practical limitations, licensing, target users, and current alternatives.
For CoTracker3, the most important criteria are:
- tracking accuracy,
- occlusion handling,
- online versus offline support,
- ease of integration,
- model availability,
- reproducibility,
- hardware requirements,
- licensing,
- and whether later methods have changed its competitive position.
You can review CompareBestAI's broader AI tool evaluation methodology.
Frequently Asked Questions
What is CoTracker3?
CoTracker3 is a transformer-based video point tracker developed by Meta AI and Oxford VGG researchers. It follows selected pixels through video frames and predicts their trajectories and visibility, including during some temporary occlusions.
Is CoTracker3 free?
Its source code and checkpoints are publicly available, but most of the repository uses a CC-BY-NC license. That makes licensing different from unrestricted commercial open-source software. Review the repository's license before commercial deployment.
What is the difference between CoTracker3 online and offline?
The online model processes video causally without relying on future frames. The offline model can use broader sequence context, making it better suited to prerecorded video where real-time processing is unnecessary.
Does CoTracker3 require a GPU?
A CUDA-capable GPU is strongly recommended by the official repository. Small workloads can run on CPU, but GPU acceleration is the practical option for substantial video tracking.
Can CoTracker3 track occluded points?
Yes. Occlusion handling is one of CoTracker3's core strengths, and its project demonstrations show trajectories surviving temporary occlusion. Results still depend on motion, appearance, texture, and video complexity.
Is CoTracker3 still state of the art in 2026?
It was introduced as a state-of-the-art tracker, but newer systems such as Track-On2 and Track-On-R have since reported stronger results on several benchmarks. CoTracker3 remains an important and practical baseline rather than an uncontested current leader.
CoTracker3 Review Verdict
CoTracker3 remains a valuable point-tracking model in 2026 because it combines strong tracking quality, online and offline variants, public checkpoints, reproducible evaluation tooling, and unusually good handling of occlusion.
Its main compromises are equally clear: this is developer-focused research software, GPU acceleration is preferable, featureless surfaces remain difficult, and the repository's CC-BY-NC license requires caution for commercial deployment.
For research, prototyping, 3D vision, robotics, and custom video-analysis pipelines, it remains highly relevant.
Before integrating it into production, verify licensing and compare it against newer trackers using your own representative video—not only public benchmark scores.
You can review CoTracker3's complete tool profile or compare AI tools side by side before choosing your computer-vision stack.
