Perception
Before a machine can act, it has to agree with us about what it is looking at. Cameras and LiDAR return points and pixels: not "a child", "a kerb", or "a doorway". Somebody has to draw that line first.
Perception is where the physical world is turned into something a policy can reason over. A LiDAR sweep is a few hundred thousand coordinates with intensity values; a camera frame is a grid of brightness. Neither contains the word "pedestrian". That meaning is supplied by people, once, carefully, and every model trained downstream inherits both the accuracy and the mistakes of that first act of interpretation.
What this needs from people
- Three-dimensional boxes drawn across continuous frames, so an object keeps its identity as it moves.
- Camera context matched to the point cloud, so the two views describe one scene.
- Edge cases handled deliberately: missing returns, distorted points, partial occlusion.
- Precise geometry where a few centimetres decide whether a manoeuvre is safe.
The research
Where this subsystem actually stands.
Talks & demonstrations
The people building it, in their own words.
Keynote — Andrej Karpathy, Tesla
PyTorch at Tesla — Andrej Karpathy
Where HSV fits
This is work HSV already delivers. Our 3D LiDAR annotation line labels continuous-frame point clouds with 3D boxes and camera context, including the missing and distorted returns most pipelines quietly drop. Our parking-slot work is the same discipline at tighter tolerance: bounding boxes plus inner corner points, with attributes chosen by slot and surface type.
See the delivery line