Skip to content

3D point cloud cuboid interface ​

When creating a dataset through the Python SDK, choose pointcloud-cuboid or pointcloud-cuboid-sequence as the task_type to use this labeling interface

Tip: use your GPU in Chrome to make sure the 3D point cloud interface runs smoothly.

This video shows you how to label cuboids in 3D point cloud sequences on Segments.ai. Specifically, we'll use the merged point cloud view to easily label static objects, and batch mode to label dynamic objects.

Create a new cuboid ​

Create a cuboid with default dimensions ​

Set default cuboid dimensions while editing each category in Dataset → Settings → Labeling.

Alternatively, edit the raw JSON in Dataset → Settings → Labeling → Raw (see Categories and attributes).

You can then create a cuboid with the current category's default dimensions by double-clicking.

  1. Select the "Create cuboid" tool by clicking on the box icon in the toolbar on the left, or by pressing the hotkey (b by default).
  2. Double click in the perspective view or in the top view to create a new cuboid. This cuboid will have the default dimensions of the current category. When you change the category, the dimensions of the cuboid will change as well, given that you have not altered the dimensions of the cuboid yet.

Draw a cuboid with custom dimensions ​

You can quickly draw a cuboid with an arbitrary rotation using our 3-click cuboid drawing method.

  1. Select the "Create cuboid" tool by clicking on the box icon in the toolbar on the left, or by pressing the hotkey (b by default).
  2. Click in the perspective view or in the top view to set the back corner of the new cuboid.
  3. When you move your mouse, the back side of the cuboid will follow your mouse position. Click to set the second corner of the cuboid.
  4. Next, you can extend the cuboid by moving your mouse. Click to complete the cuboid. After clicking, the new cuboid is added to the objects sidebar on the right.

Smart cuboid initialization ​

Instead of manually placing a cuboid, you can automatically create one from a point selection:

  1. Select the Create cuboid tool (b by default).
  2. Hold Alt (Windows/Linux) or Option (macOS) and drag to draw a selection box over the point cloud.
  3. Release to create a fitted 3D cuboid with an estimated position, dimensions, and rotation.

A "Creating cuboid..." indicator appears while the cuboid is being computed.

This works best when you select a cluster of points belonging to a single object.

Initialization modes ​

Choose a mode in Settings → Annotation tools → Cuboid init method:

  • Simple: Fits a cuboid around the selected points, with its yaw based on the current view.
  • Smart (default): Uses geometric fitting to estimate the cuboid's position, dimensions, and yaw after filtering low points and outliers. Outlier tolerance controls filtering: lower values remove outliers more aggressively.
  • AI (beta): Uses a machine learning model to predict the cuboid's position, dimensions, and rotation from the selected points.

AI mode is still in beta and can produce unexpected results.

Auto-adjust cuboids ​

Select a cuboid and press q (the default hotkey) to fit its position and dimensions to the visible point-cloud points inside it.

Auto-adjust preserves yaw and produces an upright cuboid, resetting pitch and roll. It uses geometric fitting independently of the selected initialization method.

Position and size the cuboid so it encloses the intended object before adjusting. Points outside it are not considered; if no points are found inside, it remains unchanged.

Remove a cuboid ​

  1. Select the cuboid you want to remove.
  2. Press the hotkey (Backspace by default) or click the trash icon next to the object in the objects sidebar on the right.

Select a cuboid ​

Using the "Select and edit" tool ​

  1. Select the "Select and edit" tool by clicking on the pointer icon in the toolbar on the left, or by pressing the hotkey (Esc by default).
  2. Click on the cuboid in any view.

Side view auto-centering

  • The side views auto-center on the selected annotation
  • Panning or rotating a side view automatically disables its auto-centering behavior on the active object. Other views will continue to follow the active object normally.
  • A "Re-center" button appears on the modified view to manually re-enable auto-centering.
  • Auto-centering resets automatically on track or frame change

Using the objects sidebar ​

  1. Click on the object in the objects sidebar to select it.

Change the dimensions of a cuboid ​

  1. Select the "Select and edit" tool by clicking on the pointer icon in the toolbar on the left, or by pressing the hotkey (Esc by default).
  2. Select the cuboid you want to change.
  3. Change the cuboid's dimensions by using the hotkeys (i, j, k, l, u, o by default) or by using your mouse:
    • In the perspective view: click and drag a face of the cuboid.
    • In a side view: click and drag one of the edges of the cuboid.

Auto-adjust cuboid ​

To fit the cuboid to the points inside it, use Auto-adjust cuboids.

Translate a cuboid ​

  1. Select the "Select and edit" tool by clicking on the pointer icon in the toolbar on the left, or by pressing the hotkey (Esc by default).
  2. Select the cuboid you want to translate.
  3. Translate the cuboid by using the hotkeys (w, a, s, d, Shift + W, Shift + S by default) or by using your mouse:
    • Click and drag the yellow cube in the middle of the cuboid to translate the cuboid freely.
    • Hover over the yellow cube in the middle to see the translation axes. Click and drag on an axis to translate the cuboid along the selected axis.

Rotate a cuboid ​

By default, only yaw rotation (around the z-axis) is enabled. To enable full 3D rotation, visit the dataset settings, select the "Labeling" tab, tick the "Enable 3D cuboid rotation" checkbox, and press "Save".

  1. Select the "Select and edit" tool by clicking on the pointer icon in the toolbar on the left, or by pressing the hotkey (Esc by default).
  2. Select the cuboid you want to rotate.
  3. Click and drag the red rotation plane attached to the cuboid to rotate the cuboid or use the hotkeys:
    • Rotate 1° counterclockwise (yaw): f by default
    • Rotate 1° clockwise (yaw): g by default
    • Rotate 45° counterclockwise (yaw): Shift + F by default
    • Rotate 45° clockwise (yaw): Shift + G by default

Change the heading of a cuboid ​

  1. Select the "Select and edit" tool by clicking on the pointer icon in the toolbar on the left, or by pressing the hotkey (Esc by default).
  2. Select the cuboid you want to change.
  3. Change the heading of the cuboid (indicated by a red arrow):
    • In the perspective view: double-click the face where you want the heading to be.
    • In a side view: double-click the edge where you want the heading to be.

Edit cuboid information ​

When a cuboid is selected you'll see the "Cuboid info" pane pop up at the bottom right. In here you can directly change the numeric inputs of the cuboid for more fine-grained control. In here it's also possible to propagate a certain value to the next and previous frames by clicking on the arrow buttons next to the input field. These buttons are disabled for properties that are already synced across frames.

Cuboid information pane

Change the category of a cuboid ​

If the selected cuboid was created with default dimensions, the dimensions of the cuboid will change to the default dimensions of the new category.

Using the category dropdown ​

  1. Select the cuboid.
  2. Open the category dropdown by clicking on the category name in the objects sidebar on the right, or by pressing the hotkey (c by default).
  3. Select a category by clicking on the desired category, or by using the arrow keys and pressing Enter to confirm.

Using the category hotkey ​

  1. Select the cuboid.
  2. Press the hotkey of the desired category (1 - 9).

Cycle between cuboids / select the next cuboid ​

Press the hotkey (Tab by default).

To select among objects at the same location, use Alt + click (Option + click on macOS) to cycle through annotations under the cursor. See Select overlapping annotations for plain-click behavior and editing-handle interactions.

Copy/paste a cuboid ​

Copy a cuboid ​

  1. Select the cuboid.
  2. Press Ctrl + C (Windows/Linux) or Cmd + C (macOS).

Paste a cuboid ​

  1. Press Ctrl + V (Windows/Linux) or Cmd + V (macOS). A translucent preview of the cuboid appears.
  2. Move the cursor in the 3D viewer to position the preview.
  3. Click to place the cuboid, or press Escape to cancel.

The preview shows where the cuboid will be pasted. Its horizontal position follows the cursor, while its height, dimensions, and rotation remain unchanged.

A selected cuboid and a translucent cuboid preview at the proposed paste location in a point cloud.

Move the cursor to position the cuboid preview, then click to paste.

In sequences, pasting while the selected track already has an annotation in the current frame creates a new track ID. If there is no annotation yet in the selected track, the pasted cuboid will be added to the selected track instead.

Lock a track ​

See Object locking