Sample formats

Search…
Sample formats
A sample is a data point you want to label. Samples come in different types, like an image, a 3D point cloud, or a video sequence. When uploading (client.add_sample()) or downloading (client.get_sample()) a sample using the Python SDK, the format of the attributes field depends on the type of sample. The different formats are described here.
The section Import data shows how you can obtain URLs for your assets.
Image
Supported image formats:  jpeg, png, bmp.
{
    "image": {
        "url": "https://example.com/image.jpg"
    }
}
Image sequence
Supported image formats:  jpeg, png, bmp.
{ 
  "frames": [
    {
      "image": {
        "url": "https://example.com/frame_00001.jpg"
      },
      "name": "frame_00001" // optional
    },
    {
      "image": {
        "url": "https://example.com/frame_00002.jpg"
      },
      "name": "frame_00002"
    },
    {
      "image": {
        "url": "https://example.com/frame_00003.jpg"
      },
      "name": "frame_00003"
    }
  ]
} 
3D point cloud
{
    "pcd": {
        "url": "https://example.com/pointcloud.bin",
        "type": "kitti"  // optional, "pcd" by default
    },
    "images": [
        { ... },
        { ... },
        { ... }
    ], // optional
    "name": "frame_00001", // optional
    "timestamp": "00001", // optional
    "ego_pose": {
        "position": {
            "x": -2.7161461413869947,
            "y": 116.25822288149078,
            "z": 1.8348751887989483
        },
        "heading": {
            "qx": -0.02111296123795955,
            "qy": -0.006495469416730261,
            "qz": -0.008024565904865688,
            "qw": 0.9997181192298087
        }
    },
    "default_z": -1 // optional, 0 by default
}
Name
Type
Description
pcd
​Point cloud data​
Required. Point cloud data.
images
array of camera images​
Reference camera images.
name
string
Name of the sample.
timestamp
string
Timestamp of the sample.
ego_pose
​Ego pose​
Pose of the sensor that captured the point cloud data.
default_z
float
Default z-value of the ground plane. 0 by default. Only valid in the point cloud cuboid editor. New cuboids will be drawn on top of the ground plane, i.e. the default z-position of a new cuboid is 0.5 (since the default height of a new cuboid is 1).
Point cloud data
See 3D point cloud formats for the supported file formats.
{
    "url": "https://example.com/pointcloud.bin",
    "type": "kitti"  // optional, "pcd" by default
}
Name
Type
Description
url
string
Required. URL of the point cloud data.
type
string: "pcd" | "binary-xyzi" | "kitti" | "binary-xyzir" | "nuscenes"
Type of the point cloud data. "pcd" by default. See 3D point cloud file formats for the list of supported file formats.
Camera image
A calibrated or uncalibrated reference image corresponding to a point cloud. The reference images can be opened in a new tab from within the labeling interface. You can determine the layout of the images by setting the row and col attributes on each image. If you also supply the calibration parameters, the main point cloud view can be set to the image to obtain a fused view. In this case, your image should be rectified before uploading.
Calibrated camera images are a beta feature. The definition of the camera parameters might still change. Contact us if you have any feedback.
{
    "url": "https://example.com/image.jpg",
    "row": 0,
    "col": 0,
    "intrinsics": { // optional
        "intrinsic_matrix": [
            [1266.417203046554, 0, 816.2670197447984],
            [0, 1266.417203046554, 491.50706579294757],
            [0, 0, 1]
        ]
    },
    "extrinsics": { // optional
        "translation": {
            "x": -0.012463384576629082,
            "y": 0.76486688894964,
            "z": -0.3109103442096661
        },
        "rotation": {
            "qx": 0.713640516187247,
            "qy": -0.001134052598226082,
            "qz": 0.0036449450274057696,
            "qw": 0.7005017073187271
        }
    }
}
Name
Type
Description
url
string
Required. URL of the camera image.
row
int
Required. Row of this image in the images viewer.
col
int
Required. Column of this image in the images viewer.
intrinsics
​Camera intrinsics​
Intrinsic parameters of the camera.
extrinsics
​Camera extrinsics​
Extrinsic parameters of the camera relative to the ego pose.
Camera intrinsics
{
    "intrinsic_matrix": [
        [1266.417203046554, 0, 816.2670197447984],
        [0, 1266.417203046554, 491.50706579294757],
        [0, 0, 1]
    ]
}
Name
Type
Description
intrinsic_matrix
2D array of floats representing 3x3 matrix KKK
in row-major order​
Required. Intrinsic matrix KKK
 used in the pinhole camera model.
K=[fx0ox0fyoy001]K = \begin{bmatrix} f_x & 0 & o_x\\ 0 & f_y & o_y \\ 0 & 0 & 1 \end{bmatrix}K=⎣⎡​fx​00​0fy​0​ox​oy​1​⎦⎤​
​
​fxf_xfx​
 and fyf_yfy​
​ are the focal lengths in pixels. We assume square pixels, so fx=fyf_x = f_yfx​=fy​
​. oxo_xox​
 and oyo_yoy​
 are the offsets (in pixels) of the principal point from the top-left corner of the image frame.
Camera extrinsics
{
    "translation": {
        "x": -0.012463384576629082,
        "y": 0.76486688894964,
        "z": -0.3109103442096661
    },
    "rotation": {
        "qx": 0.713640516187247,
        "qy": -0.001134052598226082,
        "qz": 0.0036449450274057696,
        "qw": 0.7005017073187271
    }
}
Name
Type
Description
translation
object: {
    "x": float,
    "y": float,
    "z": float
}
Required. Translation of the camera in lidar coordinates, i.e., relative to the ego pose.
rotation
object: {
    "qx": float,
    "qy": float,
    "qz": float,
    "qw": float
}
Required. Rotation of the camera in lidar coordinates, i.e., relative to the ego pose. Defined as a rotation quaternion.
Ego pose
The pose of the sensor used to capture the 3D point cloud data. This can be helpful if you want to obtain cuboids in world coordinates, or when your sensor is moving. In the latter situation, supplying an ego pose with each frame will ensure that static objects do not move when switching between frames.
{
    "position": {
        "x": -2.7161461413869947,
        "y": 116.25822288149078,
        "z": 1.8348751887989483
    },
    "heading": {
        "qx": -0.02111296123795955,
        "qy": -0.006495469416730261,
        "qz": -0.008024565904865688,
        "qw": 0.9997181192298087
    }
},
Name
Type
Description
position
object: {
    "x": float,
    "y": float,
    "z": float
}
Required. XYZ position of the sensor in world coordinates.
heading
object: {
    "qx": float,
    "qy": float,
    "qz": float,
    "qw": float
}
Required. Orientation of the sensor. Defined as a rotation quaternion.
3D point cloud sequence
{ 
  "frames": [
    { ... },
    { ... },
    { ... }
  ]
} 
Name
Type
Description
frames
array of 3D point clouds​
Required. List of 3D point cloud frames in the sequence.
Text
{ 
    "text": "Example text sample." 
}
Name
Type
Description
text
string
Required. Text data.
To upload text samples in bulk, see file formats.
Reference - Previous
Task types
Supported file formats
Last modified 1mo ago
Copy link
On this page
Image
Image sequence
3D point cloud
Point cloud data
Camera image
Ego pose
3D point cloud sequence
Text