Sample formats
A sample is a data point you want to label. Samples come in different types, like an image, a 3D point cloud, or a video sequence. When uploading (
client.add_sample()
) or downloading (client.get_sample()
) a sample using the Python SDK, the format of the attributes
field depends on the type of sample. The different formats are described here.Supported image formats: jpeg, png, bmp.
{
"image": {
"url": "https://example.com/image.jpg"
}
}
If the image file is on your local computer, you should first upload it to our asset storage service (using
upload_asset()
) or to another cloud storage service.Supported image formats: jpeg, png, bmp.
{
"frames": [
{
"image": {
"url": "https://example.com/frame_00001.jpg"
},
"name": "frame_00001" // optional
},
{
"image": {
"url": "https://example.com/frame_00002.jpg"
},
"name": "frame_00002"
},
{
"image": {
"url": "https://example.com/frame_00003.jpg"
},
"name": "frame_00003"
}
]
}
On Segments.ai, the up direction is defined along the z-axis, i.e. the vector (0, 0, 1) points up. If you upload point clouds with a different up direction, you might have trouble navigating the point cloud.
{
"pcd": {
"url": "https://example.com/pointcloud.bin",
"type": "kitti"
},
"images": [
{ ... },
{ ... },
{ ... }
], // optional
"name": "frame_00001", // optional
"timestamp": "00001", // optional
"ego_pose": {
"position": {
"x": -2.7161461413869947,
"y": 116.25822288149078,
"z": 1.8348751887989483
},
"heading": {
"qx": -0.02111296123795955,
"qy": -0.006495469416730261,
"qz": -0.008024565904865688,
"qw": 0.9997181192298087
}
},
"default_z": -1 // optional, 0 by default
}
Name | Type | Description |
---|---|---|
pcd | Required. Point cloud data. | |
images | Reference camera images. | |
name | string | Name of the sample. |
timestamp | string | Timestamp of the sample. |
ego_pose | Pose of the sensor that captured the point cloud data. | |
default_z | float | Default z-value of the ground plane. 0 by default. Only valid in the point cloud cuboid editor. New cuboids will be drawn on top of the ground plane, i.e. the default z-position of a new cuboid is 0.5 (since the default height of a new cuboid is 1). |
{
"url": "https://example.com/pointcloud.bin",
"type": "kitti"
}
Name | Type | Description |
---|---|---|
url | string | Required. URL of the point cloud data. |
type | string : "pcd" | "binary-xyzi" | "kitti" | "binary-xyzir" | "nuscenes" | Required. Type of the point cloud data. See 3D point cloud file formats for the list of supported file formats. |
If the point cloud file is on your local computer, you should first upload it to our asset storage service (using
upload_asset()
) or to another cloud storage service.A calibrated or uncalibrated reference image corresponding to a point cloud. The reference images can be opened in a new tab from within the labeling interface. You can determine the layout of the images by setting the
row
and col
attributes on each image. If you also supply the calibration parameters, the main point cloud view can be set to the image to obtain a fused view. In this case, your image should be rectified before uploading.{
"url": "https://example.com/image.jpg",
"row": 0,
"col": 0,
"intrinsics": { // optional
"intrinsic_matrix": [
[1266.417203046554, 0, 816.2670197447984],
[0, 1266.417203046554, 491.50706579294757],
[0, 0, 1]
]
},
"extrinsics": { // optional
"translation": {
"x": -0.012463384576629082,
"y": 0.76486688894964,
"z": -0.3109103442096661
},
"rotation": {
"qx": 0.713640516187247,
"qy": -0.001134052598226082,
"qz": 0.0036449450274057696,
"qw": 0.7005017073187271
}
}
}
Name | Type | Description |
---|---|---|
url | string | Required. URL of the camera image. |
row | int | Required. Row of this image in the images viewer. |
col | int | Required. Column of this image in the images viewer. |
intrinsics | Intrinsic parameters of the camera. | |
extrinsics |
If the image file is on your local computer, you should first upload it to our asset storage service (using
upload_asset()
) or to another cloud storage service.{
"intrinsic_matrix": [
[1266.417203046554, 0, 816.2670197447984],
[0, 1266.417203046554, 491.50706579294757],
[0, 0, 1]
]
}
Name | Type | Description |
---|---|---|
intrinsic_matrix | 2D array of float s representing 3x3 matrix in row-major order | Required. Intrinsic matrix used in the pinhole camera model.
and are the focal lengths in pixels. We assume square pixels, so . and are the offsets (in pixels) of the principal point from the top-left corner of the image frame. |
{
"translation": {
"x": -0.012463384576629082,
"y": 0.76486688894964,
"z": -0.3109103442096661
},
"rotation": {
"qx": 0.713640516187247,
"qy": -0.001134052598226082,
"qz": 0.0036449450274057696,
"qw": 0.7005017073187271
}
}
Name | Type | Description |
---|---|---|
translation | object : {
"x": float ,
"y": float ,
"z": float
} | |
rotation | object : {
"qx": float ,
"qy": float ,
"qz": float , "qw": float
} | Required. Rotation of the camera in lidar coordinates, i.e., relative to the ego pose. Defined as a rotation quaternion. We use the OpenGL/Blender coordinate convention for cameras. +X is right, +Y is up, and +Z is pointing back and away from the camera. -Z is the look-at direction. Other codebases may use the OpenCV convention, where the Y and Z axes are flipped but the +X axis remains the same. See diagram 1. |

Diagram 1: camera convention for calibrated camera images on Segments.ai.
The pose of the sensor used to capture the 3D point cloud data. This can be helpful if you want to obtain cuboids in world coordinates, or when your sensor is moving. In the latter situation, supplying an ego pose with each frame will ensure that static objects do not move when switching between frames.
{
"position": {
"x": -2.7161461413869947,
"y": 116.25822288149078,
"z": 1.8348751887989483
},
"heading": {
"qx": -0.02111296123795955,
"qy": -0.006495469416730261,
"qz": -0.008024565904865688,
"qw": 0.9997181192298087
}
},
Name | Type | Description |
---|---|---|
position | object : {
"x": float ,
"y": float ,
"z": float
} | Required. XYZ position of the sensor in world coordinates. |
heading | object : {
"qx": float ,
"qy": float ,
"qz": float , "qw": float
} |
{
"frames": [
{ ... },
{ ... },
{ ... }
]
}
Name | Type | Description |
---|---|---|
frames | Required. List of 3D point cloud frames in the sequence. |
{
"text": "Example text sample."
}
Name | Type | Description |
---|---|---|
text | string | Required. Text data. |
Last modified 11d ago