🚧

Prerequisite

Before you start, ensure you have learned how to organize Data for each dataset type in the Data and Folder Structure.

Batch and Scene are essential concepts in BasicAI to help scalably organize and distribute data.

The Batch is a collection of scenes and data, while the Sene is similar to a video constructed by Data via a timeline. Batch and Scene are optional for your zips. You can use the folder name or folder structure to define which level is a Batch or Scene.

Parse ZIP by Folder Name

By default, BasicAI recognizes folders with prefixes "batch" and "scene" as batches and scenes respectively, regardless of their folder level. A batch should be at a higher level than a scene, and a scene should be at a higher level of data.

├──batch_1 // will be recognized as a Batch with the name of bacth_1
│   ├──scene_00001// will be recognized as a Scene with the name of scene_1
│   │   ├── data_1 
│   │   ├── data_2
│   │   ├── data_3
│   ├──scene_00002
│   │   ├── data_1 
│   │   ├── data_2
│   │   ├── data_3
│   ├──scene_00003
│   │   ├── data_1 
│   │   ├── data_2
│   │   ├── data_3

📘

Data folder structure differs from datatype

Check here to learn how to organize your files into data for the different data types.

├──batch_1 // will be recognized as a Batch with the name of bacth_1
│   ├──scene_00001// will be recognized as a Scene with the name of scene_1
│   │   ├── 00.jpg // jpg will be recognized as a Data
│   │   ├── 01.jpg
│   │   ├── 02.jpg
│   ├──scene_00002
│   │   ├── 00.jpg
│   │   ├── 01.jpg
│   │   ├── 02.jpg
│   ├──scene_00003
│   │   ├── 00.jpg
│   │   ├── 01.jpg
│   │   ├── 02.jpg

├──batch_1 // will be recognized as a Batch with the name of bacth_1
│   ├──scene_00001// will be recognized as a Scene with the name of scene_1
│   │   ├── camera_config //files in the camera_config, camera_image_n, and result folder construct a data by file name
│   │   │   ├── data1.json
│   │   │   └── data2.json
│   │   ├── camera_image_0 
│   │   │   ├── data1.jpg
│   │   │   └── data2.jpg
│   │   ├── camera_image_1 
│   │   │   ├── data1.jpg
│   │   │   └── data2.jpg
│   │   ├── camera_image_2 
│   │   │   ├── data1.jpg
│   │   │   └── data2.jpg
│   │   ├── lidar_point_cloud_0 
│   │   │   ├── data1.pcd
│   │   │   └── data2.pcd
│   │   ├── result
│   │   │   ├── data1.json
│   │   │   ├── data1_lidar_point_cloud_0_segmentation.pcd
│   │   │   ├── data2.json
│   │   │   └── data2_lidar_point_cloud_0_segmentation.pcd 

├──batch_1 // will be recognized as a Batch with the name of bacth_1
│   ├──scene_00001// will be recognized as a Scene with the name of scene_1
│   │   ├── 001.MP3
│   │   ├── 002.MP4
│   │   ├── 003.AVI

Parse ZIP by Folder Structure

If you don't want to rename your folder name inside your compressed files, you can select parse ZIP by folder architecture in the configuration of upload data.

In this case, you have the option to select any combination of batch, scene, and data. In more complex scenarios, you can also customize your folder architecture.

When parsing ZIP files with a folder architecture, the first upper level will be recognized as a scene, and the second will be recognized as a batch based on the data level in your zips. For example

├──unimportant_folders // will be ignored
│   ├──level_2// will be recognized as a batch with the name of level_2
│   │   ├── level_3 // will be recognized as a scene with the name of level_3
│   │   │   ├── 001.png //will be recognized as a data
│   │   │   ├── 002.png

Parse ZIP by Customized Folder Structure

If you have a messy compressed file or face a duplicate data error, you can customize each path inside your folder structure and its names. For more details, please refer to Customized ZIP Architecture.