Upload Data
📦 Guide to mastering data uploads with BasicAI
After you have Created a Dataset, you must upload your data to the dataset before annotation. BasicAI provides four convenient methods to upload your data. Click the Upload in the top right corner.
Prepare the Data
Sample datasets
You can find sample datasets for various scenarios that can be directly uploaded to BasicAI. You can copycat our samples if you don't want to go through our detailed documentation.
📖 Image Sample Datasets
| Description | Configurations | Download Link |
|---|---|---|
| Upload images without any configs | N/A | Download📁 |
| Upload Video in Image with frame extraction | 1. Frame Extraction= true | Download📁 |
| Upload images with pre-label results(BasicAI format) | 1. Import annotations=true 2. Annotation format=Basic AI Image Format | Download📁 |
| Upload images with pre-label results(COCO format) | 1. Import annotations=true 2. Annotation format=COCO | Download📁 |
| Upload Images with pre-label results (Pascal VOC XML) | 1. Import annotations=true 2. Annotation format=Pascal VOC XML | Download📁 |
| Upload images with Batch, Scene with BasicAI desired folder names | 1. Parse ZIP by = Folder Name (default) | Download📁 |
| Upload images with customized Batch, Scene folder names | 1. Parse ZIP by = Folder Architecture(Batch, Scene, and Data) | Download📁 |
📖 LiDAR Fusion Sample Datasets
| Description | Configurations | Download Link |
|---|---|---|
| Upload LiDAR without any configs | N/A | Download📁 |
| Upload LiDAR with RGB color | N/A, but RGB value is required in PCDs | Download📁 |
| Upload LiDAR without Images | N/A, compatible with deprecated LiDAR Basic dataset | Download📁 |
| Upload LiDAR with pre-label results(Instance, BasicAI format) | 1. Import annotations=true 2. Annotation format=BasicAI LiDAR Fusion Format | Download📁 |
| Upload LiDAR with pre-label results(Segmentation, BasicAI format) | 1. Import annotations=true 2. Annotation format=BasicAI LiDAR Fusion Format | Download📁 |
| Upload LiDAR with pre-label results(Segmentation & Instance, BasicAI format) | 1. Import annotations=true 2. Annotation format=BasicAI LiDAR Fusion Format | Download📁 |
| Upload LIDAR with pre-label results(KITTI format) | 1. Import annotations=true 2. Annotation format=Kitti 3D Object Detection | Find in KITTI |
| Upload LiDAR with Batch, Scene with desired folder names | 1. Pharse ZIP by = Folder Name (default) | Download📁 |
| Upload images with customized Batch, Scene folder names | 1. Pharse ZIP by = Folder Architecture(Batch, Scene, and Data) | Download📁 |
| Upload LiDAR Fusion with 4D BEV format | 1. Data type=4D BEV | Contact us! |
📖 Audio and Video Sample Datasets
| Description | Configurations | Download Link |
|---|---|---|
| Upload audio and video without any configs | N/A | Download📁 |
| Upload audio and video with pre-label results(BasicAI format) | 1. Import annotations=true 2. Annotation format=Basic AI AV Format | Download📁 |
| Upload audio and video with Batch with BasicAI desired folder names | 1. Pharse ZIP by = Folder Name (default) | Download📁 |
| Upload images with customized Batch folder names | 1. Pharse ZIP by = Folder Architecture(Batch, and Data) | Download📁 |
📖 Text Sample Datasets
📖 Generative AI Sample Datasets
| Description | Configurations | Download Link |
|---|---|---|
| Upload generative AI datasets without any configs | N/A | Download📁 |
Organize your compressed files
If you have problems organizing your compressed files, please refer to Data and Folder Structure.
Additionally, BasicAI offers data conversions for different types of convertible and reusable data formats for enterprise users only. However, now, it will be free for all team and enterprise users. Please feel free to contact us on Slack.
Choose the Way
Datasets can be uploaded from the Local drive, through URLs, Object Storage (MinIO Bucket), and My Cloud Storage.
📖 Upload from Local drive
As a quick start, you can directly select your files from your local device and upload them to the dataset.
The maximum file you can directly upload is 1GB(1024MB). For the larger files, we highly recommend you use URLs or Cloud Storages to upload your files. Just drag or drop your data!
📖 Upload by URLs
It will be handy if you already have your data hosted on any public cloud storage, such as AWS S3, Azure Blob, or Google Cloud Platform, etc.
If you are using Cyberduck connecting with S3,
- you can directly right-click on your file and Copy as a HTTPS URL
- Then paste it here.
The way to access the file URLs of different public cloud storage services like GCP and Azure should be similar. Check the major cloud services' documentation
📖 Upload from BasicAI's Cloud
BasicAI offers free object storage of up to 10GB per team
This feature is specifically designed for teams that do not have their own cloud storage but still need to upload large files. If you need upgrade the storage, please contact us!
- Firstly, upload your files to our object storage. Please refer to the MinIO-Guide page.
- Just select and upload it!
📖 Upload from My Cloud Storage
Have your own cloud storage? Integrate it with BasicAI !
My Cloud Storage now supports AWS S3, Google Drive and Dropbox. It supports connections to a wide range of data sources where you can add them in My Team page Cloud Storage tab.
After a successful connection, you can browse the folders in the cloud storage and directly select the compressed data file to complete the upload.
Dataset Upload Tips
- For non-enterprise users, the maximum supported file size for each upload is 40GB when compressed. If you need to upload a larger file, please contact us!
- You can directly upload supported files for image, audio, and video datasets.
- If you use cloud storage or URLs, data transfer will be much faster.
- Occasionally, uploads may become stucked at 40% while waiting in a queue for other clients' compressed files to be processed.
- BasicAI only allows duplicates across the different Batch or Scene. If you encounter a duplicate data error while attempting to upload duplicate data, please refer to our Customized Zip Architecture guide.
- If you are getting errors with uploading, you can hover the error to see the error messages
Configure the Uploads
Several configurations can apply to your compressed files to define the folder structure and pre-label results format. You can click the config switch button to configure your uploads.
Parse ZIP by
This configuration defines how BasicAI recognizes batch and scene in your folder structure. You can define BasicAI to recognize batch and scene by folder name or architecture or customize the structure.
Data and annotation
Annotation format
This configuration defines how BasicAI recognizes your folder structure at the data and annotation level. formats displayed here differ from datatype. For instance, the KITTI format is only available in the LiDAR Fusion dataset, while the COCO format is only available in the Image dataset. The well-known and common formats are 100% compatible with BasicAI formats. For more details about how BasicAI converts to those formats, please refers
Customize Formats
For enterprise users, we offers free import and export scripts to meet all your import and export requirements
Import annotations
This configuration specifies if and where the annotations are imported. Specify whether the result is from the model (model runs) or from humans (ground truths), then enter a name for the result. After uploading, you can find your pre-label results in the source.
Additionally, you can add the pre-label results or results from other tasks to a specific task as their default results.
Frame extraction(Image dataset only)
You will see the frame extraction configuration when uploading a video or zip file in an image dataset. This determines how BasicAI extracts the frames from the video and converts them into data.
- By second: each second will be converted to a data
- By frame: each frame will be converted to a data
- Customized: you can customize how many frames are extracted from how many seconds.
Data type(LiDAR Fusion only)
You need to clarify a more specific data type between 4D BEV and 3D Point Cloud if you are uploading data into a LiDAR Fusion dataset. Keep it as default if you are not uploading as 4D BEV format
Updated 8 months ago
🎯 Hopefully, you have uploaded the data as desired!