5. Bring Your Own Data

This section explains how to format your datasets for use with Tiny ML ModelZoo. The toolchain supports various data formats and automatically handles preprocessing.

Contents

• 5.1. Classification Dataset Format
  • 5.1.1. Directory Structure
  • 5.1.2. Data File Format
  • 5.1.3. Supported File Types
  • 5.1.4. Annotations (Optional)
  • 5.1.5. Configuration
  • 5.1.6. Dataset Splitting Modes
  • 5.1.7. Example: 3-Class Vibration Data
  • 5.1.8. Class Balancing
  • 5.1.9. Common Issues
  • 5.1.10. Key Differences: Classification vs Regression vs Forecasting
• 5.2. Regression Dataset Format
  • 5.2.1. Directory Structure
  • 5.2.2. Data File Format
  • 5.2.3. Time Column Handling
  • 5.2.4. Annotation Files (Optional)
  • 5.2.5. Configuration
  • 5.2.6. Target Processing
  • 5.2.7. Complete Example
  • 5.2.8. Troubleshooting
  • 5.2.9. Best Practices
  • 5.2.10. Common Issues
  • 5.2.11. See Also
• 5.3. Forecasting Dataset Format
  • 5.3.1. Directory Structure
  • 5.3.2. Data File Format
  • 5.3.3. Key Difference from Regression
  • 5.3.4. Configuration
  • 5.3.5. Variable Specification Options
  • 5.3.6. Windowing Behavior
  • 5.3.7. Complete Example
  • 5.3.8. Important Notes
  • 5.3.9. Minimum Data Requirements
  • 5.3.10. Common Issues
  • 5.3.11. Annotations and Splits
  • 5.3.12. See Also
• 5.4. Anomaly Detection Dataset Format
  • 5.4.1. Folder Structure
  • 5.4.2. Concrete Example
  • 5.4.3. Data Splitting Strategy
  • 5.4.4. What If You Don’t Have Anomaly Data?
  • 5.4.5. Datafile Format (CSV)
  • 5.4.6. See Also
• 5.5. Audio and Radar Dataset Format
  • 5.5.1. Audio Classification
  • 5.5.2. Radar Classification
  • 5.5.3. See Also
• 5.6. Data Splitting
  • 5.6.1. Split Methods
  • 5.6.2. Configuration
  • 5.6.3. Annotation File Format
  • 5.6.4. Split Examples
  • 5.6.5. When to Use Each Method
  • 5.6.6. Best Practices
  • 5.6.7. Creating Annotation Files
  • 5.6.8. Verifying Splits
  • 5.6.9. See Also

5.7. Dataset Format Overview

Tiny ML ModelZoo uses different folder structures depending on the task type:

Classification Tasks

dataset_name/
├── classes/
│   ├── class1/
│   │   ├── file1.csv
│   │   └── file2.csv
│   ├── class2/
│   │   └── file1.csv
│   └── classN/
└── annotations/          # Optional - auto-generated if missing
    ├── instances_train_list.txt
    └── instances_val_list.txt

Anomaly Detection Tasks

dataset_name/
├── classes/
│   ├── Normal/           # Training data (normal operation only)
│   │   ├── file1.csv
│   │   └── file2.csv
│   └── Anomaly/          # Test-only data (fault/anomaly samples)
│       ├── file1.csv
│       └── file2.csv
└── annotations/          # Optional - auto-generated if missing

See Anomaly Detection Dataset Format for full details.

Regression & Forecasting Tasks

dataset_name/
├── files/               # MUST be named "files"
│   ├── datafile1.csv
│   └── datafileN.csv
└── annotations/         # Required for these tasks
    ├── instances_train_list.txt
    └── instances_val_list.txt

Audio and Radar Classification Tasks (new in 1.5.0)

Both use the same classes/<class_name>/ layout as time series classification. Audio classes hold .wav/.flac recordings and radar classes hold .csv/.npy files. See Audio and Radar Dataset Format.

Splits are described in Data Splitting. If no annotation files are found, the split lists are generated automatically (train, val and test).

5.8. Supported File Formats

• CSV files (.csv) - Most common, human-readable

• Text files (.txt) - Same format as CSV

• NumPy arrays (.npy) - Binary format, faster loading

• Pickle files (.pkl) - Python serialized pandas DataFrames

The list above applies to time series tasks. Audio tasks read .wav and .flac; radar tasks read .csv and .npy.

5.9. Data Sources

You can provide your dataset as:

• A local directory path

• A local ZIP file path (extracted next to the ZIP file)

• A remote URL to a ZIP file (automatically downloaded)

Example:

dataset:
  dataset_name: my_dataset
  input_data_path: '/path/to/dataset'  # or URL

For working configs see How to Create a ModelZoo Config YAML. Examples of every dataset layout are in the Examples & Applications section.