9.6 KiB
IMPORTANT: This project has been discontinued and is no longer supported by Unity. Please use the Github issues page to seek community support.
Join our Mailing List! Sign up to stay up-to-date on our latest feature release, community events, blog posts, and more!
The Perception package provides a toolkit for generating large-scale synthetic datasets for computer vision training and validation.
Getting Started with the Perception Package
🌟 Perception Synthetic Data Tutorial 🌟
Ideal for those new to either Unity, the Perception package, or synthetic data in general. Detailed instructions covering all the important steps: installing the Unity Editor, creating your first synthetic data generation project, adding domain randomization, visualizing, and analyzing your generated datasets. No prior experience required.
Quick Start Guide
Instructions for adding the Perception package to your existing project. Recommended for users already familiar with Unity and the Perception package. If this is the first time exploring our toolset, we highly recommend briefly going through the Perception Synthetic Data Tutorial below!
Human Pose Labeling and Randomization Tutorial
Step by step instructions for using the keypoint, pose, and animation randomization tools included in the Perception package. We recommend that you complete the Perception Tutorial above before diving into this guide!
Analyzing Datasets with Pysolotools
Introduction to Unity's pysolotools - a python package for parsing, converting, and analyzing a SOLO dataset. Step by step instructions on how to parse a SOLO dataset and look at the dataset statistics in a jupyter notebook.
Visualizing a Dataset with Voxel51 Viewer
We will walk through using Voxel51 with our custom SOLO importer to visualize a dataset.
Converting to COCO
We will use pysolotools to convert a SOLO dataset into the COCO format.
Frequently Asked Questions
Check out our FAQ for a list of common questions, tips, tricks, and sample code for common code patterns.
Reference Documentation
In-depth documentation on specific components of the package. For the full set of feature-based documentation, take a look at the Features page.
| Feature | Description |
|---|---|
| Labeling | A component that marks a GameObject and its descendants with a set of labels |
| Label Config | An asset that defines a taxonomy of labels for ground truth generation |
| Perception Camera | Captures RGB images and ground truth from a Camera. |
| Dataset Capture | Ensures sensors are triggered at proper rates and accepts data for the JSON dataset. |
| Accumulation / Path Tracing | Accumulation is a feature that can be used for rendering techniques that require frames to be accumulated, such as path tracing. |
| Randomization | The Randomization tool set lets you integrate domain randomization principles into your simulation. |
| Output Endpoint and Dataset Schema | An endpoint is responsible for delivering the generated data to the user. Currently supported output endpoints are: Solo endpoint (default), Perception endpoint, and No Output. |
Datasets and Sample Projects
Synthetic Homes
Head over to the Synthetic Homes repository for a 100K image dataset of annotated synthetic home interiors for the purpose of training computer vision models, as well as a configurable Unity based stand alone application for generating such datasets.
Synthetic Humans
The Synthetic Humans package gives you the ability to procedurally generate and realistically place diverse groups of synthetic humans in your Unity Computer Vision projects.
People Sans People
PeopleSansPeople is a human-centric privacy-preserving synthetic data generator with highly parametrized domain randomization. This dataset generator contains simulation-ready 3D human assets, a parameterized lighting and camera system, and generates 2D and 3D bounding box, instance and semantic segmentation, and COCO pose labels.
SynthDet
SynthDet is an end-to-end solution for training a 2D object detection model using synthetic data.
Robotics Object Pose Estimation Demo
The Robotics Object Pose Estimation project demonstrates pick-and-place with a robot arm in Unity. It includes using ROS with Unity, importing URDF models, collecting labeled training data using the Perception package, and training and deploying a deep learning model.
Community and Support
This project has been discontinued and is no longer supported by Unity. Please use the Github issues page to seek community support.
License & Citation
The Perception package is licensed under Apache License Version 2.0. If you find this package useful, consider citing it using:
@misc{unity-perception2022,
title={Unity {P}erception Package},
author={{Unity Technologies}},
howpublished={\url{https://github.com/Unity-Technologies/com.unity.perception}},
year={2020}
}