UE_SensorSimulator

Unreal Engine Plugin for Synthetic Data Generation

Features

• Realtime lidar sensor (scanning the world level via LineTracer)
• Depth map, Point cloud (optional color mapping for attached camera), Camera RGB image
• Supporting async generation
• Supporting UDP communication for the sensor data

Quick Install & Setup

1. Download Latest Release
2. Create new or choose project.
3. Browse to your project folder (typically found at Documents/Unreal Project/{Your Project Root})
4. Copy Plugins folder into your Project root.
5. Plugin should be now ready to use.

DEMO

Overview

Digital Twin

Implementing corner case scenarios such as more than 40 weather, brightness changes according to time, and extreme weather

Segmetation

Collecting synthetic data for training the Segmentation model (PIDNet) and utilizing Unreal Engine's stencil buffer functionality to extract Semantic maps, which will be used as target data.

Point Cloud

Implementing an LIDAR sensor in Unreal Engine to capture RGB and Depth information, transmitting this data to Python via UDP communication, and then implementing a Point Cloud in Panda3D.

Sensor Calibration

Calibrating the PointCloud generated by the Lidar Sensor with the imagery from the RGB Sensor to create a reference image. Using this calibrated image to enhance the reliability and accuracy of the depth estimation model.