Scaling Robot Simulation: Using AI-Generated Worlds for Domain Randomization

The traditional bottleneck in robot training has long been the manual creation of high-fidelity 3D environments. Traditionally, building a single scene in engines like Unity or Unreal Engine could take days of manual effort. By integrating World Labs Marble AI with NVIDIA Omniverse Isaac Sim, that timeline can be compressed into minutes.

Accelerating Domain Randomization

AI-driven generation allows for the rapid creation of massive environmental variation. A single world can be generated in just 6 to 10 minutes, providing a foundation for extensive randomization of several key factors:

• Lighting and atmospheric conditions
• Room layouts and spatial configurations
• Clutter placement and object textures

This approach is exceptionally effective for domain generalization, ensuring that trained policies are robust enough to handle the unpredictability of the real world.

Practical Application: Navigation and Locomotion

Testing this workflow involved adapting standard Isaac Sim tasks, such as the orange picker scene, to run within Marble-generated environments. While this already assists with domain randomization, the most significant impact lies in movement and locomotion-based tasks.

Exposure to diverse geometry, varying scales, and complex layouts is critical for training robust navigation policies. In these scenarios, the ability to iterate through hundreds of unique layouts outweighs the need for bespoke, manually authored environments.

Technical Constraints and Fidelity

While the speed of generation is a significant leap forward, there are technical trade-offs to consider regarding collision physics:

• Collision Mesh Fidelity: Major structures and large appliances generally have usable colliders. However, smaller decorative elements like flower pots or fine details remain inconsistent.
• Task Suitability: Currently, these environments are ideal for navigation. For precision manipulation tasks, collision refinement and higher-fidelity meshes are still required.

The Future of Data-Driven Robotics

Even with current fidelity limitations, this workflow represents a major advancement over manual authorship. Instead of spending days building a single environment, developers can now focus on scaling data diversity. This shift toward automated, diverse scene generation is exactly what modern robot learning pipelines require to bridge the gap between simulation and reality.