Observe
An image supplies appearance. ShapeCodec turns coarse 3D occupancy into compact text tokens.
IMAGE + COARSE GEOMETRY THINK BEFORE YOU GENERATE
ShapeMind
Native 3D Understanding and Generation
of Semantically Structured 3D Assets
1 ShanghaiTech University2 VAST
Understand the parts. Generate the geometry.
Bring the structure to life.
From visual evidence to meaningful, editable, movable parts.
ShapeVLM + Semantic PartGenTHE IDEA
Structure comes first.
3D assets need more than a surface.
They need to know what they are made of.
ShapeMind generates low-poly 3D assets with explicit semantic part structure and, where applicable, kinematic structure. ShapeVLM reasons over an image and coarse geometry encoded as ordinary text. Its predicted part boxes, labels, and descriptions guide Semantic PartGen to create native triangle meshes, with each part independently editable.
Understand
ShapeVLM predicts semantic parts, their locations, and how groups of parts move together.
PARTS + KINEMATICSBuild
Semantic PartGen jointly generates part vertices and topology in one shared object frame.
NATIVE PART MESHES01 / STRUCTURE IN MOTION
Same parts.
Same motion.
From predicted voxels to generated meshes,
the articulated structure stays connected.
A
Reference image
Visual input
B
Predicted voxels
ShapeVLM · part geometry
C
Generated mesh
Semantic PartGen · native triangles
Loading selected result…Drag to orbit · scroll to zoom · cameras stay in sync
Selected ShapeMind outputs from PhysX-Bench. Both views use the same predicted joint graph and ranges. Playback is a kinematic visualization, not a physics simulation.
02 / NATIVE GEOMETRY
Made of parts.
Built with triangles.
Explore the actual surfaces and topology.
Each color is a separate, editable component.
FROM GENERATED PARTS TO ARTICULATION
A hand that can grasp.
The original 32 meshes become five articulated fingers. Each finger bends around the centers of its generated joints, following a three-joint hierarchy.
32 semantic meshes15 rotational joints
Articulation authored on the existing generated geometry. Joint pivots are fitted to the named joint meshes; the original triangles are unchanged.
Watch the hand in motion ▷
One continuous render: the generated hand opens and closes, then reveals its original triangle edges.
Selected meshes from the supplied Blender gallery. Wireframe follows the original triangle edges; no remeshing or subdivision is applied. Exploded views only translate parts for inspection.
03 / AGENT-DIRECTED CREATION
Understand first.
Then generate.
An agent can speak the same part interface:
an image, semantic descriptions, and 3D boxes.
Independent agents design complex objects, generate reference images, plan semantic parts and joint hierarchies, then call Semantic PartGen. The generated components are bound to those agent-authored joints to create articulated assets.
DESIGN BRIEF
01
Imagine
Generated reference image
02
Plan the parts
Labels + descriptions + 3D boxes
03
Generate + articulate
Generated meshes + agent-authored joints
The agent supplies part conditions and the kinematic graph; these examples bypass ShapeVLM. Semantic PartGen generates the meshes. Joints are agent-authored and aligned to the actual output geometry. Playback demonstrates kinematics, not a dynamics simulation.