Generative AI models have made such an indelible impact on digital content creation that it’s becoming increasingly complex to remember what the Internet was like before. You can operate these AI tools for clever projects like videos and photos, but their artistic talents haven’t yet transferred to the physical world.
So why haven’t we yet seen personalized AI-powered objects like phone cases and flower pots in places like homes, offices and stores? According to researchers from the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL), the key issue is the mechanical integrity of the 3D model.
While AI can facilitate generate personalized, manufacturable 3D models, these systems often do not take into account the physical properties of the 3D model. MIT Electrical and Computer Science graduate student and CSAIL engineer Faraz Faruqi explored this trade-off, creating generative AI-based systems that can make aesthetic changes to designs while maintaining functionality, and another that modifies structures with the desired tactile properties that users want to feel.
Making it happen
Together with researchers from Google, Stability AI and Northeastern University, Faruqi has found a way to create real-world objects using artificial intelligence, creating items that are both strong and exhibit the user’s intended appearance and texture. Thanks to the artificial intelligence-basedMechStyle“, users simply upload a 3D model or select a pre-set asset such as vases and hooks, and are prompted via images or text to create a personalized version. The generative AI model then modifies the 3D geometry, while MechStyle simulates how these changes will affect individual parts, ensuring that sensitive areas remain well. If you’re satisfied with this AI-enhanced design, you can 3D print it and operate it in the real world.
You can choose the model of, say, a wall hook and the material from which you will print it (for example, plastics such as polylactic acid). You can then ask the system to create a personalized version using prompts like “generate a cactus-like hook.” The AI model will work with the simulation module and generate a 3D model resembling a cactus while having the structural properties of a hook. This green, ribbed accessory can then be used to hang mugs, coats and backpacks. Such creations are made possible in part by the styling process, during which the system changes the geometry of the model based on understanding text prompts and working with feedback received from the simulation module.
According to CSAIL researchers, 3D styling had unintended consequences. Their initial study found that only about 26 percent of the 3D models remained structurally sound after modification, meaning the AI system did not understand the physics of the models it was modifying.
“We want to use artificial intelligence to create models that can be produced and used in the real world,” says Faruqi, the lead author of the project paper project presentation. “So MechStyle actually simulates how GenAI-based changes will affect the structure. Our system allows you to personalize the tactile experience of your item, incorporating your personal style into it, while ensuring that the item will withstand everyday use.”
This computational accuracy could ultimately facilitate users personalize their belongings, creating, for example, a unique pair of glasses speckled with blue and beige dots resembling fish scales. A pill box was also created with a rocky checkered texture with pink and aqua splashes. The potential of the system extends to creating unique home and office decorations, such as a lampshade resembling red magma. It can even design assistive technology tailored to the user’s specifications, such as finger splints to facilitate with dexterity injuries and utensil holders to facilitate with motor impairments.
In the future, MechStyle may also be useful for creating prototypes of accessories and other handheld products that you can sell in a toy store, hardware store, or craft boutique. The goal, CSAIL researchers say, is for both experts and novice designers to spend more time brainstorming and testing different 3D designs, rather than manually assembling and adjusting elements.
Stay sturdy
To ensure MechStyle’s creations can withstand everyday operate, the researchers expanded their generative artificial intelligence technology to include a type of physical simulation called finite element analysis (FEA). You could imagine a 3D model of an object, such as a pair of glasses, with some kind of heat map indicating which regions are structurally feasible at a realistic weight and which are not. As AI refines this model, physics simulations highlight which parts of the model are becoming weaker and prevent further changes.
Faruqi adds that running these simulations every time a change is made slows down the AI process dramatically, so MechStyle is designed to know when and where to perform additional structural analyses. “MechStyle’s adaptive planning strategy tracks what changes occur at specific points in the model. When the genAI system makes adjustments that compromise certain areas of the model, our approach re-simulates the physics of the design. MechStyle will make subsequent modifications to ensure the model does not break after production.”
Combining the FEA process with adaptive planning enabled MechStyle to generate facilities that were 100% structurally feasible. By testing 30 different 3D models with styles resembling bricks, stones, and cacti, the team found that the most effective way to create structurally viable objects is to dynamically identify tender regions and improve the generative AI process to mitigate its effects. In these scenarios, researchers have found that you can either stop styling completely once a certain stress threshold is reached, or gradually make smaller improvements to prevent the at-risk areas from getting closer to that level.
The system also offers two different modes: a freestyle function that allows the artificial intelligence to quickly visualize different styles on a 3D model, and a MechStyle mode that carefully analyzes the impact of changes on the structure. You can explore different ideas and then try out MechStyle mode to see how these artistic additions will affect the durability of specific areas of the model.
CSAIL researchers add that while their model can ensure that the model remains structurally sound before 3D printing, it cannot yet improve 3D models that were not feasible to begin with. If you upload such a file to MechStyle, an error message will appear, but Faruqi and his colleagues intend to improve the durability of the faulty models in the future.
Furthermore, the team hopes to operate generative AI to create 3D models for users, rather than styling presets and designs submitted by users. This would make the system even more user-friendly, and those who are less familiar with 3D models or cannot find their design on the Internet could simply generate it from scratch. Suppose you want to produce a unique type of bowl and this 3D model is not available in the repository; Instead, artificial intelligence can create it for you.
“While style transfer to 2D images works incredibly well, little work has explored how this transfers to 3D,” says Google scientist Fabian Manhardt, who was not involved in the paper. “Generally, 3D is a much more difficult task because training data is scarce and changing the geometry of an object can damage its structure, rendering it unusable in the real world. MechStyle helps solve this problem by enabling 3D styling without compromising the structural integrity of the object through simulation. This empowers people to be creative and express themselves better through products tailored to them.”
Farqui wrote the paper with senior author Stefanie Mueller, who is an MIT associate professor and principal investigator of CSAIL, and two other CSAIL colleagues: researcher Leandra Tejedor SM ’24 and graduate student Jiaji Li. Their co-authors are Amira Abdel-Rahman PhD ’25, currently an assistant professor at Cornell University, and Martin Nisser SM ’19, PhD ’24; Google Researcher Vrushank Phadnis; Vice President of AI Stability Research Varun Jampani; MIT Professor and Director of the Bits and Atoms Center Neil Gershenfeld; and Northeastern University assistant professor Megan Hofmann.
Their work was supported by the MIT-Google Computer Innovation Program. It was presented in November at the Association for Computing Machinery Symposium on Computational Manufacturing.