Q: How does your team consider approaching mechanical engineering issues from an observational perspective?
Ahmed: The question we’ve been thinking about is how can generative AI be used in engineering applications? A key challenge is to incorporate precision into generative AI models. Now, in the specific work that we studied there, we utilize the idea of a self-supervised, contrastive learning approach where we effectively learn the representations of the design’s links and curves, or what the design looks like and how it works.
This is very closely related to the idea of automatic discovery: can we really discover recent products using AI algorithms? Another comment on the bigger picture: one of the key ideas, particularly around associations, but generally around generative AI and huge language models – they’re all the same family of models that we’re looking at, and precision really plays a massive role in all of them. So the lessons we draw from these types of models, with some form of data-driven learning supported by engineering simulators and joint embedding of design and performance, could potentially translate to other engineering fields as well. What we show is a proof of concept. Then people can take advantage of that and design ships and planes, and solve problems with precise image generation, and so on.
For connections, your design looks like a set of bars and how they are connected. The working principle is basically a path that they write down as they move, and we learn these common representations. So you have basic input – someone comes in and draws some path – and you try to generate a mechanism that can track it. This allows us to solve the problem in a much more precise way and much faster, with 28 times smaller error (more correct) and 20 times faster than previous state-of-the-art approaches.
Q: Tell me about the linkage method and how it compares to other similar methods.
Nobari: Contrastive learning occurs between mechanisms represented as graphs, so basically each connection will be a node in the graph, and the node will contain some features. The characteristic features are location, space and type of connections. These may be fixed joints or free joints.
We have an architecture that takes some of the basic elements of describing mechanism kinematics into account, but it’s essentially a graph neural network that computes embeddings for these mechanism graphs. We then have another model that takes these curves as input and creates an embedding for them, and then we combine these two different modalities using contrastive learning.
Then this contrasting learning structure that we train is used to find recent mechanisms, but of course we also want precision. In addition to the identified potential mechanisms, we also have an additional optimization step, during which the identified mechanisms will be further optimized to get as close to the target curves as possible.
If you’ve done the combinatorial part correctly and you’re pretty close to where you need to be to get your target curve, you can do direct gradient-based optimization and adjust the location of the connections to get super precise. This is a very vital aspect of work.
These are examples of letters of the alphabet, but they are very tough to obtain traditionally using existing methods. Other machine learning methods often can’t even do this because they are only trained on four or six ticks, which are very miniature mechanisms. However, we managed to show that even with a relatively miniature number of connections it is possible to approach these curvatures.
Previously, we didn’t know what the design limitations were for a single linkage mechanism. This is a very tough question. You can really write the letter M, right? No one has ever done it, and the mechanism is so complicated and so sporadic that you have to find a needle in a haystack. But with this method we show that it is possible.
We looked at the possibility of using ready-made generative models to create charts. In general, generative models for graphs are very tough to train and are usually not very effective, especially when it comes to mixing continuous variables, which are very sensitive to the actual kinematics of the mechanism. At the same time, you have many different ways to connect calls and connections. These models simply cannot generate efficiently.
I think the complexity of the problem is more obvious when you look at the way people approach it through optimization. With optimization, this becomes a mixed-number nonlinear problem. By using uncomplicated two-level optimizations or even simplifying the problem, they essentially create approximations of all the functions so that they can utilize mixed-integer conic programming to approach the problem. The combinatorial space connected to the continuous space is so huge that in principle it can reach as many as seven connections. Moreover, it becomes extremely tough and takes two days to create one mechanism for one specific purpose. If this were done exhaustively, it would be very tough to cover the entire design space. This is where you can’t just throw deep learning at it without trying to be a little smarter about how to do it.
State-of-the-art deep learning approaches utilize reinforcement learning. They – having a target curve – start building these mechanisms more or less randomly, basically using a Monte Carlo optimization method. This is measured by a direct comparison of the curve determined by the mechanism with the target curves entered into the model. We show that our model performs approximately 28 times better. Our approach takes 75 seconds and the reinforcement learning approach takes 45 minutes. The optimization approach is that you run it for longer than 24 hours and it doesn’t converge.
I think we’ve gotten to the point where we have a very solid proof of concept for linkage mechanisms. This is such a sophisticated problem that we see that conventional optimization and conventional deep learning alone are not enough.
Q: What is the bigger picture of the need to develop techniques such as linkages that will enable future human-AI co-design?
Ahmed: The most obvious is the design of machines and mechanical systems, as we have already shown. Having said that, I think the key contribution of this work is that it is a discrete and continuous space that we are learning about. So, if you think about the connections that exist and how those connections are connected to each other, it’s a discrete space. You’re either connected or you’re not: 0 and 1, but where each node is is a continuous space that can change – you can be anywhere in that space. Learning for these discrete and continuous spaces is an extremely tough problem. Much of the machine learning we see, much like computer vision, is continuous or the language is mostly discrete. By showing this discrete and continuous system, I think the key idea can be generalized to many engineering applications, from metamaterials to sophisticated networks, to other types of structures, and so on.
There are steps we think about straight away, and the natural question involves more sophisticated mechanical systems and more physics, such as adding different forms of elastic behavior. Then you can think about different types of components. We are also thinking about how precision can be incorporated into huge language models and some of the knowledge gained will be transferred there. We are looking at making these models generative. Right now, they’re kind of taking the mechanisms and then optimizing them based on the dataset, while generative models will generate these methods. We are also exploring the possibility of end-to-end learning when optimization is not necessary.
Nobari: There are several places in mechanical engineering where they are used, and systems of this type of inverse kinematic synthesis are very common where it would be useful. A few that come to mind are, for example, car suspension systems where a specific path of motion is required for the entire suspension mechanism. They usually model this in 2D using planning models of the entire suspension mechanism.
I think the next step that will ultimately be very useful is to demonstrate the same or similar framework for other sophisticated problems that involve combinational and continuous values.
These issues include one of the things I’ve been looking at: compliance mechanisms. For example, if we have the mechanics of continuous – instead of these discrete – stiff joints, we get a distribution of materials and movement, and one part of the material deforms the rest of the material, giving a different type of movement.
In the case of compliant mechanisms, they are used in a wide variety of places, sometimes in precision machinery for fastening mechanisms, where a specific component is needed to be held in place by a mechanism that fastens it, which can do so consistently and with very high precision . If you could automate a lot with this kind of framework, it would be very useful.
These are all tough problems that involve both combinatorial design variables and continuous design variables. I think we are already very close to this goal and ultimately this will be the last stage.