Ocean full of life. But until you approached, a significant part of the sea world can easily remain concealed. This is because the water itself can act as an effective coat: the featherlight that shines through the ocean, can bend, distract and quickly disappear when it travels through a dense water medium and reflects from a robust mist of ocean particles. This means that grasping the real color of objects in the ocean is extremely complex without imaging them from close range.
Now the MIT and Woods team Hole Oceanographic Institution (WHOI) has developed a tool for image analysis that crosses the optical effects of the ocean and generates images of underwater environments that look like water was discharged, revealing the real colors of the ocean scene. The band connected a color correction tool with a computing model, which transforms the scene images into a three -dimensional underwater “world”, which can then be examined virtually.
Scientists called the fresh tool “Seasplat”, in relation to its underwater application and the method known as 3D Gaussian Spllatting (3DGS), which takes pictures of the stage and staples them together to generate a full, three -dimensional representation that can be viewed in detail, from any perspective.
“Thanks to Seasplat, it can clearly model what water does, as a result of which it can in a sense remove water and produces better 3D models of the underwater stage,” says Mit Daniel Yang.
Scientists applied Seasplat to paintings of the sea floor made by divers and underwater vehicles, in various locations, including on the Virgin Islands. The method generated 3D “worlds” from images that were real and more bright and diverse, compared to previous methods.
The team claims that Seasplat can aid sea biologists monitor the health of some ocean communities. For example, when an underwater robot examines and takes photos of the coral reef, Seasplat would simultaneously process images and make a real colorful 3D representation that scientists could practically “fly”, at their own pace and path to check the underwater scene, for example in terms of coral whitening signs.
“Billen looks white from close -up, but it may seem blue and foggy from afar, and you may not be able to detect it,” says Yogesh Girdhar, an associate scientist in Whoi. “Balling corals and various species of coral can be easier to detect using Seasplat images to get real colors in the ocean.”
Girdhar and Yang will present Seasplat paper details at the IEEE international conference on robotics and automation (ICRA). Their co -author of the research is John Leonard, professor of mechanical engineering in MIT.
Water optics
In the ocean, the color and transparency of objects are distorted by the effects of the featherlight moving through the water. In recent years, scientists have developed color correction tools that are aimed at reproducing real colors in the ocean. These efforts included adapting tools originally developed to water from water, for example to reveal the true color of features in foggy conditions. One of the last works accurately reproduces the real colors in the ocean, with an algorithm called “Sea-Thru”, although this method requires enormous computing power, which means that it uses its apply of 3D scenes models.
At the same time, others have made progress in 3D Gaussian Spllatting, with tools that smoothly sew the images of the scene together, and intelligently fill all gaps to create the entire version of the 3D scene. These 3D worlds allow “synthesis of innovative view”, which means that someone can watch the generated 3D scene, not only from the perspective of original images, but from any point of view and distance.
But 3DGS was successfully used only for water from water. Efforts to adapt 3D reconstruction to underwater images have been hindered, mainly by two optical underwater effects: reverse dispersion and suppression. Reverse dispersion occurs when the featherlight reflects from petite particles in the ocean, forming a fog -like curtain. Damping is a phenomenon in which the featherlight of some wavelengths suppresses or disappears from a distance. For example, in the ocean, red objects seem to disappear more than blue objects when you look at further distances.
From the water, the color of the objects appears more or less the same, regardless of the angle or distance from which they are viewed. However, in the water, the color can change quickly and disappear depending on the perspective. When 3DGS methods try to sew underwater images into a coherent 3D whole, they are not able to solve objects due to water reverse dispersion and suppression effects, which distort the color of objects at different angles.
“One dream of an underwater robotic vision that we have is: Imagine, would you be able to remove all the water in the ocean. What would you see?” Leonard says.
The model is swimming
In the fresh work, Yang and his colleagues have developed a color correction algorithm that takes into account the optical effects of reverse dispersion and damping. The algorithm determines the degree to which each pixel in the image had to be distorted by reverse dispersion and suppression effects, and then basically removes these water effects and the face that the real color of the pixel must be.
Then Yang has developed a color correction algorithm in the Gauss 3D model to create Seasplat, which can quickly analyze underwater scene images and generate a real virtual 3D version of the same scene that can be examined in detail at any angle and distance.
The team used Seasplat to many underwater scenes, including paintings made in the Red Sea, in Carriban off the Curaçao coast and the Pacific Ocean near Panama. These images that the team took from a previously existing set of data present a number of ocean locations and water conditions. They also tested Seasplat in photos taken by a remote -controlled underwater robot in the US Virgin Islands.
From the paintings of each ocean scene, Seasplat generated the real 3D world, which scientists were able to virtually explore, for example, enlarging the stage and leaving some functions from different perspectives. Even while watching at different angles and distances, they found objects in each stage, they retained their real color, not fading, as it is if they are viewed by the actual ocean.
“After generating a 3D scientist, he can simply” swim “through the model, as if they dive, and look at things in detail, with real color”, “says Yang.
For now, the method requires strong computing resources in the form of a desktop computer that would be too bulky to transfer underwater robot on board. Despite this, Seasplat can act on imprisonment operations in which the vehicle, associated with the ship, can examine and take photos that can be sent to the ship’s computer.
“This is the first approach that can quickly build high -quality 3D models with exact colors, under water, and can create them and render them quickly,” says Girdhar. “This will aid estimate the biological diversity and assess the health of the coral reef and other marine communities.”
These works were partly supported by the investment in the Scientific Fund in WHOI and by the National Science Foundation.