Preserving ideas beyond the screen: How AI is changing the way we create in 3D

A handmade sculpture or perhaps an interesting object you found from one of your travels? These objects may carry a story you’d love to preserve. While you might take photographs of your object from a few choice angles, there’s no real way to truly understand the object in its entirety with just a couple of pictures. There are also limited avenues where you can integrate this object into any digital projects. Enter image-to-3D AI. With new developments in 3D modelling, users can now convert their 2D images to digital 3D objects, allowing them to study, tweak, and utilize the object for various creative uses!

Of course, 3D modelling isn’t just the domain of professional studios anymore. Thanks to image-to-3D AI, museums, teachers, as well as everyday hobbyists and artists can leverage this tool in various ways. You can gain a deeper appreciation for a 3D object and use it to develop new visual resources and personal projects. The output may not be the same depending on your source image and how complicated the object may be, but the end results can be a great jumping-off point for digital creation.

One of the great differences between a 2D image and a 3D model of the subject is perspective. Being able to see an object in 3D opens up new possibilities to the viewer. You can rotate and check the object’s proportions and even inspect the object’s surface from all kinds of different angles that aren’t possible in a 2D photograph.

Imagine you’re a history student doing a case study on ancient pottery. You may have a picture of a pottery vessel, showing its decoration and profile. However, what if you were to model that pottery vessel in 3D? The experience of studying the object in its 3D form will give you a better understanding and appreciation of the object’s shape. Likewise, if you’re an artist who’s made a handmade object and would like to document the work, a 3D model could allow you to take that image and translate that object into a 3D creative use of your own.

3D models have traditionally been hard to produce if you don’t have knowledge of the 3D software or the time to make one. Nowadays, with the advent of AI tools, 3D modeling has become more approachable in that it will read your image and extrapolate a 3D representation for you, complete with its surface and geometry.

AI-generated 3D models are best suited if you need an object to experiment with an idea or as a placeholder. It also provides an excellent opportunity for those who aren’t as experienced in 3D modelling to get a taste of what 3D modelling has to offer.

An image-to-3D AI tool works by analyzing your 2D image, be it an illustration or a photograph, and extracting the visual information from that image. AI image-to-3D systems will estimate the shape of the object based on that information and create a 3D model for you.

There’s one big caveat to an AI image-to-3D conversion; the technology will only be able to capture and translate the visual information that the 2D source provides. If your photograph of a decorative object only shows the front, the AI can only estimate what’s on the back, which may not reflect what the object actually looks like.

In order to get a better representation and understanding of your object’s proportions and shape, you may want to consider submitting multiple photos of the same object from different angles. You might also want to consider the image’s quality, such as whether there’s sufficient lighting, distinct image silhouettes, and simple backgrounds. These factors can serve as a great reference for the AI to work with.

Once you have your model, you can open it in your 3D viewer to inspect the object’s form, check to see which parts may need some adjustment, and whether it requires any touch-ups before you export the model.

If you’d like to try out AI image-to-3D modelling for yourself, consider giving Meshy’s image-to-3D feature a spin. It will generate your 2D images into textured 3D models right from your web browser. It will accept a wide range of images, from your 2D references, concepts, and characters to photographs and illustrations.

The process is quite straightforward. Once you’ve uploaded your 2D image, you can then generate a 3D model based on it. From there, you can touch up the model’s textures and use different remeshing options to modify your mesh’s polygon count and topology as needed, depending on the application you may be using it for.

Additionally, you can take advantage of physically based rendering (PBR) texture maps, which will help materials in your model respond realistically to lighting sources in your 3D software program that supports PBR. This will come in handy if you’d like to transfer your image to a 3D asset for a larger digital project you may have.

Meshy allows users to export their models in popular 3D file formats like GLB, OBJ, FBX, STL, and USDZ. Each file type caters to a variety of uses, from editing your model further to game development, web visualization, augmented reality, and even 3D printing. In other words, you can apply your AI-generated 3D model and integrate it with your current workflow, instead of using it as a complete 3D project.

What’s also great about using AI image-to-3D in education is that you can help your students gain an appreciation for how images translate into 3D space. Image-to-3D modeling can be a great classroom exercise if you’re studying something like sculpture, product design, or historical objects. Students can take that experience further and have them generate a 3D model and discuss the similarities and differences between the image and model’s proportions.

Image-to-3D can also be a boon to indie artists who need a creative resource to develop their character design, digital props, or a proof-of-concept. An independent artist might want to develop a small video game and needs a rough idea of how the objects look based on concept art. Using image-to-3D to generate initial object assets, they can refine their models in Blender or a compatible game dev environment.

You can even use AI image-to-3D models for personal projects as well. A hobbyist might want to develop their own model based on an illustration to experiment with digital design, and a maker may even want to take that same model and use it for a 3D printing project.

Keep in mind, the accuracy of an image-to-3D model will depend on how you want to apply the model to your project. An experimental art project may leave much more room for interpretation and creativity with your 3D model, but if you want to reflect an object in real life, you may want to do some corrections and inspection yourself.

With any AI-generated models, you should be wary of uneven geometry, simplified features, and details that are not in the reference image. Just because your 3D model looks impressive when previewing it, it doesn’t mean it’s ready for production use. You might have to make some touch-ups to your 3D model.

For 3D printing, you should inspect your mesh to ensure there are no structural issues that may make it unprintable. If you’re a digital artist, you may want to touch up some surfaces, textures, or even the object’s proportions to meet your artistic vision.

Finally, if you’re using an image or photo from your source, be sure to check if you have the rights to use it, especially if it’s a commercial project. Before generating your asset from an image or photo and using or distributing it, be sure to check its image licenses and permissions.

Image-to-3D is more than just an alternative route for the designer with experience. This technique allows the student, teacher, artist, and hobbyist the chance to investigate objects of interest.

With the help of a browser based program like Meshy, the photograph or drawing becomes the catalyst for the creation of an interactive 3D model. It is through the combination of high quality references and proper refining that success can be achieved.

This article was written in cooperation with Tom White