Introduction
What Is 3DAI?
Within the current surge of technological development in the field of artificial intelligence (AI), immense public, commercial, and scientific interest has been directed towards the sub-field of generative AI. This area includes large language models (LLM's) such as OpenAI's ChatGPT, Anthropic's Claude, and Google's Gemini, as well as image diffusion models such as OpenAI's DALL-E and Stability AI's Stable Diffusion. These tools have garnered significant interest for their novel applications, but another newly emerging technology in the field has shown much less publicity - 3D AI.
3D AI <- Hover for definition is the rapidly evolving field of creating 3D models and environments, either through reconstruction from images and video or through generation from text prompts. The field is extremely new, and rapidly evolving as innovative methods and techniques are researched and adopted.
This article is an interactive explainer on the field, analysing and comparing the current methods, discussing some current and proposed applications, and considering the wider implications that this technology brings. Specifically, this article three common methods used in 3D AI - those of Neural Radiance Fields, Gaussian Splatting, and Multi-View Diffusion.
Chapter 4
Implications
Generative AI comes with baggage, and 3D AI is no different. While the implications of this technology may be less immediately apparent than say, Deepfakes, there are still concerns for how these tools will be used.
First, some of these methods require training, and training requires datasets. Currently the field is still in its infancy, largely being pushed forward by open-source, public domain researchers at various publicly funded research institutions. This means the research is open, and experiments can be monitored and reviewed with ethics boards. Should this technology find commercial interest, these obligations are removed, and so go the grounds for transparent ethical reviews. Other forms of generative AI that have found this commercial interest have so far been largely developed through the widespread use of unauthorized and non-consensual data collection techniques. This is a dangerous precedent that could be continued and expanded in the form of 3D AI.
There is also some concern for the trust placed in these systems in safety-critical environments. Some development has been put towards utilizing reconstructed 3D AI geometry in critically important environments, such as medical imaging and automotive sensor inputs. In these environments, these experimental and newly emerging technologies should only be incorporated with the utmost caution. There is very little research into the interpretation of these models, and so we are not aware of what biases are present or where these models will fail.
The more typical use cases for this technology also pose a risk for increasing the realism of generated content. Several current methods of detecting AI generated content rely on tracking physical properties and detecting subsequent inconsistencies. This could be mitigated with the widespread use of 3D reconstructions, where these physical properties would be consistent with real media.