Artificial Intelligence and Design

Problem Statement

This experimental project aims to explore present AI tools and techniques throughout the design process, resulting in the creation of a unique artifact or intelligent application. Initially, experimental tools are playfully utilized, particularly during the ideation phase. Professional tools will also be employed to refine designs or generate variations. Ultimately, the project entails developing an application using custom scripts and models, with a focus on exploring both the possibilities and limitations of artificial intelligence in design.

Idea and Concept

Following an intensive initial research phase focusing on the fundamentals of neural networks, we have decided to explore our problem using a video game as a case study. In a second research phase, we investigated and experimented with various AI systems. Utilizing the insights gained, we then attempted to create a game using the Unreal Engine. By employing Stable Diffusion and our proprietary model, we aimed to recreate given assets from 2D images and reconstruct them within a 3D environment. This project allowed us to apply theoretical knowledge practically and assesses the capabilities of AI within a dynamic, interactive environment.

Implementation

Initially, we used ChatGPT 3.5 for idea generation and Stable Diffusion for creating visual materials. For 3D modeling, we tested systems like SloydAI, ThreeStudio, and MasterpieceX, but found that without funding, the technology isn’t yet suitable for our needs. Using ChatGPT 3.5, ChatGPT 4, and BlackboxAI, we tried creating blueprints in Unreal Engine, but the technologies proved inadequate for visual coding. However, we successfully implemented sound effects and music with Audiocraft and Bark. To utilize the assets available, we trained a model to generate templates.