How neural networks can help game developers and whether they will replace them in the future

Contents:

• How neural networks work
• Can neural networks replace artists?
• The main neural networks working with art?
• Interesting cases, as well as personal experience.

In August 2022, a Japanese developer nicknamed Nao_u released a simple scrolling shooter in which all art was created using the Midjourney neural network. Although setting up the game required considerable effort, and the project itself cannot be considered full-fledged, the results of the author's work are impressive. This project demonstrates the potential of using neural networks in the gaming industry and opens new horizons for developers.

A month after its launch, the two-person studio Cute Pen Games released a dating simulator with puzzle elements called This Girl Does Not Exist on Steam. A fascinating feature of the game is that both the script and the voice acting were generated entirely using neural networks. This makes the project unique in the world of indie games and highlights the capabilities of modern technologies in content creation.

This is just the beginning. With the development of technologies such as Midjourney, regularly updated with new functionality, discussions are again reemerging about the possibility of neural networks soon replacing artists.

In this text, we will take a closer look at whether this is really the case, clarify the principles of neural networks, and analyze several interesting examples of their application. We will delve into the mechanisms underlying neural networks and learn how they solve complex problems in various fields.

How Neural Networks Work

Let's leave aside the philosophical reflections on artificial intelligence and technological singularity, which cause fears among supporters of various conspiracy theories, and turn to practical aspects. Currently, neural networks lag significantly behind the human brain and are incapable of performing truly creative tasks. This statement remains true if we do not delve into the concept of creativity itself. Artificial intelligence technologies continue to develop, but their capabilities in the field of creativity are still limited.

Neural networks are already providing significant benefits, which is their main purpose. Their effectiveness directly depends on computing power, which explains why the development of neural networks has only become possible in recent years. As the name of the technology suggests, neural networks mimic the functioning of neurons and synaptic connections in the human brain. This allows them to identify and process a vast number of non-obvious connections, the patterns of which the network determines independently. These capabilities open new horizons in various fields, from medicine to finance, making neural networks an important tool for solving complex problems.

The more connections in a neural network, the more efficiently it functions. Achieving high efficiency requires significant computing resources, making the use of neural networks an expensive process. This is important to consider when developing and implementing artificial intelligence-based technologies.

Currently, most neural network services that generate images based on text queries operate on a credit system. The business models of these services vary, but they all rely on the purchase of credits. Users purchase a certain number of credits and spend them on image creation, as well as on additional options, such as upgrading to HD quality or increasing the number of steps the neural network performs when creating a picture. This allows users to tailor the generation process to their needs and achieve higher-quality results.

Currently, the main exception is the popular Midjourney, which is in beta testing and operates through the Discord platform.

Servers with video cards are significantly more expensive than standard servers due to their high performance and the need to run neural networks. When neural networks depend on video cards, software development costs increase significantly. Unfortunately, it is not always possible to achieve the desired results, and this becomes an additional problem for businesses. Servers require payment for the time they are running, so many companies prefer a loan-based financing model. As a result, the longer the server is operational, the higher the financial costs.

Denis Shiryaev is the CEO and Chief Product Officer at neural.love. His professional experience and leadership qualities contribute to the development of innovative solutions in the field of artificial intelligence and generative technologies. Under Denis's leadership, the neural.love team is working to create cutting-edge tools that transform how users interact with artificial intelligence.

Currently, all neural networks for artists operate under a public domain license, allowing them to be freely used for commercial purposes. You can create unique art for a T-shirt or use it in your game—there are no restrictions. Neural network services occupy the same niche as Pinterest and other similar platforms, which have traditionally served as excellent sources of reference for creative projects. This opens up new opportunities for artists and developers, allowing generated images to be easily integrated into various products and services.

Neural.love, which uses Stable Diffusion technology, aims to create a platform similar to a photo stock, where every user can find the images they need. Neural network services provide open access to the work of their algorithms, allowing anyone to explore vast archives and find the materials they need. This approach promotes the dissemination of creative content and simplifies the process of finding images for various needs.

There is a common misconception that neural networks create their work using existing images online. In fact, neural networks generate unique images based on given queries, transforming random noise data into visual forms based on the training they have received. Thus, neural networks do not simply combine elements of other works of art, but create original images on their own.

An artist knows what a forest looks like, having not only visited it but also studied it through photographs and the works of other artists. A neural network learns in a similar way: it is provided with images of the forest in various styles and works by other artists. This allows it to create its own paintings, depicting the forest in different interpretations. Thus, both the artist and the neural network use visual knowledge to create unique works of art.

One of the main problems of modern neural networks is the low resolution of the latent space, which is only 64x64 pixels. The resulting images are then enlarged using upscaling. This is why neural networks often demonstrate insufficient quality when generating images of faces and, for example, fingers. Improving the resolution of the latent space can significantly improve the quality of the generated images and expand the applicability of neural network models in various fields.

The next step is to move to a higher-resolution latent space. This will ensure unrivaled quality. This is expected to happen either at the end of this year or early next year.

Denis Shiryaev is the CEO and Chief Product Officer at neural.love. Under his leadership, the company develops innovative technologies in the field of artificial intelligence and machine learning. Denis is actively involved in strategic planning and product management, which allows the company's neural network solutions to successfully compete in the market. His experience and industry knowledge contribute to the continuous improvement of the products and services offered by neural.love.

In order for the neural network to correctly interpret a user's request, it is necessary to enter keywords, known as prompts. Correctly choosing and formulating prompts play a key role in obtaining accurate and relevant answers from the neural network. The clearer and more specific the keywords, the more effectively the system can process the query and provide the desired information.

Different neural networks process incoming queries differently, notes Alexander Dobrokotov, creative director of the advertising agency DADA and an expert in artificial intelligence. For example, the Stable Diffusion generative model requires a detailed prompt. It's important to specify angles, scales, lighting, color palettes, styles, artist names, and mood. In some cases, it's necessary to add a negative prompt so the algorithm clearly understands what shouldn't be in the image. In contrast, in Midjourney, a simple query like "beautiful cat" is enough to generate a satisfactory result. Understanding the differences in how neural networks work allows you to create visual content more effectively.

Midjourney can be enhanced by adding additional words to clarify the desired result. This will help produce more accurate and high-quality images that meet your needs. Using additional descriptive phrases and keywords significantly increases the likelihood that the final image will reflect your intentions and preferences.

Mentioning a style, such as Claude Monet, Simon Stålenhag, or Rudi Giger, as well as a genre, such as cyberpunk or retrofuturism, significantly impacts the final result. It is important to consider the type of image, whether it is a render, a photograph, or a drawing. These elements shape perception and create a unique atmosphere, making them critical to achieving the desired effect in visual content. Choosing the right style and genre can significantly increase the appeal and memorability of an image.

When describing lighting, you can use terms such as professional studio lighting or a bright purple neon light. It's important to mention technical details, such as 50mm lenses and diffusion filters, and specify the camera model. It's also worth considering different angles and poses, which will add dynamism and originality to the image. Such details make the description more appealing and informative to the target audience and also contribute to improved SEO.

Artem Netyagin is an indie developer and AI enthusiast. He actively creates unique software solutions and applications using modern technologies and approaches. His work reflects a deep understanding of algorithms and a passion for innovation in development. Artem continues to explore new trends in AI, striving to implement them in his projects and share his knowledge with the developer community.

Some services offer prompt engineering features. For example, on the neural.love platform, users can choose categories such as Cyberpunk, Creepy, Anime, and others. According to Alexander Dobrokotov, key recommendations include separating semantic blocks with commas, using English, and actively experimenting with wording. These approaches can significantly improve content generation and achieve greater creative results.

Can neural networks replace artists?

A definitive answer to this question is unlikely, but all our speakers are unanimous in their opinion: these tools are currently useful resources that can significantly simplify and diversify artists' creative work. Using modern technologies and tools helps artists expand their capabilities, improve their efficiency, and find new inspiration for creativity.

Neural networks do not diminish the importance of artists' work and cannot be considered art. They are a useful tool for achieving quick results with minimal time and effort. However, neural networks are not capable of creating complex and specific works that require careful attention to detail and nuance. Furthermore, they are limited in their ability to make adjustments to certain aspects of the image. Therefore, neural networks can only serve as an aid in the creative process, but cannot replace a professional artist.

Artem Netyagin is an indie developer and AI enthusiast. He actively creates unique projects using modern technologies and approaches. His work reflects a deep understanding of both programming and AI principles. Artem strives not only to develop his skills but also to share his knowledge with others, inspiring a new generation of developers to experiment with technology and creativity.

Denis Shiryaev argues that artists should actively embrace neural networks rather than reject them. In his opinion, neural networks cannot replace creative people who use this tool for inspiration and the creation of new works. The replacement will only affect those who are unprepared to adapt to change, as is the case with any other technology. The use of neural networks in art opens up new horizons and opportunities for self-expression, allowing artists to expand their creative boundaries.

Artists expect high results, as modern audiences have a wide range of experience perceiving art. A simple image of a girl with a flower in her hair cannot evoke the desired delight in viewers. To attract attention, works must possess depth, originality, and a unique approach. In a highly competitive environment, artists must create works that are not only beautifully executed but also evoke emotion, make people think, or tell a story. The quality of works by renowned artists is undoubtedly significantly higher. Every year, people's awareness of art grows, and it is becoming accessible to everyone. This phenomenon represents an important step toward the democratization of art, allowing people around the world to enjoy works of art and discover new directions. Art is no longer the preserve of a small circle, but is becoming a part of everyday life, which contributes to its development and popularization. Denis Shiryaev is the CEO and Chief Product Officer at neural.love. He actively develops and implements innovative solutions in the field of artificial intelligence and neural network technologies. Under his leadership, the company strives to improve the user experience and expand the capabilities of its products, thereby contributing to the development of AI and making neural networks more accessible to a wider audience. Denis has significant experience in project management and strategy development, which allows him to effectively cope with the challenges of the modern market.

VOKI Games Art Director Pavel Pilovets draws parallels between the emergence of neural networks and the time when traditional artists criticized the use of graphics tablets, claiming they undermined art. Similar comments were later made about Photoshop. In both cases, new technologies became tools that artists began to actively use. "These tools expand the possibilities of creators, possibly lowering the barrier to entry and, to some extent, devaluing old working methods. However, the industry itself remains alive, as does the artistic profession," Pilovets emphasizes.

In the future, concept artists will likely begin to more actively use neural networks in their work. A new profession, such as "neural network operator," is expected to emerge, which could change the creative services market. Briefs from clients, art directors, and directors will likely become more detailed thanks to the use of neural networks. This will enable visualizers to generate a large number of unique images with various stylistic approaches. They will then be able to commission artists to implement projects in specific styles and with specified parameters. For now, technical specialists skilled in project completion need not worry about the future. Concept artists are simply adapting their workflows, which will open new horizons in the creative industry.

Pavel Pilovets is the Art Director at VOKI Games. He is responsible for the artistic direction and visual design of games, creating unique concepts and images that capture players' attention. With his experience and creative approach, VOKI Games successfully develops projects that are distinguished by high quality and originality. Pavel Pilovets is actively involved in the game creation process, from character development to world design, making him a key figure on the team.

Neural networks are a powerful tool for artists today, allowing them to quickly generate a variety of images and use them in their work. They can be used to visualize even the boldest ideas by capturing a few key points as a prompt. Instead of wasting time searching for inspiration on image aggregators, you can use neural network services that offer unexpected and original results that can inspire new creative solutions.

One of the less obvious applications of neural networks is the rapid visualization of ideas. This tool can be useful not only for artists but also for game designers. Instead of explaining your concepts to the team verbally or trying to draw them, you can use a generated image. This makes it easier to convey the style and atmosphere you want to create for your project. Neural networks allow for the effective communication of visual ideas, accelerating the development process and improving understanding between team members.

A neural network can be used to generate various types of assets, such as paintings, book and magazine covers, advertising billboards, and portraits. This does not infringe copyright and significantly simplifies the content creation process. This reduces the need to engage a dedicated artist for such tasks, although professional artists still play an important role. Indie teams are now able to solve many problems without the involvement of artists, as proven by successful cases.

Artem Netyagin is an indie developer and AI enthusiast. He creates unique projects that combine modern technology and a creative approach. His work is distinguished by its originality and high quality, attracting the attention of both users and industry professionals. Artem actively explores the possibilities of AI, applying them in his developments, which contributes to the creation of innovative solutions.

The Main Neural Networks Working with Art

Currently, there are three main image generation technologies: Stable Diffusion, DALL-E 2, and Midjourney. There are also neural networks developed in China, but they have not yet achieved the popularity and quality of the above. These technologies are actively developing and are used in various fields, including art, design, and marketing, providing the opportunity to create unique visual solutions.

The advantage of Stable Diffusion is that it is a long-open technology, which is the basis for many neural network services, including neural.love. The community is actively developing this technology, while various companies adapt and modify it for their specific needs. For example, neural.love recently improved its algorithms, which allows it to create more accurate portraits. This makes Stable Diffusion an attractive choice for developers and users seeking quality and innovation in image generation.

DALL-E 2 has remained a closed technology for a long time, but its developers have now made its code available to a wider audience. This neural network demonstrates outstanding results in the field of photorealism, making it a promising tool for artists, designers, and developers. Opening the source code of DALL-E 2 will allow users to more actively use its capabilities in a variety of projects, pushing the boundaries of creativity and innovation in digital art.

Midjourney is a closed system that actively borrows best practices and ideas from other sources. Currently in beta testing and functioning through the Discord platform, this unique platform allows users to interact with artificial intelligence to create visual content.

Midjourney initially focused on the "make beautiful" function. The results were indeed aesthetically pleasing, but often quite abstract. This led to the formation of a unique style known as the "Midjourney style." This approach allowed users to create visually appealing images, but it wasn't always clear how to interpret their meaning. As the platform evolved, the emphasis shifted to more detailed and specific results, expanding users' capabilities in creating visual content.

After the Stable Diffusion developers made the code available, the Midjourney team integrated their work, significantly improving the quality of image generation and understanding of user requests. However, this proved insufficient, and a fourth version of the algorithm was soon released. The specific details of how this new version works remain unknown, but there is a suspicion that it may be based on DALL-E 2 technologies.

Alexander Dobrokotov is the creative director of DADA and is actively interested in artificial intelligence. His experience in creative solutions and passion for new technologies make him an important figure in the development of innovative projects. Alexander is inspired by the interplay of design and technology, which allows him to create unique concepts that combine creativity and functionality. In the rapidly changing world of technology, his expertise in AI opens new horizons for business and creativity.

In the latest version of Midjourney, according to Dobrokotov, there is significantly increased coherence and consistency of images. This term, called coherence, refers to an improvement in the logic and integrity of visual content. The higher the level of coherence, the more harmonious the images become. This means that the image generation process eliminates illogical elements, such as multiple legs on a single character, making the results more realistic and appealing to users.

Among the image generation technologies, besides Stable Diffusion, DALL-E 2, and Midjourney, there are other solutions, but they have not yet achieved a similar level of popularity. The open source nature of these technologies allows them to be modified to suit specific needs, but for artists without programming skills, using ready-made services is a more convenient and accessible option. Currently, there are quite a few such services, simplifying the process of creating unique images for creative professionals.

The differences between DALL-E 2, Midjourney, and Stable Diffusion can be summarized as follows. DALL-E 2 is a powerful tool for processing photographic images, offering capabilities comparable to Photoshop but at a higher level. However, its results are often less successful when used for artistic creation. Midjourney, on the other hand, is excellent for generating art, demonstrating high quality and creativity in this area. However, when working with photographic images, it exhibits certain weaknesses, resulting in images that resemble paintings more than actual photographs. Stable Diffusion is a cross between DALL-E 2 and Midjourney, possessing both strengths and weaknesses in both categories. Results for artistic images can be unpredictable, and shortcomings can also arise when creating photographs.

Pavel Pilovets is the art director at VOKI Games. In his role, he is responsible for the visual design and artistic direction of the studio's projects. Pavel has significant experience in the gaming industry and strives to create unique and engaging game worlds, allowing VOKI Games to stand out from the competition. His creative approach and attention to detail contribute to the successful implementation of ideas and concepts, which has a positive effect on the overall perception of the studio's games.

Interesting cases, as well as personal experience

The Internet presents many examples of the use of neural networks in art. One neuroartist updated and detailed the pixelated characters of the famous game Fallout 2, another developed interactive concept art using the Midjourney neural network and Unity. A third won a fine arts competition. However, such projects are often perceived as entertainment rather than a serious application of new technologies.

The work of artist Osh Punch, whose creation process he presented on the DTF platform, deserves special attention. In his practice, he uses the playgroundai service, which is based on Stable Diffusion and DALL-E 2 technologies. These modern tools enable the creation of unique works of art by combining generative design algorithms and artificial intelligence. Osh Punch demonstrates how innovative technologies can expand the boundaries of traditional artistic creation, opening up new possibilities for self-expression.

Osh Punch used a neural network to create a 2D character model, to which he then added skeletal animation and integrated into the game engine. However, the artist did not limit himself to working with the neural network: he used his own developments, refining details and uploading updated versions back to the system. This process demonstrates the successful collaboration between the artist and the neural network, which led to an outstanding result.

After generating a large number of images, it's time to select the most suitable ones. In some cases, I used only the heads and corrected any imperfections that arose during the neural network's operation. In other cases, I simply cut out the necessary elements. Hands usually turn out to be stumps, so it's often better to refine them yourself. It's also worth experimenting with the source image, adding schematic objects to the doll for clearer generation. This will help achieve better quality in the final images and improve their visual perception.

Artist Osh Punch shares his work and thoughts on his blog at DTF. His work attracts attention thanks to his unique style and original ideas. Osh Punch explores a variety of themes, from everyday life to deep philosophical questions, making his artistic expression multifaceted and relevant. The blog features not only his artwork but also his reflections on the process of creation, inspiration, and the significance of art in the modern world. Osh Punch's posts on DTF allow art fans and new readers to better understand his vision and approach to creativity.

Indie studio Xlab Digital is a shining example of the use of modern technology in game development. They are creating the VR game Strangers World: The Swarm, relying solely on Midjourney's graphics generation capabilities. Initially, the developers attempted to engage artists, but faced difficulties due to a limited budget. The services of a qualified artist are quite expensive, which became an obstacle to the implementation of their project. Using Midjourney allowed the team to streamline the process of creating visual content while maintaining high graphics quality.

For $30 For $100 per month, which is the price of the paid version of Midjourney, you get access to an artist who is reliable and won't let you down. They are capable of creating hundreds of works of art, surprising you with their ideas. Of course, the generation process can take time, and sometimes a single image can take a whole day. But can you find an artist who will complete your order in a day and for that price? Most likely not.

Andrey Trunov is an indie developer at Xlab Digital, a studio that creates unique game projects. In his work, he combines creativity with technical skills, which allows him to develop innovative solutions in the field of video games. Xlab Digital, under Andrey's leadership, is committed to creating high-quality content that attracts players and stands out from the crowd of mass-market releases. The studio actively experiments with new ideas and technologies, opening new horizons for the indie industry.

Xlab Digital's neural network is used to create concept art, which subsequently serves as the basis for 3D modeling. The developers showcased a number of works, including key art for the game, all generated using Midjourney. Neural networks like Xlab Digital and Midjourney are opening new horizons in digital art and game development, enabling quick and efficient visualization of ideas.

The pistol, the default weapon for players, is truly impressive. Developer Andrey Trunov shares his vision: "Initially, we planned to create a unique weapon using alien crystals, but the AI ​​exceeded all expectations. I imagined it differently, but Midjourney made the model much more impressive than I could have imagined. Of course, the final version won't be exactly the same, but the similarity will be 80-90%." This highlights how artificial intelligence can inspire and enhance the creative process of game development, creating unique elements that attract players' attention.

In my experience, I'm actively creating art for a splatterpunk side-scrolling shooter that includes elements of blood, flesh, and bone. In this process, I use neural networks as my primary source of reference. I cut some elements directly from art generated by neural networks, which allows me to significantly speed up and improve the creative process. This approach not only saves time, but also opens up new possibilities for the visual design of games in the splatterpunk genre.

Unfortunately, my requests turned out to be too narrowly specialized, as was the chosen art style, so I often have to refine the neural network results. Nevertheless, I received a lot of useful references. After spending several hours experimenting with prompts and presets, I collected ideas for creating a dozen unique enemies for my game.

During the experiment, I used Stable Diffusion to generate the image of one of the bosses - an astronaut. At that time, I had not developed his attack patterns or combat mechanics. These elements emerged while working with the neural network, allowing us to create a unique gaming experience.

After choosing a suitable result, I immediately began developing the mechanics for the boss. It will have a huge claw that strikes across the entire screen, adding a bullet hell element. In the second phase of the fight, the boss will remove its helmet and begin pursuing the player's ship in the air. The remaining details will depend on further technical implementation.

An attempt to refine the image using neural networks was unsuccessful. Midjourney failed to produce the desired results, producing images in its characteristic style, regardless of the settings. Stable Diffusion, even with the use of various additional services, showed more satisfactory results, but ultimately turned out to be ineffective for implementing the plan.

To complete the astronaut, I'll need to completely rework him myself. I also needed a flying meat fish with a disco ball on its back and a Gothic church with eyes. In this process, I collected many suitable references that will help in creating a unique and memorable visual image.

The conclusion is clear: the more complex the task, the lower the likelihood of obtaining a high-quality result, so I doubt that neural networks will be able to completely replace artists in the future. However, they show good results in creating simple art objects. Neural networks can effectively create icons, various chests, backgrounds, and even character faces. This makes them a useful tool for simplifying processes in graphic design and digital art.

This scenario is valuable for artists involved in creating storyboards and still shots for films and cinematics. Using a single image, you can generate multiple cinematic compositions while maintaining a consistent style. This requires only slight changes to the description depending on the plot. This approach allows for efficient scene visualization and adds originality to projects.

Pavel Pilovets is the Art Director at VOKI Games. In this role, he is responsible for the visual design and artistic direction of the studio's projects. Under his leadership, the team creates unique game worlds and characters, which enhances the games' appeal to a wider audience. Pavel's experience and creative vision allow VOKI Games to maintain a competitive position in the video game market.

This method is ideal for designers seeking new forms based on familiar objects, such as household appliances. This technique was actively used, for example, in Star Wars. A neural network can create a concept for futuristic transport using images of a toaster or printer as input. Similarly, fashion designers and architects can also significantly speed up their processes thanks to the capabilities of neural networks. The use of such technologies opens new horizons for creative professionals, allowing them to experiment and find unique solutions.