In addition, Japanese domestic companies are entering the Metaverse area one after another.In October 2021, Facebook made headlines around the world by changing its company name to Meta.
This time, Mr. Taketomi, an AI Lab researcher working on CG research at CyberAgent, Inc., which provides services using digital twins and CG such as “Digital Twin Label” and “Polar Predictive AI Human”, and Virtual Human at a subsidiary We interviewed Mr. Ashida, director of CyberHuman Productions Co., Ltd., which is in charge of 3DCG production, about the efforts related to the Metaverse and future challenges.
|Mr. Naoki Ashida
Director of CyberHuman Productions Co. , Ltd. Joined
CyberAgent Co., Ltd. as a new graduate in 2013. After working as an advertising planner, in June 2017, he established CG Changer Co., Ltd., which specializes in creative production of video advertisements using 3DCG, and assumed the position of CEO. Current position since August 2019.
CyberAgent, Inc. AI Lab Research Scientist
Engaged in research related to augmented reality, virtual reality, and computer graphics. In March 2011, received a Ph.D. from the Graduate School of Information Science, Nara Institute of Science and Technology. 2011-2018 Assistant Professor, Graduate School of Information Science, Nara Institute of Science and Technology. 2018-2020 Senior Engineer, Huawei Technologies Japan Co., Ltd. Tokyo Research Laboratory. Current position since November 2020.
Table of Contents
- Metaverse Digital Twin Research by CyberAgent
- CyberAgent’s vision for the future of Internet advertising
- Future prospects
- Digital Twin Ai Tominaga’s first ad casting! ｜First initiative using a digital twin label
- All sounds are automatically generated! ｜AI Lab’s AI speech synthesis technology
Metaverse Digital Twin Research by CyberAgent
First, let’s review the differences between the Metaverse and the Digital Twin.
The meaning of Metaverse is a coined word combining “meta” and “universe”.
Metaverse refers to a virtual space where many people can participate and the participants can act freely in it. For example, there are already many services such as the virtual Isetan Shinjuku store and the service “Horizon Worlds” provided by Meta.
A digital twin is the reproduction of a real space in a virtual space using data collected from the real space. Unlike conventional virtual spaces, digital twins are characterized by being able to reproduce the real world in real time.
ーーCyberAgent is researching many fields, including the Metaverse, through AI Lab. What kind of research are you doing now?
Mr. Taketomi: Recently, I’m mainly working on research on generating Metaverse space content by generating 3D models .
Various contents are necessary to enter the Metaverse business. However, now we are completely dependent on creators for content creation, so we are researching how to use AI technology to reduce the burden on creators as much as possible and automate it.
Specifically, we are working on technology that generates 3D models from photographs of people’s faces . Since the shape of a human face is fixed to some extent, it is easy to create a statistical model, so we started by automating the relatively simple 3D conversion of the face.
ーーHow does CyberAgent use digital twins and metaverses in its business?
Mr. Ashida: At Digital Twin, we will start a service called “Digital Twin Label” from August 2021, which creates official 3DCG models of celebrities such as talents and artists, and casts digital twins that will be the “alter-ego” of celebrities. I’m here. We aim to produce and cast digital twins of 500 celebrities by 2023.
Furthermore, in the Metaverse field, on February 25, 2022, CyberMetaverse Productions Co., Ltd., which specializes in virtual store development, will be established to create stores in the Metaverse space and provide a new shopping experience. announced. We plan to establish a virtual store in the metaverse space as a new sales channel that is different from real stores and EC, and plan to support the creation of digital content using NFT and the issuance of unique crypto assets.
In addition, although the challenges differ depending on the retail industry, there is a subsidiary called CA Unmanned Store Co., Ltd. , which provides unmanned store solutions for retailers using AI. We have received requests to leave some of the customer service that is relatively easy to automate, such as inventory availability, to digital humans.
ーーI think the importance of NFT technology will increase in order to prove ownership when buying things in the Metaverse space. How will you proceed with your business in this field?
Mr. Ashida: As you say, proof of ownership is required when buying and selling in the digital space, and I think NFT may be used.
CyberAgent’s business partners include many companies and brands that are active in the real world. To go. In addition to that, digital items are added as accessories, adding value that can only be experienced in the Metaverse space.
Also, if it becomes possible to buy and sell using digital space and 3D data, it will be possible to check the actual size and color more than ever before.
In this way, I would like to expand the use of virtual stores in the form of expanding the core business of companies and brands that are active in the real world.
CyberAgent’s vision for the future of Internet advertising
ーーThere are almost no examples of digitized models being used in commercials and advertisements. What is your reason?
Mr. Ashida: Realism is guaranteed, but the problem is that it takes too much time.
Hollywood and Netflix have a long production period and a production budget of billions of yen, and it may be possible to create a character model. However, there are not many production studios that can pay that much to produce.
In recent years, the time required to create a digital twin has been shortened to about one month, but it takes time to make adjustments if you try to make it move without discomfort. At present, it takes five or six people several months to create even a few seconds of speaking scenes.
Taketomi: The reason why it takes so much time is that the cause of the discomfort (so-called uncanny valley) when moving the human model is not clarified, and it takes time to reproduce the unconscious movement. I think it’s one.
Currently, CG creators create the appearance and unconscious movements, but it would take a huge amount of time if the creators were to try to solve the problem by themselves. We are developing technology that can reflect this.
ーーPlease tell us about the challenges you want to tackle in the future.
Mr. Ashida: In video advertisements such as YouTube, everyone sees the same video, but in the future we would like to deliver videos that suit each individual’s taste. However, in addition to the current situation of spending a lot of time on one, it is difficult to prepare more variations.
Mr. Taketomi: The creation of video advertisements using 3DCG has the advantage of being able to freely change clothes, hairstyles, backgrounds, etc., but the current situation is that production takes a lot of time. On the other hand, image generation technology using GAN (Generative Adversarial Network) has a limited degree of freedom in editing, but it is possible to control facial expressions at low production costs without using 3DCG.
Our company provides a service that generates an original fictional AI person model suitable for each company or brand with an effect prediction AI called ” Kiwami Prediction AI Human “. I would like to work on the development of technology that can efficiently produce a large number of variations of person video advertisements by taking advantage of the advantages of .
ーーHow will CyberAgent utilize CG and Metaverse technologies in the advertising field?
Digital Twin Ai Tominaga’s first ad casting! ｜First initiative using a digital twin label
On July 28, 2022, CyberAgent’s digital twin, Ai Tominaga, made her first advertising casting. Here, we introduce an example of an initiative using a digital twin label.
CyberAgent and Mitsubishi Jisho Residence Co., Ltd. will open a special site for the virtual space “SUPER MODEL ROOM” of the newly built condominium “The Parkhouse” where digital twin Ai Tominaga will move in on July 28, 2022. Did.
On the special site, Ms. Ai Tominaga, a digital twin who lives in the “SUPER MODEL ROOM” will serve as a room tour guide and introduce “a special lifestyle”. In addition to the spacious “living room” and the comfortable and simple “kitchen”, you can freely explore a total of nine spaces within the browser.
All rooms are tied to the time in the real world, so users can experience the changing atmosphere of the room, such as the angle of the sunlight that changes between day and night, depending on the time they visit the special site.
In addition, one of the functions of “SUPER MODEL ROOM” is a chatbot that allows you to ask questions and post comments about your favorite furniture to Ai Tominaga, the digital twin.
All sounds are automatically generated! ｜AI Lab’s AI speech synthesis technology
The voice of Ai Tominaga, the digital twin played on the special site this time, is realized by AI speech synthesis technology of “AI Lab”, and all voices are reproduced by intonation technology that can express slight differences in nuance. It’s automatically generated.
AI Lab’s speech synthesis system collects a large number of voices in its own system with the aim of capturing the characteristics of general speaking styles. Synthesize that data with a base model created by machine learning to build natural utterances. As a result, Ai Tominaga learned her own voice more efficiently and was able to speak naturally without additional recording. In addition, when generating lip-synced video, we use technology that estimates vowels from the synthesized audio data and automatically applies natural mouth movements according to the timing of utterances.
CyberAgent has newly appointed CyberMetaverse Productions Co., Ltd., which supports the sales activities of companies in the Metaverse space, and Kengo Kuma, an architect, as an advisor to research, plan, and produce virtual buildings and space designs in the Metaverse space. We have started full-scale entry into the Metaverse business, such as establishing the ” Metaverse Architecture Lab “.
The Metaverse is a field that can be expected to further develop in the future through the combination of NFT art and AI technology, and many celebrities predict that the Metaverse will become a trend in 2022.
There is a possibility that many companies will enter the Metaverse business in the future, and I think it will become one of the technologies that attract the most attention. I hope this article will get you thinking about the metaverse market.