Senior MS&T experts discuss digital twin technology

By Shane Klestinski, Associate Editor

 

A group of experts on digital twin technology explored the concept of the Performance, Immersive Experience, and Entertainment Lab (PIXEL) during a panel discussion held at the Dr. Phillips Center for the Performing Arts in Orlando, Florida, April 6.

 

Wayne Fogel, research program director for the University of Central Florida’s (UCF) Institute for Simulation and Training, moderated the discussion. During his introductory comments, Fogel said that UCF (where PIXEL would be located) has had “an incredible history” involving digital twins.

 

“We’ve done amazing, groundbreaking, innovative work within academia, industry and the government,” Fogel said. “What we’ve never done – what we’ve never been a part of – is a performing arts center and its associated digital twin.”

 

The panel consisted of Grace Bochenek, Ph.D., executive director for the Pegasus Research Institute and director for UCF’s School of Modeling, Simulation and Training; Michael Tschanz, engineering consultant and former director of engineering technology and analysis for The Walt Disney Company; and Eileen Smith, program director for applied research and director for UCF’s E2i Creative Studio.

 

Bochenek began the discussion by noting that we all live in a digital world, pervaded by technology everywhere. She went on to provide a basic introduction to a largely nontechnical audience on digital twin technology, saying that a digital twin is “nothing more than an exact digital replica of a system, a product [or] a process.” She explained that digital twins allowed for simulation, as well as analyzing data and information to optimize “whatever you’re thinking about.” Given the connected nature of the world with data and information, a digital twin could also receive real-time information for future planning and other considerations.

 

Throughout the talk, the speakers emphasized the multiple benefits and applications of digital twin technology applied to multipurpose facilities.

 

“Digital twins coexist together [with the real-world counterpart they reflect to provide] feedback,” Bochenek said. “How is all this data and information being used and fed back into the real system? With a real, physical thing like this building, we can create the digital replication of it, instrument it, and have data and information in real time – and wow, does it open up opportunities in terms of decision-making. We’re excited because this is what Orlando’s about: It’s about modeling and simulation. It’s about human performance. It’s about arts and humanities and the blending of all that.”

 

Referencing his experience observing the impact of digital twins linking physical and digital representations, Tschanz said that Disney started applying modeling and simulation to develop all of its attractions “about 20 years ago.” He went on to explain that an immersive experience’s path from the design concept stage, to where it actually becomes something guests experience, is “extremely complex.”

 

“We do custom applications and we build prototypes, and a lot of [Disney attractions] are industrial systems that we put people on,” Tschanz said. “Understanding how those are going to run and operate in advance is very important so that when you deliver them and they are in operation, you know exactly how they are going to run. From our standpoint, it always has to start with the model.”

 

According to Tschanz, such processes involve bringing in different data sets, trying them out, looking at what that model ultimately does, honing in on an answer, optimizing it, and running it over and over again. He noted that the aerospace industry has worked with digital twins for decades, roughly 30 to 50 years.

 

“They used analog computers prior to the digital computers that are used now, but the concepts are still the same,” Tschanz said. “You start with a very rigorous engineering model, you validate it, verify it, and then you start running data sets through it. Once you have that model, you take it and reuse it over and over again.”

 

Tschanz described an example that involved using a digital twin for predictive maintenance. Maintenance workers could observe their “robotic systems,” see how those systems are running, and then compare them to what is actually in operation. Based on that comparison, they can determine how well the overall systems are working to make predictive estimates on future requirements to keep operations optimal.

 

“We’d try to figure out the best way to apply these modeling methods to come up with an optimal solution and then bring them to life better, faster and cheaper,” Tschanz said. “What I always like to say about digital twins when people say, ‘Among better, faster and cheaper, pick two,’ the beauty of modeling and simulation… is that you can do all three.”

 

Smith’s background is in technology, but her degree is in social-based theater. She outlined the benefits of theatrical applications in maximizing a facility like PIXEL, specifically in how users could employ interactive media to create, as well as develop various training packages.

 

“As we look at PIXEL, it’s really the fusion of traditional theater and other ways to use a space [with] dynamic layouts, ways to use different interactions, real-time adaptability to new layouts, uses and techniques,” Smith said. “The digital twins really enhance both live performances and training.”

 

Smith emphasized three areas of benefits in digital twin usage: planning and design, operations, and audience engagement. When considering planning and design, for someone who works in theater (director, lighting designer, set designer, etc.), being able to see the stage, or a digital copy, can provide a helpful platform to explore the space before committing to final decisions. This option helps make “everything ready to go on the first day,” according to Smith, thanks to the objective data this approach provides.

 

“It’s optimizing that use of space before I use the space,” Smith said. “As an actor, you get your blocking, you go home and think about it… But, the ability to look at the space and realize that they could spring backwards five feet without hitting a wall, or knocking the set over, is a critical thing to know if you’re a stage actor.”

 

Smith went on to elaborate on the significance of audience engagement, and how certain factors that are important to theater workers in maintaining an audience’s attention, also share relevant overlap in conducting military training.

 

Not all required training for Defense Department personnel involves rigorous physical activity. Over the years, military members have cited “death by PowerPoint,” as commentary on how being a captive audience member enduring multiple, consecutive briefings can also be physically draining in its own way – and result in wasted time due to ineffective instruction.

 

“The digital twin in an audience experience doesn’t start [and end] when the curtain opens and the curtain shuts,” Smith said. “The idea is understanding the audience from the transitions from stage to stage, the idea of the building and audience flow, and what are the amenities and accessibilities that we want to ensure for everyone. From when they get out of their car to when they exit the building, whether it’s a theatrical use or a training use, we have engaged them, and we understand how to use their systems and their time in the most optimum fashion.”

 

Smith recalled that when audience members’ heads drooped forward in the past, it usually meant they were bored and/or falling asleep. Nowadays, the same body posture often indicates they are looking down at their phones during a performance. Either way, “we’ve lost them,” because they aren’t engaged with the presented content. Smith said that directors might use preview audiences, among other tactics, to ensure a play keeps an audience’s attention.

 

“Imagine being able use a digital twin where [a director can say,] ‘Stop – cut that by two [minutes], and I want different things over here,’” Smith said. “All of a sudden, when you run it again, everyone is staying engaged. Points in time, over time, are what make up the data that makes a digital twin really rich.”

 

Fogel later asked Bochenek about the directions digital twin-based innovations could go in other industries, far beyond theatrical capabilities. She responded that the technology was “leading the way” and could also have applications to the first-responder community.

 

“If I’m in law enforcement or the fire department, I’m using these tools – these same models that are validated – that are helping us understand operations and predictions,” Bochenek said. “I’m using it for my needs [just as a performer would use them] to train or practice.”

 

Smith expanded on Bochenek’s comments in the context of large training events, in which training coordinators could blend physical and digital aspects in the same space to provide trainees with the most beneficial experiences. She went on to reference a “burning house” visual in the presentation that was left over from an event UCF previously did with Orange County Fire Rescue.

 

“Think about all of our homes in Orange County – the lieutenant on the first truck is the fire commander, even if it’s his first fire,” Smith said. “We started with simulation about 10 years ago [and] modeling and simulation has been marching toward digital twins. What we haven’t had – until now – is a lot of the data connectivity that allows us to actually start turning that idea of objective data over time. That’s something very important because the ability to do that is what opens up some of these other opportunities for use of the space.”

 

Bochenek continued the conversation of PIXEL’s potential digital twin capabilities in the direction of military training.

 

“If you’re in the Defense Department, you’re training individual soldiers, airmen or Marines, or you’re training a group of people, and sometimes you’re training across geographical distances,” Bochenek said. “So today, you’re training from a common data source, a common model, or a common environment, and you create consistency in training. Consider having a large performing arts [facility] that could transform itself to support any kind of media or national security or defense exercise. You could have 20, 30, 40 – maybe more people – in a virtual space, all doing different operations and training as a unit together. It’ll have huge impacts.”

 

The panel later discussed digital twin applications that could enhance STEM (science, technology, engineering, math) education. Smith explained that such applications could involve middle and high school learners in multiple levels of participation. For example, a use case would provide rich exploration using data, interactivity and simulation capabilities to “build a rich picture” of different people within a team using the digital twin in different ways. Similar to roles in theater (such as a director, a lighting designer or a stage prop manager) different team members may have distinctive perspectives and requirements, which the digital twin could replicate with different dashboards and different views to help the entire team.

 

“We’ve got to train the future generations in STEM and the arts,” Tschanz said. “There’s going to be such a big labor demand, but there will also be fewer people available to accomplish these tasks. We’ll need people who are extremely well trained, and we’ll need tools that create force multipliers, and [digital twin tech] does both.”

 

He explained that the digital twins create an environment where people can explore different ideas in how they would do different things in science or math, or present different opportunities that say, “If you follow down this path, you could study these things, or you could do these [other things] – or you could work on Broadway productions.” Tschanz said users could create a variety of what-if scenarios for people to consider when visualizing what their future careers might look like and help develop them in such environments.

 

“We’ve got to have tools like this because demographics are changing so rapidly, and we need an environment like this to help train our future engineers, scientists, medical doctors and arts producers so they can get up to speed so much quicker because of the demand in the labor force,” Tschanz said.

 

Tschanz emphasized that in his time working with Disney, when the company was recruiting prospective engineers, it always looked at whether a candidate had some kind of arts background. Although it was not always a requirement, Disney found that such a blending provided “an understanding of what’s going on,” and a creative side that included problem-solving skills for difficult challenges that weren’t found in a textbook.

 

“Innovation is what drives economics and business development,” Tschanz said. “It is the biggest multiplier when you think about returns on investment – it’s always innovation. You can always replicate things you’ve done over and over, but at the end of the day, innovation is what’s going to solve problems.”

 

Near the end of the program’s formal discussion, the panel addressed the importance of asset libraries.

 

“If you go back and you think about digital models that are creating these digital twins, they’re lifecycle assets that… will get new data, they’re going to get enriched, and they’ll get used for all kinds of purposes, so there is that lifecycle growth of data and information that gets richer… and provides better decisions,” Bochenek said. “The future is now. We’re fortunate because we’re in it, and we’re all going to be moving it forward.”