Integration of AI Tools in Production of a SOKO Stuttgart Episode

The rise of generative AI tools and 3D reconstruction algorithms brings changes to multiple industries, including the film and television industry. In our cooperation project with Bavaria Fiction, we wanted to answer the question: "Are generative AI and 3D reconstruction tools already feasible enough to be integrated in the production of a VFX scene for a TV series? In this case, we investigated and tested the integration of generative AI tools for 2D and 3D assets, computervision-based motion capturing, and Gaussian splats in an episode of the detective series SOKO Stuttgart broadcasted by ZDF. In particular, we found out that these tools are already capable of creating visual appealing content (in the case of a TV series), but the whole process is still in need of human creativity and technical knowledge to produce meaningful results. My focus in this research project was to sketch a workflow that integrates generative AI tools for 2D and 3D assets as well as the computervision-based motion capturing tool making it usable in combination with the Unreal Engine.

Launch of the first Virtual Production Setup at HFF Munich

Virtual Production, in particular In-Camera VFX, is a newly established filmmaking technology, which was introduced in the Disney series "The Mandalorian". It merges real-time rendered CGI backgrounds with live-action sets by utilizing game engines (in our case the Unreal Engine) and an LED volume. Our team's mission was to integrate this technology to the HFF Munich and its curriculum. In the course of two years, we acquired the needed hardware and software resources with the help of specialists from Macom and Pro Video, educated ourselves with the support of Sony and Brompton Technology, executed a test production as a proof of concept, and introduced this new technology to the students with several hands-on seminars. As HCI research, I took care of software as well as hardware and focused on making the heterogenous tool landscape as accessible as possible. During seminars, I gave students insights of working with Unreal Engine and how the softwares manage hardware components. In our test production (organized by Fineline Film) I took over the role of the VP Operator, the person that manages the whole system during film production and is responsible for the CGI background to work properly during every shot.

Collaborative Augmented Reality TV for Kids

Based on my research work, I had a look into how a concrete mobile application for AR-enhanced edutainment for kids could look like. The prototype is part of the pilot episode concept which introduces kids to basic programming skills. Their learning goal is to get familiar with simple statements and to navigate a robot to its home environment after it gets lost in an interdimension. The kids are guided by a tutor in their television and use their phone to work on their task.

Standardized Interaction Concept in VR Trainings

Standardized interactions and a maintainable base framework are crucial in virtual reality trainings that are developed for multiple customers with various use cases to guarantee a consistent software quality and training experience for every industry need. Investigating the customers' needs, existing interaction concepts in VR games/simulations, we (together with a project owner and a fellow UX designer) at Accenture worked on a base for a standardized concept planning to use it as a template for all upcoming industrial trainings for our customers PAG and Airbus.

Digitalized Credit Decisions

More efficient credit decisions for various credit grantors with the help of a digital tool that provides all necessary customer data, digital documents and an streamlined digital communication pipeline to mortgage counselors. As UX Designer I worked with an interdisciplinary team of software engineers, product owners, service designers, and fellow UX colleagues in an agile environment.

Fast digital transmission of loan documents from home via HOME

Digitalization of the document fetch process integrated in the customer portal "Interhyp Home" enabling customers to send documents from home and getting instant feedback by their mortgage consultant. As a fullstack engineer and later as UX designer I worked on this vision of a responsive web application following a mobile-first strategy as part of an agile interdisciplinary team with software engineers, product owners, service designers, and fellow UX colleagues.

46 million digital pages of loan documents processed in 2021

A document manager tool that enables mortgage counselors and their customers to work with digital documents leading to a faster work and less paper. I worked on this project in the role of a fullstack engineer and later as UX designer with an agile interdisciplinary team of software engineers, product owners, service designers, and fellow UX colleagues.

MunixTour - A Robot-Guided City Tour

How could a city tour through the city of munich look like when tourists are guided by a robot? During our Design Workshop II. We wanted to answer this question by following a whole design thinking process. We started by creating moodboards that combined the visual style of the city of munich with modern shapes. Based on these moodboards we drew several first design ideas and three user personas that we presented to a focus group. With the help of this focus group we gained insights about user needs, must have and nice to have features, as well as preferences regarding the visual design. The final product of the following design iterations is a video prototype showcasing the city tour of a family of three experiencing munich in an interactive and playful way.

Exhibition in VR

During a design course, we worked out a whole exhibition room in VR that guides visitors through the subject of NFT avatars. Leading to a whole experience from proactive creating an own avatar to visual, spatial and auditive design of the educational material and the whole room experience.

OrbitsTouch Camera

After getting promising results in our user studies regarding the applicability of interaction technique OrbitsTouch on mobile, it was time to integrate this novel touch interaction technique into a daily use case. We chose the camera app as it provides a reasonable context where it can be supportive to use indirect touch interaction. With this application we investigated also four different design variations of OrbitsTouch targeting the submenu-structure of the main settings in the camera (e.g. flash, timer).

Want some meme? - A Meme Generator

In the course of three months, we conceptualized and implemented a web-based meme generator based on a given feature list and techstack in a group of four. The fullstack application includes a ReactJS frontend where users can login, edit their profile and create their own memes (and share them on a global feed). The backend includes a ExpressJS middleware, a NodeJS server and a MongoDB database.

"Hamsterkäufe" - Data Visualization of panic buyings during the covid pandemic

During the Covid pandemic, panic buyings made the headlines. As part of our practical task in the module "information visualizations", we investigated if there was a correlation between the local incidence value and buying power. We compared eight countries and took a detailed look into eight products. As a final result, we built an interactive web application to visualize the data we collected.

Safari Tangram - Augmented Reality Puzzles for Kids

This AR application's aim was to play around with different types of AR image tags. Using the classic tangram puzzle with a twist, we implemented an AR mobile app for kids to introduce them to Augmented Reality in a playful way.

Multiplayer Jump 'n' Run Web Game: Beat the Bugs

Two programmers spent all night writing their code when suddenly they were sucked into their PC and become part of their own programm. Sadly their code is broken because all the semicolon got lost. In this co-op multiplayer jump 'n' run webgame implemented by a group of five students, two players have to work together to collect the missing semicolons, fight against bugs, and enter a combat against the compiler.

VR Game: Fly me to the mooncake

"Fly me to the mooncake" is a short singleplayer VR Rail-Looter which came to life during a university course. We developed a concept that introduces body motion as player input letting the player navigating an UFO by moving their body. The aim of the game is to rescue astrocats (astronaut cats) that got lost in the milky way and to survive the travel until arriving at a planet in the shape of a mooncake.

Less discussions in band practices

"What song should we practice next?" - This question came up a lot during our band rehearsals. So I tried to find a solution to save time instead of discussing what to play next. "What's next?" decided which song to play next by tapping a button or shaking the smartphone. The Firebase database made it possible to create a whole songlist and share the list with other devices.