CognitiveXR

Humans perceive their environment through their senses. However, our environment, especially in urban areas, is becoming increasingly complex. Augmented reality can support humans by providing additional digital and contextual information and offers "cognitive assistance", for example, in confusing traffic or in the case of visual impairments. The research project CognitiveXR focuses on developing a platform that enables cognitive augmentation in the smart city domain by seamlessly integrating augmented reality, edge computing, and artificial intelligence.

Dreigeteiles Bild. Ganz links: Autosymbol in Augmented Reality, das mithilfe der AR-Brille bereits im Vorfeld anzeigt, dass ein Auto von links kommt. Mittig die Straße, auf der das Auto auftauchen wird. Rechts oben: Eine Userin mit HoloLens.

Use case of how AR and cognitive assistance can improve safety in urban traffic: a car that is actually still out of sight is "announced" to the user via HoloLens with the help of real-time traffic data.

Überblicksgrafik über Funktionsweise und Möglichkeiten der CognitiveXR-Plattform

The CognitiveXR platform enables the scalable operation of services for cognitive assistance applications on a distributed, city-wide edge infrastructure. Among other things, software is provided that allows AR developers transparent access to the platform.

Blick auf einen Menschen, der durch ein weiß umrandetes Quader via Hololens hervorgehoben wird

View through the HoloLens: Through targeted object recognition, the CognitiveXR project supports the development of augmented reality solutions for cognitive assistance systems.

Blick durch eine AR-Brille auf eine Kühltheke. In der Bildmitte zeigt ein grünes Rechteck sowie die Information darüber, dass sich an dieser Stelle Joghurt befindet.

Use case of how AR and cognitive assistance can help visually impaired people in everyday life, for example, during shopping by recognizing, naming and locating individual items.

The unique selling point of CognitiveXR is the development of a novel edge computing infrastructure for urban environments. As first prototypes of Smart Lamppost, conventional webcams are connected to edge computing nodes to track objects or users and provide position information for augmented reality cognitive assistance applications.

Cognitive assistance applications have been an important field of research in the extended reality sector for many years to support people in their everyday lives. Through augmented reality technology, the lives of urban and digitalized populations can be enhanced in a very tangible way and in real-time, for example, by supporting citizens to cycle through the city or helping visually impaired people move around in everyday life or to go shopping. Modern wearable augmented reality (AR) devices, recent advances in artificial intelligence, and the wide availability of sensory environmental data and digital information are now opening up entirely new possibilities, but also entail quite a few challenges, for example, in the area of computing power.

This is where the research project CognitiveXR comes in. The platform developed as part of the project, which combines edge computing technology with artificial intelligence and augmented reality, makes it possible to move freely in an area of any size, from laboratory environments to smart cities, using common AR wearables, while receiving information in real-time.

The CognitiveXR platform provides straightforward integration capabilities for the latest mixed reality wearable technology such as the Microsoft HoloLens, Vuzix Blade, or nReal. This makes it easy for AR developers to build services based on CognitiveXR infrastructures that use AR glasses for cognitive assistance solutions. These solutions can, for example, use real-time traffic data to increase the traffic safety of cyclists in the city by pointing out dangerous situations, sharpen the situational awareness of emergency forces through background information, or even support people with visual impairments in their everyday lives through AI-supported object recognition.

The unique selling point of CognitiveXR is the combination of AI and state-of-the-art edge computing technology to optimize speed and processing power. This not only decisively increases the usage radius of AR wearables - opening up entirely new possibilities for use in smart city infrastructures - but also allows relevant information for users to reach them faster than ever before.

VRVis' main task in this project is the optimization of augmented reality technology for the implementation in cognitive assistance systems according to specific target groups. Our know-how in this area is based on years of experience in the development of VR and AR solutions for safety training or simulation scenarios, e.g., for barrier-free accessibility for people with eye diseases.

The CosmoWeather research project aims to develop VR and AR solutions for experts and the general public to make the research field of space weather more accessible and understandable.

The goal of the project MARAMT is to develop a software framework to significantly reduce the effort required to work with existing and future complex cyber-physical systems.

The applied research project Lightbox 2.0 focuses on the development of a photogrammetric 3D scanner for automatic and deep learning-based modeling of all kinds of keys.