Digi­tal time tra­vel with ZAUBAR

The AI-sup­port­ed Aug­men­ted Rea­li­ty crea­tor plat­form making immersi­ve tours possible

Using a smart­phone or tablet to return to the past of memo­ri­al sites or to get a glim­pse of urban plan­ning of the future: this can now be easi­ly rea­li­sed thanks to the Media­Tech Hub start­up ZAUBAR’s AR plat­form “We are tel­ling loca­ti­on-based sto­ries with our AR app. Whe­re­ver the­re is a limi­t­ed phy­si­cal loca­ti­on that can be digi­tal­ly enhan­ced,” is how COO Anne-Sophie Pan­zer descri­bes the appli­ca­ti­on. She foun­ded ZAU­BAR tog­e­ther with CEO Ste­fan Marx in 2019. What ori­gi­nal­ly star­ted as a sto­rytel­ling tool for jour­na­lists at MIZ Babels­berg has now deve­lo­ped into an app used by cul­tu­ral insti­tu­ti­ons, tou­rism com­pa­nies, fac­to­ries, foot­ball sta­di­ums and public trans­port systems. 

ZAU­BAR not only uses immersi­ve tech­no­lo­gy to crea­te pre­cis­e­ly locata­ble AR expe­ri­en­ces in pro­ject-spe­ci­fic col­la­bo­ra­ti­ons, but also offers indi­vi­du­als with no pri­or tech­ni­cal know­ledge the oppor­tu­ni­ty to crea­te their own AR tours on the crea­tor plat­form. Until now, Aug­men­ted Rea­li­ty expe­ri­en­ces had been dis­tri­bu­ted via apps or tech­ni­cal­ly infe­ri­or web AR. This year, howe­ver, has seen ZAU­BAR deve­lo­ping a new solu­ti­on via App Clips and Instant Apps, a mix­tu­re of app and web com­bi­ning the best of both worlds: instant expe­ri­en­ces wit­hout com­pro­mi­sing on quality. 

A digi­tal look back into the past

The pro­ject “The Libe­ra­ti­on”, which ZAU­BAR deve­lo­ped tog­e­ther with Baye­ri­scher Rund­funk, tells visi­tors coming to the Dach­au Con­cen­tra­ti­on Camp Memo­ri­al Site about the day when the camp was libe­ra­ted by the US Army on 29 April, 1945. The AR expe­ri­ence is pri­ma­ri­ly aimed at young peo­p­le. When visi­ting the camp, they can expe­ri­ence cer­tain sec­tions direct­ly on site and simul­ta­neous­ly in digi­tal form to see per­so­nal sto­ries of the inma­tes. The bar­racks at the Dach­au camp - of which only some are now pre­ser­ved and still stan­ding at the Memo­ri­al Site - housed around 200,000 pri­soners, many clo­se to star­va­ti­on and with 32,000 peri­s­hing. What was hap­pe­ning the­re? What did it look like back then? In the app, life-size black-and-white images cap­tu­red by a pho­to­grapher on the day of libe­ra­ti­on are super­im­po­sed onto the sur­roun­dings. This crea­tes a dif­fe­rent and direct way of com­mu­ni­ca­ting history. 

The col­la­bo­ra­ti­on with the team of the Memo­ri­al Site and the cli­ent Baye­ri­scher Rund­funk was hand­led with par­ti­cu­lar sen­si­ti­vi­ty. The “The Libe­ra­ti­on” app, which appeared in 2020 to mark the liberation’s 75th anni­ver­sa­ry, was also nomi­na­ted for the Grim­me Online Award. 

Ano­ther pro­ject with a his­to­ri­cal back­ground not only recei­ved a nomi­na­ti­on, but then took home the Grim­me Online Award in the “Know­ledge and Edu­ca­ti­on” cate­go­ry in 2023: ZAU­BAR joi­n­ed forces with WDR on the con­cept and rea­li­sa­ti­on of the “Stumb­ling Stones” pro­ject in North Rhi­ne-West­pha­lia. The 16,000 or so stumb­ling stones in the fede­ral sta­te are now regis­tered online. They remem­ber the last resi­den­ces of peo­p­le who were per­se­cu­ted and depor­ted during the Natio­nal Socia­list era. Anyo­ne sear­ching the streets with a mobi­le pho­ne can use AR in front of the mark­ed hou­ses to find out more about the depor­ted resi­dents from that time. 

Digi­tal gui­de with an enter­tain­ment factor

ZAU­BAR is curr­ent­ly working with Hamburg’s Hoch­bahn trans­port com­pa­ny on a fun tou­rist gui­de. Ham­burg has more day visi­tors than Paris. The pilot pro­ject aims to help Hamburg’s day visi­tors find the right way out of the S-Bahn and U-Bahn net­work. It’s easy to get lost at the Jung­fern­stieg sta­ti­on next to the Inner Als­ter Lake with its 14 dif­fe­rent exits. “To put it rather blunt­ly, we are trans­forming the Jung­fern­stieg into a laby­rinth. The enter­tain­ment fac­tor aims to appeal par­ti­cu­lar­ly to the youn­ger gene­ra­ti­on who are inte­res­ted in AR and VR. Enter Way­ment - Jung­fern­stieg - is the name given to the pro­ject with a play on the words enter­tain­ment and way­fin­ding,” says CEO Ste­fan Marx. The­re are also playful ele­ments enab­ling one to coll­ect points with the app in addi­ti­on to the simp­le navi­ga­ti­on ele­ments. Hamburg’s Hoch­bahn will be making the AR appli­ca­ti­on available for down­loa­ding from 6 December. 

The tech­no­lo­gy behind the AR expe­ri­en­ces is loca­ti­on-based. The ZAU­BAR team visits the loca­ti­ons in per­son on seve­ral occa­si­ons. “It’s important for us to iden­ti­fy with the site and space,” says Anne-Sophie Pan­zer. “The tran­si­ti­on from the loca­ti­on - the phy­si­cal space - to the over­lay of digi­tal con­tent - should be seamless.” 

The social com­po­nent is also important. While one can com­ple­te­ly dis­ap­pear into the digi­tal world with VR, AR makes it pos­si­ble to inter­act with others as well as to look at and expe­ri­ence some­thing tog­e­ther. One is chal­len­ged to make the move to lea­ve one’s home and go out­side. All you need for this inter­ac­tion is a mobi­le pho­ne that ever­y­bo­dy is now car­ry­ing with them. In future, incre­asing use will be made of Visi­on Pro and other hard­ware, but the focus at the moment is on pro­vi­ding a lean and very user-fri­end­ly technology. 

Loca­li­sa­ti­on via VPS or mar­kers with QR codes

ZAU­BAR uses VPS Visu­al Posi­ti­on Sys­tem tech­no­lo­gy in public places. If you scan a cer­tain side of a street, the sys­tem will reco­g­ni­se it and can visua­li­se whe­re someone is stan­ding right down to the nea­rest cen­ti­met­re. The team works with mar­kers and QR codes at loca­ti­ons like the Dach­au Con­cen­tra­ti­on Camp Memo­ri­al Site and pri­va­te pro­per­ties. In the case of an exhi­bi­ti­on at the Kunst­pa­last Düs­sel­dorf, the pain­tings them­sel­ves beca­me mar­kers in the app. Ano­ther pro­ject with the Hohen­schön­hau­sen Memo­ri­al Site shows AR in the per­ma­nent exhi­bi­ti­on in the cel­lars - the for­mer place of work for the women impri­so­ned the­re. The­re should be as litt­le inter­fe­rence as pos­si­ble with such a his­to­ri­cal site, and that’s why volu­metric cap­tu­re of con­tem­po­ra­ry wit­nesses can also be acces­sed via mar­kers on the walls. 

As an alter­na­ti­ve, ZAU­BAR recom­mends its cus­to­mers to use the simp­ler opti­on of film­ing with green screen, which anyo­ne can do them­sel­ves the­se days using a green back­ground and image editing tool. Volu­metric cap­tu­re may be very rea­li­stic in its 3D scan­ning of peo­p­le, but even a two-dimen­sio­nal repre­sen­ta­ti­on of con­tem­po­ra­ry wit­nesses can be fit­ting to tell a sto­ry for VR. This means that ever­yo­ne can crea­te their own vir­tu­al rea­li­ty tour in just a few easy steps - as the Ber­lin city gui­de Jeff Man­nes did. He uses ZAU­BAR for his the­med tours such as the “Histo­ry of LGBTQ Ber­lin” which he runs in the dis­trict around Nol­len­dorf­platz. Par­ti­ci­pan­ts are able to use AR to call up images and por­traits of the most important prot­ago­nists of times gone by. AR is used to replace the orga­nic super­mar­ket on the cor­ner with an image of the El Dora­do club, the Berg­hain of the 1920s, which stood on the same spot at that time. 

Regu­lar meet­ups for the XR com­mu­ni­ty in the capi­tal region

Con­tent crea­ti­on is often time-con­sum­ing and expen­si­ve. The­re will be grea­ter deploy­ment of AI in the future. This is some­whe­re whe­re the ZAU­BAR team sees a gre­at oppor­tu­ni­ty in beco­ming auto­ma­ted in the crea­ti­on of worlds and con­tent in 3D. As Anne-Sophie Pan­zer notes: “We are expe­ri­men­ting a lot with gene­ra­ti­ve AI alt­hough the terms for AR and VR are beco­ming incre­asing­ly blur­red. And we are working on deve­lo­ping stan­dard tech­no­lo­gies, trai­ning them for us and inte­gra­ting them into our CMS.” 

ZAU­BAR is working clo­se­ly with Deut­sche Tele­kom to give a boost to the local com­mu­ni­ty in Ber­lin and help build up a net­work around Mixed Rea­li­ty. They orga­ni­se regu­lar meet­ups in Telekom’s Hub­raum Space in Schö­ne­berg. Here peo­p­le have the chan­ce to try out Visi­on Pro, test pro­ducts and exch­an­ge ide­as with other XR deve­lo­pers and enthusiasts. 

All tho­se inte­res­ted are very wel­co­me to attend. Infor­ma­ti­on about the next meet­up can be found at meetups@​zaubar.​com.

About MTH Blog

The media technologies of the future are already being used today – not only in the entertainment sector, but also in a wide variety of industries. Christine Lentz meets up with tech enthusiasts, established companies and researchers for our monthly MediaTech Hub Potsdam blog to tell the stories behind the innovative business models.