Post­pro­duc­tion 4.0

How arti­fi­ci­al intel­li­gence and meta­da­ta are revo­lu­tio­ni­s­ing the film industry

The film indus­try in the 21st cen­tu­ry: strea­ming ser­vices are ubi­qui­tous, vir­tu­al rea­li­ty is making inroads ever­y­whe­re, and films in 3D have lite­ral­ly rea­ched new dimen­si­ons. But, if you look at the work­flows in the back­ground, the film indus­try is still a long way from being in the digi­tal age. “Film must be thought of as IT – in that respect, the indus­try is still stuck in the last cen­tu­ry,” says Hol­ger Leh­mann, CEO of Rotor Film.

The pro­duc­tion com­pa­ny is an expert for all stages of post-pro­­duc­­ti­on from dai­lies during shoo­ting, through image and sound design to the film mas­ter. They have work­ed both natio­nal­ly and inter­na­tio­nal­ly on such fea­ture film pro­duc­tions as The Squa­re, Iron Sky 2 or Gun­der­mann and for strea­ming por­tals like Net­flix and Ama­zon Prime (Deutsch­land 89 and Bad Banks). But, in addi­ti­on to clas­si­cal post-pro­­duc­­ti­on, Rotor Film has made it its goal to rethink post-pro­­duc­­ti­on for the future. “This would requi­re the auto­ma­ti­on of the com­plex pro­ces­ses that have been almost fac­­to­ry-based and lar­ge­ly manu­al until now,” Leh­mann explains.

Nor­mal­ly, a film will be shot, edi­ted, the sound and images are pro­ces­sed, a mas­ter ver­si­on is deli­ver­ed, and then dis­tri­bu­ted and exploi­ted. The stages in the pro­duc­tion are often a logi­sti­cal and tem­po­ral chall­enge. The­re are peo­p­le working simul­ta­neous­ly at many dif­fe­rent places in both the inter­na­tio­nal and natio­nal film indus­tries to com­ple­te a film. This results in num­e­rous arran­ge­ments being rea­ched bet­ween play­ers some­ti­mes in dif­fe­rent count­ries. The con­­tent-rela­­ted crea­ti­ve work is mark­ed by the tech­ni­cal con­di­ti­ons and vice ver­sa: for exam­p­le, when lar­ge amounts of data are trans­mit­ted, sound pro­ces­ses and colour spaces are being coor­di­na­ted, and last-min­u­­te chan­ges should still be done befo­re the final appr­oval of the film.

This is whe­re sim­pli­fied col­la­bo­ra­ti­ve working methods are nee­ded with ever­yo­ne having access to the same data. Pre­vious pro­ces­ses cost a lot of time and resour­ces. Time, as we all know, is money and resour­ces like skil­led workers are fini­te. So, how can post-pro­­duc­­ti­on 4.0 be rea­li­sed? Which addi­tio­nal busi­ness models can be gene­ra­ted around more data-dri­­ven struc­tures? This is exact­ly what Rotor Film is working on in its rese­arch alli­ance with DWerft in Babelsberg.

Meta­da­ta play a major role here: infor­ma­ti­on which is alre­a­dy gathe­red during the film pro­duc­tion and dis­tri­bu­ti­on pro­ces­ses and embedded in the film. A tech­ni­cal sub­li­ne, as it were, is added, hel­ping to clas­si­fy cer­tain ele­ments or make them quick to find. After all, the bet­ter a film is enhan­ced with meta­da­ta, the easier it is to find and all the more explo­ita­ble. The­re is a wide ran­ge of meta­da­ta that can be inte­gra­ted into the film - whe­ther it be tech­ni­cal infor­ma­ti­on, infor­ma­ti­on about its dis­tri­bu­ti­on or con­­tent-rela­­ted infor­ma­ti­on. An ide­al case would see the data con­nec­ting and inte­gra­ting ever­y­thing to do with pro­duc­tion, post-pro­­duc­­ti­on, dis­tri­bu­ti­on and archiving.

“The meta­da­ta are thus beco­ming the “fuel” for arti­fi­ci­al intel­li­gence and a vital pre­re­qui­si­te,” is how Hol­ger Leh­mann descri­bes their signi­fi­can­ce. “But the ques­ti­on is: how can we expand on the ide­as for the use of meta­da­ta? I could store infor­ma­ti­on about secon­da­ry or ter­tia­ry use in other count­ries or use it to ana­ly­se the opti­mal nar­ra­ti­ve strand for a roman­tic come­dy. Par­ti­cu­lar places in the dia­lo­gue, addi­tio­nal infor­ma­ti­on about film loca­ti­ons or music rights as well as lan­guage ver­si­ons can be spe­ci­fi­cal­ly cate­go­ri­sed. Moreo­ver, meta­da­ta can pro­vi­de the basis for a glo­bal con­tent data­ba­se.” And that is only part of the future possibilities.

Clas­si­cal meta­da­ta models are insuf­fi­ci­ent for such appli­ca­ti­ons. Rotor­film is the­r­e­fo­re working on inno­va­ti­ve methods to per­mit such data pro­ces­sing from the film shoot through to its dis­tri­bu­ti­on. For this pur­po­se, they are deve­lo­ping adap­ters and plug-ins faci­li­ta­ting the exten­si­ve gathe­ring and map­ping of this data and their strin­gent trans­mis­si­on. After all, apart from deve­lo­ping new busi­ness models, the aim is to trans­fer out­da­ted working methods into new digi­tal work­flows. Manu­al stages in post-pro­­duc­­ti­on such as noi­se reduc­tion or the dele­ti­on of visu­al con­ti­nui­ty errors could then be auto­ma­ted in the future with the help of arti­fi­ci­al intelligence.

This is the value chain Hol­ger Leh­mann at Rotor Film would now like to focus on deve­lo­ping tog­e­ther with his busi­ness part­ner Mar­tin Früh­mor­gen: “Our visi­on is to beco­me one of the lea­ding meta­da­ta pro­ces­sors for the moving image”.

By Chris­ti­ne Lentz

More blog artic­les can be read here.

About MTH Blog

The media technologies of the future are already being used today – not only in the entertainment sector, but also in a wide variety of industries. Christine Lentz meets up with tech enthusiasts, established companies and researchers for our monthly MediaTech Hub Potsdam blog to tell the stories behind the innovative business models.