How the dwerft research alliance is advancing structural change in the film and TV industry
Hasn’t the film world been at the forefront of digitisation with its digital visual effects and streaming platforms since works like Avatar or Inception and platforms à la Netflix? But even if we have long since said goodbye to 35 mm and can create entire galaxies with bits and bytes: the production chain in the background is anything but digital.Research findings to be presented at the MediaTech Hub Conference
The research project dwerft - linked metadata for media wants to change this and has been researching metadata and artificial intelligence in the film and television industry at its Babelsberg location for the past three years. The research project for innovative media technologies, which was funded by the Federal Ministry of Education and Research, is now in its final phase. The findings and first prototypes will be presented for the first time during the MediaTech Hub Conference in Potsdam on 10 and 11 November.
“What is going to come after artificial intelligence? This is the credo we adopted,” says dwerft project coordinator Claudia Wolf when explaining their approach. Artificial intelligence (AI) is already being applied to finished content and can be trained to search for images, sound sequences and text segments, for example. But metadata is needed in order to be able to operate the search algorithms correctly. And the better this is prepared, the better the AI can evaluate it. It is only then that the audiovisual value chains can be fully exploited and production processes well automated.
Metadata is thus the determining factor when it comes to the future of film distribution, marketing and automation. It is relevant for the end user when you are talking about film suggestions on streaming platforms or very specific search queries: romantic films taking place on Valentine’s Day? A nice autumn film with scenes in Central Park? Or the best films with a soundtrack set to music of the 70s? All this can be prepared and displayed with the right metadata linking.
Automated production processes and distribution thanks to metadata
More important for film production, however, is the metadata which already exists during shooting and pre-production. A script by itself already contains information about actors, costumes, times, weather and locations, but this information is no longer available as processed data later on in the production and often gets lost. This metadata has to be prepared correctly and synchronously during the production process so that it can be accessed at a later stage for the appropriate deployment.
Claudia Wolf: “There are currently data silos in the production phases that should actually be linked together. The dwerft solution aims at storing metadata during the actual production and ensuring that there is a suitable value chain for this data. Our goal is to have the best possible preparation so that it can be made available to all activities at any time.”
The metadata of soundtracks, images, costumes, etc. are recorded, categorised and aggregated. Technical linking is needed in order to be able to really make use all of this additional information. To this end, the dwerft project has developed structures and researched semantic linking. The participating partners - DRA Stiftung Deutsches Rundfunkarchiv, filmwerte GmbH, Interlake Media GmbH, Rotor Film and transfermedia production services - have contributed their expertise and partly their own software solutions: the Deutsches Rundfunkarchiv for archiving, filmwerte with video-on-demand, or Rotor Film with its own plug-in for post-production. The Linked Media Data Cloud is the core technology providing the basis for other software solutions. The data is mapped there so well that it can be accessed depending on the purpose - in other words, a knowledge database where a lot of semantic groundwork has already been done.
Use and marketing of music and social media
The dwerft managers will also be presenting a special case with GEMA at the MediaTech Hub Conference. Film music is an important factor in the production and exploitation of films. To start with, every music right for a soundtrack has to be requested separately from GEMA. This involves manual lists being prepared and submitted. But then there are films which often have their own compositions, and these in turn have to be sent to GEMA for rights exploitation. dwerft was involved in working on enabling producers to link their source music directly in future to the GEMA database which then provides them with all of the necessary information. Every type of music is linked to a specific time code so that it can be identified at the right place. This is something that doesn’t now have to be done at the end of the production process, but runs directly in an automated format during production.
Metadata during the production process also has an influence on future film and television productions. So-called predictive analytics based on artificial intelligence automatically suggest leading actors and locations or make comparisons with similar films and their marketing - even before a shoot has taken place. This also includes social media marketing. “In the case of film distribution, a lot of data is generated on social media, for example, about directors, actors, stories or locations. With the database, our dwerft ontology, we link them together and give them a structured form,” says Claudia Wolf. dwerft will also be joined by cooperation partners on the panel at the MTH Conference in November 2021 to present such a case study as well as other business processes associated with the Linked Media Data Cloud.
With the funding phase now coming to an end, the research project is going out into the big wide world and will develop the individual prototypes to market maturity. Among other things, the dwerft alliance will be one of the exhibitors in the German Pavilion at the IBC in Amsterdam from 3 to 6 December, 2021.
About MTH Blog
The media technologies of the future are already being used today – not only in the entertainment sector, but also in a wide variety of industries. Christine Lentz meets up with tech enthusiasts, established companies and researchers for our monthly MediaTech Hub Potsdam blog to tell the stories behind the innovative business models.
More blog articlesAbout MTH Blog
The media technologies of the future are already being used today – not only in the entertainment sector, but also in a wide variety of industries. Christine Lentz meets up with tech enthusiasts, established companies and researchers for our monthly MediaTech Hub Potsdam blog to tell the stories behind the innovative business models.