Case Study: How Vintage Cloud Uses Clarifai’s Visual Recognition API To Archive & Document Old Films

Vintage Cloud Smart Indexing

How Vintage Cloud uses Clarifai to help customers transform the value of their film archive assets

The history of film goes back over 100 years – to the 1890s, in fact. Since that time, it’s estimated that more than half a million movies have been created. It wasn’t until 1925 that John Logie Baird gave his first demonstration of television – and it’s probably conservative to say that over 40 million hours of content have been created in the 90+ years since then.

But movies and TV aren’t the only sources of shows and films that were committed to film. Think of all the commercials, all the documentaries, all the in-house productions, all the public information footage that’s been created – and that’s just scratching the surface.

And here’s the thing. Much of that material has been lost – but huge amounts of it remain, in a wide variety of film archives around the world – both commercial and non-commercial. Those huge amounts of content have significant social and historical value. More than that: that content also has real monetary value.

The challenge
There are, however, difficulties in working with this archive material. The first challenge is that film degrades over time. It deteriorates if it hasn’t been stored in ideal conditions; it gets scratched through mis-handling; and sprocket holes start to fray through repeated playing. It’s valuable – but incredibly fragile. For many organisations, that means that capturing it in digital form has become a matter of urgency – before its physical condition means that it becomes inaccessible.

Rising to that first challenge is relatively straightforward. The second challenge is much more difficult: understanding exactly what content is included within the recording. A documentary producer might be looking for, for example, footage of a steam train in the 1930s. How to locate samples of the appropriate footage?

Vintage Cloud is a company that was created to address these challenges – and the opportunities that solving those challenges presents. It specializes in transferring old film archives into digital files. In itself, that’s a fairly well-understood process – so long as appropriate care is taken of the fragile source material.

What’s much harder than simply converting the content into computer-storable, computer-retrievable files is to make sense of what those files contain – so that their value can be preserved and, perhaps, monetized. What’s needed is ‘metadata’ – information about the content, in as much detail as possible. Unfortunately, with much archive material, that metadata just doesn’t exist. Where it does exist, it’s often in rudimentary, unstructured, written form that requires substantial editing.

The solution
Vintage Cloud quickly realised that the secret to successful transfer of archive material to the digital world lay not just in capturing the source – but making the extraction/creation of metadata from the source as simple, straightforward and fast as the physical transfer. The company quickly discovered that starting from scratch to solve the problem was the only way forward.

A period of investigating the possible alternative approaches led Vintage Cloud to Clarifai and its artificially intelligent visual recognition system – which, in effect, can take the place of a human being viewing all the footage in its entirety while frantically taking notes.

“We found the tagged results amazing,” says Peter Englesson, CEO of Vintage Cloud. “The Clarifai vision recognition API found objects that human eyes never would have found – especially not in real time. The results were stunning – even though much of our footage consists of “vintage” images, those items that you can’t find in today’s world. Take something like helmets, for example. Helmet design has changed radically over the last hundred years – but Clarifai AI is able to identify helmets from 1920 up to today. That’s astonishing.”

That ability to automatically detect objects that appear in the archived material – not only with the highest accuracy, but also far faster than a human eye could ever achieve the feat – means that the creation of easily searchable metadata is transformed, as is the value of the archive asset.

Vintage Cloud has built a platform that automatically creates a keyframe every third second, while digitizing old film content, and sends the image to the cloud – returning it, fully indexed, in real time.

“Thanks to my background as a film editor, I recognise the power of keyframes,” says Englesson. “It allows you to overview a massive amount of film footage without the need to view hours of video.”

Clarifai AI produces an enormous amount of tags while indexing. The Vintage Cloud “Smart Indexing” platform has built-in efficient “search” functionality to filter out, for the time being, tags that are not needed.

“This function makes it very easy for our clients to search throughout their whole catalogue for items or tools,” continues Englesson. “For example: if the client searches for ‘vintage car’, he’ll find all the instances within the archive where cars built between 1920 and, say, 1970 can be found.”

Vintage Cloud’s Smart Indexing capability is an integral part of its Steenbeck Digitizer – but it also works as a standalone application for material that has already been digitized. The fact that Vintage Cloud is a leader in helping owners of film archives derive the true value of their assets is due in no small part to the remarkable technology that it leverages from Clarifai.

WordPress Video Lightbox Plugin