Top Video Artificial Intelligence and Machine Learning at NAB 2018

Video artificial intelligence was a massive theme at NAB 2018 with a majority of the video publishing technology companies showing off some form of AI integration. In my previous post How Artificial Intelligence Gives Back Time time is money in the video publishing business. AI is set to be a very important tool why all the big guns like AWS (Elemental), Google (Video Intelligence), IBM (Watson) and Microsoft (Azure) had digital AI eye candy to share. There was a feeling of a meet too with all of them were competing to weave their video annotation/labeling – speech to text APIs into a variety of video workflows.

Top Video AI use cases:

  1. Labeling – The ability to label the elements within a video specific scene selection, people, places, and things.
  2. Editing – Segmenting by relevance, slicing up the video into logical parts and producing.
  3. Discovery – Using both annotation and speech to text to expand metadata for funding specific scenes within video libraries.

Challenges

One of several challenges is this ALL or nothing situation. Video publishers assets can be on many hard drives or encoded without lots of metadata. There are companies that provide services like Axel to index those videos and make them searchable with a mixed model of on-prem tech and cloud services. Dealing with live feeds requires hardware and bigger commitments. Most publishers are not willing to forklift over their video encoding and library to another provider without a clear ROI justification. The other big ROI challenge is video publishers don’t have a lot of patience and the pressure to increase profits on video is higher now with more competition in the digital space across all channels. Selling workflow efficiency in AI won’t be a big enough draw over AI generating substantial revenue solving a specific problem. The pain isn’t high enough to make a big AI investment. There are lots of POC right now in the market, however, not one product creates a seamless flow within a video publisher’s existing workflow. Avid and Adobe are well positioned for the edit team since their products are so widely used. Other cloud providers are enabling AI technology not a specific solution.

AI Opportunity

Search and discovery was the biggest theme using AI to do image and speech to text analysis. Compliance with Closed Caption to make video accessible in digital will be mandated driving faster adoption. Editing video via AI is in its early phase, however, the technology is emerging fast. There are some exciting examples of AI created video but at scale is another. Of the many talks at NAB some exciting direction on AI in Video were discussed around video asset management. Here are a few examples of what we demoed at NAB 2018 showing promise in the video intelligence field.

Adobe Sensi

Adobe Video SegmentingAdobe had a big splash with their new editing technologies and using AI to enhance the video editing process. Todd Burke presented Adobe Sensi their AI entry into video intelligence. The video labeling demo and scene slicing we’re designed to help editors create videos faster and simplify the process. The segmenting was just a prototype and video labeling demonstrated the API extension integrated within Sensi. Adobe Labeling Demo

IBM Watson

IBM Watson Video SegmentingIBM’s demo was slick and pointed to the direction of using machine learning to process large amounts of video to pull out interesting parts of the video. Doing the announcer and crowd response analysis added another layer of segmentation. You can see a live demo of their AI highlight for the Master. They did the same for Wimbelton slicing up the live feed they were powering for the events and creating “cognitive highlights”. It wasn’t clear if these highlights were used by the edit team or if this was a POC. Regardless there was both image and text analysis of the steams occurring and demonstrated the power of AI in the video.

Avid

Avid Video analysisThe Avid demo was just that. They created a discovery UI on top of API’s like Google Vision to assist in the video analysis for search and supporting edit teams. Speech to text and annotation in one UI has its advantages. It’s wasn’t clear how soon this was going to be made available over a development tool. Avid Labeling

Google Vision

Google Vision Zoro labelingThe team over at Zora had by far the slickest video HUB approach. I believe the play for Google is more around their cloud strategy trying to attract storage of the videos and leverage their Video Intelligence to enable search over all your video assets. Google’s video intelligence is just getting started and their opening up of the AI foundation Tensorflow makes them one of the top companies committed to video AI. I like what Zora is doing and can see editing teams benefiting from this approach. There was a collaborative element too.

Microsoft Azure

Azure GreyMeta2GrayMeta UI was slick and their voice to text interface was amazing. This was all powered by Azure. Azure Video Indexer is the real deal and ability to identify face identification has broad use cases. Indexing voice isn’t new but having a fast and slick UI  helps enable adoption of the technology. They can pinpoint parts of the video just on the text along. There is a team collaboration element around the product have a Slack feel. The approach was making all media assets searchable.

AWS Elemental

There were several cool examples of possibilities with Amazon Rekognition - video analysis, facial recognition and video segments. Elemental (purchased by Amazon) core technology is a video ad stitching whereby video ads are inserted into the video directly. They created UI extension demonstrating some possibilities with Rekognition.  It wasn’t clear what was in production over the demo. The facial recognition around celebrities looked solid. AWS Singular Analysis Tracking PeopleElemental had a cool real-time object detect bounding boxes showing up on sports content too. This has many use cases, however, creating more data for video publishers to access when the amount of data they can manage needs to be addressed before adding another data firehosed. AWS Elemental label celebrity words SM

Conclusion

Video artificial intelligence is just getting started and will only improve with greater computing advancements and new algorithms. The guts of what’s needed exist to achieve scale.  The major use cases around video discovery and search are set to improve dramatically with industry players opening up more API’s. Video machine learning has great momentum utilizing these API’s to crack open the treasure trove of data locked away inside of video. The combination of video AI and text analysis truly creates a massive metadata for the multitude of use cases where voice computing can play are roll. Outside of all the AI eye candy there needs to be more focus on clear business problems vs. Me Too. More like what’s the end product and how will it make the video publisher more revenue?

FORBES: InfiniGraph Invents Video Thumbnail Optimization

Bruce Rogers ForbesBruce Rogers, FORBES STAFF
I’m Forbes’ Chief Insights Officer & write about thought leadership.
Originally posted on Forbes

A Series of Forbes Insights Profiles of Thought Leaders Changing the Business Landscape: Chase McMichael, Co-Founder and CEO, InfiniGraph

Optimizing web content to drive higher conversion rates, for a long time, meant only focusing on boosting the number of click-throughs, or figuring out what kinds of static content got shared most often on social media sites.

But what about videos? This key component of many sites went largely overlooked, because there simply wasn’t a good way to determine what actually made viewers want to click on and watch a given video.

Chase_McMichael_Video_Machine_Learning_Headshot_2_2016

Chase McMichael, Co-Founder and CEO, Infinigraph

In an effort to remedy this problem, says entrepreneur Chase McMichael, brand managers may have, at most, tried to simply improve the video’s image quality. Or, in a move like a Hail Mary pass, they might have splashed up even more content, in the hopes that something, anything, would score higher click-to-play rates. Yet even after all that, McMichael says, brands often found that some 90% of viewers still did not watch the videos posted on their sites.

As it turns out, the “thumbnail” image (static visual frame from the video footage) has
everything to do with online video performance. And while several ad tech companies were already out there, using so-called A/B testing to determine how to optimize the user experience, no one had focused on optimizing video thumbnail images. Given video’s sequencing speed with thousands of images flashed up for milliseconds at a time, it meant that measuring the popularity of thumbnails was simply too complex.

Sensing a challenge, McMichael, a mathematician and physicist with an ever-so-slight east Texas drawl, set out to tackle this issue. He’d already started InfiniGraph, an ad tech firm aimed at tracking and measuring people’s engagement on brand content. But as his company grew, he found that customers began asking more and more about how they might best optimize web videos in order to boost viewership.Panthers_Video_Machine_Learning_iPhoneKRAKEN (1)

Viewership, of course, is key: Higher video viewership translates into more shares; more shares means increased engagement. And that all translates into more revenue for the website. Premium publishers are limited in their ability to create more inventory because the price of entry is so high. These new in house studios are producing quality content, but getting scale is a huge challenge.

When he started looking into it, McMichael says, he often found that the thumbnails posted to attract viewers usually fell flat and the process for choosing thumbnails hasn’t changed in 15 years. And the realization that the images gained little to no traction among viewers came as something of a surprise: Most of the time, the publishers and brand managers themselves had selected specific images for posting with no thought at all into optimizing the image.

According to McMichael, the company’s technology (called “Kraken”) solves for two critical areas for publishers: it creates inventory and the corresponding revenue while also increasing engagement and time spent on site.

Timing, it turns out, was everything for McMichael and InfiniGraph. Image- and object-recognition software had been improving to the point where those milliseconds-at-a-time thumbnails could be slowed down and evaluated more cheaply than in the past. Using that technology along with special algorithms, McMichael created Kraken, a program that breaks down videos into “best possible” thumbnails. Using an API, Kraken monitors which part of the video, or which thumbnail, viewers click on the most. Using machine learning, Kraken then rotates through and posts the best thumbnails to increase the chances that new users will also click on the published thumbnail in order to watch an entire video.

This process is essentially crowd-sourced, says McMichael—the images that users click on the most are those that Kraken pushes back to the sites for more clicks. “What’s fascinating is we’ve had news content, hard news, shocking, all the way up to entertainment, music, sports and it’s pretty much universal,” he says, “that no one [person] picks the right answer”—only the program will provide the best image or images that draw in the most clicks. On its first few experimental runs, InfiniGraph engineers discovered something huge: By repeatedly testing and re-posting certain images, InfiniGraph saw rates of click-to-play increase by, in some cases, 200%. Says McMichael: “It was like found money.”

InfiniGraph is a young and small company, even for a start-up: The Silicon Valley firm has eight employees in addition to a network of technicians and specialty consultants he scales on and as-needed basis, and has boot-strapped itself to where it is today. McMichael says he’s built a “very revenue-efficient company” because “everything is running in two data centers and images distributed across a global CDN.” His goal is to be cash-flow positive by this summer. Right now InfiniGraph works exclusively with publishers but the market is ripe for growth, especially in mobile devices, McMichael says.

Recently, Tom Morrissy, a publishing leader with extensive experience in both publishing (Entertainment Weekly, SpinMedia Group) and video ad tech (Synaptic Digital, Selectable Media) joined InfiniGraph as a Board Advisor.

“So many companies claim to bring a ‘revenue generating solutions that is seamlessly integrated.” This product creates inventory for premium publishers and is the lightest tech integration I’ve seen. I was completely impressed with Chase’s vision because he truly thought through the technology from the mindset of a publisher. Improve the consumer experience and the ad dollars always follow” says Tom Morrissy

The son of a military officer father and registered nurse mother, McMichael grew up in the small town of New Boston, Texas, located just outside of the Red River Army Depot. A self-described “Brainiac kid,” McMichael says he was always busying himself with science experiments, with a special interest in superconductors, or materials that conduct electricity with zero resistance. Though he’d been accepted to North Texas, McMichael still took a tour at the University of Houston, mainly because the work of one physics professor who discovered high temperature superconductivity had grabbed his attention. “So I went to Paul Chu’s office and said, ‘hey, I want to work for you.’” It was the craziest thing, but growing up I was always told, ‘If you don’t ask for it, you won’t know.’”

That spawned the beginning of seven-year partnership with Chu during which time the University built a ground-breaking science center. McMichael spent seven years in DARPA funded applied science, but decided to leave for the business world. A friend of McMichael’s worked at Sun Microsystems and encouraged him to leverage his programming knowledge. His first job out of college was creating the ad banner management system for Hearst. “So I got sucked into the whole internet wave and left the hard-core science field,” he says. He also worked at Chase Manhattan Bank in the 90s, building out its online banking business.

As for the future for InfiniGraph?

McMichael says his mission is “to improve the consumer experience on every video across the globe, and it’s an ambitious plan. But we know that there are billions of people holding a phone right now looking at an image. And their thumb is about to click ‘play,’ and we want to help that experience.”

Bruce H. Rogers is the co-author of the recently published book Profitable Brilliance: How Professional Service Firms Become Thought Leaders - Originally posted on Forbes