The Pentagon is hunting ISIS using big data and machine learning

Its newly-minted Algorithmic Warfare Cross Functional Team is taking the lead.

AFP/Getty Images

Military and civilian intelligence analysts "overwhelmed" by the sheer volume of video surveillance data recorded over the America's numerous military incursions against ISIS will soon get some relief. The Pentagon announced on Monday that it is tasking its newly-minted Algorithmic Warfare Cross Functional Team (AWCFT), dubbed Project Maven, with using big data and machine learning to accelerate the process of discovering actionable intelligence in all that aerial imagery.

There's simply too much incoming information for the Pentagon's stable of analysts, says. Air Force Lt. Gen. John N.T. "Jack" Shanahan, director for defense intelligence for warfighter support. "We're not going to solve it by throwing more people at the problem...That's the last thing that we actually want to do. We want to be smarter about what we're doing," he told Defense One.

The idea is to reduce the amount of time analysts spend performing administrative tasks, like manually entering data into spreadsheets whenever they find something of interest, and more time actually analyzing. To that end, the AWCFT will leverage and improve upon existing Processing, Exploitation, and Dissemination (PED) capabilities and automate the data-labelling process.

The group has 90 days to complete its first phase: to either develop the algorithms itself or acquire them from the private sector. The second and third phases, each also 90 days, involve procuring the hardware needed to run the algorithms and implementing the completed system.

The need for a fast turnaround is essential, Shanahan argued. "It's no good if it's just shown and demonstrated in a research lab somewhere. We want to deliver -- in this calendar year -- some capabilities out to the warfighters that make a difference."

Should this program meet with success, Shanahan hopes to expand Project Maven's machine learning efforts throughout the rest of the armed services. "We see all sorts of things for intelligence, for targeting, for collection management, for sensor fusion. For the department... logistics, command and control, communications," Shanahan told Defense One. "Everything that [the tech industry] is working on has some applicability throughout the entire department." Hopefully that success will also cajole Congress into approving a $70 million transfer of funds until the unit gets its own budget.