Advertisement

FBI using Twitter's raw 'firehose' data to monitor users

With Dataminr, the bureau has "near real-time" access to live tweets.

AP Photo/Patrick Semansky

The FBI is using a Twitter tool called Dataminr to track criminals and terrorist groups, according to documents spotted by The Verge. In a contract document, the agency says Dataminr's Advanced Alerting Tool allows it "to search the complete Twitter firehose, in near real-time, using customizable filters." However, the practice seems to violate Twitter's developer agreement, which prohibits the use of its data feed for surveillance or spying purposes.

It's not clear if Twitter has authorized the bureau to access Dataminr, but Engadget has reached out for comment. It cut off another company, Geofeedia, for using its raw feed to track users in a similar way. However, Twitter has a closer relationship with Dataminr; it owns five percent of the firm and has given it a virtual monopoly on its data feed. As the FBI points out in a "limited source justification" document, "Dataminr is the only certified Official Twitter Data Partner with direct proprietary access to the full 'Twitter firehose' which allows it to provide near real-time alerts of breaking news and events."

US intelligence owns a venture capital firm called In-Q-Tel that invested in Dataminr and other social media analytics firms. Twitter reportedly authorized a pilot program that let the CIA and other agencies scan users, but cut access off after it ended. Like Apple, Google and other US tech companies, it wanted to avoid being associated with spying on both foreign and domestic users, according to the WSJ.

Dataminr is the only certified Official Twitter Data Partner with direct proprietary access to the full 'Twitter firehose' which allows it to provide near real-time alerts of breaking news and events.

During his campaign, President-elect Donald Trump promised to force social media companies to give law enforcement the encryption keys needed to access to private user data. US tech companies, especially Apple, have argued vociferously against that, saying it would undermine confidence in their platforms. Foreign regions like Europe are especially sensitive to US spying after discovering that the NSA used server back doors and other tricks to spy on its citizens (and leaders).

It's not far-fetched for entire nations to ban a social network, as Russia cut off 5 million LinkedIn users just last week. And the European Union is increasingly wary of Facebook, Google and other US tech firms.