Since its founding in 2004, Palantir has managed to grow into a billion dollar company while being very surreptitious about what it does exactly. Conjecture abounds. The vague facts dredged up by reporters confirm that Palantir has created a data mining system used extensively by law enforcement agencies and security companies to connect the dots between known criminals.

TechCrunch has received a private document from 2013 which reveals the company’s extensive trove of data analysis tools and lists many of its key clients. The document is currently being passed around as an investor prospectus for a new secondary round.

In short, the description above is in part correct. But, thanks to this leaked information, we now know far more about the secretive company.

Palantir’s data analysis solution targets three industries: government, the finance sector and legal research. Each of these industries must wrestle with massive sets of data. To do this, Palantir’s toolsets are aimed at massive data caches, allowing litigators and the police to make connections otherwise invisible. For example, a firm hired by the Securities Investment Protection Corporation used Palantir’s software to sort through the mountains of data, over 40 years of records, to convict Ponzi schemer Bernie Madoff (of all things).

Palantir’s software sits on top of existing data sets and provides users with what seems like a revolutionary interface. Users do not have to use SQL queries or employ engineers to write strings in order to search petabytes of data. Instead, natural language is used to query data and results are returned in real-time.

Clients include the Los Angeles Police Department which used Palantir to parse and connect 160 data sets: Everyone from detectives to transit cops to homeland security officials uses Palantir at the LAPD. According to the document, Palantir provides a timeline of events and has helped the massive police department sort its records.

The leaked report quotes Sergeant Peter Jackson of the LAPD stating: “Detectives love the type of information it [Palantir] provides. They can now do things that we could not do before. They can now exactly see great information and the links between events and people. It’s brought great success to LAPD. It supports the cops on the streets and the officers doing the investigations. It is a great tool. They are becoming more efficient and more effective cops. Palantir is allowing them to better serve the public.”

Palantir explains that it is a toolset for use in human analysis on its website. However, we now understand that the service is a smarter way of displaying data for analysis by humans. It is capable of building comprehensive models of activity to detect suspicious anomalies and is even able to provide immunity to fraud thanks to strategies the founders learned while still at PayPal.

Palantir’s anti-fraud system uses algorithms to detect and isolate patterns designated by analysts. This approach was inspired by combating adaptive threats at PayPal, the leaked document states. Four out of the five people on the Palantir management team worked at PayPal. Palantir co-founder Peter Thiel was also a PayPal co-founder.

The document confirms that Palantir is employed by multiple US Government agencies. One of the company’s first contracts was with the Joint IED Defeat Organization in 2006. From 2007-2009 Palantir’s work in Washington expanded from eight pilots to more than 50 programs.

As of 2013, Palantir was used by at least 12 groups within the US Government including the CIA, DHS, NSA, FBI, the CDC, the Marine Corps, the Air Force, Special Operations Command, West Point, the Joint IED-defeat organization and Allies, the Recovery Accountability and Transparency Board and the National Center for Missing and Exploited Children. The Centers for Medicaid and Medicare Services were planning on pilot testing the use of Palantir in 2013 to investigate tips received through a hotline. A second test was run by the same organization to identify potentially fraudulent medical providers in the Southern region of the US.

However, as of 2013, not all parts of the military used Palantir. The U.S. Army developed its own data analysis tool called the Distributed Common Ground System at a cost of $2.3 billion, but it is believed that it is not very popular. The leaked document cites a 2012 study where 96% of the surveyed war fighters in Afghanistan recommended Palantir.

The prospectus holds that the US military used Palantir with great success. The Pentagon used the software to track patterns in roadside bomb deployment and was able to conclude that garage-door openers were being used as remote detonators. With Palantir, the Marines are now able to upload DNA samples from remote locations and tap into information gathered from years of collecting fingerprints and DNA evidence. The results are returned almost immediately. Without Palantir, the suspects would have already moved onto a different location by the time the field agents received the results.

Samuel Reading, a former Marine who works in Afghanistan for NEK Advanced Securities Group, a U.S. military contractor, was quoted in the document as saying It’s the combination of every analytical tool you could ever dream of. You will know every single bad guy in your area.”

The U.S. spy agencies also employed Palantir to connect databases across departments. Before this, most of the databases used by the CIA and FBI were siloed, forcing users to search each database individually. Now everything is linked together using Palantir. In fact, cyber analysts working for the now-defunct Information Warfare Monitor used the system to mine data on the China-based cyber groups GhostNet and The Shadow Network.

Yet Palantir is not exclusively used by governments or law enforcement agencies. The company’s data solution works equally as well in more pedestrian pursuits.

The International Consortium of Investigative Journalists uses Palantir to gain insight into the global trade and illegal trafficking of human tissue. And, as we mentioned before, the K2 Intelligence firm was employed by the SIPC to conquer the 20 terabytes of data in its case against Bernie Madoff. The leaked report quotes Jeremy Kroll, CEO and Co-founder of K2, saying that Palantir was able to construct a story around several key events in the Madoff saga in just a couple of hours.

In the business of dealing with some of the world’s most sensitive sets of data, secrecy is clearly important to Palantir’s success. This document likely only gives a glimpse into Palantir’s true capability and reach, especially since it was current just over a year and a half ago. There’s probably a great deal of Palantir information still out there, waiting to be discovered — More than a Madoff’s worth.

Middle image via aki51