Total Information Awareness Is Back

By Bruce Schneier
October 31, 2006

Remember Total Information Awareness?

In November 2002, the New York Times reported that the Defense Advanced Research Projects Agency (DARPA) was developing a tracking system called “Total Information Awareness” (TIA), which was intended to detect terrorists through analyzing troves of information. The system, developed under the direction of John Poindexter, then-director of DARPA's Information Awareness Office, was envisioned to give law enforcement access to private data without suspicion of wrongdoing or a warrant.

TIA purported to capture the “information signature” of people so that the government could track potential terrorists and criminals involved in “low-intensity/low-density” forms of warfare and crime. The goal was to track individuals through collecting as much information about them as possible and using computer algorithms and human analysis to detect potential activity.

The project called for the development of "revolutionary technology for ultra-large all-source information repositories," which would contain information from multiple sources to create a "virtual, centralized, grand database." This database would be populated by transaction data contained in current databases such as financial records, medical records, communication records, and travel records as well as new sources of information. Also fed into the database would be intelligence data.

The public found it so abhorrent, and objected so forcefully, that Congress killed funding for the program in September 2003.

None of us thought that meant the end of TIA, only that it would turn into a classified program and be renamed. Well, the program is now called Tangram, and it is classified:

The government's top intelligence agency is building a computerized system to search very large stores of information for patterns of activity that look like terrorist planning. The system, which is run by the Office of the Director of National Intelligence, is in the early research phases and is being tested, in part, with government intelligence that may contain information on U.S. citizens and other people inside the country.

It encompasses existing profiling and detection systems, including those that create “suspicion scores” for suspected terrorists by analyzing very large databases of government intelligence, as well as records of individuals' private communications, financial transactions, and other everyday activities.

The information about Tangram comes from a government document looking for contractors to help design and build the system.

DefenseTech writes:

The document, which is a description of the Tangram program for potential contractors, describes other, existing profiling and detection systems that haven't moved beyond so-called "guilt-by-association models," which link suspected terrorists to potential associates, but apparently don't tell analysts much about why those links are significant. Tangram wants to improve upon these methods, as well as investigate the effectiveness of other detection links such as "collective inferencing," which attempt to create suspicion scores of entire networks of people simultaneously.

Data mining for terrorists has always been a dumb idea.

And the existence of Tangram illustrates the problem with Congress trying to stop a program by killing its funding; it just comes back under a different name.