2008-05-11

Measuring the Effectiveness of Bulk Data Collection

While decompressing from a brutal day of studying for a crypto final, I came across an article on BBC which argues that "huge investment in closed-circuit TV technology has failed to cut UK crime." My first thought was, did they really expect it to?

A lot has been made by the media and bloggers of the efforts in London to deploy thousands of CCTV cameras, much of it surrounding civil liberties of British citizens. I'm going to set aside civil liberties concerns for now and focus on more objective measurements (not that these issues are not important, but rather they aren't important to my point here).

To sell or design a widespread CCTV system on some notion that the thought of Big Brother will somehow keep the citizenry well-behaved is so tragically Orwellian that I don't think it warrants another mention. However it was sold to the public or government, and regardless of these silly claims, measuring its success in terms of crime reduction belies the real investigative benefit of such a system: as a forensic tool.

To bring this into an area which I have more expertise, I think of CCTV in the same way that I think of full packet capture on an important network segment. How much sense would it make to have an analyst sit and watch every packet, every flow, every session that blows by this sensor? How much would I expect detection of malicious activity to increase? Not at all. Even if it were possible for an analyst to keep up with the data rate of the sensor (which is the case with CCTV), so few things happen in the timespan of the human attention span that have investigative prima facie meaning that I would expect the results to be negligible. However, when placed in the context of a known attack, suddenly benign or minute details become significant. That white van parked in a parking spot that leaves 1 minute after a robbery a block away now has some meaning. That weird base64-encoded comment in HTML is now of concern.

Active monitoring of these dragnet systems is ludicrous. If some correlative system can be built to reduce data - and that's a big if - then some limited monitoring might make sense, but we are nowhere close to having a technique that will allow us to do so and this argument is moot.

The bigger story is that only 3% of London's street robberies [are] being solved by security cameras. This is certainly concerning, but this is one slice of crime. How do these tools assist in other crimes? The information provided in that article is limited. I would like to see a comprehensive study on the forensic use of this tool by London police - perhaps one is available that I haven't seen. Both the police and the media should start focusing their attention on this aspect of the system - for critique, improvement, and measuring success. That's what we'll be doing as we build a full packet capture system at work, and how we'll be measuring its success.

1 comment:

CCTV Training said...

In Europe and the UK there will be interesting changes for both private security and national security – data protection is important.