Privacy advocates write to Interception of Communications Commissioner

Today ORG, Liberty, Privacy International, Big Brother Watch and Professor Ross Anderson of University of Cambridge will write to the Interception of Communications Commissioner to ask about apparent mistakes in his 2011 report into how effective the RIPA oversight regime is.

In his report the Commissioner tries to calculate the ‘error percentage’ in RIPA requests. Which is basically a way of trying to say how often mistakes are made by those with powers to request data under RIPA. The consequence of these sort of mistakes is information potentially being disclosed when it should not.

The figure has been used by the Home Office to demonstrate how few errors there are and how well RIPA works to guard against unauthorised use – for example the 2010 figure (which was 0.3%) is cited in their Privacy Impact Assessment for the Snooper’s Charter (or to give it its official name the draft Communications Data Bill). In his 2011 report, the Commissioner states that the figure is 0.18%. 

However, we’re pretty sure this is incorrect. The figure seems to have been worked out by dividing the number of errors he has discovered or had reported to him by the total number of RIPA requests. But the IoCC and his team don’t look at every single request. They take a sample. And the sample size is not published.

As we say in our letter, that means the reported error figure of 0.18% is effectively useless. Assuming we’re correct, it only identifies the error percentage rate for the total number of RIPA requests if the Commissioner is confident that there are zero further errors in the uninspected requests.

We have already asked for more information about this. The IoCC said they could not publish it. Further, the Prime Minister’s Office have acknowledged they hold the relevant information but consider it exempt from the FOI Act for national security reasons, and are considering the public interest in disclosure.

A clear picture of the error percentage is important to help us judge whether the powers to collect and access communications data are working. At the moment, this problem is getting in the way of a proper consideration of the draft Communications Data Bill – which is proposing to extend the current oversight regime to a much broader set of data.

So it needs clearing up. You can read the full letter below.

Friday 24th August 

Dear Sir Paul,

We are writing to you about the number of errors you discovered through your team’s inspections, and to express concerns about the conclusions you draw regarding the overall ‘error percentage’ in RIPA requests for communications data.

We welcome the increased breadth of information disclosed in your 2011 Annual Report. Transparency is an important part of any effective scrutiny regime, and at no time is this function more vital than when safeguarding against the unlawful access of private communications data.

Of the 494,078 requests for communications data in the reporting year 2011-2012, you state that “895 communications data errors were reported to my office by public authorities”. Later in the document, you disclose that 99 of those errors were identified by your own inspectors, and not reported by public authorities. Thus, 11% of all errors identified within the Report were only uncovered following your inspections, which examined a random sample of those 494,078 requests. This figure demonstrates the importance of independent scrutiny, and we laud your transparency in permitting its disclosure within the report.

We note, however, that you do not detail the size of the sample inspected, making further accurate independent analysis of this aspect of your report impossible. Based upon those 895 identified errors, you declare that the “overall error percentage rate” is 0.18%; a conclusion we assume to have been reached by the following calculation:

(895/494078) x 100 = 0.18%

Your inspectors have not examined each of the 494,078 requests but, rather, a subset of that total. Thus, with respect, your ‘error percentage rate’ cannot be correct: the calculation assumes that within the uninspected remainder there are no further errors.

A more accurate (although still imperfect) calculation would establish the “error percentage rate” of the random sample, and apply that percentage to the total number of requests. If we assume, for example, that 10,000 requests were scrutinised by your team, the 99 errors identified would equate to an “overall error percentage rate” five times greater than your conclusion within the report:

(99/10000) x 100 = 0.99%

On this assumption, there remain a further 4784 undiscovered errors within the pool of 494,078 requests.

A clear picture of the error percentage is critical for determining the necessity and proportionality of powers used to collect and access communications data. It facilitates a proper understanding of the likely ‘collateral intrusion’, and helps us to understand the likely frequency of false positives. 

We are concerned that a lack of clarity, or imprecision, in the analysis of error rates under the current RIPA regime may be inhibiting proper scrutiny of the draft Communications Data Bill. For example, the overall error percentage rate from 2010 (0.3%) is cited on page 11 of the Home Office’s Privacy Impact Assessment for the draft Bill as evidence of how robust the current oversight regime is. As we explain above, we are unable to accept the accuracy of this figure.

Accordingly, we appeal to you to clarify how your calculations are made and what advice on statistics you have had, and to disclose the number of requests your team inspected.

Yours sincerely,

Professor Ross Anderson FRS FREng, University of Cambridge
Gus Hosein, Executive Director, Privacy International
Jim Killock, Executive Director, Open Rights Group
Nick Pickles, Director, Big Brother Watch
Rachel Robinson, Policy Officer, Liberty