“But I expect they’ll be familiar with the correct use of /dev/random”

If you will forgive the pun “the probability is not high”.

I have as you may remember more than a passing interest in RNGs in their various forms for generating “noise” for various activities involving money systems (electronic purses / hand held electronic betting devices / transfer authentication systems /anonymity systems / micropayment systems ).
And sadly I was successfully attacking both the hardware and software RNG’s in them back in the 1980’s with external RF sources modulated with various “fault injection” wave forms (and I can safely say that “naff all” has happend since then in the way of improvment or understanding of the issues).

(For those who want to see some recent published work have a hunt on this site there is some work showing a supposadly secure 32bit RNG can be trivialy reduced down to an 8bit RNG, without the use of modulating “fault injection” waveforms also a bit of discussion for future directions for students wanting to get published papers).

Just about every system I have looked at since is gamable in some way or another.

One thing that is rarely considered is how “efficiency” compramises “security”. The simple fact is the more efficient a system is designed to be the more avenues for attack or inadvertant side channels occur.

One documented problem is that of a CPU “cache” leaking AES key bits onto the network.

What is not made clear in the majority of “papers” is just how easily time bassed attacks move up and down the system stack in modern systems.

Which brings me around to your comment,

“Recall that they’re running their system on a “real” computer, not some trivial embedded component. They will have access to considerable amounts of entropy.”

Hmm I admire your confidence in making such a statment, especialy considering the work of various of your colleagues at the lab.

Without technical details it is very difficult to say, and I very much doubt they would provide them to me (as I will certainly not sign NDA’s these days having had one been used to “gag me” in the past).

I suspect that the entropy in the system is most likley to be from external “controlable” events like network traffic. And unless speciffic precautions have been taken by the designers then controlling the data rate into the system can remove all the entropy from that and also allow internal entropy from clock drift etc to be very acurately predicted.

One of the reasons I prefer “embeded” components over “real” computers is they are much easier to design to reduce the oportunities for “gaming”.

Most OS’s in “real” computers are designed for “specmanship” including the seperation kernal OS’s for high assurance systems are still relativly easy to game in one way or another via covert channels (usually but not always time based).

The question many are likley to ask at this point is “so what? Who’s going to go to the effort? there’s no incentive / pay off”.

Unfortunatly history shows that “re-use” is a very major security weakness (see Microsoft et al ad nusium).

The system they have designed is unlikley to be gamed, however having gone to the effort of developing their “key” managment system they are likley to carry it forward into many other products.

The probability is that some of these future systems will have incentive / pay off thus if there are any weaknesses now they will be carried forward untill the weaknesses become publicaly and painfully clear.

As has been observed hundreds of years ago,

“Oft’ the ship is lost for a ha’penth of tar”,

With the obvious prevention of,

“A stich in time saves nine”.

However from the little that has been said by you and others on the Detica system, and the “open work” carried out by others at Cambridge Labs should give significant pause for thought.

My view on it is that there is way to little information to say the system is not gameable and thus the primise for the systems “anonymity” ability is at best highly questionable, based on industry experiance to date.

However if Detica want to get in contact with me in an “open way” I’d be more than happy to look over their design as the process if done in an “open way” would raise everybodies boat.

However if the “key generation process” is determanistic and you wrote the process then chances are you can reproduce the input to the “key generation process” at any time and thus “re-create” the key (or do you disagre with this?).

If it’s deterministic that’s obviously an issue. Detica are of course the people to ask if they have been completely incompetent or not. But I expect they’ll be familiar with the correct use of /dev/random

Recall that they’re running their system on a “real” computer, not some trivial embedded component. They will have access to considerable amounts of entropy.

“viz: a simple hash is indeed useless (as I think is obvious to every reader); so that’s why there’s a key!”

When I said,

‘For example I take known details and “hash” them with MD5 or whatever. Provided I don’t release the hash and delete it at some point then it will meet the criteria of the statment. However If I know the input to the hash algorithm I can reproduce the hash at any time I wish.’

I was assuming that you would be bright enough to realise I was talking of a simple example to generate a key…

The point is for there to be a “key” there has to be a “key generation process” (or do you disagree with this on principle?).

That process can be one way via a hash or whatever and thus cannot be reversed (or do you disagree with this?).

However if the “key generation process” is determanistic and you wrote the process then chances are you can reproduce the input to the “key generation process” at any time and thus “re-create” the key (or do you disagre with this?).

Now is that sufficiently well described for you?

Oh and with regards your,

“However, it doesn’t fit any of the rest of description provided here and on the Christopher Parsons blog,”

There is no technical description of a “key generation process” on either your posting or Christoper’s that can be evaluated in any meaningfull way, so it is again one of your rather pecunish arguments.

Oh and whilst I remeber a happy new year to you and all at the Labs, may it be fruitfull for all.

A known “one way” process fits that description

However, it doesn’t fit any of the rest of description provided here and on the Christopher Parsons blog, viz: a simple hash is indeed useless (as I think is obvious to every reader); so that’s why there’s a key!

Thus if the key change period is short or made at a busy time then the results could be significantly skewed.

A substantive point (at last). But not a very significant one, since you can easily adjust for this bias in and amongst all the other scaling that is going on… all you need to do is to establish the ratio between the key rollover period and the average length of a measured P2P connection.

“The keys are never made available outside of the device, and once a set of keys for a given time period are discarded they cannot be recovered – the process is irreversible.”

That is as good a “weasle statment” as I have ever heared

A known “one way” process fits that description but it can easily be shown that the information whilst “not be disclosed” and being “irreversible” does not preclude somebody else reproducing it…

For example I take known details and “hash” them with MD5 or whatever. Provided I don’t release the hash and delete it at some point then it will meet the criteria of the statment. However If I know the input to the hash algorithm I can reproduce the hash at any time I wish.

As I said the “devil is in the details”.

There is also the question of “lost information” leading to incorrect results.

If I change the key I use to make the information anonymous during a monitored transaction how do I know that I’m not,

1, Lossing one or more recordable events.
2, Double counting one or more recordable events.

Thus if the key change period is short or made at a busy time then the results could be significantly skewed. How much is dependant on the average number of events in the time period between key changes and the average number of events at the key change time.

Likewise it is also possible that an event in progress can be tracked across a key change. That is if the data being downloaded by an individual is known then the anyonomous string will change across the key change but the data identifier will not.

There are many other such problems with this sort of system as I well know and they are very difficult to get right…

although christopher does Not state the Detica persons’ name or position inside the company OC, so it could be some high ranking PR personel, with no way to confirm or corroborate that at this time.

http://www.christopher-parsons.com/blog/privacy/update-to-virgin-media-and-copyright-dpi/#more-1483
“In terms of the CView system, let’s first address the concern of anonymization. Specifically, we have to ask how stringent the anonymization system actually is.

When I asked Detica about this process, they informed me that because the CView device is intended to produce a Copyright Infringement Index (aka the ‘Piracy Index’) by evaluating the overall filesharing on a network that identity information isn’t required for this objective.

IP addresses are anonymized at the source/DPI device using a pseudo-random replacement algorithm, which also entails ignoring the external IP addresses.

The key generation system is managed automatically by the device (and thus an ISP can’t muck around with the system), and keys are periodically cycled and redistributed.

The keys are never made available outside of the device, and once a set of keys for a given time period are discarded they cannot be recovered – the process is irreversible.

On this basis, we can argue that no subscriber ID is associated with the randomized replacement algorithm, there is no way to associate a subscriber ID with the pseudo-random number after the fact, and as such the anonymization system should serve its purpose.

Of course, there is a concern that there are no such things as anonymization processes – as noted by Paul Ohm ”
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1450006
“Broken Promises of Privacy: Responding to the Surprising Failure of Anonymization

Paul Ohm
University of Colorado Law School

August 13, 2009

University of Colorado Law Legal Studies Research Paper No. 09-12 “

“Yes it does … it says you’re not going to brute force it !”

No it does not, not at all.

All,

“A modern crypto algorithm with a key of substantial length”

says in “somebodies view” the key is of a “substantial length”.

Some people would say 128bits for a modern algorithm like AES is ok, but 128bits for RSA most definatly not.

So as I said you have not actually said anything by which a

“a judgment qualative or otherwise can be made”.

For instance it is not clear if it is a statment from their PR people or your considered opinion after due consideration of the relavant facts.

Saying,

“it says you’re not going to brute force it”

Is only saying that you think the key space is too large to mount the least optimal practical attack.

Thus even if the key is of sufficient length for the algorithm to prevent a “british museum attack” it does not state anything about how the key is selected in use.

That is how is it generated, how often is it changed etc etc etc.

Would all have to be known before a judgment could be made.

As I said the devil is in the details.

Now maybe you do not know the details, maybe you know only some of them, maybe they have been shown to you under an NDA or maybe they are open to review to any interested party.

All we know from your posting is insufficient for “a judgment qualative or otherwise can be made”.

With regard to,

“…ask you to make a suitable distinction between a product that Detica is trying to create… …and correcting some false impressions given by their PR team.”

Hmm all you appear to have realy said on this is,

“The links that they [Detica] monitor need not be carrying all of the ISP’s traffic — they merely hope that it will be a statistically significant sample.”, and,

“The statistics system then scales up its numbers (to adjust for any sampling at the earlier stages) and generates reports and graphs that give information such as the total amount of P2P traffic;” and,

“then the proportion that is “unlawful file sharing” can only be guessed at. Also, the system cannot even be totally sure that the transfer of a copyrighted file is in fact unlawful”, and,

“Detica are giving the impression that ISPs will be happy to see their new product. I’m less sure that ISPs actually want to measure this traffic quite so exactly.”

Let me paraphrase,

Detica have a system that sees some of an ISP’s traffic. Which they hope will will be statistically sufficient. This they then scale up to give a figure of the total amount of P2P traffic on the ISP’s network…

But it only recognises some P2P traffic and may not be able to tell the content and if they can if it is illegal or not…

Thus, ‘the proportion that is “unlawful file sharing” can only be guessed at.’

And ‘Detica are giving the impression that ISPs will be happy to see their new product.’.

Hmm, you give the impression that you are close to saying their product cannot do what they claim, and that you don’t think their product has a market currently.

And my original concern was that there was a lack of information, and that this “information vacum” would be filed by unwarented speculation, that could be averted by technical information that would reduce fears and if required the methods involved cold be easily strengthend.

Ahh but I forgot you also said,

“Finally, the paranoid will observe that minor tweaks to the software will deliver up a first-class monitoring system that can generate reports about unlawful activity by individual users;”

Some (Detica PR for instance) might think you where actually trying to “fan the flames”.

And you invited me to do this review of what you had said, knowing that I had said,

“I think you will therfore understand why there might be more than the odd questioning eye brow raised at such apparantly inconsistant statments.”

Hmm…

It convays no usefull information by which a judgment qualative or otherwise can be made.

Yes it does … it says you’re not going to brute force it !

The normal implication of “encryption” is that it is of necessity reversable.

This is precisely why I made the point about the key not leaving the box, so that you won’t be able to reverse it unless you have access to the inside of the box. That’s why I think a comparison with a hash is helpful — it’s certainly not a public key system, which was the context in which I made that comparison.

I think you will therfore understand why there might be more than the odd questioning eye brow raised at such apparantly inconsistant statments.

I will just invite you to read the whole article again … and perhaps when doing so ask you to make a suitable distinction between a product that Detica is trying to create (doubtless they’d be delighted to answer all your detailed questions, and listen with great interest to your description of their motives); and my role in reporting some of its more interesting aspects — and correcting some false impressions given by their PR team.

‘A modern crypto algorithm with a key of substantial length.’

That is a little like saying,

“The wiring in my house is of modern design with a conductor of substantial CSA”

It convays no usefull information by which a judgment qualative or otherwise can be made.

As I said,

‘I know from designing similar “anonymising key” systems the devil is most definatly in the details.’

Which for an unstated reason you have not provided details.

Which obviously you or Detica are entitled to do.

However you also have to grant a similar right to others not to take your unsupported statments as being of any assurance.

But please do not think I’m being rude or insensitive, I’m concerned that the “vacum principle” may be applied.
Which may give rise based on past applications, to the notion that there is a questionable motive for keeping such details from the public (as has proved to be the case with phorm).

This is not helped by your apparently inconsistant statments. You say “encryption” not “hash” in your article but say,

‘No, more like a crytographic hash ( or “digest” ).’

In reply to one set of questions but,

‘A modern crypto algorithm with a key of substantial length.’

To another.

The normal implication of “encryption” is that it is of necessity reversable.

However the normal implication of “hash” is that it is not of necessity reversable.

And the normal implication of “crytographic hash” is that it is not reversable (except under special circumstances).

I think you will therfore understand why there might be more than the odd questioning eye brow raised at such apparantly inconsistant statments.