When firmware attacks! (DDoS by D-Link)

April 7th, 2006 at 17:12 UTC by Richard Clayton

Last October I was approached by Poul-Henning Kamp, a self-styled “Unix guru at large”, and one of the FreeBSD developers. One of his interests is precision timekeeping and he runs a stratum 1 timeserver which is located at DIX, the neutral Danish IX (Internet Exchange Point). Because it provides a valuable service (extremely accurate timing) to Danish ISPs, the charges for his hosting at DIX are waived.

Unfortunately, his NTP server has been coming under constant attack by a stream of Network Time Protocol (NTP) time request packets coming from random IP addresses all over the world. These were disrupting the gentle flow of traffic from the 2000 or so genuine systems that were “chiming” against his master system, and also consuming a very great deal of bandwidth. He was very interested in finding out the source of this denial of service attack — and making it stop!

Poul-Henning had already identified that the bad packets were the ancient NTPv1 format, whereas the traffic he wanted to handle was exclusively NTPv4. He supplied me with some extensive packet dumps and asked me to try and find out the cause of his problems. There was quite a lot of material (about 1.4GB of .gz files) to look at — on a typical day he’d receive 3.2 million bad packets (that’s 37 a second!). Analysing the patterns (and lack of patterns) within these dumps strongly suggested that the traffic wasn’t synthetic, viz: it was coming from where it said it was; or it was being spoofed to look as if it had come from real addresses using algorithms that were an order of magnitude better than any I had seen before.

If the source were genuine, then it was worthwhile contacting some of the endpoints. I was quickly able to isolate the traffic coming from AS2529 which is used by Demon Internet, a UK ISP for whom I regularly consult. I approached their “abuse team” to determine if any of the source addresses within their network were known to be sources of email spam, had been reported as being a part of a botnet, or had been known to have been infected with a virus or worm in the past. A handful had been implicated, but most of the rest were “clean”. So it didn’t look as if I was looking for a botnet herder with a grudge against the Danes!

At that point I realised that one of the sources was known to me. A decade ago, in another life, developing Turnpike, I’d dealt with a problem Nick Wedd was having with his machine corrupting files. I hoped that he’d remember me, and so I sent him an email to ask if he knew of any reason why his system would suddenly be sending NTPv1 packets to a machine in Denmark.

To cut a very long story short, it turned out that he did remember me, but that he had a complex network with four or five machines, routers, wireless links and all sorts. Over a week or so, by a process of elimination (he kept track of which of his machines were switched on) and monitoring (his stepson David Prime ran a copy of Ethereal to dump any of the v1 packets); we eventually narrowed the packet source down to his D-Link DI-624 wireless router.

I purchased a DI-624 on eBay and found that its exact operation depended upon the particular firmware version it was running. Typically the firmware contains a list of 50 or so NTP time servers and it will choose one at random and ask it for the time – by sending out a naively constructed NTPv1 packet. If there is no answer (because the remote server doesn’t reply, or the response is firewalled off) then after 30 seconds it will choose another server and try again. If it does get a reply then it won’t ask again for a hour or so.

And it’s not just the DI-624. Many other D-Link products have the same behaviour, and over the years they have shipped tens of millions of devices. So all of these enquires add up (especially the unanswered ones)… to about 37 packets a second on each of the world’s stratum one timeservers!

This isn’t how NTP is meant to work. Consumer devices should ask one of their ISP’s time service machines (probably running at stratum 3), the ISP will synchronise these to a stratum 2 device that is firewalled off from customers, and that machine will chime with some nearby (same continent) stratum 1 machines. Leaving aside the denial-of-service issues there’s not much point in consumers sending packets half-way across a continent to a stratum 1 machine — network variability will mean that they get as good or better results from a nearby box.

Now D-Link do provide a way of configuring a DI-624 to contact a nearby NTP server — and setting this will mean no extraneous stratum 1 traffic. However, the default configuration is to access the stratum 1 timeservers and hence the stage is set for a DDoS attack on part of the key infrastructure of the Internet.

If this story sounds familiar then it is. Back in May 2003 the University of Wisconsin – Madison found itself under a DDoS attack of hundreds of thousands of packets a second. In that case it was Netgear routers that were configured to send Simple NTP (SNTP) packets to a single server. To make matters worse, if they didn’t get an answer within 5 seconds they tried again. Hence, after a network outage (or perhaps a widespread power outage) there would be so much network congestion that answers would be lost — and so the congestion would continue for hours.

Although no-one discusses legal negotiations in public, shortly after Dave Plonka worked out what was causing the incoming traffic, Netgear spontaneously made a generous gift of $375,000 over three years “to improve wireless security on campus and to build out our campus network”. Which was nice.

However, in the current case, D-Link don’t seem to be feeling quite so generous. Poul-Henning reports in an open letter to D-Link (which means that I can finally report the material above) “I have been accused of extortion. I have been told that I have no claim, been told that I exaggerate the claim.”

In my own opinion, shipping equipment that generates 37 packets a second to Poul-Henning (and hence about 2K packets/second to all of the stratum 1 servers as a whole — that’s about a T1 of traffic) is hardly trivial. If D-Link were running their own time servers, as in my opinion they should be, it would cost them about $1000/month for the bandwidth alone.

Poul-Henning has a particularly strong case because in the canonical list of NTP stratum 1 servers his machine is listed as "Service Area: Networks BGP-announced on the DIX; Access Policy: open access to servers, please, no client use;" so access by random consumers who own D-Link routers is clearly not permitted. Indeed, many of the other stratum 1 NTP servers used by D-Link have similar rules — either setting geographic limits or, often, requiring that only stratum 2 servers should connect. So D-Link, who must have consulted a similar list when they were writing their firmware, appears to be entirely ignoring well-publicised access restrictions.

Of course, even if D-Link immediately saw the error of their ways, they’ve nailed these lists of time servers into the firmware of all the devices they’ve shipped. Consumers don’t generally re-flash their kit (the instructions tend to be pretty scary) and so it will be years before the traffic to Poul-Henning starts to die down even if, as he may be forced to, he removes his server from the DIX.

Those dumb little D-Link boxes in the corner of the lounges all over the world (I’ve heard them called “clocksuckers”) will still be demanding, whether or not they get an answer, to know what time it is. And this is just so that they can timestamp their logs correctly; even though no-one will ever look at their contents… What a waste, quite literally, of time!

Entry filed under: Security economics

130 comments Add your own

  • 1. Markus Kuhn  |  April 7th, 2006 at 18:33 UTC

    The solution could be so simple. Clients could pick their NTP server in exactly the way in which they pick their DNS server: via DHCP. Is this the classic chicken and egg problem? As long as ISPs do not reliably advertise their NTP servers via DHCP, clients will not use this information, and as long as no client does, why bother and advertise it? The only difference between DNS and NTP is that if DNS fails, every web browser fails instantly, whereas if NTP fails (especially on a router), only the most experienced time-keeping geek will notice. Which makes it less of a priority for both ISPs and equipment vendors.

  • 2. Jeff Shultz  |  April 7th, 2006 at 23:43 UTC

    “Clients” normally have no clue what NTP is (although they’d probably think it was cool if they did).

    We provide static IP addresses and Static DNS – no DHCP servers for our network.

    In fact, the only time I think we’ve been asked about it was when we had an ACL in one of our routers that was blocking NTP return traffic from some of the Stratum 2 or 3 providers that a consultant used on his customers boxes.

    But maybe it’s time to add a new field to the piece of paper we hand out to customers with all the information on it.

  • 3. .$author.  |  April 7th, 2006 at 23:43 UTC

    [...] When Firmware Attacks! (DDoS by D-Link) [...]

  • 4. Daniel Senie  |  April 8th, 2006 at 02:20 UTC

    Sadly, the D-Link products don’t even use the time sync for anything much. They don’t act as time servers for PCs, Microsoft syncs everyone’s PCs to servers they provide (by default, can be overridden).

    D-Link, like Netgear before, are in the wrong here and need to fix their boxes and pay for the damage done.

  • 5. .$author.  |  April 8th, 2006 at 10:43 UTC

    [...] Update: Richard Clayton who helped Poul-Henning Kamp in this issue has written When Firmware Attacks! (DDoS by D-Link) [...]

  • 6. Jeffrey A. Reimer  |  April 8th, 2006 at 16:05 UTC

    I’ve always seen d-link products as the choice for users that only want something to work out-of-the-box, but some things just shouldn’t be hard-coded into firmware…one of them being NTP lists.

    Unfortunately, many vendors appear to be doing this, and most don’t appear to keep their lists in line with the connection terms of the NTP servers.

    In my opinion, there are 6 choices a network equipment vendor should make in the implementation of NTP:

    1)
    Point the request to a server in their networks, which could then decide from a list which servers to use. This would allow the company to change the list to conform to NTP server use guidelines…especially in the event that they change.

    This option would cost the companies for service use, and server maintenance, and would be reliant on the company to make the changes as necessary.

    2)
    Host their own NTP servers, and point the NTP service on their devices to those locations only.

    This option would cost the companies for service use, and server maintenance, but would not require reconfiguration over time.

    3)
    Integrate a service in their devices that allows the device itself to query servers at their locations for firmware upgrades, and apply them automatically.

    This option would cost the companies for service use,, and would be reliant on the company to make the changes in firmware as necessary.

    On the other hand, it would ensure that they could deploy changes (such as updating the NTP list) in a relatively short amount of time after the firmware is published.

    And before those of you that use d-links in an administered infrastructure yell at me for suggesting opening up a possible security hole, I agree. The service would be ‘on’ by default for the end-users that use it out-of-the-box. But for those a little more net-savvy, there would be an option to disable this service…as you probably check firmware revisions yourself fairly regularly anyways.

    4)
    Allow receipt of NTP server addresses via DHCP service. This option could be bundled with options #5 & 6 for best results.

    This option would cost the companies nothing, but would be reliant on the end-users to have an ISP that does this, or and NTP server of their own.

    5)
    Allow the manual entry of NTP addresses by the end-user. This option could be bundled with options #4 & 6 for best results.

    This option would cost the companies nothing, but would make the end-user solely responsible for compliance to the server use guidelines.

    The issue of user compliance could be mitigated by the companies having a list of NTP servers, that the user can legitimately use, in their support pages.

    6)
    Allow the user to manually set the time. This option could be bundled with options #4 & 5 for best results.

    This would negate any expense and hassle for anyone, and a minor inconvenience for the average home end-user…most of which don’t even care if their device has the right time.

    As a network administrator–and by no means do I claim to know everything–I can certainly sympathize with anyone who gets constantly bombarded with unnecessary traffic, but when it’s as a result of the poor decision making process of a large corporate entity, it makes my blood boil.

    I just hope that d-link, and any other producer of networking equipment that takes the automation of the end-users product over the livelihood of any responsible individual (or group), would wake-up and realize that there’s a world beyond themselves.

    They say “a word to the wise”, and here I’ve gone and written 21 paragraphs.

  • 7. Philipp  |  April 8th, 2006 at 21:18 UTC

    Spread this issue! Everybody should know about it (even if some don’t care about it as much as phk does…).

  • 8. .$author.  |  April 8th, 2006 at 22:32 UTC

    [...] Finally, Richard Clayton, who helped Poul-Henning track down the source of the NTP flood, has published an article explaining how he did it.  The article also provides addition technical details, such as his deductions about D-Link’s NTP implementation and the rate of illegitimate NTP requests from D-Link routers (37 pps at gps.dix.dk alone, probably a similar amount for every other server on their list). [...]

  • 9. .$author.  |  April 8th, 2006 at 22:56 UTC

    [...] Men … Hvorfor gøre det rigtige, når man kan skylle den sunde fornuft ud i toilettet? I flere D-Link routere er der en fast liste over timeservere. Pyt så med at man spørger en server på den anden side af jorden om klokkeslettet. D-Link mener åbenbart ikke at det er deres problem at deres produkter misbruger tidsservere i en grad at det nærmer sig et distribueret denial of service angreb. Richard Clayton har skrevet et indlæg om probelmet. [...]

  • 10. Shep  |  April 9th, 2006 at 01:09 UTC

    Unreal.

    As of now I will no longer use d-link products and replace every one that is existing
    in all of my clients that can take on the cost.

    In no way will I add to his expense if I don’t have to
    .

    To all Admin’s

    Join me, explain to your client / employer why its the right thing to do.
    What are we talking here, $100 (usd) for most?

    Yeh, I know – some places this isn’t an option.

    1) exempted
    1a) home users – this is beyond us, only d-link can solve this
    1b) financially strapped – if you cant, you cant

    2) Should
    2a) medium size – none of you should be looking at replacing more than a half dozen.
    - at current prices (retail even), under $500. (if your using 6 devices, your organization is of size
    where 500 isn’t much).

    Talk to them, explain how this just isn’t right. Most would probably agree.

    3) Large Companies
    3a) no excuse acceptable
    – as most products (from quick scanning of list) with the problem are designed for home use, your use of said products will be very limited at best. Replace them asap, its the right thing to do,.

    Rob

  • 11. Joseph A Nagy Jr  |  April 9th, 2006 at 01:16 UTC

    This shouldn’t even be an issue. The router shouldn’t be using NTP at all, period. Such precise time-keeping is unnecessary for log files.

    Boo to d-link, I’ll never use another one of their routers again. Now I’m going to go tell them why.

  • 12. Shep  |  April 9th, 2006 at 01:29 UTC

    Joseph A Nagy Jr

    Thats 2.

    How many more can we get. What we do mere wont be much but if it saves him a buck, it sames him a buck.

    Send this world wide.

    Whan to get D-Link’s attention, hit them where it hurts – SALES

    BOYCOT TILL FIXED

    people – spreed this

    Shep

  • 13. .$author.  |  April 9th, 2006 at 02:08 UTC

    [...] As many of the more tech-savvy of you know, D-Link makes a line of products dealing with Internet and network connectivity including wired and wireless routers. A link has come to my attention that needs your immediate attention. D-Link’s routers, specifically several models outlined on the previously linked site, do not respect the restrictions on the use of the NTP protocol. Now it isn’t the fault of the products themselves, they are inanimate objects that operate as they are programmed to operate, so cursing at them will not help. It is the fault of the policies over at D-Link that allows such unauthorized and rude behavior in the first place. So in response, I urge you to contact D-Link as I have (the contents of the email I sent can be seen below). Such irresponsible behavior on the part of D-Link must not be tolerated. There is a blog entry here as well. To Whom It May Concern: [...]

  • 14. Peter  |  April 9th, 2006 at 07:02 UTC

    I’m shocked.

    In Poul-Henning’s second letter, to the NTP community, it looks to me, like it is all stratum 1 servers that could be being misused by D-link.

    I’m not buying products from D-Link again, and I’m going to advise my collegues to keep away from them too.

    This sure is an ugly case of the big guy picking on the little guy!

    Go get them Poul-Henning!

  • 15. sam_dal  |  April 9th, 2006 at 08:22 UTC

    In my DI-524 they provide a field to enter the ntp server but it is optional. They should have made it compulsory/required field, initially filled with something like ntp.dlink.com etc. I use pool.ntp.org

  • 16. Joseph A Nagy Jr  |  April 9th, 2006 at 08:48 UTC

    D-Link doesn’t seem to care about playing by the rules, so lets stick it to them where it hurts.

  • 17. moozilla  |  April 9th, 2006 at 08:56 UTC

    pool.ntp.org enuff said.

    I also emailed the Dlink UK press contact complaining about my hardware being used as a DDoS and asking if they will refund me any money if these services are turned off and my equipment does not function as purchased for (EU Law)

  • 18. Dan  |  April 9th, 2006 at 10:40 UTC

    2 very simple solutions, pick 1:

    1. Assign 0.pool.ntp.org, 1.pool.ntp.org and 2.pool.ntp.org as time servers (see http://www.pool.ntp.org

    2. give a dropdown list in the router config with a list of countries. use more local NTP servers based on the country selected or fall back onto global NTP servers (see point 1)

    As a final point, if the router is going to the trouble of processing NTP packets it might as well act as a time server for the local network (see ntp.org and OpenNTPD instead of writing your own naive implementation).

  • 19. .$author.  |  April 9th, 2006 at 13:50 UTC

    [...] D-Link sells tens of millions of network equipment with this abusive firmware, and this has effectively created a malicious botnet constantly DDoS’ing the Stratum 1 NTP servers with traffic. Read more about the detective work the operators had to go through in order to discover the source of these DDoS attacks. [...]

  • 20. Alan C  |  April 9th, 2006 at 16:48 UTC

    I don’t understand any of the techie stuff, but I get the gist of the problem.
    Potential purchase of Dlink router and NIC’s cancelled. I’ll go back to Linksys, even if it costs few £’s more. I hope I’m doing the right thing.
    Are there any other manufacturers of network kit who don’t know what they should be doing?

  • 21. ptup  |  April 9th, 2006 at 16:51 UTC

    Hope we can all help solve this before he has to shutdown the server… Hope I will get the message to Portugal owners od those DDOS Routers
    [...]A empresa D-Link não assume a sua responsabilidade, faz como Pilatos e lava as mãos e nem consegue mudar as listas inseridas no hardware dos equipamentos.[...]
    [...] Pode ajudar modificando o seu router D-Link, veja se é um dos premiados com um router abusador no link em baixo, se for o caso,

    Configure o seu router para actualizar a hora nestes servidores de NTP
    0.pool.ntp.org, 1.pool.ntp.org and 2.pool.ntp.org
    [...]

  • 22. .$author.  |  April 9th, 2006 at 21:12 UTC

    [...] D-link has been less than compliant with his wishes so he’s finally seen no other solution but to publish an open letter to flush out the responsible at D-link and getting a closure to the ordeal this has been for him. The open letter is published here. There’s also a detailed story of the investigation into what caused the problems in the first place. [...]

  • 23. LZW2006  |  April 9th, 2006 at 21:12 UTC

    Thinking of buying a D-Link router…. D-Think!

  • 24. Chris C  |  April 9th, 2006 at 21:35 UTC

    I’ve never used a D-Link router (Netgear and now Vigor), and I now won’t and will pass on the message. However, the suggestion to allow automatic updates is not workable, even for domestic users, I had a Netgear router where the default firmware was fine but the latest version had a severe bug, if it had upgraded automatically I’d have been lost.

  • 25. A.P. Veening  |  April 9th, 2006 at 23:17 UTC

    I suggest that every owner of a D-Link product displaying this behaviour change the optional setting to a D-Link server (preferably not a time server >:) ). Let’s see how long it takes to blast D-Link from the net and to its senses.

  • 26. Joseph A Nagy Jr  |  April 10th, 2006 at 04:00 UTC

    Chris, the only thing that would be automatically updated would be the list, but even that is more complex then it needs to be. If D-Link desperately wants to use a stratum 1 server for whatever asinine reason, let them set up and maintain one themselves that only their routers can access.

    Since that will never happen I see the best solution as being using 0.pool.ntp.org, as mentioned previously by others.

  • 27. Jay Gee  |  April 10th, 2006 at 05:36 UTC

    I have changed almost 100 D-link routers to USR and Linksys routers at my client locations. I am putting these routers in a compactor and crushing them, I could have thrown them away but Someone else could have used them and the problem could have remained.

    From now on D-link, which used to be my favourite, is on my bad list. If someone at D-link is reading this please pass my ,essage to your superiors “You are the most F#$%&*G Idiot person who do not have regard for seeking permission to use someones property. I wish D-link go Bankrupt and you have to work as a NTP Admin whose Server shall always be down, and all of us can then laugh at you…Amen”

    NO MORE D-LINK PRODUCTS.

  • 28. Jonas  |  April 10th, 2006 at 08:34 UTC

    I’m already in a strict boycott of D-Link, not because of this issue but because they’ve caused me much grief with crappy firmware. Even their more expesive managed switches hung on broken packets, ports dying and then coming back to life etc. This episode with NTP doesn’t surprise me at all.

  • 29. D. de Jonge  |  April 10th, 2006 at 08:41 UTC

    First of all I would like to express my support to Poul-Henning Kamp and my great concern about the atitude D-link is displaying on this matter.

    Fortunately I am in the position to influence a lot of current/future users in buying network kit so D-link does not have to expect much bisiness from my end.

    I am shocked by the lack of response from D-link, one would expect a major player on the market to take their responsibilities and work on a solution instead of not acting at all.

    Alas it would be near imposiible to render aal products useless because a lot of home users lack the knowledge to even perform an update, but somehow D-link would have to persuede them (turn in your old device and (for a discount) take this new updated device). Furthermore D-link would have to build the already suggested NTP network and maintain it.

    If all the Stratum 1 server administrators come together and sue D-link as one would better the chances to success…

    Dennis

  • 30. Alex  |  April 10th, 2006 at 10:15 UTC

    27. Jay Gee | April 10th, 2006 at 05:36

    I have changed almost 100 D-link routers to USR and Linksys routers at my client locations. I am putting these routers in a compactor and crushing them, I could have thrown them away but Someone else could have used them and the problem could have remained.

    Have you considered filming the computer-crushing exercise and uploading it to YouTube or comparable video host?

  • 31. duude  |  April 10th, 2006 at 12:14 UTC

    boycotting d-link?
    great!!
    here in france i`ll spread the word: D-link sucks, and any other crappy brand will be an improvement,

    as for home users;
    distributing (uc)linux-firmware for those d-link boxes wich support it (mayebe with a d-link ntp adress hardcoded to show em how it feels…)
    peace
    (but not for d-link)

  • 32. .$author.  |  April 10th, 2006 at 13:33 UTC

    LOL! D-Link NTP PR SNAFU (and quick beard)…

    Welcome back to IT Blogwatch, in which D-Link is accused of misusing the Internet’s timekeeping infrastructure. Not to mention time-lapse face fungus…

  • 33. Pedro Venda  |  April 10th, 2006 at 14:05 UTC

    Incopetence, Arrogance, Disrespect for protocols, standards and rules and lack of user care: All these seem to become increasingly common amongst big companies. D-Link is no exception. I’m sure that not everyone at D-Link is as described, but the company frontend seems to be like described.

    As a simillar example, both Linksys and USRobotics (I make reference to these because thei’re widely known) had to be “reminded” of what GPL means – that is GNU Public License Agreement. Only after that (eventually through open letters) they did what had to be done and released the source code for their linux based firmwares. This is a fixable issue and they did fix it.

    Now the D-Link case is much more serious, because the problem they’ve created has a negative long term impact on the (generous) hosting organization, the (free) NTP service (GPS.dix.dk) and on Poul himself. The worst of this all is that there’s no quick or direct fix. The lack of will to even talk is just to obvious.

    Unfortunately, I don’t foresee a good ending to this episode but with a firm (expensive) lawsuit.

    Best of luck,
    Pedro Venda.

  • 34. Shep  |  April 10th, 2006 at 14:16 UTC

    A lawsuit would only reach deeper into Poul’s pockets. Why should he have to take this alone.

    Stop buying their products, and spreed it around for others to do the same, and why.

    Hurt sales and they will take notice.

    Rob

  • 35. Chris Hallman  |  April 10th, 2006 at 14:23 UTC

    I’ve thrown my 2 cents in with D-Link. I emailed D-Link customer service. Maybe enough emails will get them to act.

    I am a D-Link DI-624 owner. I will make sure I have the latest firmware and not querying the DIX server.

    Chris

  • 36. ptup  |  April 10th, 2006 at 14:30 UTC

    By the time they will take notice Poul’s will have shutdown his server and damage all the Danish community.
    Since he is saying he doesn’t accept any donations….Maybe he could accept donations just from the community he is serving in Denmark to fight for THEIR rights…

  • 37. Chris Hallman  |  April 10th, 2006 at 14:38 UTC

    I beg to differ. I recall seeing a figure of $62,000 USD in his post. That is a paltry sum for a business to pay to resolve a serious misuse of a service. They could be sued for much more in our litigious America. If enough D-Link customers see this or his article and take action (updating their firmware, changing NTP configuration and complaining to D-Link) then maybe they will act to avoid embarassment for being so inept.

  • 38. Amish  |  April 10th, 2006 at 15:41 UTC

    How much bandwith does one NTP request generate?

  • 39. Markus Kuhn  |  April 10th, 2006 at 15:48 UTC

    About 0 Hz.

  • 40. Steve Yates  |  April 10th, 2006 at 15:58 UTC

    For those of you discussing replacing or boycotting D-Link, it appears that relatively current (~6 months) firmware lets you use the default list of NTP servers, or to specify one. It seems like one could rather quickly specify a specific NTP server, or turn off automatic time sync completely if so desired.

    – Steve Yates
    – ITS, Inc.
    – Hire teenagers while they still know everything.

    ~ Taglines by Taglinator 4 – http://www.srtware.com ~

  • 41. Shep  |  April 10th, 2006 at 16:15 UTC

    Steve Yates – your assuming home users going to have a clue how and know to. I dont share this thinking.

    Maybe personal boycotting of ink products and writing the company in protest will not accomplish much, but its a start.
    A sale lost is cash lost.

    Maybe complaints should include copies to every major worldwide seller of d-link products. Most might find it interesting that many a company has been named as co-defendants in class action lawsuits for selling products that infringe on others rights, regards of the fact they didn’t make the products. Here in the good old USA, results like this are not uncommon. Just imagine the effect on the stock price if just one distributor was to stop shipment, and publicly announce why. This would be resolved in 24 hours.

    Shep

  • 42. Steve Yates  |  April 10th, 2006 at 16:59 UTC

    Hi Shep,

    I actually wasn’t targeting my comments towards home users but to the comments above such as the person who says they have replaced 100 routers already. Knowing how to change that setting won’t fix all the routers in the world, but it’s something “we” techie-types can do fairly easily and at zero cost to the end user.

    – Steve Yates
    – ITS, Inc.
    – Cop tag: Another game not quite as fun as it sounds.

    ~ Taglines by Taglinator 4 – http://www.srtware.com ~

  • 43. Joseph A Nagy Jr  |  April 10th, 2006 at 17:09 UTC

    But that won’t generate the bad press and boycott needed to get others aware of this problem and to cause D-Link to fix this problem.

  • 44. Pedro Venda  |  April 10th, 2006 at 17:21 UTC

    Hi,

    Regarding my comment and the lawsuit… I didn’t say I liked or preferred that way – make no mistake: *I don’t*.
    What I meant was that I fear that the only way to get *real* attention from the company is through a lawsuit.

    Cheers,
    Pedro Venda.

  • 45. Don Marti  |  April 10th, 2006 at 20:40 UTC

    D-Link doesn’t listen? How about writing to the analysts who cover the company?

    http://www.corpasia.net/taiwan/2332/irwebsite/index.php?secid=15&version=e&mod=analyst

    Citigroup Global Markets Securities
    Dale Gai
    886-2-8725-1888
    dale.gai@citigroup.com

    CLSA
    Robert Cheng
    886-2-2717-0737
    robert.cheng@clsa.com

    Credit Suisse First Boston
    Wanli Wang
    886-2-2715-6388
    wanli.wang@csfb.com

    Deutsche Bank Securities
    William Bao Bean
    852-2203-6247
    william.bao.bean@db.com

    Lehman Brothers Asia
    Spencer Leung
    852-2252-6227
    spencer.leung@lehman.com

    UBS Warburg
    Eve Jung
    886-2-8722-7200
    eve.jung@ubs.com

    Yuanta Core Pacific Securities
    Chia-Lin Lu
    886-2-2718-1234
    chialin.lu@yuanta.com.tw

    KGI Securities Co. Ltd.
    Renee Tsai
    886-2-2181-8888
    reneet@kgi.com.tw

  • 46. Thom  |  April 11th, 2006 at 00:02 UTC

    Might also generate some heat by contacting the D-Link IR people below.

    If you have any request for information or suggestion about D-Link Investor Relations (IR) services, please contact our IR officers.

    Gavin Lee
    Deputy Manager, Investor Relations & Corporate Communications
    886-2-6600-0123
    ir@dlink.com.tw

    Tracy Wang
    Media Contact, Investor Relations & Corporate Communications
    886-2-6600-0123
    ir@dlink.com.tw

    A.P. Chen
    CFO
    886-2-6600-0123
    ir@dlink.com.tw

  • 47. Timma  |  April 11th, 2006 at 01:36 UTC

    Just a slightly cheeky suggestion, does anyone know if D-Link have their own NTP server? don’t throw out your D-Link gear, put it to good use, put in the D-Link NTP server.
    Then they can only blame themselves for hurting themselves.

  • 48. David Kirkby  |  April 11th, 2006 at 03:23 UTC

    I think home-users should be encouraged to return units under warranty. I have asked on the uk.legal newsgroup where one might stand on this issue:

    see Warranty status – Dlink router & the time server problem

    If home users start returning units to suppliers after 11 months demanding fixes under warranty it will hit dlink more than comments on slashdot on here ever will.

  • 49. Kiva  |  April 11th, 2006 at 10:23 UTC

    Very nice artikkel :thumbsup:
    Gives me a warm feeling when people cares!

    Kiva

  • 50. Tim Keating  |  April 11th, 2006 at 14:13 UTC

    D-link has known about this problem for a long time..
    (I reported it back in Mar 2003.. ,over 3 years ago)
    (Dlink’s query time server every hour or so. (way overkill). more frequently if they can’t make contact..)

    ===========
    Dlink case no.. PTS919573 Mar 31, 2003..

    Text of problem report…

    Problems in latest firmware(2.18, but probably in all of them). with implementation of NTP in routers.

    1st problem.. Routers (both 614, 604)query time way too often!! Most tier-2 public NTP servers only allow one(1) query per day for a max of three(3) systems/queries ! (give us control, over freq of queries)

    2nd problem.. Your internal public list is hitting polluted systems and setting date to April 2069!! (give us control, over list.. allow inside NTP server, allow NTP function to be disabled without filling up logs).
    ================

    Go get em… Sue dlink’s ass for damages..
    I’ll be a witness in court, Ive archived several email exchanges between myself and dlink to prove the case. They can’t claim that they haven’t been warned..

  • 51. David Kirkby  |  April 11th, 2006 at 14:46 UTC

    Tim Keating says Most tier-2 public NTP servers only allow one(1) query per day for a max of three(3) systems/queries

    But I just looked at
    http://ntp.isc.org/ and see most say:

    AccessDetails: Open access for up to 20 queries per hour (one-day average) from any one address, others by arrangement.

    That was the case for all the time servers my Dlink DWL-700AP wireless access point connects to, except the one where I could find no information.

    I think the *** will really hit the fan when a retailer or credit card company is forced to give a refund following a customers request for a model that does not keep him to keep accessing a server against his wishes and against the wishes of the servers owner. If Dlink will not fix the firmware, then I would think it likely refunds will occur.

    That does not address of the issue of payment for those in the field that are causing the problem though. Perhaps Dlink will find it cheaper to pay Poul-Henning Kamp for the costs he incurs.

    dave

  • 52. Steve Yates  |  April 11th, 2006 at 14:49 UTC

    >Dlink’s query time server every hour or so

    DI-604 firmware 3.51(November 2004) allows one to specify an interval (along with an NTP server), however, the longest the interval can be set is 72 hours.

    For several D-Link devices there are different versions of the hardware (and hence firmware), so the new firmware features may not be available on older hardware.

    – Steve Yates
    – ITS, Inc.
    – The trouble with being punctual is that nobody’s there to appreciate it.

    ~ Taglines by Taglinator 4 – http://www.srtware.com ~

  • 53. Richard Clayton  |  April 11th, 2006 at 15:20 UTC

    As noted, different D-Link firmware has different functionality and different retry behaviour. Also, the device may or may not retry at the same frequency if it receives an answer compared to when it does not (and note that many of the stratum 1 servers DON’T answer NTPv1 queries or have changed identity since D-Link nailed their names in to the firmware). There also seem to be wide variations in how different models of device are set up — suggesting that D-Link’s codebase may be more fragmented than anyone would wish.

    Also, all this talk of refunds etc is, I’m afraid, nonsense. If you specify an NTP server to contact (and I have not encountered any D-Link devices that talk NTP and don’t have this option described in their manuals) then the device will talk to that one and that one alone. So the DEFAULT behaviour is quite unsuitable — but it is straightforward for the knowlegeable to correct for. The tragedy is that most customers will not have that knowledge and it doesn’t seem likely they will get it any time soon. Note that one of Poul-Henning’s requests was for D-Link to seek to better inform existing customers — not just to compensate him for his losses.

  • 54. Tim Keating  |  April 11th, 2006 at 15:21 UTC

    Dlink downloads for any model of DI-604 are not availible at this time.. (I just tried it.. nothing but a big loop back to the FAQ.)

    As I recall 2.18 (maybe 2.20) was the last rev made avail for Rev A, B and C models of DI-604..

    I support a combination of rev B, C, and rev E DI-604’s at various client sites. The only realistic solution was to set up my own time server and point them all to use my own NTP server which follows the rules and queries a pair of stratum 2 servers no more than once a day.

    As for Jurisdiction.. Dlink is full of baloney about jurisdiction of Cali courts.

    Mr. Kemp (Victim) has NO contract with Dlink.. thus is NOT bound by any of dlink’s contract or shrink wrap restrictions.
    He has no obigation to sue Dlink in US court.. Their products are attacking his server in Demark..

    Mr. Kamp can sue D-link in ANY competant Danish court of his convience. He can use Danish Anti-hacking laws to his advantange. (Failing to abide by advertised NTP terms and restrictions becomes “Un-authorised access”, with Millions of dollars of punitative damages possible. )

    D-link better wake up and smell the roses or this one is going to bite them back… Hard…

  • 55. Steve Yates  |  April 11th, 2006 at 15:36 UTC

    >Dlink downloads for any model of DI-604 are not availible at this time

    Yes, looks like an error on their support pages. Our older router has v2.20, and it has a spot to enter an NTP server.

    Ha! Check this out, I tried the DI-614, and it shows shiny new firmware:

    Firmware 2.34 ¤ Fixed NTP 3/20/2006

    :)

    – Steve Yates
    – ITS, Inc.
    – Who knows what evil lurks in the hearts of purple dinosaurs?

    ~ Taglines by Taglinator 4 – http://www.srtware.com ~

  • 56. David Kirkby  |  April 11th, 2006 at 15:47 UTC

    Richard,
    I can’t find any way to specity an NTP server in my Dlink DWL 700-AP wireless access point. That seems to be hardcoded in with no means to override it that I can see. If that is so, refunds are not out of the question, although in this case it does not connect to Poul-Henning’s server and it is out of warranty. But if it was in warranty and connected to his server, I think there would be a case for seeking a refund from the retailer.

  • 57. Richard Clayton  |  April 11th, 2006 at 17:50 UTC

    Inspecting the strings within the DWL-700AP strongly suggests that it uses SNTP not NTP. However, you’d need to dump the traffic to be sure.

    Further analysis indicates that this model only connects to a very short list of sites, all of which are said to permit open access. More details in this Usenet article.

  • 58. Steve Yates  |  April 11th, 2006 at 18:18 UTC

    FWIW I asked D-Link support about the DI-604 firmware and they said it was “in development” and would be posted when ready. So I’m guessing they pulled the existing versions off their site temporarily.

    – Steve Yates
    – ITS, Inc.
    – Graphic Artist seeks Boss with vision impairment.

    ~ Taglines by Taglinator 4 – http://www.srtware.com ~

  • 59. Chris C  |  April 11th, 2006 at 18:41 UTC

    Just a thought re, Poul’s problem (doesn’t solve anyone else’s, though), but since his clients are all ISPs and presumably clued it might be possible for him to change IP address and notify them that the old one would be going away. (I’d also be tempted to put a redirector on the old address sending the packets to D-Link, but then I’m a bastard like that and Poul sounds like a nice guy.) That doesn’t solve the root cause, but with the number of D-Link routers out there mostly in the hands of people who have no concept of NTP the only thing which will solve that is time, waiting for the things to die…

  • 60. Richard Clayton  |  April 11th, 2006 at 18:59 UTC

    It would be worthwhile studying the problem carefully before proposing solutions… rather as it is unwise to educate your parent’s mothers on the obtaining of sustenance from avian ova :(

    The D-Link devices are resolving the NAME of Poul-Henning’s machine and only then sending traffic to the IP address that results from that DNS lookup. Furthermore, changing IP address is not an option per se. If you read his “open letter” you will see that he considers contacting the 2000+ current legitimate users of his system to be infeasible.

    Finally, if he wished to “redirect” packets a “redirector” (presumably rewriting the destination IP address) is hardly necessary — all that is needed is to have the name of his server resolve to something else. Making this 127.0.0.1 would of course be best for the Internet as a whole.

  • 61. Shep  |  April 11th, 2006 at 23:34 UTC

    What if he was to filter out all non Danish ip block’s, alowing only one country access.

    Granted this would still alow local abuse, but it sure should cut a large chunk out of band width..

    Shep

  • 62. Richard Clayton  |  April 11th, 2006 at 23:53 UTC

    Once again, I commend looking carefully at Poul-Henning’s statement of his problem before solving it for him… the equipment operated at the ingress to his AS doesn’t have simple to apply capabilities to filter the packets (and complex filtering sends the CPU load through the roof). Anyway, by that point it is pretty much already too late, because the bandwidth has already been consumed on the links to the AS, which may well be where DIX applies its charging algorithm.

    You will note that I started by article by describing Poul-Henning as a guru… if the solution was cheap, trivial and/or obvious then I think it would have been spotted by now.

    and please don’t forget… Poul-Henning is running just one of a large number of stratum 1 servers who all have the same problem; and will not have identical network configurations.

  • 63. NTPNinja  |  April 12th, 2006 at 01:58 UTC

    I configured http://www.dlink.com as my NTP time source, of course it doesn’t sync but perhaps they will get the message over time.

    I’ll stay on this crusade until my system clock drifts more than 3 seconds from NIST (probably a month or two).

  • 64. Timma  |  April 12th, 2006 at 05:06 UTC

    Another idea, would it be possible for Mr Kamp to put up a website that allowed you to put in your name, email address, d-link device model number and a few other details and spat out a letter for you to take to your local computer retailer, saying something along the lines of this device is contributing to a DDOS attack on Mr Kamp’s NTP Server.
    As d-link refuses to fix the majority of them, then there is no choice but to return them as they are taking part in an illegal activity (the DDOS, and the use of servers outside their access policy restrictions) against not only his, but all stratum 1 NTP servers and that by you continuing to use these devices on your network would be knowingly taking a part in that illegal activity.

  • 65. Peter "bug" Philipp  |  April 12th, 2006 at 11:58 UTC

    Well I hate to mention this but can’t you
    just turn back time to 2001/09/11 on all
    those logs? Alternatively turn the clocks
    to 2001/04/01 to find an offsetting balance.
    I mean, an upsetting balance rather. This
    whole issue has me running hot you know!
    yeah boyyyeee!

  • 66. ab  |  April 12th, 2006 at 19:16 UTC

    I already make it a point to avoid D-Link’s shoddy products and this news certainly cements that decision for me. I don’t know how they can be so inept or irresponsible, but I hope they get what’s coming to them…

  • 67. CJC  |  April 12th, 2006 at 20:13 UTC

    Just an idea for a possible workaround. I understand the firmware gets the IP from DNS, but is their reslover smart enough to handle CNAMEs? Given the awful kludges and programing standards this and similar events have revealed, I’d say its worth the easy test. I’d try if I had one of the affected devices. You could break the DNS lookup by these dumb boxes with (hopefully) no collateral damage.

  • 68. Tom Watson  |  April 12th, 2006 at 22:39 UTC

    I’ve got a DI-624 router, which I’ve configured as an access point (no WAN connection). It works well, but does “wierd things” like restarting on its own every once in a while. Since it wants to access time using the WAN port (not connected in my case), it sits at April 1, 2002 (as I recall) until I set the time manually, then it back dates everything. Complaining to D-Link does nothing (what do you expect). While this has little to do with the problem, I suggest something very simple: It seems that the routers use SNTP v.1 (or something more or less obsolete). Detect THIS packet and return RANDOM time stamps. Yes, truly random time stamps. This option should have immediate effects on peoples logs, since the router has an option to send logs to addresses. This may not trip up everyone, but with time going forward AND backward, somebody will realize there is a problem. I would also suggest the the proper users of the NTP service convert over to a different DNS name, and have this alias to the current server for a while. Given the problem, I suspect that getting the code written to poison the returns for a particular class of clients would not be difficult.

    I wish you luck. It looks like you will need a LARGE hammer to cure the problem.

  • 69. Richard Clayton  |  April 12th, 2006 at 23:23 UTC

    As made clear in the orginal article, the DI-624 uses NTP v1, not SNTP.

    Returning the wrong time will NOT reduce the traffic to Poul-Henning … unless you assume that a lot of consumers are paying attention to how well their wireless routers work. But if you assumed that, then you could assume that they will upgrade to a less unfriendly version of the firmware or — which will completely fix the problem — or fill in the address of a local (probably stratum 3) time server which will also prevent spurious traffic. As to the suggestion, put forward yet again, that the legitimate users (all 2000+ of them) should reconfigure their systems… please read Poul-Henning’s description of his problem before proposing solutions that he has already rejected as impractical.

  • 70. Steve Crook  |  April 13th, 2006 at 00:15 UTC

    I agree that D-Link are in the wrong on this issue and should make every effort to resolve it. Should we boycott them? Well that’s only an option if the competing players in the market space are innocent of equally stupid practices.

    Linksys are one of my favourite vendors, but recently they changed the chipset in their WRT54G range of products which broke compatibility with various open source code. They did this without changing the product code which should be a capital offence IMO. I also have a Linksys PSUS4 IPP Print Server sat on my desk that is incompatible with Linux IPP but works with Windows. It makes a good paperweight.

    Netgear changed from the Prism chipset on their Wireless cards without changing the product code. This broke compatibility with the code incorporated in the Linux kernel from the prism54 project. How can it possibly be legal to sneak in such a significant specification change?

    No doubt this list could go on and on. The fact is that home networking manufacturers are often guilty of misconduct. They get away with it because 95% of their market don’t understand and don’t care. “Hey, I plugged in my D-Link DSL-G604T and it worked. It even has the right time on it!”

    My point is that boycotting D-Link is not the answer. The answer is to make adherence with standards and good-practice commercially valuable. Products need an independent certification stamped on the box that says, “This is a good product”. Would you buy a babies car seat that didn’t have a kite stamp?

  • 71. bob  |  April 13th, 2006 at 09:10 UTC

    err, am I missing something ?
    why are people throwing away D-link routers…?
    - just upgrade the firmware, if there is no suitable update for your model, then send it back to D-link as faulty.

  • 72. Joseph A Nagy Jr  |  April 13th, 2006 at 12:08 UTC

    That would be awesome, demand refunds AND tell them your boycotting. Oh D-Link, the solution was so simple, yet you waited until this happened. Have a day.

  • 73. Rej Sharp  |  April 13th, 2006 at 12:59 UTC

    Is this issue only related to D-Links routers? I have some DCS-2100+ web cameras, and they time-stamp the images, does anyone know if they are up to similar mischief? They have an option for automatic NTS but but no visible list. I have set mine to synch with the PC just in case.

  • 74. Adrian M. Whatley  |  April 13th, 2006 at 13:07 UTC

    Everyone should go to http://support.dlink.com/faq/view.asp?prod_id=1228&question=NTP and fill in the comment box.

  • 75. Richard Clayton  |  April 13th, 2006 at 13:26 UTC

    The DCS-2100+ firmware lists:

    ntp-cup.external.hp.com; open access, restricted service area
    clock.isc.org; open access, restricted service area
    bitsy.mit.edu : it’s discontinued
    chronos.csr.net; open access, restricted service area
    clock.nc.fukuoka-u.ac.jp; open access, restricted service area

    so failing to set the configuration option for a specific NTP server will mean your little webcam is accessing 3 or 4 (you might possibly be within one of the access areas) stratum one time servers that do not wish to provide you with service… sigh!

  • 76. Rej Sharp  |  April 13th, 2006 at 13:36 UTC

    Thanks Richard. I will certainly leave the DCS-2100+ setting on “Synch with PC”.

    I had already blasted off a polite but direct webform asking them the same question. I will share their response – if any. R.

  • 77. Richard Clayton  |  April 13th, 2006 at 13:44 UTC

    The story has now been picked up by the BBC:

    http://news.bbc.co.uk/1/hi/technology/4906138.stm

  • 78. Joseph A Nagy Jr  |  April 13th, 2006 at 13:57 UTC

    A fluff piece at best, but at least the word is out.

  • 79. .$author.  |  April 13th, 2006 at 14:11 UTC

    Awesome programming cock-up…

    D-Link one of the world’s biggest manufacturers of networking and Internet equipment have had a bit of a shocker. NTP (Network Time Protocol) is a simple protocol designed to allow computer equipment to keep an up-to-date time clock – it’s …

  • 80. Michael Tuckniss  |  April 13th, 2006 at 15:02 UTC

    If D-Link have any sense, they will publicize the problem themselves (there is nothing on their web page about it, some hours after BBC story appeared) and offer a solution.

    My D-604 router has the 22 Nov 2004 firmware, version 3.51. It has the option first to set a single NTP (72 hours checking time limit), but also to set the time manually using the computer’s clock, avoiding all NTP activity. For all home users the last should be more than adequate.

  • 81. Adrian M. Whatley  |  April 13th, 2006 at 15:43 UTC

    Try putting ‘NTP’ in the search box at the top of the http://www.dlink.com homepage. There’s still ‘No news found matching “NTP”.’

  • 82. Tom Watson  |  April 13th, 2006 at 16:45 UTC

    In a previous message I suggested that the server return random time intervals for those requests that can be identified as coming from the router. Someone mentioned that this might not solve the problem as to the traffic originating from the routers. While true on the face of it, there are some people who WILL discover that their router now has a problem (time incorrect). Since D-Link provides the router, their customer service will be subject to the DOS attack. This SHOULD prompt them to issue a solution to their problem, which is also the main problem. No, it won’t DIRECTLY reduce the traffic to the server (not much will), but it most likely WILL prompt D-Link to clean up their act!

  • 83. Dr. David Kirkby  |  April 13th, 2006 at 17:14 UTC

    As much as I dispise D-link’s attitude, configuring time-servers to send the wrong time is not the answer. It would make Poul-Henning appear the guilty party and D-link the victims.

    Many would start mistrusting time servers – it would bring the science of timekeeping into disrepute.

    If this ever goes to court, that would play into the hands of D-link.

    I suspect others will do things to hurt D-link, but Poul-Henning should not do it. Having known him from the time-nuts mailing list before this incident, I don’t think he would do it either.

    True scientists don’t do this sort of thing.

  • 84. Martin  |  April 13th, 2006 at 17:29 UTC

    It might be a drop in the ocean but as a direct result of this news D-link have just lost their place as a preferred supplier on a contract due to be let at the end of this month. Roughly Euro 80k’s worth of business WILL go elsewhere.

  • 85. Andy West  |  April 13th, 2006 at 18:30 UTC

    I read the list of time servers those D-Link routers have been abusing. Just the inadvertant DDoS of these servers alone…

    time.nist.gov, ntp.nasa.gov, ntp2.usno.navy.mil

    …will land D-Link in a world of pain. We are talking at least civil litigation, possibly criminal charges by the U.S. Federal government against the company. But even if D-Link manages to evade these, their corporate reputation will be damaged for years.

    Let’s see D-Link try to accuse the United States Government of extortion!

  • 86. Harry  |  April 13th, 2006 at 22:37 UTC

    This may not be a forum s/he would visit but it’s always possible…

    I’d like to know the PoV of the d-link lawyer who’s been dealing with this matter for the past 5 months until Mr Poul-Henning Kamp was driven to publishing an open letter due to lack of progress in resolving the issue.

    This has got the mark of a pretty good demonstration that even pots kept firmly on the back burner in the expectation that they eventually lose interest & go away (to mix the metaphor) can boil over.

  • 87. Bored Huge Krill  |  April 14th, 2006 at 00:07 UTC

    question – when I saw this I went and looked at my D-Link gateway at home, and it turns out that it’s one of the ones on the list (a DI624). By the way, it was supplied to me by Verizon when they installed my fiber service…

    So what do I do?

    Looking at the “time” page on the “tools” tab, there is nothing in the field for “default NTP server”. FW revision is 2.43DDM.

    Suggestions much appreciated…

  • 88. Richard Clayton  |  April 14th, 2006 at 00:18 UTC

    To avoid sending unnecessary traffic to stratum 1 timeservers, fill in the “default NTP server” entry with a suitable nearby server.

    Your ISP may run one (this is best), or you could use 0.pool.ntp.org (see http://www.pool.ntp.org/use.html) for details of this project.

  • 89. Dean Kent  |  April 14th, 2006 at 05:22 UTC

    Probably another drop in the bucket, but I posted a comment a few days ago on my forum (for those interested: Forum Post Link). It did spark some interesting replies, particularly those who claimed it would be ‘easy’ for Mr. Kamp to fix the problem at his end. Seems some people find it very easy to miss the point. :-( .

    I hope D-Link stops missing the point and does the right thing…

    Regards,
    Dean

  • 90. Jan-Pieter Cornet  |  April 14th, 2006 at 13:31 UTC

    One easy thing to do, to dodge the traffic, is to move GPS.dix.dk into an AS that is only routed within DIX. Some other Exchanges have a similar setup – an internal non-routable AS that’s only reachable from exchange members, because the AS peers with every member, but has no uplinks.

    In this case, the IP address in question would be unreachable from the entire world, except from DIX members, so the traffic would never reach him. Given the usage restrictions in the list of NTP serves, this should not harm any of his legitimate “customers”.

    Of course, I don’t know if such a setup exists at DIX, but it would be relatively easy to make.

  • 91. Richard Clayton  |  April 14th, 2006 at 18:10 UTC

    This is a better suggestion than many since it would cause the unwanted traffic to be discarded somewhere other than at the stratum 1 server.

    However, Internet exchange points don’t really work this way… the international members of DIX (such as MCI, Colt, Telenor etc etc) would also peer with the DIX’s AS (if it had one). So it would be necessary to provide an AS for the timeserver alone — which might be a hard sell at RIPE.

    Conversely, it may well be that there are many Danish ISPs who are not DIX members and Poul-Henning might in future be prepared to offer them service because of their geographical location (although he does currently restrict himself to networks announced on DIX, assuming (presumably) that traffic will only flow to them via DIX). Although they are likely to be in ASs that can be identified as specifically Danish — it’s quite likely that a fair number will be part of some Europe-wide AS operated by one of the backbone providers.

    Also, as a solution, it doesn’t really address the problems of all the other stratum 1 timeservers that D-Link devices are sending traffic to.

  • 92. David Farb  |  April 14th, 2006 at 19:58 UTC

    Since D-Link is basically screwing up the internet, I suggest a US based class action lawsuit on behalf of all internet users. The only thing that will cause action on the part of a corporation is the prospect of a large financial loss.

    Make D-Link take all the violating products back, make D-Link do a product recall. Who cares what it costs them?

  • 93. David Kirkby  |  April 14th, 2006 at 23:31 UTC

    Here is an odd one – Reuters lists the D-link share price on the Taiwan Stock Exchange. It is in Taiwanese Dollars. It has actually gone up this week, starting at about NT$ 34.9 on Monday 10th April and is currently $NT 35.75 at 2 pm (Taiwan time) on Friday 14th April. That rather surprises me. But looking at other information on that page:

    Risk Alert: NONE
    News No news stories for this company within the last 30 days.

    It seems this cock-up by D-link has either not propogated to the financial markets or they don’t care. Both seem unlikely.

  • 94. Richard Clayton  |  April 14th, 2006 at 23:38 UTC

    The story has yet to hit the mainstream media as Google indicates.

  • 95. A Deeley  |  April 15th, 2006 at 03:30 UTC

    Will go round my clients that use DLINK and fix where possible. Where not possible will advise them to contact their suppliers about a product which breaks EU law. I never advise Dlink because I have had too many odd things happen on the WiFi side. There are other manufacturers at almost the same price with more predictable products.

  • 96. Jan Mortensen  |  April 15th, 2006 at 11:23 UTC

    Been following this case since the beginning. It might be that mainstream news haven’t pickup on this. But my google query has gone from 16000 to 22000+ in 3 days

    http://www.google.com/search?sourceid=navclient&ie=UTF-8&rls=GGLD,GGLD:2004-51,GGLD:en&q=%22Open+Letter+to+D%2DLink+about+their+NTP+vandalism%22

  • 97. Justin Mason  |  April 15th, 2006 at 12:00 UTC

    Actually, looks like it’s just hit the mainstream — namely the Beeb: http://news.bbc.co.uk/1/hi/technology/4906138.stm .

    It appears D-Link are finally getting their ass in gear, too, removing the firmware image from their website…

  • 98. Markus Kuhn  |  April 15th, 2006 at 13:27 UTC

    The fact that DLINK has hardwired a number of Stratum-1 servers hostnames into its firmware has repeatedly been called “illegal” or “breaking EU law” in this discussion. While we surely all agree that DLINK’s firmware choices regarding NTP can hardly be called nice, appropriate, desirable, or good engineering, I have some doubts that this practice actually violates any current criminal law (or whether we even should advise our parliamentarians to introduce legislation to that effect!).

    It hardly looks like a deliberate, malicious attempt to cripple these NTP servers, but more like a missunderstanding of the full implications of their implementation. I don’t see how existing computer-misuse legislation could apply here.

    Although these laws make it illegal to interfere with a computer’s operation against its owner’s consent, they also make such interference illegal subject to the condition that this involves by-passing some form of conditional-access mechanism (password check. etc.). None of this is happening here. These NTP servers are offered as a free public resource. In a sense, both these servers and DLINK have become a victim of their own success.

    If what DLINK did were illegal, then the same could be said for anyone who posts a URL to a small and easily overloaded website on slashdot.

    If we look at the larger picture of what computer-misuse legislation should or should not punish, I’d rather blame DLINK for bad engineering and lack of common sense here, than for “illegal” practices. Sadly, they are not alone …

  • 99. ptup  |  April 15th, 2006 at 13:43 UTC

    About the free use…
    That’s why there is a listing in http://www.eecis.udel.edu/~mills/ntp/clock1a.html
    that states:

    DK Denmark GPS.dix.dk (192.38.7.240)
    Location: Lyngby, Denmark
    Geographic Coordinates: 55:47:03.36N, 12:03:21.48E
    Synchronization: NTP V4 GPS with OCXO timebase
    Service Area: Networks BGP-announced on the DIX
    Access Policy: open access to servers, please, no client use
    Contacts: Poul-Henning Kamp (phk@FreeBSD.org)
    Note: timestamps better than +/-5 usec.

    Regards

  • 100. Richard Clayton  |  April 15th, 2006 at 14:32 UTC

    Markus has a strong point about computer misuse legislation. It’s one of the issues I address in my recent document on “Complexities in Criminalising Denial of Service Attacks (http://www.cl.cam.ac.uk/~rnc1/complexity.pdf). I discuss “authorisation” and “intent” and indicate that these are complex areas which may only be a matter that a court could decide upon, given the individual circumstances of each particular case.

    Also, although the isc.org documentation (and also the list maintained by David Mills) sets out Poul-Henning’s restrictions, it is far from obvious that either is the source of D-Link’s database of servers. There is NOT a one-to-one match between the servers listed and those nailed into the firmware, nor do they appear in the same order as these documents… I’ve done some extensive searching to see if I could work out what list they looked at — and so far, I have drawn a blank.

  • 101. Markus Kuhn  |  April 15th, 2006 at 14:39 UTC

    IANAL, but I would not be surprised if the legal profession made some difference between a polite request (“please, no client use”) on some web page, and a clearly worded, mutually agreed licence contract that is enforced by access-control mechanisms. Is DLINK impolite and causing problems? Sure! Are they doing anything “illegal”? Not so sure.

  • 102. ptup  |  April 15th, 2006 at 14:51 UTC

    Maybe not in so many words like Illegal, but they are for sure making damage to someone… So is it legal to abuse some one’s server? Is ddos not a legal activity?

  • 103. David Kirkby  |  April 15th, 2006 at 15:47 UTC

    I’m not a lawyer, but I believe you don’t have do something illegal to be liable for damages. Sure D-link will be liable for damages, but I doubt their engineering incompetence was illegal.

    Failing to correct this misuse in a timely manner once bought to their attention 5 months ago might well be illegal. (I’ve no idea what law that might break, if any). That is a very different issue though.

    I can’t see what law the original problem would break in the EU and I doubt in most other countries too.

  • 104. Dewayne Ferris  |  April 15th, 2006 at 17:19 UTC

    Hello,
    As the owner of a D-Link router, I was interested & concerned to read this. My router (DI-624 revC, FW ver. 2.71, 16 Sep 2005) does have an option to set both a “Customized NTP” address and to change the polling interval. I’ll set those to “us.pool.ntp.org” & “4 hours” which should (to my understanding) keep time reasonably accurate without burdening Stratum 1 servers.
    One comment – the DI-624 offers parental control features, including filters that are activated automatically for certain time spans during the day. I use those to gently persuade my son to get off Xbox Live and go to bed on school nights (grin). So for me at least, modest accuracy in the DI-624’s time of day clock is very useful.

  • 105. David Kirkby  |  April 15th, 2006 at 18:53 UTC

    Unless there is a very serious issue with the clock, there really is no need to get the time every 4 hours.

    The router can probably run for a week and not be more than 30s out, which I would have thought was sufficent for parental controls. In any case, assuming the time can be set with a resolution no better than 1 second, there is not much point in doing it before the error is 1 second.

    If you have multiple machines it can sometimes be important to have them all reading the same time. For example, if you run a C compiler on one computer but your data is saved to the disk on another via a NFS mount, programs such as ‘make’ will get in a mess, as files can appear to be created in the future if the times are not very close. For this reason, and to conserve network bandwidth, one of my machines using NTP to set its time and all others sync to that.

  • 106. Matt Edgar  |  April 15th, 2006 at 19:48 UTC

    I am surprised by the number of people who feel the the victim in this saga should be the one to fix the problem.

    Perhaps they are lacking empathy?

    Let me make it personal for you:

    The local taxi company accidentally include your home phone numberas one of the choices in adverts and on the cards that they leave in clubs and bars.

    You are disturbed at all hours of the day until you end up unplugging your phone to get some sleep.

    You complain to the company and they ignore you because it would cost them to fix it, and isn’t costing them not to (people just try one of the other numbers when you don’t pick up).

    You find that, despite your complaints the company are still distributing their cards with your number on – because they got a good deal on getting millions printed. Again, it would cost them to fix it, so why should they?

    What do you do now? Go public and hope to shame them into fixing their mess?

    Or, as the victim would you do one of the things suggested here, which amount to “Get caller ID, only answer if it’s for you”, “Change your number and tell all your friends” etc.

    Would you really feel that it was your responsibility to fix it?

  • 107. A Deeley  |  April 16th, 2006 at 02:37 UTC

    Anyone know if DLink DI 714P+ has this design flaw please?

  • 108. JR  |  April 16th, 2006 at 16:16 UTC

    take a look at the awards they have at http://presslink.dlink.com/awards/

    Do the award givers know about all this?

  • 109. Lazlo  |  April 18th, 2006 at 01:10 UTC

    Unless there is a very serious issue with the clock, there really is no need to get the time every 4 hours.

    I recently discovered that my DI-624’s internal clock will drift several minutes per hour if it can’t reach an NTP server.

    So now I’ve crossed Linksys (random packet corruption), Netgear (exploding capacitor) and D-Link off my hardware list. Sigh.

  • 110. Jiml8  |  April 18th, 2006 at 01:24 UTC

    After reading these articles, I checked my DI-524. I can tell you that it has NTP capability, which was enabled. I cannot say that it was contributing to the problem, but it seems likely.

    I disabled NTP on that box. It doesn’t need to know the time anyway.

  • 111. AT  |  April 18th, 2006 at 15:43 UTC

    I don’t get – why Paul servers are accesilbe from outside of DIX in first place ?

    According to DIX FAQ international traffic is regulated by separate transit agreements.

  • 112. Richard Clayton  |  April 18th, 2006 at 16:10 UTC

    Paul-Henning’s system has an IP address within AS1385 (Forskningsnettet, the Danish network for Research and Education). This AS will be peered with many other ISPs over DIX (they have a rule preventing paid transit from transitting the exchange) but available as transit via a number of backbone networks who will have private peerings at DIX, or have other connectivity arrangements elsewhere. Hence the system is accessible to anyone on the planet, but only peering traffic will go across DIX (in theory – in practice I expect they find it hard to police, LINX dropped an equivalent rule many years ago because it was unreasonably complex to detect violations).

    If you want to know more about why blocking access is expensive and complicated, then I commend Poul-Henning’s own pages to you. He explains the difficulties he has encountered.

    Also, please remember this traffic is going to dozens of other stratum 1 timeservers as well — who will far more complex connectivity arrangements; and will also have bandwidth issues of their own to address.

  • 113. Bob Johnson  |  April 18th, 2006 at 20:45 UTC

    I can think of a few ways to move the process along a bit:

    1. California has pretty strong computer crime laws. Each limited-access NTP server operator on D-Link’s list should file a complaint with the California State Attorney General. Accuse D-Link of unauthorized use of network services and of launching a Denial Of Service attack against these servers.

    2. File DoS attack (or unauthorized access) complaints with the ISPs that are hitting limited access stratum-1 servers. Explain that the problem is likely the use of a D-Link product, and that owners of the equipment should be asked to contact D-Link for a solution to the problem. Let the owners of those products deal with D-Link. A fairly effective automated system could probably be set up to do this.

    The best way to fight a granfalloon is to get another granfalloon to attack it.

  • 114. Pedro Dias  |  April 18th, 2006 at 20:52 UTC

    I think that is a reasonable proposal…
    But can he do that not being a US Citizen?

  • 115. Bob Johnson  |  April 18th, 2006 at 21:22 UTC

    He doesn’t have to be a citizen to file the complaint. It is up to the state attorneys to decide whether to actually pursue the matter, which is why I suggest that all of the server operators being hit by D-Link should each file a complaint. Because it almost certainly violates California’s computer crime laws, it becomes a matter of D-Link vs the State.

    As to why I (unlike some others) believe it is a criminal issue: as soon as D-Link became aware of the problem and refused to attempt to fix it, it became an intentional act, so any legal requirement that it be deliberate has been satisfied. Because it is costing him real money to mitigate the damages they are causing, it rises above the level of a mere nuisance that the state attorney would not find worth investigating. Because there exists a LOT of documentation that says that end-user systems should not be synchronizing directly to stratum-1 NTP servers, it is a matter of D-Link violating accepted standards in order to cause problems for these server operators. Finally, California law in these matters is frequently more strict than most. I think it is worth the relatively minor effort required to contact the California Attorney General’s Office.

    I think the second part of my strategy is important, too. It accelerates the process of getting the bad D-Link firmware off the Internet. It is probably only practical for those server operators who can clearly tell from IP number whether the connection comes from an authorized user, because they can automate the process of sending out notifications.

    And, with regard to another statement in PHK’s open letter, don’t understand why he can’t sue D-Link in Denmark (other than not wanting to pay even more in legal fees than he already has). That is certainly not up to D-Link: only Danish law can decide that. Whether he’d actually be able to collect any money from them if he won the case would be a different issue — that alone may make it not worth the cost of suing in Denmark.

  • 116. Rick Nelson  |  April 19th, 2006 at 13:49 UTC

    I thought that as a trade-press editor I might be able to get specific comments from D-Link about an article I want to write about this issue, but the company would only issue a statement saying it is investigating in an effort to achieve a full resolution. Here are the specific questions I asked (18 April 06):

    First, do you believe the problem exists, as described by timeserver Poul-Henning Kamp here:
    http://people.freebsd.org/~phk/dlink/

    If not, what part of the analysis by Kamp and Richard Clayton (www.lightbluetouchpaper.org/2006/04/07/when-firmware-attacks-ddos-by-d-link/) do you dispute?

    If so, what steps are you taking to resolve the situation, if any?

    If you agree with Kamp and Clayton’s analysis and are not taking any steps, why not?

    Clayton estimates that you have shipped tens of millions of devices that may be inappropriately querying timeservers. Is this “tens of millions” figure a realistic estimate of the numbers of products you have shipped with NTP capability?

    Kamp reports that one or more D-Link representatives have accused him of extortion. Is that correct? If so, what was the manner of this extortion?

    Is there anything you would like to add about this issue?

  • 117. David Kirkby  |  April 21st, 2006 at 22:59 UTC

    According to an an article ‘Time geek’ hopeful of deal with D-Link dated 17/4/2006, Poul-Henning Kamp and D-link might be close to a settlement. To quote from there:

    * Kamp said in an interview Friday. “With a little luck
    * we should be able to settle it by next week,” he said.

    I’ve not seen anything on Poul-Henning Kamp’’s page to indicate this, but no doubt he will wait until it is settled.

  • 118. .$author.  |  April 27th, 2006 at 10:07 UTC

    [...] All the fuss about D-Link’s usage of the Danish-based stratum 1 time server seems to have had one good result. Poul-Henning Kamp’s web page has the following announcement this morning: [...]

  • 119. .$author.  |  April 27th, 2006 at 16:21 UTC

    [...] I have an update for anyone intereseted on the D-Link situation. Its hard to believe there are those who would question the efficiency of the threat of boycott coupled with a letter writing and blogging campaign. [...]

  • 120. Scott Daugherty  |  May 6th, 2006 at 15:30 UTC

    I found this a good read for I have a D-link wireless router. I had the router for some time now but notice I could not set ntp when I bought it. Last year I set up a vpn and converted the d-link to access point duties now but I had updated the firm ware at the time. D-link states the firm ware update fixes the NTP issue. I then pointed the NTP to a local host computer. I just checked it and it is still correct. I hope D-link owns up to being responsible as indicated above. D-link should. Now that I know this I will update firm ware on other networks I know of using d-link products too.

  • 121. Tony Gore  |  May 7th, 2006 at 11:22 UTC

    The ISPs could be more helpful and give their NTP server details prominently to their customers. Sometimes they require a lot of searching for.

    The Belkin router I have only has a list of IP addresses to choose for NTP; the default for Europe is 129.132.2.21 which resolves to swisstime.ee.ethz.ch and the secondary default is 130.149.17.8 which is ntps1-1.cs.tu-berlin.de. There is no way to actually change these, as they are a drop down list. It does seem that unless these people have agreed to “host” NTP for Belkin, then Belkin are guilty of abuse – just as any SMTP server originally could be used for sending mail, and abuse by spammers put a stop to that.

    Microsoft in http://support.microsoft.com/kb/262680/en-us gives a list of public timeservers. At no point does it mention that your ISP timeserver is better placed to do the job. If you run a Windows server system, time is important to you – clients cannot log on to the server if their time and date vary too much from the server, at least with SBS2003.

    Routers may not be the only culprit – the Symantec hardware frewalls e.g. my 200R use “default timeservers” without saying which ones they are. However, you can enter your own timeserver.

    There are other devices as well – my D-link DCS-900 wired camera has an NTP server – for this, you have to enter an IP address. (I had been a good boy and resolved ntp.demon.co.uk – my ISP and entered that one).

    It seems to me, coming back to the ISP point, that ISPs should produce a list of things to use – both by name and by IP address. They should state that the name should be used in preference to the IP address (and equipment manufactures should be pushed to use names, not IP addresses).

    After all, it is in their best interests – very NTP packet that goes out of the ISP’s network costs them more than if they handled it themselves.

  • 122. Al Bounds  |  May 8th, 2006 at 13:48 UTC

    D-Link’s attitude in this matter is very simmilar to how they treated me when my D-Link WAP died for a third time while under warranty. Glad it was mine and not one of my customers! I made the descision to stay away from D-Link a few months ago and this story supports that descision. Bad product, bad company, bad attitude.

  • 123. .$author.  |  June 7th, 2006 at 21:32 UTC

    [...] Dedicated readers will recall my article about how I tracked down the “DDoS” attack on stratum 1 time servers by various D-Link devices. I’ve now had a paper accepted at the 2nd Workshop on Steps to Reducing Unwanted Traffic on the Internet (SRUTI’06) which runs in California in early July. [...]

  • 124. Criggie  |  June 15th, 2006 at 20:47 UTC

    I routinely set NTP update intervals to 1440 or a few minutes less. The recommended update time of 300 minutes seems too short… and if my clocks drift by some seconds then no great loss.

    I also run a local timeserver just for work’s network.

    What would make life much more useful would be if DHCP could carry information on all settings… ideally http_proxy as well as time_server and so on.

  • 125. They do the same to DynDns  |  December 13th, 2006 at 02:52 UTC

    Many DLink routers also display the same lack of respect for the DynDns service (Dynamic IP’s).

    DynDns was forced some years ago to block all traffic from DLink routers because they would not obey the procedures laid out for updating one’s dynamic host name. (One update only ever 28 days or so, and ONLY if your IP address has changed). DLink also do not provide a valid ID string when sending the requests.

    On the subject of my DI-604’s NTP setting: sure, I can change it, along with the other fields on the form, but after submitting the changes, I see no evidence that the information was kept. The form reloads with the same default information, leading me to wonder if the DI 604 is even using the configuration I entered.

    DI 604 rev A, B & C last firmware was 2.20. I feel there is almost no chance there will ever be another official update patch. Indeed, I found this page by looking for a hacked firmware update.

  • 126. David Klaus  |  January 7th, 2007 at 05:04 UTC

    Just a heads up. I purchased a new piece of DLink hardware and they now show the following NTP servers in a setup list:

    ntp1.dlink.com
    ntp.dlink.com.tw

    I checked and both seem to be working.

  • 127. Pasi  |  June 13th, 2007 at 06:16 UTC

    When I change the time mode to manual in my dlink 614+ it still tries to contact ntp in 30 seconds interwall. Logfile is filled with “get Time fail” and “NTP: timeout, no receive any data”. And yes, I have updated the firmware to the latest with revision info “Fixes an issue with NTP server” but it still keeps trying that NTP.

    Is this firmware update just a removal of Kamps NTP-server from the NTP list or what?

  • 128. Richard Clayton  |  June 13th, 2007 at 07:40 UTC

    Some time ago, when I last looked at the firmware, almost all the stratum 1 time servers were still present :-( My recommendation is to set a local NTP server (usually at your ISP) in by hand… if it doesn’t respond then you should be looking at firewall policies.

  • 129. Dave Baxter  |  March 27th, 2009 at 10:51 UTC

    Interesting this NTP stuff.

    I have a slight issue with NTP at the moment (March 2009) as my HF Beacon monitoring is being screwed arround with by unreliable NTP servers of and via my home ISP (Demon)

    How easy/dificult is it in reality to setup a LAN based GPS driven NTP server for Faros (what produces the data the url links to) to use?

    I have seen several web pages relating to this, but I don’t know Linux well enoug to have the confidence to do the same, with a current issue of FreeBSD.

    Comments welcome.

    Regards.

    Dave G0WBX/G8KBV

  • 130. jm  |  August 17th, 2012 at 00:57 UTC

    Di-604, latest firmware 3.52: a bug is present that causes the router to ignore the update interval setting, updating once per hour regardless. At least now I can put in ntp1.dlink.com.

    The router has a bug that causes the WAN ping and remote admin to fail due to having reply packets addressed from the router’s LAN IP out the WAN port, so no connection can be made.

Leave a Comment

Required

Required, hidden

Some HTML allowed:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Subscribe to the comments via RSS Feed


Calendar

April 2006
M T W T F S S
« Mar   May »
 12
3456789
10111213141516
17181920212223
24252627282930