← 返回首页
大洋洲 · 综合新闻 · The Age

One router, one bug. How routine upgrades caused worldwide chaos

来源:The Age · 澳大利亚 1 小时前
Microsoft’s email and office software crashed after an upgrade meant to add capacity.AP Photo/Kirsty Wigglesworth
Microsoft’s email and office software crashed after an upgrade meant to add capacity.AP Photo/Kirsty Wigglesworth · The Age

详细摘要

ASIO, councils and supermarket suppliers lost email for most of two days, all because of an upgrade meant to add capacity.

Microsoft wanted more capacity. America’s aviation regulator wanted to swap ageing copper cables for fibre. Last week, both got chaos.

Microsoft’s email and office software went dark for millions of people from shortly before 5am AEDT on Friday, after an upgrade meant to add capacity. A day later, US authorities grounded every departing flight in the country for five hours after a contractor installed a router at a remote site in Arkansas.

The two failures were unrelated and Microsoft says it has no indication its services played any part in the grounding.

The upgrades themselves were defensible: old systems commonly have to be replaced, and software has to be patched – that’s normal. What failed was containment: one fault in one place was allowed to course through an entire network.

Australians felt it within hours. ASIO and the federal police lost email and so did councils, courts and schools. Australia Post pulled its phone payment system, while suppliers couldn’t tell supermarket distribution centres what was on the way. In Germany, 15 hotel guests were reportedly hurt by chlorine gas after a pool safety system that sent its warnings by email went silent.

A tech company like Microsoft understands the risk as well as anyone. “There are no low-risk deployments to production,” its guidance to engineers warns, meaning no change to a live system is ever truly safe. Its advice is to change things in small steps, roll them out slowly, and stop at the first wobble.

On Friday the company said the outage “was introduced by an issue during an earlier network capacity upgrade that surfaced” after a later network change. Its preliminary notices say traffic moved around during that maintenance and exposed a hidden bug in the systems that connect people to their mailboxes. Requests banked up until the servers choked.

Then the dominoes fell. Work from the broken servers spilled onto healthy ones, which buckled in turn.

The flight grounding was similar: it started with a single router…

文章开头,摘自The Age。如需阅读全文,请前往媒体网站。

在 The Age 阅读全文 登录后可收藏

MarketNews 仅展示媒体发布的标题和摘要,全文请在出版方网站阅读。 (www.theage.com.au)

推荐阅读