One router, one bug. How routine upgrades caused worldwide chaos
详细摘要
ASIO, councils and supermarket suppliers lost email for most of two days, all because of an upgrade meant to add capacity.
Microsoft wanted more capacity. America’s aviation regulator wanted to swap ageing copper cables for fibre. Last week, both got chaos.
Microsoft’s email and office software went dark for millions of people from shortly before 5am AEDT on Friday, after an upgrade meant to add capacity. A day later, US authorities grounded every departing flight in the country for five hours after a contractor installed a router at a remote site in Arkansas.
The two failures were unrelated and Microsoft says it has no indication its services played any part in the grounding.
The upgrades themselves were defensible: old systems commonly have to be replaced, and software has to be patched – that’s normal. What failed was containment: one fault in one place was allowed to course through an entire network.
Australians felt it within hours. ASIO and the federal police lost email and so did councils, courts and schools. Australia Post pulled its phone payment system, while suppliers couldn’t tell supermarket distribution centres what was on the way. In Germany, 15 hotel guests were reportedly hurt by chlorine gas after a pool safety system that sent its warnings by email went silent.
A tech company like Microsoft understands the risk as well as anyone. “There are no low-risk deployments to production,” its guidance to engineers warns, meaning no change to a live system is ever truly safe. Its advice is to change things in small steps, roll them out slowly, and stop at the first wobble.
On Friday the company said the outage “was introduced by an issue during an earlier network capacity upgrade that surfaced” after a later network change. Its preliminary notices say traffic moved around during that maintenance and exposed a hidden bug in the systems that connect people to their mailboxes. Requests banked up until the servers choked.
Then the dominoes fell. Work from the broken servers spilled onto healthy ones, which buckled in turn.
The flight grounding was similar: it started with a single router…
文章开头,摘自WAtoday。如需阅读全文,请前往媒体网站。
MarketNews 仅展示媒体发布的标题和摘要,全文请在出版方网站阅读。 (www.watoday.com.au)



