Had a case where I tested a triple-redundancy-system hundreds of times (during production time, zero hiccups), but during a presentation to higher-ups… it fails spetacularly, due to a combination of a failover fail and a legit loss – Someone scheduled a truck crash ripping two fibers just at the time of demonstration, and a bug in the code made so that if it ever lost ALL connections it'd softlock.
A hundred of workers without systems and thousands of customers without service because I was feeling confident and proud of my accomplishment. And a truck freak.
Top Comments
Odie
Aug 18, 2019 at 10:49PM EDT
LurkerOfFiveYears
Aug 21, 2019 at 03:40AM EDT