rssLink RSS for all categories
 
icon_red
icon_green
icon_red
icon_red
icon_blue
icon_green
icon_green
icon_red
icon_red
icon_red
icon_orange
icon_green
icon_green
icon_green
icon_green
icon_blue
icon_green
icon_orange
icon_red
icon_green
icon_red
icon_red
icon_green
icon_red
icon_red
icon_red
icon_red
icon_orange
icon_green
 

FS#5803 — FS#9719 — UPS 2

Attached to Project— Datacenters
Incident
Strasbourg 1
CLOSED
100%
We have had an issue with a UPS in Strasbourg. The power supply to some racks was cut for a few minutes.
Date:  Wednesday, 20 November 2013, 00:28AM
Reason for closing:  Done
Comment by OVH - Tuesday, 19 November 2013, 12:08PM

During deployment of a new generator in Strasbourg (http://status.ovh.co.uk/?do=details&id=5796), we conducted a temporary generator test. The UPS 2 shut down during this test.
We don't yet have an explanation for this shutdown, we are making enquiries with the manufacturer.


Comment by OVH - Tuesday, 19 November 2013, 18:54PM

42 more servers down, a good part of which are in check filesystem


Comment by OVH - Tuesday, 19 November 2013, 18:58PM

The manufacturer has been on site for 2 hours and is continuing to extract the logs and try and understand with our teams.
It seems that the outage was intervened the second we switched onto the temporary generator.


Comment by OVH - Wednesday, 20 November 2013, 00:28AM

Thereafter deep investigations, it turned out that 2 failures have gotten us to this situation:

- During the last intervention on UPS2, the UPS manufacturer has made an error / oversight on a bit of the UPS setting. This bit seems being at the origin of the UPS output cutoff. No explanation other than the human error by the manufacturer.

- During the setting of the temporary generator, the electrician installer didn't check the tension level issued by the temporary generator. This was sensitively out of range for 2 minutes, which couldn't cause any issue if the UPS was properly set...

The UPS setting issue is fixed.
The generator's tension level check will be looked after tomorrow (setting into production the temporary generator has been postponed for tomorrow).