rssLink RSS for all categories
 
icon_red
icon_green
icon_red
icon_red
icon_blue
icon_green
icon_green
icon_red
icon_red
icon_red
icon_orange
icon_green
icon_green
icon_green
icon_green
icon_blue
icon_red
icon_orange
icon_red
icon_red
icon_red
icon_red
icon_green
icon_red
icon_red
icon_red
icon_red
icon_orange
icon_green
 

FS#4290 — FS#8224 — Room23

Attached to Project— Datacenters
Incident
Roubaix 2
CLOSED
100%
We have problems in room 23 (23xxx racks)
Our teams are intervening.


Date:  Wednesday, 13 March 2013, 11:21AM
Reason for closing:  Done
Comment by OVH - Tuesday, 12 March 2013, 17:28PM

Work is progressing, but it's slow :(


Comment by OVH - Wednesday, 13 March 2013, 11:21AM

20 people worked for 16 hours to fix all servers in rooms 21/22/23/24. Everything is up.

The RBX2 roof has VVVVV shape like everywhere in Northern France. With the vertical wind, about 10T of snow accumulated in part of the roof. The weight of the snow caused a slight deformation of the roof and the creation some infiltrations in the first floor of RBX2. Rooms 21-24 are on the ground floor, with hot air blown from the ground floor to the first floor we have avoided the worst but two racks were still affected and tens of servers here and there in the 4 rooms where there are 4x 5500 servers.

We intervened on 250 servers in total and replaced about 180 CM. This took some time: check each server, one by one, check the component to replace then restart the server.

Congratulations to the team who managed this exceptional failure!

The roof will be reinforced / changed as soon as it is warm enough.