Get webhook notifications whenever Network & Infrastructure creates an incident, updates an incident, resolves an incident or changes a component status.
Date: 2012-04-16 16:36:37 UTC The situation is stable. We're looking
to fix the problem in the next days.
Date: 2012-04-16 16:35:50 UTC vss-6 A and B are back. The temperatur is correct
The compressors of the 2 AC systems were stopped
but not disjunted. We did not have any alarms.
We had no information about the alarms of the sudden
temperatur increase in the routing rooms. We have a system
which calculates the temperatur every 60 seconds and gives the
information in the datacenter through MARCEL.
It did not work neither.
To restart them, we had to stop them a few seconds and then
restarted them. The temperature was down. We are checking
why the 2 systems are stopped but not disjuncted.
We are checking also why it had an impact on the 2
vss-6 A and B only. At worst a room is impacted and
so on of the 2 routers.
The other routers were hot but had no problem.
In short, a mega SPOF ! that we are going to fix !
Date: 2012-04-16 11:50:07 UTC Temperatures are back to normal.
vss-6a is up again. The routing is restored.
vss-6b completes its boot sequence.
Date: 2012-04-16 11:45:37 UTC vss-6a has just crash again. Networks behind vss-6a / b are cut.
We have implemented an emergency ventilation in the room to dissipate the heat. In //, we managed to re-route the air conditioning system. The temperature gradually decreases in the room.
Date: 2012-04-16 11:44:28 UTC The 2 routers are down again at the same time.
this is a problem of air conditioning in the routing
rooms of RBX4. Apparently we have a SPOF due to
the bad internal reflection.
We try to stabilize the situation and then we will
review it!
Date: 2012-04-16 11:22:39 UTC We have a problem of temperature in the room, Our team work on the problem.
Date: 2012-04-16 11:22:03 UTC vss-6b has crashed at the same time. The 2 routers were out
at the same time ..
Date: 2012-04-16 11:21:01 UTC vss-6b ne redémarre pas:
Apr 16 11:46:10 GMT: %FABRIC-SP-5-CLEAR_BLOCK: Clear block option is off for the fabric in slot 5.
Apr 16 11:46:10 GMT: %FABRIC-SP-5-FABRIC_MODULE_ACTIVE: The Switch Fabric Module in slot 5 became active.
Apr 16 11:46:11 GMT: %CPU_MONITOR-3-PEER_EXCEPTION: CPU_MONITOR peer has failed due to exception , reset by [5/0]
*** System received a Software forced crash ***
signal= 0x17, code= 0x24, context= 0x46644dd4
PC = 0x42da4ebc, SP = 0x44954918, RA = 0x413ea2bc
Cause Reg = 0x00003820, Status Reg = 0x34008002
Routing has resumed by vss-6a.
Date: 2012-04-16 11:20:10 UTC vss-6a is up but vss-6b has just crashed.