OVHcloud Network Status
Current status
Legend
- Operational
- Degraded performance
- Partial Outage
- Major Outage
- Under maintenance
FS#10705 — MacSec
Scheduled Maintenance
Report for Network & Infrastructure
Completed
In order to guarantee the confidentiality of data exchanged on our network,
we are gradually activating data encryption passing across the backbone.
The encryption is done on the hardware without increasing latency,
in 128 bits AES-GCM.
Update(s):
Date: 2014-05-01 10:16:00 UTC
We have changed the 3 cards and the problem persists...
.. the supervisor or the baby bug ..
Date: 2014-04-30 07:40:01 UTC
That doesn't fix the problem. Changing the
2 linecards on nwk-2-6k
Apr 30 08:15:07 GMT: %CONST_DIAG-3-HM_PORT_TEST_FAIL: Module 4 TestNonDisruptiveLoopback Port(s)[8] failed. System operation continues.
12w1d: %HA_EM-6-LOG: Mandatory.go_nondislp.tcl: GOLD EEM TCL policy for TestNonDisruptiveLoopback
Apr 30 08:15:21 GMT: %CONST_DIAG-3-HM_PORT_TEST_FAIL: Module 7 TestNonDisruptiveLoopback Port(s)[3] failed. System operation continues.
Date: 2014-04-30 07:28:11 UTC
The same thing for the nwk-1-6k router slot 2
nwk-1-6k-SUP2T#sh diagnostic result module 2 failure
Current bootup diagnostic level: complete
Module 2: DCEF2T 8 port 10GE
Overall Diagnostic Result for Module 2 : MINOR ERROR
Diagnostic level at card bootup: complete
Test results: (. = Pass, F = Fail, U = Untested)
38) TestNonDisruptiveLoopback:
Port 1 2 3 4 5 6 7 8
----------------------------
F . U U F U . U
nwk-1-6k-SUP2T#hw-module module 2 reset
Proceed with reload of module?[confirm]
% reset issued for module 2
Date: 2014-04-30 07:20:33 UTC
The same thing for slot 4
nwk-2-6k-SUP2T#hw-module module 4 reset
Proceed with reload of module?[confirm]
% reset issued for module 4
Date: 2014-04-30 07:19:57 UTC
We're rebooting card 7 to see if that fixes
the problem. If not, we will replace it.
nwk-2-6k-SUP2T#sh diagnostic result module 7 failure
Current bootup diagnostic level: complete
Module 4: DCEF2T 8 port 10GE
Overall Diagnostic Result for Module 7 : MINOR ERROR
Diagnostic level at card bootup: complete
Test results: (. = Pass, F = Fail, U = Untested)
38) TestNonDisruptiveLoopback:
Port 1 2 3 4 5 6 7 8
----------------------------
. . U U U U U F
nwk-2-6k-SUP2T#hw-module module 7 reset
Proceed with reload of module?[confirm]
% reset issued for module 7
nwk-2-6k-SUP2T#
Date: 2014-04-30 03:04:59 UTC
Another linecard isn't working properly. We will also launch a TAC with Cisco in order to replace the linecard.
Apr 29 14:39:47 GMT: %CONST_DIAG-3-HM_PORT_TEST_FAIL: Module 2 TestNonDisruptiveLoopback Port(s)[7] failed. System operation continues.
Apr 29 14:50:45 GMT: %CONST_DIAG-3-HM_PORT_TEST_FAIL: Module 2 TestNonDisruptiveLoopback Port(s)[5] failed. System operation continues.
Apr 29 15:03:34 GMT: %CONST_DIAG-3-HM_PORT_TEST_FAIL: Module 2 TestNonDisruptiveLoopback Port(s)[7] failed. System operation continues.
Apr 29 15:15:44 GMT: %CONST_DIAG-3-HM_PORT_TEST_FAIL: Module 2 TestNonDisruptiveLoopback Port(s)[1] failed. System operation continues.
Apr 29 15:19:49 GMT: %CONST_DIAG-3-HM_PORT_TEST_FAIL: Module 2 TestNonDisruptiveLoopback Port(s)[7] failed. System operation continues.
12w1d: %HA_EM-6-LOG: Mandatory.go_nondislp.tcl: GOLD EEM TCL policy for TestNonDisruptiveLoopback
Apr 29 15:19:49 GMT: %PM-4-ERR_DISABLE: diagnostics error detected on Te2/7, putting Te2/7 in err-disable state
Apr 29 15:19:50 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/7, changed state to down
Apr 29 15:19:50 GMT: %LINK-3-UPDOWN: Interface TenGigabitEthernet2/7, changed state to down
Apr 29 15:20:43 GMT: %CONST_DIAG-3-HM_PORT_TEST_FAIL: Module 2 TestNonDisruptiveLoopback Port(s)[1] failed. System operation continues.
12w1d: %HA_EM-6-LOG: Mandatory.go_nondislp.tcl: GOLD EEM TCL policy for TestNonDisruptiveLoopback
12w1d: %HA_EM-6-LOG: Mandatory.go_nondislp.tcl: GOLD EEM TCL policy for TestNonDisruptiveLoopback
Apr 29 15:40:43 GMT: %CONST_DIAG-3-HM_PORT_TEST_FAIL: Module 2 TestNonDisruptiveLoopback Port(s)[5] failed. System operation continues.
12w1d: %HA_EM-6-LOG: Mandatory.go_nondislp.tcl: GOLD EEM TCL policy for TestNonDisruptiveLoopback
Apr 29 15:40:44 GMT: %PM-4-ERR_DISABLE: diagnostics error detected on Te2/5, putting Te2/5 in err-disable state
Date: 2014-04-30 03:03:42 UTC
A port isn't working properly. We will replace the linecard
Apr 29 15:01:14 GMT: %CONST_DIAG-3-HM_PORT_TEST_FAIL: Module 7 TestNonDisruptiveLoopback Port(s)[3] failed. System operation continues.
12w1d: %HA_EM-6-LOG: Mandatory.go_nondislp.tcl: GOLD EEM TCL policy for TestNonDisruptiveLoopback
Apr 29 15:01:14 GMT: %PM-4-ERR_DISABLE: diagnostics error detected on Te7/3, putting Te7/3 in err-disable state
Apr 29 15:01:14 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet7/3, changed state to down
Apr 29 15:01:14 GMT: %LINK-3-UPDOWN: Interface TenGigabitEthernet7/3, changed state to down
Date: 2014-04-29 14:08:35 UTC
- 20 Gbps Telia links between the nwk-2-6k and chi-2-6k router: done
20 Gbps remains between the nwk-2-6k and chi-2-6k router. We need to move a link.
Date: 2014-04-29 14:08:28 UTC
- 30 Gbps Level3/Telia links between the nwk-1-6k and chi-1-6k router: done
Date: 2014-04-29 14:07:54 UTC
- 80 Gbps transatlantic links between the ldn-1-6k and nwk-1-6k router: done
Date: 2014-04-29 07:50:20 UTC
We have activated the authentication and encryption
of 80 Gbps transatlantic links between the ldn-5-6k
and nwk-2-6k router.
we are gradually activating data encryption passing across the backbone.
The encryption is done on the hardware without increasing latency,
in 128 bits AES-GCM.
Update(s):
Date: 2014-05-01 10:16:00 UTC
We have changed the 3 cards and the problem persists...
.. the supervisor or the baby bug ..
Date: 2014-04-30 07:40:01 UTC
That doesn't fix the problem. Changing the
2 linecards on nwk-2-6k
Apr 30 08:15:07 GMT: %CONST_DIAG-3-HM_PORT_TEST_FAIL: Module 4 TestNonDisruptiveLoopback Port(s)[8] failed. System operation continues.
12w1d: %HA_EM-6-LOG: Mandatory.go_nondislp.tcl: GOLD EEM TCL policy for TestNonDisruptiveLoopback
Apr 30 08:15:21 GMT: %CONST_DIAG-3-HM_PORT_TEST_FAIL: Module 7 TestNonDisruptiveLoopback Port(s)[3] failed. System operation continues.
Date: 2014-04-30 07:28:11 UTC
The same thing for the nwk-1-6k router slot 2
nwk-1-6k-SUP2T#sh diagnostic result module 2 failure
Current bootup diagnostic level: complete
Module 2: DCEF2T 8 port 10GE
Overall Diagnostic Result for Module 2 : MINOR ERROR
Diagnostic level at card bootup: complete
Test results: (. = Pass, F = Fail, U = Untested)
38) TestNonDisruptiveLoopback:
Port 1 2 3 4 5 6 7 8
----------------------------
F . U U F U . U
nwk-1-6k-SUP2T#hw-module module 2 reset
Proceed with reload of module?[confirm]
% reset issued for module 2
Date: 2014-04-30 07:20:33 UTC
The same thing for slot 4
nwk-2-6k-SUP2T#hw-module module 4 reset
Proceed with reload of module?[confirm]
% reset issued for module 4
Date: 2014-04-30 07:19:57 UTC
We're rebooting card 7 to see if that fixes
the problem. If not, we will replace it.
nwk-2-6k-SUP2T#sh diagnostic result module 7 failure
Current bootup diagnostic level: complete
Module 4: DCEF2T 8 port 10GE
Overall Diagnostic Result for Module 7 : MINOR ERROR
Diagnostic level at card bootup: complete
Test results: (. = Pass, F = Fail, U = Untested)
38) TestNonDisruptiveLoopback:
Port 1 2 3 4 5 6 7 8
----------------------------
. . U U U U U F
nwk-2-6k-SUP2T#hw-module module 7 reset
Proceed with reload of module?[confirm]
% reset issued for module 7
nwk-2-6k-SUP2T#
Date: 2014-04-30 03:04:59 UTC
Another linecard isn't working properly. We will also launch a TAC with Cisco in order to replace the linecard.
Apr 29 14:39:47 GMT: %CONST_DIAG-3-HM_PORT_TEST_FAIL: Module 2 TestNonDisruptiveLoopback Port(s)[7] failed. System operation continues.
Apr 29 14:50:45 GMT: %CONST_DIAG-3-HM_PORT_TEST_FAIL: Module 2 TestNonDisruptiveLoopback Port(s)[5] failed. System operation continues.
Apr 29 15:03:34 GMT: %CONST_DIAG-3-HM_PORT_TEST_FAIL: Module 2 TestNonDisruptiveLoopback Port(s)[7] failed. System operation continues.
Apr 29 15:15:44 GMT: %CONST_DIAG-3-HM_PORT_TEST_FAIL: Module 2 TestNonDisruptiveLoopback Port(s)[1] failed. System operation continues.
Apr 29 15:19:49 GMT: %CONST_DIAG-3-HM_PORT_TEST_FAIL: Module 2 TestNonDisruptiveLoopback Port(s)[7] failed. System operation continues.
12w1d: %HA_EM-6-LOG: Mandatory.go_nondislp.tcl: GOLD EEM TCL policy for TestNonDisruptiveLoopback
Apr 29 15:19:49 GMT: %PM-4-ERR_DISABLE: diagnostics error detected on Te2/7, putting Te2/7 in err-disable state
Apr 29 15:19:50 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/7, changed state to down
Apr 29 15:19:50 GMT: %LINK-3-UPDOWN: Interface TenGigabitEthernet2/7, changed state to down
Apr 29 15:20:43 GMT: %CONST_DIAG-3-HM_PORT_TEST_FAIL: Module 2 TestNonDisruptiveLoopback Port(s)[1] failed. System operation continues.
12w1d: %HA_EM-6-LOG: Mandatory.go_nondislp.tcl: GOLD EEM TCL policy for TestNonDisruptiveLoopback
12w1d: %HA_EM-6-LOG: Mandatory.go_nondislp.tcl: GOLD EEM TCL policy for TestNonDisruptiveLoopback
Apr 29 15:40:43 GMT: %CONST_DIAG-3-HM_PORT_TEST_FAIL: Module 2 TestNonDisruptiveLoopback Port(s)[5] failed. System operation continues.
12w1d: %HA_EM-6-LOG: Mandatory.go_nondislp.tcl: GOLD EEM TCL policy for TestNonDisruptiveLoopback
Apr 29 15:40:44 GMT: %PM-4-ERR_DISABLE: diagnostics error detected on Te2/5, putting Te2/5 in err-disable state
Date: 2014-04-30 03:03:42 UTC
A port isn't working properly. We will replace the linecard
Apr 29 15:01:14 GMT: %CONST_DIAG-3-HM_PORT_TEST_FAIL: Module 7 TestNonDisruptiveLoopback Port(s)[3] failed. System operation continues.
12w1d: %HA_EM-6-LOG: Mandatory.go_nondislp.tcl: GOLD EEM TCL policy for TestNonDisruptiveLoopback
Apr 29 15:01:14 GMT: %PM-4-ERR_DISABLE: diagnostics error detected on Te7/3, putting Te7/3 in err-disable state
Apr 29 15:01:14 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet7/3, changed state to down
Apr 29 15:01:14 GMT: %LINK-3-UPDOWN: Interface TenGigabitEthernet7/3, changed state to down
Date: 2014-04-29 14:08:35 UTC
- 20 Gbps Telia links between the nwk-2-6k and chi-2-6k router: done
20 Gbps remains between the nwk-2-6k and chi-2-6k router. We need to move a link.
Date: 2014-04-29 14:08:28 UTC
- 30 Gbps Level3/Telia links between the nwk-1-6k and chi-1-6k router: done
Date: 2014-04-29 14:07:54 UTC
- 80 Gbps transatlantic links between the ldn-1-6k and nwk-1-6k router: done
Date: 2014-04-29 07:50:20 UTC
We have activated the authentication and encryption
of 80 Gbps transatlantic links between the ldn-5-6k
and nwk-2-6k router.