OVHcloud Network Status

Current status
Legend
  • Operational
  • Degraded performance
  • Partial Outage
  • Major Outage
  • Under maintenance
FS#16833 — Switch /Cloud gra-z1b*-a70
Scheduled Maintenance Report for Network & Infrastructure
Completed
To be ready for the vRack on GRA /CLOUD, we will update all TOR switches of the infrastructure.

The switch will be updated in hitless mode, this will be transparent for client.

We starting the switches update at 23:00 CET.

Update(s):

Date: 2016-03-22 22:22:19 UTC
gra-z1b7-a70 to gra-z1b11-a70 and gra-z1b13-a70 to gra-z1b15-a70 are done.

Date: 2016-03-22 21:15:26 UTC
We start the maintenance.

Date: 2016-03-22 13:57:54 UTC
We will continue the migration tonight 22 March 2016 at 22h00 GMT +1

Date: 2016-02-29 23:16:59 UTC
gra-z1b6-a70 is back online without loss. Done.

We will stop maintenance for tonight.


Date: 2016-02-29 23:07:39 UTC
gra-z1b7-a70 done

Date: 2016-02-29 23:05:37 UTC
gra-z1b3-a70 done
gra-z1b4-a70 done
gra-z1b5-a70 done
gra-z1b6-a70 issue:
Mar 1 00:02:45 CET gra-z1b6-a70 ProcMgr-worker: %PROCMGR-6-PROCESS_RESTART: Restarting 'StrataL3' immediately (it had PID=4336)
2016-03-01T01:02:45+02:00 92.222.60.253 gra-z1b6-a70.fr.eu 2016 a6 [local4.info] === 2016 Mar 1 00:02:45 CET gra-z1b6-a70 ProcMgr-worker: %PROCMGR-6-PROCESS_STARTED: 'StrataL3' starting with PID=4395 (PPID=1921) -- execing '/usr/bin/StrataL3'
2016-03-01T01:02:47+02:00 92.222.60.253 gra-z1b6-a70.fr.eu 2016 a6 [local4.info] === 2016 Mar 1 00:02:47 CET gra-z1b6-a70 Strata: %HARDWARE-6-PERR_CORRECTABLE: Correctable parity error detected in a Strata switching ASIC (chip: Linecard0/0, table: 2529 (L3_IPMC), index: 6647, address: 0x402419f7).
2016-03-01T01:02:52+02:00 92.222.60.253 gra-z1b6-a70.fr.eu 2016 a6 [local4.info] === 2016 Mar 1 00:02:52 CET gra-z1b6-a70 Strata: %HARDWARE-6-PERR_CORRECTABLE: Correctable parity error detected in a Strata switching ASIC (chip: Linecard0/0, table: 2039 (ING_L3_NEXT_HOP), index: 11428, address: 0x40102ca4).
2016-03-01T01:02:52+02:00 92.222.60.253 gra-z1b6-a70.fr.eu 2016 a6 [local4.info] === 2016 Mar 1 00:02:52 CET gra-z1b6-a70 Strata: %HARDWARE-6-PERR_CORRECTABLE: Correctable parity error detected in a Strata switching ASIC (chip: Linecard0/0, table: 2529 (L3_IPMC), index: 6663, address: 0x40241a07).

We reloading it totally.

Date: 2016-02-29 22:47:49 UTC
We had an issue with gra-z1b2-a70:

2016-03-01T00:28:01+02:00 92.222.60.189 gra-z1b2-a70.fr.eu 2016 a4 [local4.warning] === 2016 Feb 29 23:28:01 CET gra-z1b2-a70 Launcher: %STAGEMGR-4-STAGE_SLOW: Asu Hitless boot stage L3SmsReconcile timed out
2016-03-01T00:28:01+02:00 92.222.60.189 gra-z1b2-a70.fr.eu 2016 a3 [local4.err] === 2016 Feb 29 23:28:01 CET gra-z1b2-a70 Launcher: %STAGEMGR-3-FATAL_ERROR_REBOOT_REQUEST: Agent stageMgr encountered a fatal error. The system will restart now. (L3SmsReconcile timed out)

The impact is 5 minutes, the switch has been updated. We investigate the issue.



Date: 2016-02-29 22:21:33 UTC
gra-z1b2-a70 done

Date: 2016-02-29 22:21:15 UTC
gra-z1b1-a70 done
Posted Feb 29, 2016 - 22:20 UTC