OVHcloud Network Status

Current status
Legend
  • Operational
  • Degraded performance
  • Partial Outage
  • Major Outage
  • Under maintenance
FS#18040 — 37.187.28.248/249
Incident Report for Network & Infrastructure
Resolved
Switches 37.187.28.248 and 37.187.28.249 have rebooted one after the other.

Reason: Reset triggered due to HA policy of Reset
Service: eth_port_sec hap reset

This bug is well-known by Cisco. We are planning to update the switch this week.

Update(s):

Date: 2016-05-16 14:46:54 UTC
Everything seems to be back to normal. We are monitoring.

Date: 2016-05-16 14:46:15 UTC
sw.37.187.28.248# install all kickstart n5000-uk9-kickstart.7.1.3.N1.2a.bin system n5000-uk9.7.1.3.N1.2a.bin

Verifying image bootflash:/n5000-uk9-kickstart.7.1.3.N1.2a.bin for boot variable \"kickstart\".
[####################] 100% -- SUCCESS

Verifying image bootflash:/n5000-uk9.7.1.3.N1.2a.bin for boot variable \"system\".
[####################] 100% -- SUCCESS

Verifying image type.
[####################] 100% -- SUCCESS

Extracting \"system\" version from image bootflash:/n5000-uk9.7.1.3.N1.2a.bin.
[####################] 100% -- SUCCESS

Extracting \"kickstart\" version from image bootflash:/n5000-uk9-kickstart.7.1.3.N1.2a.bin.
[####################] 100% -- SUCCESS

Extracting \"bios\" version from image bootflash:/n5000-uk9.7.1.3.N1.2a.bin.
[####################] 100% -- SUCCESS

Extracting \"fexth\" version from image bootflash:/n5000-uk9.7.1.3.N1.2a.bin.
[####################] 100% -- SUCCESS

Performing module support checks.
[####################] 100% -- SUCCESS

Notifying services about system upgrade.
[####################] 100% -- SUCCESS



Compatibility check is done:
Module bootable Impact Install-type Reason
------ -------- -------------- ------------ ------
1 yes disruptive reset Incompatible image
3 yes disruptive reset Incompatible image
100 yes disruptive reset Incompatible image
101 yes disruptive reset Incompatible image
102 yes disruptive reset Incompatible image
103 yes disruptive reset Incompatible image
104 yes disruptive reset Incompatible image
105 yes disruptive reset Incompatible image
106 yes disruptive reset Incompatible image
107 yes disruptive reset Incompatible image
108 yes disruptive reset Incompatible image
109 yes disruptive reset Incompatible image
111 yes disruptive reset Incompatible image



Images will be upgraded according to following table:
Module Image Running-Version New-Version Upg-Required
------ ---------------- ---------------------- ---------------------- ------------
1 system 6.0(2)N2(2) 7.1(3)N1(2a) yes
1 kickstart 6.0(2)N2(2) 7.1(3)N1(2a) yes
1 bios v3.6.0(05/09/2012) v3.6.0(05/09/2012) no
1 power-seq v2.0 v3.0 yes
1 SFP-uC v1.1.0.0 v1.0.0.0 no
3 power-seq v2.0 v2.0 no
100 fexth 6.0(2)N2(2) 7.1(3)N1(2a) yes
101 fexth 6.0(2)N2(2) 7.1(3)N1(2a) yes
102 fexth 6.0(2)N2(2) 7.1(3)N1(2a) yes
103 fexth 6.0(2)N2(2) 7.1(3)N1(2a) yes
104 fexth 6.0(2)N2(2) 7.1(3)N1(2a) yes
105 fexth 6.0(2)N2(2) 7.1(3)N1(2a) yes
106 fexth 6.0(2)N2(2) 7.1(3)N1(2a) yes
107 fexth 6.0(2)N2(2) 7.1(3)N1(2a) yes
108 fexth 6.0(2)N2(2) 7.1(3)N1(2a) yes
109 fexth 6.0(2)N2(2) 7.1(3)N1(2a) yes
111 fexth 6.0(2)N2(2) 7.1(3)N1(2a) yes
1 microcontroller v1.2.0.1 v1.2.0.1 no


Switch will be reloaded for disruptive upgrade.
Do you want to continue with the installation (y/n)? [n] y

Date: 2016-05-16 14:45:48 UTC
sw.37.187.28.249# System is still initializing
Configuration mode is blocked until system is ready

Date: 2016-05-16 14:45:10 UTC
Pre-loading modules.
[This step might take upto 20 minutes to complete - please wait.]
[*Warning -- Please do not abort installation/reload or powercycle fexes*]
[####################] 100% -- SUCCESS

Finishing the upgrade, switch will reboot in 10 seconds.

Switch 249 is rebooting.

Date: 2016-05-16 14:44:29 UTC
The FEX image preload is in progress. Once the preload is done, the switch will reboot in the latest version and we will upgrade the primary switch.

Date: 2016-05-16 14:42:31 UTC
Secondary update is in progress of breakdown.

sw.37.187.28.249# install all kickstart n5000-uk9-kickstart.7.1.3.N1.2a.bin system n5000-uk9.7.1.3.N1.2a.bin ?

ssi Boot-variable name

sw.37.187.28.249# install all kickstart n5000-uk9-kickstart.7.1.3.N1.2a.bin system n5000-uk9.7.1.3.N1.2a.bin

Verifying image bootflash:/n5000-uk9-kickstart.7.1.3.N1.2a.bin for boot variable \"kickstart\".
[####################] 100% -- SUCCESS

Verifying image bootflash:/n5000-uk9.7.1.3.N1.2a.bin for boot variable \"system\".
[####################] 100% -- SUCCESS

Verifying image type.
[####################] 100% -- SUCCESS

Extracting \"system\" version from image bootflash:/n5000-uk9.7.1.3.N1.2a.bin.
[####################] 100% -- SUCCESS

Extracting \"kickstart\" version from image bootflash:/n5000-uk9-kickstart.7.1.3.N1.2a.bin.
[####################] 100% -- SUCCESS

Extracting \"bios\" version from image bootflash:/n5000-uk9.7.1.3.N1.2a.bin.
[####################] 100% -- SUCCESS

Extracting \"fexth\" version from image bootflash:/n5000-uk9.7.1.3.N1.2a.bin.
[####################] 100% -- SUCCESS

Performing module support checks.
[####################] 100% -- SUCCESS

Notifying services about system upgrade.
[####################] 100% -- SUCCESS



Compatibility check is done:
Module bootable Impact Install-type Reason
------ -------- -------------- ------------ ------
1 yes disruptive reset Incompatible image
3 yes disruptive reset Incompatible image
100 yes disruptive reset Incompatible image
101 yes disruptive reset Incompatible image
102 yes disruptive reset Incompatible image
103 yes disruptive reset Incompatible image
104 yes disruptive reset Incompatible image
105 yes disruptive reset Incompatible image
106 yes disruptive reset Incompatible image
107 yes disruptive reset Incompatible image
108 yes disruptive reset Incompatible image
109 yes disruptive reset Incompatible image
111 yes disruptive reset Incompatible image



Images will be upgraded according to following table:
Module Image Running-Version New-Version Upg-Required
------ ---------------- ---------------------- ---------------------- ------------
1 system 6.0(2)N2(2) 7.1(3)N1(2a) yes
1 kickstart 6.0(2)N2(2) 7.1(3)N1(2a) yes
1 bios v3.6.0(05/09/2012) v3.6.0(05/09/2012) no
1 power-seq v2.0 v3.0 yes
1 SFP-uC v1.1.0.0 v1.0.0.0 no
3 power-seq v2.0 v2.0 no
100 fexth 6.0(2)N2(2) 7.1(3)N1(2a) yes
101 fexth 6.0(2)N2(2) 7.1(3)N1(2a) yes
102 fexth 6.0(2)N2(2) 7.1(3)N1(2a) yes
103 fexth 6.0(2)N2(2) 7.1(3)N1(2a) yes
104 fexth 6.0(2)N2(2) 7.1(3)N1(2a) yes
105 fexth 6.0(2)N2(2) 7.1(3)N1(2a) yes
106 fexth 6.0(2)N2(2) 7.1(3)N1(2a) yes
107 fexth 6.0(2)N2(2) 7.1(3)N1(2a) yes
108 fexth 6.0(2)N2(2) 7.1(3)N1(2a) yes
109 fexth 6.0(2)N2(2) 7.1(3)N1(2a) yes
111 fexth 6.0(2)N2(2) 7.1(3)N1(2a) yes
1 microcontroller v1.2.0.1 v1.2.0.1 no


Switch will be reloaded for disruptive upgrade.
Do you want to continue with the installation (y/n)? [n] y

Install is in progress, please wait.

Date: 2016-05-16 14:41:37 UTC
%SYSMGR-2-SERVICE_CRASHED: Service \"eth_port_sec\" (PID 4853) hasn't caught signal 6 (core will be saved).
%SYSMGR-2-HAP_FAILURE_SUP_RESET: System reset due to service \"eth_port_sec\" in vdc 1 has had a hap failure

Date: 2016-05-16 14:41:23 UTC
We still have problems with the switches. We are upgrading on last version.

Date: 2016-05-16 14:40:54 UTC
Same problem: hap_reset.

Reason: Reset triggered due to HA policy of Reset
Service: eth_port_sec hap reset

If this problem continúes within the next hours, we will proceed with the update.

Date: 2016-05-16 14:39:24 UTC
We have encountered the same problem again. We are investigating to determine the urgency of the upgade.
Posted May 16, 2016 - 14:38 UTC