OVHcloud Network Status
Current status
Legend
- Operational
- Degraded performance
- Partial Outage
- Major Outage
- Under maintenance
FS#8422 — 178.33.62/24 - switchs upgrade
Scheduled Maintenance
Report for Network & Infrastructure
Completed
Currently this network is using Nexus 5010.
We will change it and implement more performing switches, Nexus 5548.
Update(s):
Date: 2013-04-11 23:49:36 UTC
The upgrade was completed but did not fix the problems on the ipmi vlan and the vracks' netboot .
Regarding IPMI, we established a workaround and we are operating normally.
Netboot problems for vrack customers persist for now. We will continue our investigations with Cisco engineers.
Date: 2013-04-11 23:45:57 UTC
We will reboot the second switch in order to continue the update.
Traffic to the network 178.33.62 will be impacted for a few minutes.
Date: 2013-04-11 23:44:50 UTC
We resumed the upgrade of the platform.
Date: 2013-04-11 23:44:09 UTC
The quick upgrade went wrong, the situation is that a switch was upgraded and the other one was not. The FEX and primary switch are still using the old release.
If the quick upgrade fails again on the second switch, there is a risk of a longer downtime because the images of the new release should be downloaded by the switch on all FEX. We will therefore postpone the upgrade till later in the evening to minimize the impact on our customers.
We are planning to resume the intervention at 23:00.
Date: 2013-04-11 23:38:45 UTC
The ISSU upgrade went wrong and traffic to the servers was impacted.
sw-n5-62.249# sh install all failure-reason
Install has failed. Return code 0x40930039 (aborting due to failed upgrade)
The infrastructure is now up but in a degraded status.
Date: 2013-04-11 23:37:36 UTC
All FEX is offline:
100 FEX0100 Offline Sequence N2K-C2232PP-10GE SSI15420EDH
101 FEX0101 Offline N2K-C2232PP-10GE SSI15410357
102 FEX0102 Offline N2K-C2232PP-10GE SSI15410358
103 FEX0103 Offline Sequence N2K-C2232PP-10GE SSI15420EHQ
104 FEX0104 Offline N2K-C2232PP-10GE SSI15500JLL
105 FEX0105 Offline N2K-C2232PP-10GE SSI15460ARY
106 FEX0106 Offline Sequence N2K-C2232PP-10GE SSI15460AXP
107 FEX0107 Offline N2K-C2232PP-10GE SSI160700NJ
111 FEX0111 Offline N2K-C2248TP-1GE SSI154706RN
Date: 2013-04-11 23:37:10 UTC
We have also netboot problems for vrack customers.
After consultation with Cisco, we will upgrade the software on this pair.
Date: 2013-04-11 11:50:14 UTC
Following the update, we are having an impi problem.
We are investigating.
Date: 2013-04-11 02:35:59 UTC
Done
We will change it and implement more performing switches, Nexus 5548.
Update(s):
Date: 2013-04-11 23:49:36 UTC
The upgrade was completed but did not fix the problems on the ipmi vlan and the vracks' netboot .
Regarding IPMI, we established a workaround and we are operating normally.
Netboot problems for vrack customers persist for now. We will continue our investigations with Cisco engineers.
Date: 2013-04-11 23:45:57 UTC
We will reboot the second switch in order to continue the update.
Traffic to the network 178.33.62 will be impacted for a few minutes.
Date: 2013-04-11 23:44:50 UTC
We resumed the upgrade of the platform.
Date: 2013-04-11 23:44:09 UTC
The quick upgrade went wrong, the situation is that a switch was upgraded and the other one was not. The FEX and primary switch are still using the old release.
If the quick upgrade fails again on the second switch, there is a risk of a longer downtime because the images of the new release should be downloaded by the switch on all FEX. We will therefore postpone the upgrade till later in the evening to minimize the impact on our customers.
We are planning to resume the intervention at 23:00.
Date: 2013-04-11 23:38:45 UTC
The ISSU upgrade went wrong and traffic to the servers was impacted.
sw-n5-62.249# sh install all failure-reason
Install has failed. Return code 0x40930039 (aborting due to failed upgrade)
The infrastructure is now up but in a degraded status.
Date: 2013-04-11 23:37:36 UTC
All FEX is offline:
100 FEX0100 Offline Sequence N2K-C2232PP-10GE SSI15420EDH
101 FEX0101 Offline N2K-C2232PP-10GE SSI15410357
102 FEX0102 Offline N2K-C2232PP-10GE SSI15410358
103 FEX0103 Offline Sequence N2K-C2232PP-10GE SSI15420EHQ
104 FEX0104 Offline N2K-C2232PP-10GE SSI15500JLL
105 FEX0105 Offline N2K-C2232PP-10GE SSI15460ARY
106 FEX0106 Offline Sequence N2K-C2232PP-10GE SSI15460AXP
107 FEX0107 Offline N2K-C2232PP-10GE SSI160700NJ
111 FEX0111 Offline N2K-C2248TP-1GE SSI154706RN
Date: 2013-04-11 23:37:10 UTC
We have also netboot problems for vrack customers.
After consultation with Cisco, we will upgrade the software on this pair.
Date: 2013-04-11 11:50:14 UTC
Following the update, we are having an impi problem.
We are investigating.
Date: 2013-04-11 02:35:59 UTC
Done