OVHcloud Private Cloud Status

Current status
Legend
  • Operational
  • Degraded performance
  • Partial Outage
  • Major Outage
  • Under maintenance
FS#6506 — Network worktasks pcc-1a/b-n7 - 19-20/03/12
Scheduled Maintenance Report for Hosted Private Cloud
Completed
We are performing tonight worktasks on the PCC network infrastructure. We need to replace two cards F1 by M1 in order to advance the infrastructure with mac. that's why we will move some uplinks to other cards and use only M1 for the peer-links (monitoring links between the switches) and uplinks to routers.

We start replacing the cards tonight from 11:30 pm.Then we will have a sequence where we will completely isolate one of the infrastructure 's two switches in order to restore vPC (Virtual PortChannel) on the new card M1. We will finish by migrating uplinks.

The CPC infrastructure is completely redundant, we do not anticipate any impact on the service 's accessibility.The network and pcc team are still mobilized in order to respond immediately in case of problems during the intervention.

Update(s):

Date: 2012-03-20 04:25:49 UTC
Switching uplinks is completed. Remains the last part of the worktask planned: we will reset the sup5 of the pcc-1a in which a minor fault is detected.

Date: 2012-03-20 04:22:17 UTC
Switching on the uplinks pcc-1a completed.It's in progress for pcc-1b.

Date: 2012-03-20 04:21:09 UTC
The verifications of the infrastructure are still in progress.
According to preliminary analyzes, the pcc-1b would have cut all its vPC Po when the peer-link was reconnected. Remains to know why,because this was not planned. We work with Cisco in order to understand what happened.
In //, we finish the worktask by switchning uplinks on the new cards.

Date: 2012-03-20 03:29:31 UTC
We reactivated the peer-link but we have had a fortuitous, for some reason that we should identify,the switches have exchanged their roles (primary / secondary) and this caused a cut-off in traffic for about 2min on the infrastructure.
We disabled immediately the vPC port-channels on pcc-1a. From a network point of view, the infrastrucuture is nominal operating back.
Verifications on the infra CPC are in progress.

Date: 2012-03-20 03:13:55 UTC
The switch pcc-n7-1a is completely isolated,we reconfigure the peer-link.

Date: 2012-03-20 03:12:39 UTC
All Po vPC are cut-off. Now we cut the peer-link between the two main switches of the pcc infrastructure, pcc-1a and 1b.

Date: 2012-03-20 03:08:21 UTC
The new cards are implemented. We start cutting manually vPC port-channels on pcc-1a-n7

Date: 2012-03-20 03:01:56 UTC
We started the worktask.
swapping cards F1 by M1.
Posted Mar 19, 2012 - 22:01 UTC