rssLink RSS for all categories
 
icon_red
icon_green
icon_red
icon_red
icon_blue
icon_green
icon_green
icon_red
icon_red
icon_red
icon_orange
icon_green
icon_green
icon_green
icon_green
icon_blue
icon_green
icon_orange
icon_red
icon_green
icon_red
icon_red
icon_green
icon_red
icon_red
icon_red
icon_red
icon_orange
icon_green
 

FS#1087 — FS#5096 — Updating of the pCC

Attached to Project— Network
Maintenance
Whole Network
CLOSED
100%
We will update the software on the network equipment
of the pCC.

This will allow us to test the good (or the bad)
functioning of the network. We need to have still some
adjustments to add.

We will start with the core of the network.
Date:  Tuesday, 08 February 2011, 18:52PM
Reason for closing:  Done
Comment by OVH - Wednesday, 02 February 2011, 13:01PM

The A side is ongoing...


Comment by OVH - Wednesday, 02 February 2011, 13:01PM

Compatibility check is done:
Module bootable Impact Install-type Reason
------ -------- -------------- ------------ ------
1 yes non-disruptive rolling
2 yes non-disruptive rolling
5 yes non-disruptive reset
6 yes non-disruptive reset



Images will be upgraded according to following table:
Module Image Running-Version(pri:alt) New-Version Upg-Required
------ ---------- ---------------------------------------- -------------------- ------------
1 lc-d-n7k 5.1(1) 5.1(2) yes
1 lc-d-n7k 5.1(1) 5.1(2) yes
1 bios v1.10.13(03/15/10): v1.10.13(03/15/10) v1.10.14(04/02/10) yes
2 lc-d-n7k 5.1(1) 5.1(2) yes
2 lc-d-n7k 5.1(1) 5.1(2) yes
2 bios v1.10.13(03/15/10): v1.10.13(03/15/10) v1.10.14(04/02/10) yes
5 system 5.1(1) 5.1(2) yes
5 kickstart 5.1(1) 5.1(2) yes
5 bios v3.22.0(02/20/10): v3.22.0(02/20/10) v3.22.0(02/20/10) no
5 cmp 5.1(1) 5.1(1) no
5 cmp-bios 02.01.05 02.01.05 no
6 system 5.1(1) 5.1(2) yes
6 kickstart 5.1(1) 5.1(2) yes
6 bios v3.22.0(02/20/10): v3.22.0(02/20/10) v3.22.0(02/20/10) no
6 cmp 5.1(1) 5.1(1) no
6 cmp-bios 02.01.05 02.01.05 no


Comment by OVH - Wednesday, 02 February 2011, 13:01PM

Mod Sw Hw
--- -------------- ------
1 5.1(1) 1.0
2 5.1(1) 1.0
5 5.1(1) 1.8
6 5.1(2) 1.8

"Switching over onto standby".

64 bytes from pcc-1a-n7.routers.ovh.net (178.33.123.249): icmp_seq=50 ttl=250 time=4.28 ms
64 bytes from pcc-1a-n7.routers.ovh.net (178.33.123.249): icmp_seq=51 ttl=250 time=4.27 ms
64 bytes from pcc-1a-n7.routers.ovh.net (178.33.123.249): icmp_seq=52 ttl=250 time=4.25 ms

64 bytes from pcc-1a-n7.routers.ovh.net (178.33.123.249): icmp_seq=55 ttl=250 time=1008 ms
64 bytes from pcc-1a-n7.routers.ovh.net (178.33.123.249): icmp_seq=56 ttl=250 time=475 ms
64 bytes from pcc-1a-n7.routers.ovh.net (178.33.123.249): icmp_seq=57 ttl=250 time=4.38 ms

et pas de coupure dans le service

Mod Sw Hw
--- -------------- ------
1 5.1(2) 1.0
2 5.1(2) 1.0
5 5.1(2) 1.8
6 5.1(2) 1.8


Comment by OVH - Wednesday, 02 February 2011, 13:02PM

The A side of the core was finished.

We will restart the B side of the core.


Comment by OVH - Wednesday, 02 February 2011, 13:03PM

Done.

We will pitch in the racks:
A Side


Comment by OVH - Wednesday, 02 February 2011, 14:40PM

Compatibility check is done:
Module bootable Impact Install-type Reason
------ -------- -------------- ------------ ------
1 yes non-disruptive reset
100 yes non-disruptive rolling
101 yes non-disruptive rolling
102 yes non-disruptive rolling
103 yes non-disruptive rolling
104 yes non-disruptive rolling
105 yes non-disruptive rolling
106 yes non-disruptive rolling
107 yes non-disruptive rolling
108 yes non-disruptive rolling
109 yes non-disruptive rolling
110 yes non-disruptive rolling
111 yes non-disruptive rolling



Images will be upgraded according to following table:
Module Image Running-Version New-Version Upg-Required
------ ---------- ---------------------- ---------------------- ------------
1 system 5.0(2)N1(1) 5.0(2)N2(1) yes
1 kickstart 5.0(2)N1(1) 5.0(2)N2(1) yes
1 bios v1.3.0(09/08/09) v1.3.0(09/08/09) no
100 fexth 5.0(2)N1(1) 5.0(2)N2(1) yes
101 fexth 5.0(2)N1(1) 5.0(2)N2(1) yes
102 fexth 5.0(2)N1(1) 5.0(2)N2(1) yes
103 fexth 5.0(2)N1(1) 5.0(2)N2(1) yes
104 fexth 5.0(2)N1(1) 5.0(2)N2(1) yes
105 fexth 5.0(2)N1(1) 5.0(2)N2(1) yes
106 fexth 5.0(2)N1(1) 5.0(2)N2(1) yes
107 fexth 5.0(2)N1(1) 5.0(2)N2(1) yes
108 fexth 5.0(2)N1(1) 5.0(2)N2(1) yes
109 fexth 5.0(2)N1(1) 5.0(2)N2(1) yes
110 fexth 5.0(2)N1(1) 5.0(2)N2(1) yes
111 fexth 5.0(2)N1(1) 5.0(2)N2(1) yes
1 power-seq v1.2 v1.2 no


Comment by OVH - Wednesday, 02 February 2011, 14:41PM

2011 Feb 2 13:00:05 pcc-10b-n5 %VPC-2-VPC_ISSU_START: Peer vPC switch ISSU start, locking configuration
2011 Feb 2 13:02:49 pcc-10b-n5 %SATCTRL-FEX100-2-SATCTRL_IMAGE: FEX100 Image update in progress.
2011 Feb 2 13:02:59 pcc-10b-n5 %SATCTRL-FEX101-2-SATCTRL_IMAGE: FEX101 Image update in progress.
2011 Feb 2 13:03:09 pcc-10b-n5 %SATCTRL-FEX102-2-SATCTRL_IMAGE: FEX102 Image update in progress.
2011 Feb 2 13:03:18 pcc-10b-n5 %VPC-2-VPC_ISSU_END: Peer vPC switch ISSU end, unlocking configuration
Feb 2 13:04:51 pcc-10b-n5.routers.ovh.net : 2011 Feb 02 13:04:28 CET: %SATCTRL-FEX100-2-SATCTRL_IMAGE: FEX100 Image update failed [/isan/plugin_img/fexth.bin]: File transfer error
Feb 2 13:05:13 pcc-10b-n5.routers.ovh.net : 2011 Feb 02 13:04:50 CET: %SATCTRL-FEX102-2-SATCTRL_IMAGE: FEX102 Image update failed [/isan/plugin_img/fexth.bin]: File transfer error
Feb 2 13:05:21 pcc-10b-n5.routers.ovh.net : 2011 Feb 02 13:04:58 CET: %SATCTRL-FEX101-2-SATCTRL_IMAGE: FEX101 Image update failed [/isan/plugin_img/fexth.bin]: File transfer error


Comment by OVH - Wednesday, 02 February 2011, 16:39PM

We did not finish the boot of FEX. The update does not function.
We will update the B side.

2011 Feb 2 15:55:50 pcc-10a-n5 %VPC-2-VPC_ISSU_START: Peer vPC switch ISSU start, locking configuration
2011 Feb 2 15:59:25 pcc-10a-n5 %VPC-2-VPC_ISSU_END: Peer vPC switch ISSU end, unlocking configuration
2011 Feb 2 15:59:31 pcc-10a-n5 %VPC-2-PEER_KEEP_ALIVE_RECV_FAIL: In domain 156, VPC peer keep-alive receive has failed
2011 Feb 2 16:06:17 pcc-10a-n5 %SATCTRL-FEX109-2-SATCTRL: FEX-109 Module 1: Cold boot
2011 Feb 2 16:06:32 pcc-10a-n5 %PFMA-2-FEX_STATUS: Fex 109 is online
2011 Feb 2 16:06:32 pcc-10a-n5 %NOHMS-2-NOHMS_ENV_FEX_ONLINE: FEX-109 On-line
2011 Feb 2 16:06:36 pcc-10a-n5 %PFMA-2-FEX_STATUS: Fex 109 is online
2011 Feb 2 16:05:45 pcc-10a-n5 %SATCTRL-FEX106-2-SATCTRL: FEX-106 Module 1: Cold boot
2011 Feb 2 16:07:01 pcc-10a-n5 %PFMA-2-FEX_STATUS: Fex 106 is online
2011 Feb 2 16:07:01 pcc-10a-n5 %NOHMS-2-NOHMS_ENV_FEX_ONLINE: FEX-106 On-line
2011 Feb 2 16:07:03 pcc-10a-n5 %PFMA-2-FEX_STATUS: Fex 106 is online
[...]


Ports interrupted.

The servers have switched to the second network card with an interruption of 60 seconds.


Comment by OVH - Wednesday, 02 February 2011, 16:44PM

UP.

Not terrible.

Basically, the Nexus 5000 with the FEX are not ISSU in dual attach.
The N5 which is updated with the latest version can not update
the FEX with this version because the other N5 has the precedent
version.

We'll try an update with a single attach. For this we
will cut one of two sides and do the update of the remaining running
side. We'll see if it is ISSU.


Comment by OVH - Wednesday, 02 February 2011, 18:46PM

Shit.

The 2nd pair N5 was updated with lots of breakdowns. The N5 with FEX does not work now.

Well, we'll do it differently with the 3rd pair. we cut all ports on the 2 N5. Then we will update them.


Comment by OVH - Thursday, 03 February 2011, 11:31AM

We have the method:
as each host is connected to two networks A and B. and each network is
in a double-attach. We interrupt port A from the server and we update
the double attach A. First all, the 1st N5 A then when it is up, the 2nd
N5 A. once the first double-N5 is updated, the port is reactive.
and we can do the same operation on the port B. Thus, there is no
interruption in service pCC.


Comment by OVH - Monday, 07 February 2011, 15:50PM

We are going to test another method on the remaining double-N5.