rssLink RSS for all categories
 
icon_red
icon_green
icon_red
icon_red
icon_blue
icon_green
icon_green
icon_red
icon_red
icon_red
icon_orange
icon_green
icon_green
icon_green
icon_green
icon_blue
icon_green
icon_orange
icon_red
icon_green
icon_red
icon_red
icon_green
icon_red
icon_red
icon_red
icon_red
icon_orange
icon_green
 

FS#2363 — FS#6329 — Upgrade Leclerc

Attached to Project— Dedicated Cloud
Modernization
Backend / Core
CLOSED
100%
We are going to upgrade all the datastores of the pcc customers in order to use the hybrid technology.

Sdd disks will be added in each one of the leclerc where the writing cache will be stored.

The data will still be stocked on the classical disks, dedicated for each customer.

This will allow to increase sensibly the performances (and the latencies) in writing.

This upgrade is expected with no impact for the final customer and without no reconfiguration
from his part.

First of all, we are going to reorganise the used disks on the leclerc in order to insert those
new ssd disks.


After that, we are going to restart the master head and the slave head of each leclerc in order
to update the configuration of the disk controls This configuration change is necessary to
ensure the consistency of the data during a future switch between the 2 heads.


Then, we are going to configure the ssd disks to use them in cache of the existing datastores.
Date:  Thursday, 29 March 2012, 18:19PM
Reason for closing:  Done
Comment by OVH - Sunday, 05 February 2012, 18:30PM

The reorganisation of the used disks is done. We freed the necessary spots to allow the insertion
of the new ssd disks.
The first step of the intervention is done.


We are going to spread the seconde step (configuration change) over two nights this week.
It's the most sensitive operation. We planned this procedure on the papr but the task force
will be also there in order to react in case of problem.


Comment by OVH - Tuesday, 07 February 2012, 09:45AM

We start the second step this night.
Beginning of the operations predicted in the midnight.


Comment by OVH - Tuesday, 07 February 2012, 09:53AM

1/3 of heads have been restarted with the new configuration.
The manual switch procedure is perfectible to avoid some problems at the level of umount/mount/share between ZFS and NFS.

We will follow upgrades the next night.


Comment by OVH - Wednesday, 08 February 2012, 09:28AM

We will begin upgrades in 20 minutes.
We improved the procedure of switch to have a faster recovery by service.
We have also included probes to get the state of the vm and the datastore before during and
after intervention.




Comment by OVH - Wednesday, 08 February 2012, 09:30AM

The change of the first leclerc was performed properly.
The probes of the vm and datastores are green.
We continue the change of the following Leclerc.





Comment by OVH - Wednesday, 08 February 2012, 09:31AM

80% of planned changes were performed.
We continue.


Comment by OVH - Wednesday, 08 February 2012, 09:36AM

During the change of head-42 slave to head-42 master, the master crashed during the import.
3 datastores were above: pcc-001095, pcc-001096 and pcc-001097.
We created tickets with concerned customers and we restarted the impacted vms.

We will fix the hardware of head-42 Masters before returning it to service.
We leave the concerned datastore in service on the slave.


Comment by OVH - Wednesday, 08 February 2012, 09:41AM

We completed the upgrade of leclerc heads configuration.
2 heads are left on the Slaves (head-42 and head-150).
We ill Check the hardware of masters before returning them to service.
The most sensible part of upgrades is completed.

Next step: inserting the ssd that we will perform tomorrow during the day.


Comment by OVH - Thursday, 23 February 2012, 10:32AM

We start the addition of ssd cache in storages.
Important: At each cache addition, we will send you an email to inform the client of the concerned change.


Comment by OVH - Thursday, 23 February 2012, 10:33AM

Today we have upgraded about 1/3 of pcc customers storages.
We continue from tomorrow.


Comment by OVH - Friday, 24 February 2012, 11:05AM

Last Thursday, we continued upgrades, and there are about 50% of storages that are now migrated.


Comment by OVH - Tuesday, 28 February 2012, 06:26AM

After a day of upgrades, we are at about 70% process.
Simultaneously we notified by e-mail the few customers who can not be migrated on the spot and to whom storage should be replaced by a new one.


Comment by OVH - Thursday, 08 March 2012, 21:54PM

The upgrades are going on smoothly.

Right now , about 75% of the whole storages of the infrastructure were switched.

The storages that we could not warm switch were replaced by hybrid storages.


Comment by OVH - Friday, 23 March 2012, 13:45PM

About 80 % of the infrastructure is impacted.



Comment by OVH - Thursday, 29 March 2012, 15:27PM

Up until now, around 95% of storage has been switched.
We are dealing with the few left.