rssLink RSS for all categories
 
icon_red
icon_green
icon_red
icon_red
icon_blue
icon_green
icon_green
icon_red
icon_red
icon_red
icon_orange
icon_green
icon_green
icon_green
icon_green
icon_blue
icon_green
icon_orange
icon_red
icon_green
icon_red
icon_red
icon_green
icon_red
icon_red
icon_red
icon_red
icon_orange
icon_green
 

FS#14019 — FS#20238 — CHI <> PAL

Attached to Project— Network
Incident
Backbone US
In progress
0%
We lost 10x10G between Chicago and Palto Alto. Our provider confirm a fiber cut.

Comment by OVH - Thursday, 08 September 2016, 00:12AM

09/07/2016 21:32:52 GMT - The Transport NOC suspects that a fiber issue between Oasis, NV and Elko, NV is impacting unprotected services. Field Services is engaged to dispatch to the site, with an estimated time of arrival of 01:00 GMT on September 8, 2016 due to the remote location of the fault. Once on site, Field Services will perform fiber testing to isolate the fault. The next update will be provided upon arrival, or if new information becomes available. The information conveyed hereafter is associated to live troubleshooting effort and as the discovery process evolves through to service resolution, ticket closure, or post incident review, details may evolve.

Sep 7, 2016 10:02 PM GMT

09/07/2016 21:56:36 GMT - It has been confirmed that the damage was caused by construction and an approximate location has been identified. Field Services remains en route for further isolation testing, with an estimated time of arrival of 01:00 GMT on September 8, 2016. Repair crews are also engaged to commence repairs upon isolation of the fault. The next update will be provided upon arrival, or if new information becomes available.


Comment by OVH - Thursday, 08 September 2016, 03:39AM

*** CASCADED EXTERNAL NOTES 08-Sep-2016 01:26:50 GMT From CASE: 11432200 - Event
Field Services remains en route the testing site in Oasis, but an updated estimated time of arrival is unavailable.


Comment by OVH - Thursday, 08 September 2016, 03:58AM

We shutdown the EQUINIX peering to save some bandwidth on west coast.


Comment by OVH - Thursday, 08 September 2016, 05:05AM

09/08/2016 1:52 AM GMT


*** CASCADED EXTERNAL NOTES 08-Sep-2016 01:50:37 GMT From CASE: 11432200 - Event
A partial cut has been confirmed, but the services that are not completely cut are severely degraded. As such, it is necessary to fully cut the cable and repair the damage to restore services to their optimum level. All crews are expected to arrive on site to commence repairs by 04:30 GMT. The next update will be provided at that time, or if new information becomes available.


Comment by OVH - Thursday, 08 September 2016, 13:24PM

*** CASCADED EXTERNAL NOTES 08-Sep-2016 10:48:48 GMT From CASE: 11432200 - Event
Field Services have completed splicing, and alarms have cleared. The Technical Service Center is currently confirming restoral with customers. If further issues are experienced, please contact the Technical Service Center at 1-877-4-LEVEL3.


Comment by OVH - Thursday, 08 September 2016, 13:38PM

We reactivating the traffic on EQUINIX and local-pref on ANY2


Comment by OVH - Thursday, 08 September 2016, 14:53PM

All is back to normal.