News: NETWORK ISSUE in MANCHESTER (Lumen) **SOLVED**

Published: 09/20/2023

Hello,

we are aware of the network issue in Manchester,UK.

Notes History:

September 22, 2023, 09:45 GMT - Fix Action: The fiber was repaired, thus restoring services to a stable state.

2023-09-22 09:53:44 GMT - Splicing efforts are ongoing. Some of the alarms are starting to clear as splicing progresses.

2023-09-22 08:08:15 GMT - Splicers report that splicing is complete on 2 of 6 tubes. The next update will be provided by 10:10 GMT, or as new information is received.

2023-09-22 05:17:54 GMT - Splicing is ongoing. Due to the extensive work involved the next update will be provided by 08:20 GMT, or as new information is received.

2023-09-22 03:40:19 GMT - Splicers have begun cutting the cables to splice in the replacement fiber.

2023-09-22 02:42:43 GMT - Splicers have been granted access to the track and are preparing the cable for cutting and splicing.

2023-09-22 02:12:02 GMT - Splicers are awaiting access to the tracks to commence repairs.

2023-09-22 00:15:09 GMT - Cable has been installed, and splicers are en route.

2023-09-21 23:46:57 GMT - The line has been blocked and crews have resumed installation of the replacement cable.

2023-09-21 23:27:09 GMT - Field Operations are awaiting further assistance from railroad signalers and safety personnel to block the lines and allow for the resumption of fiber installation.

2023-09-21 21:37:16 GMT - Crews on site are pulling replacement cable in 20-30 minute stints as the railroad is able to block the line and allow access.

2023-09-21 17:47:44 GMT - Field personnel have been granted access to begin repairs at 19:00 GMT. Once on-site technicians will begin installing the replacement cable and is expected to be completed by approximately 22:30 GMT. Once the replacement cable has been installed splicing can commence. An Estimated Time of Restoral (ETR) of approximately 04:00 GMT on September 22, 2023 has been advised.

2023-09-21 15:36:45 GMT - The Outside Plant Engineering Team is working with the Railroad Authority to obtain site access and permits to begin repair efforts. Please be advised updates will be provided every four (4) hours or as new information is obtained.

2023-09-21 14:58:41 GMT - The NOC continues in efforts of repair details from the Outside Plant Team.

2023-09-21 14:09:39 GMT - The Lumen NOC is contacting the Outside Plant Team for a status of repair teams and approvals to begin repair efforts.

21 Sep 2023 11:18 a.m. EEST


*** CASCADED EXTERNAL NOTES 2023-09-21 08:18:30 GMT From CASE: 27658193 - Event
The location of the fault is making it difficult to formulate a plan of action because repair crews will need to be granted access by the railroad again. The next update will be provided by 14:20 GMT, or as new information is received.
21 Sep 2023 10:17 a.m. EEST


*** CASCADED EXTERNAL NOTES 2023-09-21 07:17:07 GMT From CASE: 27658193 - Event
After several hours of testing each individual strand of fiber it has been determined there are no viable spare fibers to use for repairs. Crews are now working to determine a plan of action.
21 Sep 2023 8:24 a.m. EEST


*** CASCADED EXTERNAL NOTES 2023-09-21 05:24:20 GMT From CASE: 27658193 - Event
Testing spare fibers and splicing efforts remain ongoing. The repair crew reports that half of the fibers have been tested up to this point. The next update will be provided by 07:25 GMT, or as new information is received.
21 Sep 2023 6:25 a.m. EEST


*** CASCADED EXTERNAL NOTES 2023-09-21 03:25:04 GMT From CASE: 27658193 - Event
Splicing is ongoing at the railroad splice point.
21 Sep 2023 5:20 a.m. EEST


*** CASCADED EXTERNAL NOTES 2023-09-21 02:20:24 GMT From CASE: 27658193 - Event
Crews are testing spare fibers from the splice joint, and resplicing the fibers according to test results.
21 Sep 2023 4:32 a.m. EEST


*** CASCADED EXTERNAL NOTES 2023-09-21 01:32:08 GMT From CASE: 27658193 - Event
Crews are being granted to the railroad at this time to resume testing.
21 Sep 2023 4:05 a.m. EEST


*** CASCADED EXTERNAL NOTES 2023-09-21 01:04:51 GMT From CASE: 27658193 - Event
Crews are pending safe access from the railroad signaler.
21 Sep 2023 3:11 a.m. EEST


*** CASCADED EXTERNAL NOTES 2023-09-21 00:11:31 GMT From CASE: 27658193 - Event
Crews are on site, awaiting access to the railroad.
21 Sep 2023 3:00 a.m. EEST


*** CASCADED EXTERNAL NOTES 2023-09-20 23:59:55 GMT From CASE: 27658193 - Event
The NOC is working to confirm the arrival of the replacement team for testing.
20 Sep 2023 9:07 p.m. EEST


*** CASCADED EXTERNAL NOTES 2023-09-20 18:07:28 GMT From CASE: 27658193 - Event
The NOC has advised repair teams have left the site for mandatory rest and pending access to the railroad from the railroad company to continue isolation efforts. Lumen Repair teams will return to the site on September 21, 2023, at 00:01 GMT, when access will be granted.
20 Sep 2023 8:28 p.m. EEST


*** CASCADED EXTERNAL NOTES 2023-09-20 17:27:57 GMT From CASE: 27658193 - Event
Repair efforts continue to restore services as soon as possible.
20 Sep 2023 7:35 p.m. EEST


*** CASCADED EXTERNAL NOTES 2023-09-20 16:35:00 GMT From CASE: 27658193 - Event
Teams continue repair efforts.
20 Sep 2023 5:48 p.m. EEST


*** CASCADED EXTERNAL NOTES 2023-09-20 14:48:06 GMT From CASE: 27658193 - Event
Teams have returned to the site and are isolating the exact point of failure. The next update will be provided in 2 hours.
20 Sep 2023 1:11 p.m. EEST


*** CASCADED EXTERNAL NOTES 2023-09-20 10:11:44 GMT From CASE: 27658193 - Event
Teams were able to obtain access to the site for fiber testing. Due to weather conditions in the area causing moisture on the equipment and rail traffic approaching peak hours, restoral efforts are currently on hold. Please be advised, the next update will be provided at 18:00 GMT, or as new information becomes available.
20 Sep 2023 10:12 a.m. EEST


*** CASCADED EXTERNAL NOTES 2023-09-20 07:12:44 GMT From CASE: 27658193 - Event
Efforts to secure the area and coordinate access with the railway remain in progress. Please be advised, the next update will be provided once access is granted, or as new information becomes available.
20 Sep 2023 9:16 a.m. EEST


*** CASCADED EXTERNAL NOTES 2023-09-20 06:16:11 GMT From CASE: 27658193 - Event
The original access point has been deemed inaccessible. As such, splice teams are in the process of relocating farther down the track to an accessible entry point.
20 Sep 2023 8:20 a.m. EEST


*** CASCADED EXTERNAL NOTES 2023-09-20 05:20:38 GMT From CASE: 27658193 - Event
Splice crews and the Outside Plant Team remain on site awaiting authorization from the railway. Once access is granted, teams will assess the damage and formulate a plan of repair.
20 Sep 2023 7:25 a.m. EEST


*** CASCADED EXTERNAL NOTES 2023-09-20 04:25:26 GMT From CASE: 27658193 - Event
Splice teams are on site and awaiting access from the railway.
20 Sep 2023 5:26 a.m. EEST


*** CASCADED EXTERNAL NOTES 2023-09-20 02:26:36 GMT From CASE: 27658193 - Event
Field Operations has completed light level measurements and observed a fault approximately 59 kilometers from the location. The splice team remains en route with the provided estimated time of arrival (ETA) of 04:00 GMT. The next update will be provided by 04:30 GMT, or as new information becomes available.
20 Sep 2023 4:45 a.m. EEST


*** CASCADED EXTERNAL NOTES 2023-09-20 01:44:54 GMT From CASE: 27658193 - Event
Field Operations has arrived onsite and is performing light level measurements to determine the presence of a fiber fault and the approximate location.
20 Sep 2023 3:27 a.m. EEST


*** CASCADED EXTERNAL NOTES 2023-09-20 00:27:41 GMT From CASE: 27658193 - Event
Field Operations remains en route to a facility and is expected to be arriving shortly. Once onsite, they will perform light level measurements to determine the location of the failure. The Lumen NOC has engaged a splice team for dispatch to assist with the repair efforts, which is expected to arrive by 04:00 GMT.
20 Sep 2023 1:26 a.m. EEST


*** CASCADED EXTERNAL NOTES 2023-09-19 22:26:39 GMT From CASE: 27658193 - Event
The Lumen Network Operations Center (NOC) was engaged to investigate the failure. Initial investigations suggested a fiber fault to be the source of the service interruption. As such, the Lumen NOC engaged Field Operations for dispatch to a facility to assist with isolation efforts. Field Operations is en route and expected to arrive by 23:45 GMT. The next update will be provided by 00:30 GMT, on September 20, 2023 or as new information becomes available.