/
/
Upstream Provider Network Disruption Dual Path Failure Impacting IIG & NIX

Upstream Provider Network Disruption Dual Path Failure Impacting IIG & NIX

October 14, 12:08 am - 
October 14, 12:23 am

Affected Area

IIG and NIX network.

Incident Type

Network Outage

Root Cause

On October 14, 2024, internet services via IIG and NIX experienced an outage caused by the upstream provider issue. Upon investigation, the upstream provider reported a double fault affecting both primary and backup network paths:

  • Main Path: The upstream provider conducted planned maintenance under CR00025689 between JASMINE and THECLOUD from 00:06 to 00:06 (UTC+7) on October 14, 2024, rendering the main path unavailable.
  • Protection Path: During the maintenance, a high-loss issue occurred on the protection path between JASMINE and THECLOUD from 00:06 to 00:20 (UTC+7), resulting in a simultaneous failure of both paths.

Resolution

The upstream provider adjusted the optical power level on protection path for circuit restoration.

Prevention

  1. Since the issue arose from the DWDM link between CSLoxinfo The Cloud and Jasmine Tower, we plan to replace the DWDM link with a Dark Fiber connection featuring fault-tolerant architecture to enhance reliability.
  2. We are proceeding with our plan to connect additional NIX and IIG providers at Telehouse datacenter and configure network redundancy between two datacenters