Networking Incident
Incident Report for Digital Pacific
Postmortem

Root Cause:
The incident was traced back to the failure of an edge router interface card, which resulted in a brief interruption of public traffic. Thankfully, our automated systems efficiently detected the issue and seamlessly switched over to our backup router, thereby minimising the impact on your public-facing services.

However, during this process, a configuration issue arose that affected the routing for our private network. As a result, some of our customers experienced temporary difficulties accessing private network resources.

Resolution and Mitigation:
Our dedicated team of experts promptly identified the root cause and diligently worked to rectify the configuration issue. We have taken the necessary steps to ensure that similar incidents will not occur in the future. Rest assured, we have enhanced our monitoring systems and implemented additional checks to safeguard against similar configuration-related incidents.

Next Steps:
We deeply apologise for any inconvenience this outage may have caused you. We understand the critical nature of our services to your operations and are committed to maintaining the highest standards of reliability and performance.

In the spirit of transparency, we will continue to review our systems and processes, striving for continuous improvement to prevent future incidents. Your feedback and insights are invaluable to us, and we encourage you to share any observations or suggestions to further enhance our services.

Our support team is available around the clock to address any questions or concerns you may have. Please don't hesitate to reach out to us.

Thank you for your understanding and continued trust in our services.

Posted Aug 03, 2023 - 21:15 AEST

Resolved
This incident has been resolved.
Posted Aug 03, 2023 - 21:15 AEST
Update
The private network connectivity between Sydney and Melbourne hosts has been successfully restored. However, we are continuing to work on resolving a small segment of the private network issue that persists between the Sydney pods.

Rest assured, our team is actively working on resolving this remaining issue.
Posted Aug 03, 2023 - 19:06 AEST
Identified
We have identified and are actively working on an incident which is impacting network connectivity in our data centre.
The issue is being worked on as a top priority.
Further information will be provided here as updates become available.
Posted Aug 03, 2023 - 14:58 AEST
This incident affected: Network.