Ferrum IT - Outbound and inbound phone calls – Incident details

Following investigation, the outage was traced to a failure of the virtual machine (VM) hosting our internal phone systems within our third-party hosting partner’s infrastructure.

Details:

• The VM experienced a critical fault that required a full restoration from a recent snapshot image.

• After the snapshot restoration was completed, our Network Security Team conducted a thorough review and reapplied verified firewall rules to ensure secure communication between the phone system and related services.

• This firewall verification and rule reapplication process added additional time to the recovery effort to ensure that all systems were properly secured before bringing services back online.

• Once validation was completed, all phone and voice communication services resumed normal operation.

Preventive Actions:

• Implementing additional monitoring on hosting partner virtual machines for early fault detection.

• Reviewing and automating firewall rule revalidation post-restoration to minimize recovery time.

• Evaluating redundancy improvements to reduce dependency on a single virtual machine instance.

Current Status:
✅ All phone and voice communication systems are operational and stable.

We have successfully applied the fix, and all phone and voice communication systems are now fully operational.

• Normal inbound and outbound calling functionality has been restored.

• Support lines are no longer being routed through AI processing platforms.

• Teams have resumed standard communication methods via phone, SMS, chat, email, and portal.

We’ll continue to monitor performance to ensure continued stability.

Thank you for your patience and understanding during this incident.

Our team has identified the root cause of the phone system outage and has begun the remediation process.

During this time:

• All support lines have been forwarded to our AI processing platforms, which continue to log cases automatically into our ticketing system.

• Our support teams are proactively reaching out via SMS, chat, email, and through the customer portal to ensure uninterrupted communication.

• We will provide another update once remediation progress reaches completion or if there are significant changes in impact.

Thank you for your continued patience as we work toward full restoration.