Partial Email Service Outage
Incident Report for Enom
Postmortem

Summary: On May 2, 2024, one of the components within our Hosted Email infrastructure failed. We upgraded the software to fix the problem when the initial recovery effort failed. This situation coincided with a Denial of Service attack, which greatly increased traffic levels, leading to an unresponsive webmail. Mitigation efforts were then taken to block abusing IPs, which allowed the authentication service to recover. Webmail response times returned to expected levels that day, and to ensure the actions we took resolved the issue, we continued to monitor the event until May 6. Once confirmed, we closed this incident.

Client Impact: Webmail and login were slow or unresponsive. Email delivery was delayed due to a short-term IMAP/POP failure.

Remediation: We’ve taken the following actions to prevent a future recurrence:

  • Authorization service reengineering: We’re reducing wait times to expected levels by reengineering the software to scale to higher levels of traffic. We’ll also provide better tools to manage the authentication process. .
  • Proactively arranging maintenance windows: A set of quarterly change windows will be scheduled to address any software updates and preventative maintenance required on the hosted email infrastructure. We'll ensure you're aware of updates/maintenance windows and their impact in advance, and we'll schedule them to occur when our server traffic is typically at its lowest.
  • Increased abuse mitigation efforts: We’re exploring new options to reduce occurrences and the impact of abuse.

Thank you,

Tucows Domains Operations Team

Posted May 10, 2024 - 07:37 PDT

Resolved
We've confirmed that this issue has been resolved now email services have been restored.
Posted May 06, 2024 - 06:38 PDT
Update
We continue to monitor and work towards a full restoration of the Webmail service, which is experiencing interruptions. We appreciate your patience.

Further updates to come as they are received.
Posted May 03, 2024 - 15:23 PDT
Update
Our team is still monitoring webmail as delays persist, all activity is expected to return to normal soon. Additional updates will be provided as they become available.

We appreciate your patience.
Posted May 03, 2024 - 07:33 PDT
Update
Our team continues to work on the webmail responsiveness.

We will provide further updates as they become available.
Posted May 02, 2024 - 20:31 PDT
Update
Our team continues to monitor webmail responsiveness.

We will provide further updates as they become available.
Posted May 02, 2024 - 13:49 PDT
Monitoring
As Pop and IMAP (mail client) connection is restored, with webmail access improving, we are moving this to monitoring.

We will provide an update on this within one hour.
Posted May 02, 2024 - 12:27 PDT
Update
We’ve implemented a fix and are monitoring the improvements this is having to Pop/IMAP connectivity and responsiveness.
Webmail access is currently slow but responsive.

We will provide an update on this within one hour.
Posted May 02, 2024 - 12:18 PDT
Update
We are working on a fix for this issue, and appreciate your patience.
We do see improvements for those affected accounts, and will continue until it is fully restored.

The next update will be within one hour.
Posted May 02, 2024 - 11:13 PDT
Update
Our team is diligently addressing the performance issues that some users may be encountering with our webmail service. We sincerely appreciate your ongoing patience.
Posted May 02, 2024 - 10:24 PDT
Update
Our team is continuing to work on slowness that some users are experiencing with webmail. We thank you for your continued patience.
Posted May 02, 2024 - 08:53 PDT
Update
Our team is continuing to work on slowness that some users are experiencing with webmail. We thank you for your continued patience.
Posted May 02, 2024 - 06:19 PDT
Identified
Our engineers have identified the issue impacting the email service and have begun implementing a fix.
Posted May 02, 2024 - 03:33 PDT
Investigating
We are currently experiencing a known issue impacting our email service.

At this time you may not be able to log in via Webmail, IMAP, or POP3.

Our engineers are aware of the situation and working to resolve it.

We will post updates as soon as more information is available.

Thank you for your patience.
Posted May 02, 2024 - 03:05 PDT
This incident affected: Basic Email Service.