On Saturday, October 14th, Redtail performed the first phase of an upgrade process for our hosted email platform. The first phase was to update our LDAP, MTAs, and a small number of mail servers. This process being used was vetted through our development environment which includes a fully functional small scale replica of our email environment. The upgrade process was fully documented and tested prior to use in our production environment.
During the upgrade process, we ran into some unforeseen issues that caused the process to take longer than the scheduled maintenance window. We made the decision to end the upgrade and turn to triage to get the system back to a working state to accommodate our users’ business needs. Due to the vital nature of email to daily operations of our end users, every change to get to a full resolution had to be weighed and carefully drawn up to guarantee system and data integrity.
The root cause(s) have been identified and our hosted email environment is currently stable and healthy partially running on the new version. We are currently planning adjustments for the upgrade process and an updated timeline. Once finalized, we will move forward with the completion of the upgrade in the near future.
We at Redtail would like to extend to you our humble and sincere apology for any negative impact these issues had on you and your business. We understand how critical it is that we deliver maximum uptime to support your daily operations, and will increase our efforts to meet and exceed your availability expectations. Please rest assured that we will do everything we can to learn from this event and use it to strive for improvement across all of our services.