The data center team has said that they have completed the maintenance. We are still monitoring the situation, but all services are back to normal. Any messages scheduled to be sent during the connectivity issues caused by the maintenance have been fully processed and sent.
Posted Apr 08, 2015 - 05:20 EDT
Update
CPU is stabilizing on the routers, things are returning to normal.
Posted Apr 08, 2015 - 05:08 EDT
Update
We appear to be in the most disruptive part of the maintenance, as the network aggregation routers recalculate the network topology. Message sending is currently delayed, but will resume as connectivity is restored.
Posted Apr 08, 2015 - 04:57 EDT
Update
Maintenance is still ongoing, with occasional connectivity issues caused by the router upgrades.
Posted Apr 08, 2015 - 04:32 EDT
In progress
The maintenance is in progress at the data center. So far we have seen a few unexpected connectivity blips caused by the maintenance, but we are in contact with the data center team should there be major issues.
Posted Apr 08, 2015 - 00:47 EDT
Scheduled
On Wednesday, April 8th between midnight and 6am ET, our data center will be performing emergency maintenance on their network switches.
Last week's network incident was caused by critical bugs identified by Cisco on their NX-OS software release on the aggregation switches in our data center, which caused high CPU utilization and suboptimal spanning-tree calculations. Our data center will be upgrading the switches and configuring VPCs on multiple switches in our data center.
This maintenance should not impact customers. However, Appboy will be online and working in conjunction with the networking team at Rackspace during the maintenance, should any issues arise.