We have been extensively monitoring the system and have found the following details.
At around 3:00 PM UTC on Mondy 23rd Arctitan started receiving massively elevated connections via SMTP. Likewise, the flow of data into the system via SMTP has increased, this situation has been ongoing until around 3:30 AM UTC Thursday 26th.
The traffic levels have for the moment returned to normal historical values.The increased number of connection requests has the impact of consuming more memory on the SMTP servers and also generating a high queue for the requests to be serviced.
During this event, we have scaled up the number of SMTP servers which has mitigated the problem somewhat but not completely. The very high request rate results in some senders having their connections dropped when trying to deliver emails to the archive.
Now that connection rates have returned to normal, the James servers are at full performance and have consumed all emails waiting at the senders' systems. The SMTP servers processing emails very quickly once connections could get through.
At 22:00 UTC Wednesday 25rd we enabled detailed logging of inbound connection requests to the Amazon load balancer.
We have been able to analyse the IP addresses during the logged time and identified SIUMED to be the top IP address. We are leaving the Amazon Request logging system in place. If the elevated traffic were to resume, we will be able to identify the senders' IP addresses.
We will continue to monitor the system over the coming days, however, for the moment the incident appears to be over.
