AWS late yesterday was hit by a sustained DDoS attack, which appears to have lasted some 8hours. The incident hit its Route 53 DNS web offering, knocking down other services, and raises many questions about the nature of the attack and about AWS’s own DDoS mitigation service, “Shield Advanced”.
Google Cloud Platform (GCP) had a range of issues at a similar time. The two are not understood to be linked. In a status update GCP cited interruptions to “multiple Cloud products including Google Compute Engine, Cloud Memory store, Google Kubernetes Engine, Cloud Bigtable and Google Cloud Storage” at a similar time. A Google spokesperson told us: “Our service disruptions were unrelated to any kind of DDoS attempt.”
The attack on AWS left many customers
struggling to access AWS’s S3 services, with many AWS services relying on external DNS queries, including its Relational Database Service (RDS), and Elastic Load Balancing (ELB). The US East Coast appears to have been particularly severely hit. (AWS described the impact of the attack as only affecting a “small number of specific DNS names”).
AWS users on Reddit said they had found
Aurora (a MySQL and PostgreSQL-compatible database) clusters also unreachable, with many complaining that their customers had been left
unable to use cloud services for several hours.
An AWS status update reads: “Between 10:30 AM and 6:30 PM PDT, we experienced intermittent errors with resolution of some AWS DNS names. Beginning at 5:16 PM, a very small number of specific DNS names experienced a higher error rate. These issues have been resolved.”
An email to customers pointed the finger at a Distributed Denial of Service (DDoS) attack. As widely shared on Reddit, Twitter, and reported by the Register, the email notes: “We are investigating reports of occasional DNS resolution errors. The AWS DNS servers arebcurrently under a DDoS attack.
“Our DDoS mitigations are absorbing the vast majority of this traffic, but these mitigations are also flagging some legitimate customer queries at this time.
It added: “We are actively working on
additional mitigations, as well as tracking down the source of the attack to shut it down.”
Amazon’s own Shield Advanced DDoS mitigation offering dealt with much of the attack, but the mitigations were also flagging some legitimate customer queries as malicious, meaning they were unable to connect. Given the sheer size of AWS and the traffic it handles at any given time, the attack must have been significant. It is not clear if a more
detailed autopsy will be forthcoming. (Critics noted that AWS’s Route 53 Service Level Agreement (SLA) promises 100 percent uptime…
[tweet https://mobile.twitter.com/0xdabbad00/status/1186740011758014464 width=100]
AWS had not commented further nor answered specific questions from Computer Business Review about the attack as we published. Customers were able to resolve the issue by updating the configuration of their clients
accessing S3 to specify the specific region that their bucket is in when making requests to
mitigate impact: e.g. specifying “mybucket.s3.us-west-2.amazonaws.com
rather than “mybucket.s3.amazonaws.com”.
