Himeo
Vyemm Raider
- 3,263
- 2,802
They've been targeted by the largest DDOS attacks of all time. They have multiple ISP's and the attacks have been so big riot never sees the traffic because it crashes the ISP servers. This isn't an issue on riot, it's a group of hackers being cunts.
Recent DDoS and Malicious Attacks
Riot Gradius, an Associate Information Security Engineer, hopped into a forum thread to shed some light on the recent DDoS attacks that have been causing disruptions of service lately:
Hello, just wanted to see if I could drum up some conversations about the nature of DDOS attacks and what kind of scale we're seeing. The attacks that have recently taken place on many different Internet services are quite large because of a very specific flaw in the NTP protocol. These attacks are called reflection attacks. (http://en.wikipedia.org/wiki/Denial-...Spoofed_attack).
What's difficult to deal with when it comes to DDOS attacks is that the larger the attacks become, the easier it is for the attacker to completely consume all bandwidth available for a specific provider. What this means is that any Internet service that this provider has under them is also unable to communicate back to the Internet. As the attacks become larger, we block them farther and farther up the provider levels, until we're working with Tier 1 ISPs to implement access control lists that prevent these attacks. Here's some more information from Cloudflare about the recent NTP Reflection DDOSes (http://blog.cloudflare.com/understan...d-ddos-attacks)
Us networking guys at Riot are not only working internally to find a solution to this problem, but we're also working with the Information Security industry as a whole to improve the situation. We're working to find a tech solution to block the attacks as they're ongoing, and fixing the underlying problem of open/unpatched NTP servers on the Internet.
Please feel free to toss any questions my way that you might have, I would love to have a chat with everyone about this and answer as much as I can.
He continued:
"Yup, I've wanted to find the right post to start a conversation in... but so many of them are "RIOT FIX YOUR SERVERS" .. and I really wanted to have a dialog, rather than a storm of that sort of message.
I don't mind chatting with you all (I'd actually prefer it over staying quiet), and I'd really like to address any questions that I can. I'll be honest, it's a ****ty situation to be in, and myself along with a lot of other Rioters, as well as other industry professionals are trying to solve this problem as a whole rather than slowly crawling our ACLS up our provider lists. It really does make me sad that any of your games get ruined because of a DDOS, which is even more motivation to kick our butts into high gear and get this problem solved."
Riot Gradius also commented that Rioter isn't always the one being targeted, it's usually their providers:
"The problem with that is a lot of the time it's not even our network that's being destroyed by the malicious traffic, it's usually our providers or even their providers. This is why as a whole, network operations is moving towards a solution of being closer to providing their own Internet connections to our players. I'll find the post on the EUW forums that references things like the AMS datacenter and Riot Direct and edit it in here."
He continued:
"The biggest problem with blocking NTP at the provider level is that we're still working up the chain. The amount of traffic coming through is so huge, that it's taking down multiple provider links. This is why part of our attack strategy is building up relationships with multiple providers and others in the industry. This isn't something we're going to solve overnight, and definitely not something we'll solve by ourselves. Working with many parties can cause inherent lag in the process, but we're kicking ass and taking names in terms of making friends all over the industry."
When asked about the role loss prevented plays in these situations and an ETA on a fix, he commented:
"Loss prevented is intended to be an automatic thing when our service level goes below that of what we'd expect. You should see those changes apply fairly quickly (though, I'm not an expert in that area, I mostly just see the emails fly by with "Loss prevented is being enabled")
As for a timeline, that's hard to say since we're looking at a problem that's a huge scale. Just know that we're working tirelessly to get this under control. We want the network to be stable as bad as everyone else does"
When asked "why can't Riot just throw money at it", Riot Gradius explained:
"It's not that this a new issue, it's an old issue with new scale. Technically, we could throw money at it and buy more bandwidth, but that's only a temporary solution, and doesn't buy anyone much in terms of stability. As your bandwidth grows, so does the attacker's. This is why we have to find more solutions than just throwing money at it. This is why we have to work with other companies and groups in the Information Security industry, because of the magnitude of these recent attacks. We want this to be more complex than a bandwidth arms race."
In response to a summoner concerned about account security due to all this, Riot Triggs offered some general advice but noted DDoS attacks aren't an attempt to access personal info:
"You should always use a secure, long, hard to guess password. Using this site:https://howsecureismypassword.net/(probablywant to get past 1 month.) Also, use two factor auth anywhere you can. However, this has nothing to do with them access your personal information. DDoS attacks basically jam too many packets into a pipe. Like trying to put too many cars on a freeway."
Riot Gradius, an Associate Information Security Engineer, hopped into a forum thread to shed some light on the recent DDoS attacks that have been causing disruptions of service lately:
Hello, just wanted to see if I could drum up some conversations about the nature of DDOS attacks and what kind of scale we're seeing. The attacks that have recently taken place on many different Internet services are quite large because of a very specific flaw in the NTP protocol. These attacks are called reflection attacks. (http://en.wikipedia.org/wiki/Denial-...Spoofed_attack).
What's difficult to deal with when it comes to DDOS attacks is that the larger the attacks become, the easier it is for the attacker to completely consume all bandwidth available for a specific provider. What this means is that any Internet service that this provider has under them is also unable to communicate back to the Internet. As the attacks become larger, we block them farther and farther up the provider levels, until we're working with Tier 1 ISPs to implement access control lists that prevent these attacks. Here's some more information from Cloudflare about the recent NTP Reflection DDOSes (http://blog.cloudflare.com/understan...d-ddos-attacks)
Us networking guys at Riot are not only working internally to find a solution to this problem, but we're also working with the Information Security industry as a whole to improve the situation. We're working to find a tech solution to block the attacks as they're ongoing, and fixing the underlying problem of open/unpatched NTP servers on the Internet.
Please feel free to toss any questions my way that you might have, I would love to have a chat with everyone about this and answer as much as I can.
He continued:
"Yup, I've wanted to find the right post to start a conversation in... but so many of them are "RIOT FIX YOUR SERVERS" .. and I really wanted to have a dialog, rather than a storm of that sort of message.
I don't mind chatting with you all (I'd actually prefer it over staying quiet), and I'd really like to address any questions that I can. I'll be honest, it's a ****ty situation to be in, and myself along with a lot of other Rioters, as well as other industry professionals are trying to solve this problem as a whole rather than slowly crawling our ACLS up our provider lists. It really does make me sad that any of your games get ruined because of a DDOS, which is even more motivation to kick our butts into high gear and get this problem solved."
Riot Gradius also commented that Rioter isn't always the one being targeted, it's usually their providers:
"The problem with that is a lot of the time it's not even our network that's being destroyed by the malicious traffic, it's usually our providers or even their providers. This is why as a whole, network operations is moving towards a solution of being closer to providing their own Internet connections to our players. I'll find the post on the EUW forums that references things like the AMS datacenter and Riot Direct and edit it in here."
He continued:
"The biggest problem with blocking NTP at the provider level is that we're still working up the chain. The amount of traffic coming through is so huge, that it's taking down multiple provider links. This is why part of our attack strategy is building up relationships with multiple providers and others in the industry. This isn't something we're going to solve overnight, and definitely not something we'll solve by ourselves. Working with many parties can cause inherent lag in the process, but we're kicking ass and taking names in terms of making friends all over the industry."
When asked about the role loss prevented plays in these situations and an ETA on a fix, he commented:
"Loss prevented is intended to be an automatic thing when our service level goes below that of what we'd expect. You should see those changes apply fairly quickly (though, I'm not an expert in that area, I mostly just see the emails fly by with "Loss prevented is being enabled")
As for a timeline, that's hard to say since we're looking at a problem that's a huge scale. Just know that we're working tirelessly to get this under control. We want the network to be stable as bad as everyone else does"
When asked "why can't Riot just throw money at it", Riot Gradius explained:
"It's not that this a new issue, it's an old issue with new scale. Technically, we could throw money at it and buy more bandwidth, but that's only a temporary solution, and doesn't buy anyone much in terms of stability. As your bandwidth grows, so does the attacker's. This is why we have to find more solutions than just throwing money at it. This is why we have to work with other companies and groups in the Information Security industry, because of the magnitude of these recent attacks. We want this to be more complex than a bandwidth arms race."
In response to a summoner concerned about account security due to all this, Riot Triggs offered some general advice but noted DDoS attacks aren't an attempt to access personal info:
"You should always use a secure, long, hard to guess password. Using this site:https://howsecureismypassword.net/(probablywant to get past 1 month.) Also, use two factor auth anywhere you can. However, this has nothing to do with them access your personal information. DDoS attacks basically jam too many packets into a pipe. Like trying to put too many cars on a freeway."
