DDoS – BD | Application and Network Security Blog

Brute force attacks can be painful especially if your application has no logic to help mitigate the attack. Since December of 2015 I have seen several brute force attacks that have traversed multiple business verticals.

I know what you are thinking, “But Brian, we have a cloud based WAF so we are protected.” In every vertical listed above, they all had a cloud based WAF, specifically one of the following: Akamai Kona, Imperva Incapsula, and CloudFlare WAF. To the customers surprise, these services were not able to mitigate these attacks.

Customer feedback on Akamai Kona:

“They (Akamai Kona) are supposed to be our cloud WAF. Until recent events, we thought we were covered. Turns out, they can’t stop simple HTTP POST attacks!” CISO

Customer feedback on Imperva Incapsula:

“We were originally led to believe that Incapsula was their SecureSphere product in the cloud–it is actually nothing more than a modified version of ModSecurity.

Note: Incapsula is NOT a modified version of ModSecurity–the lack of features in Incapsula led them to believe it was.

When we asked why they didn’t detect the attack they told us that the 4,000 to 6,000 IPs that did the brute force attack (we had to provide the IPs too) weren’t classified in Incapsula as known bad actors. They do not have any mechanisms available to detect bot traffic based on behavior, only IP. The tricky part of this Bot Net was how it behaved, it wasn’t a ton of HTTP POSTS, the Bot behaved as a regular user and only performed 33 to 50 POSTs over a 48 hour period. ” -Security Correlation Specialist

Edit: I had a reference to Skyfence in the above paragraph, copy/paste mistake on my behalf. Credit Ofer Gayer.

Customer feedback on CloudFlare WAF:

“I think CloudFlare’s WAF most innovative feature is holding customers hostage with their crippling low self esteem. Seriously, they are like the prettiest Denny’s waitress of cloud based services and their attitude is “Appreciate what you got, even if what you got is a woman built like mashed potatoes.”” -Network Engineer

Not all cloud based WAFs are created equal. F5 Siverline WAF is the only cloud based WAF (F5 ASM) that is recognized in the Gartner Magic Quadrant for Web Application Firewalls. What does this mean to you? Well for starters, you get:

Consistent Security Policy for both on prem and in the cloud
Perfect forward secrecy so you can obtain that A+ rating on SSL Labs
Complete managed solution, just wear your architect hat if you want to
Positive logic security model; ability to learn every HTTP method, header, url, parameter, referrer, cookie, etc
Behavioral based detection for botnets, not just IP based
The power of iRules!

Since each customer had F5 on prem, they were able to mitigate these attacks when their Cloud WAF couldn’t. One popular technique that was effective was F5 Proactive BOT Defense and CAPTCHA. Using the iRule below, when the site was being brute forced, we were able to throw a CAPTCHA page to the BOTs and successfully mitigate the attack.

when BOTDEFENSE_ACTION {
    # LOGGING OFF  = 0
    # LOGGING ON   = 1
    set DEBUG_LEVEL 0
    if {[BOTDEFENSE::action] eq "allow"} {
        if {[BOTDEFENSE::captcha_status] ne "correct"} {
                set res [BOTDEFENSE::action captcha_challenge]
                if {$res ne "ok"} {
                    if { $DEBUG_LEVEL == 1} { log local0. "Unable to send captcha_challenge: \"$res\"""}
                    BOTDEFENSE::action tcp_rst
                }
        }
    }
}

While this iRule was used to mitigate the attack on prem, if the customer wanted to use the exact same iRule with F5 Silverline WAF they could. Pretty sweet.

Regards,

Denial of Service (DoS Attack) or Distributed Denial of Service (DDoS attack) is still a hot topic in 2016 even though it was discovered in 1994 and began being seen in the wild in September of 1996. There are many solutions that counter these attacks via on premise and off premise solutions. Let’s expand on on-premise solutions.

A DoS attack is an attack that is not necessarily volumetric in nature. Instead is it focused on overwhelming the on premise gear but not saturating the full internet pipe. A good example of this would be a syn flood that fills up the connection table on your firewall. I know what you are thinking, “But Brian, we have a NGFW that is capable of moving 40Gbps!” but let’s look at this a bit closer. Yes, your firewall is capable of moving a 20GB file but what happens if you break up that 40GB into millions of connections? The real attack vector here is not throughput, it’s connections.

Connections per second (CPS):

Connections per second (CPS) refers to the rate at which a device can establish state parameters for new connections. A stateful device must create and manage connection information on all unique IP streams that transit the device. Typically, the device must handle the first packet of a new connection differently than all subsequent packets so that the device can establish the state parameters for the new connection. Because this process is specialized, it usually occurs in the software process of the devices, as opposed to the normal hardware-based forwarding process.

What firewall administrators usually do not do is to check the max CPS numbers their Internet facing link can carry. This number depends on the Internet link speed. It is the same number as the max frames per second a link can carry. Frames per second is calculated as following:
(Link speed)/(84 Byte * 8bit/Byte) = max frames per second.

Each frame can be a new connection (UDP), therefore max FPS = max CPS Here an overview of how many CPS a link can transport related to link speed:

Link Speed	Max CPS
100 Mbps	148, 809
1,000 Mbps (1Gbps)	1,488,906
10,000 Mbps (10Gbps)	14,880,960

Maximum Concurrent Connections (MCC)

The number of concurrent connections is directly related to the max CPS number. When a Firewall hits it’s concurrent connection limits it does not accept new connections for any service and all services behind this firewall are not accessible for new requests. This number shows also the time a DDOS attack needs to take a firewall down and also the time the firewall has to take countermeasures against the attack. If for example, a firewall processes a DDOS mitigation run every 30 seconds but the firewall is overloaded in 10 seconds then it is too late for countermeasures.

So, why does it bother me? My Firewall does 40Gbps!

Well, what firewall vendors advertise is the throughput of the system for transferring large files. Unfortunately DDOS attacks do not use regular traffic. The frontend Internet facing firewall should be designed to handle the Internet facing threat potential. This is the reason why Administrators do not choose a 200 Mbps firewall to protect against a 1 Gig link. The same sizing guides should be taken into account for CPS.

Action Item for you: Go to your Firewall vendor product specification list and check the CPS numbers they publish.

What is the result?

Your Firewall cannot protect you. Even more: even if you upgrade to the strongest appliance model your Firewall vendor offers, it will not protect your datacenter because it can not handle the CPS threat. Please note, this post isn’t here to pick on your current firewall. At the end of the day, these are complementary solutions. Most likely you did not buy your NGFW for DoS/DDoS mitigation however advertised speeds tend to give a false sense of security.
Here are some real life examples (state of Dec 2015):

Vendor Name	1 Gbps Threat Protection	10 Gbps Threat Protection
Cisco ASA 5585-X CPS: 350,000 CC: 10M	23.5% threat protection Exhausted in 7 sec	2.3% threat protection Exhausted in 0.7 sec
Juniper SRX 5800 CPS: 450,000 CC: 100M	30.2% threat protection Exhausted in 67 sec	3% threat protection Exhausted in 6,7 sec
Checkpoint 61000 CPS: 3,000,000 CC: 210M	100% threat protection Exhausted in 141 sec	20.1% threat protection Exhausted in 14 sec
Fortinet Fortigate 5001D CPS: 565,000 CC: 23M	38% threat protection Exhausted in 16 sec	3.8% threat protection Exhausted in 1.6 sec

Please note: all the Firewalls are High End Firewalls and partially marked as Carrier class devices, which means, they are also priced as carrier class devices.

How do I protect my perimeter firewall?

F5 is the only vendor that offers high speed Firewall protection at midsize appliances. Here an overview of possible F5 Firewall solutions:

	BIG-IP 5205v	BIG-IP 12250v	VIPRION B2250	VIPRION B4300
Throughput	30 Gbps	80 Gbps	320 Gbps	640 Gbps
CPS	700k	1.5M	4.0M	11.2M
CC	24M	80M	192M	288M
SYN Cookie	40M	80M	240M	640M

It’s interesting to note that 5250v (smallest appliance) is only a 2U that is capable of cleaning just about anyone’s internet circuit. I am unaware of anyone in the southwest with pipe greater than 20Gbps.

Do I migrate my existing perimeter firewall to a F5 firewall?

In short, no. A F5 System could be installed in parallel with the existing Firewall. However, in recent efforts, I have deployed the F5 as a L2 Transparent DoS/DDoS mitigation device in front of the existing firewalls. The draw for this is the deployment is simple, easy to manage, you keep your existing policy on your firewalls, and utilize the F5 for mitigation only.

Credit: Vladimir Bojkovic

In 2015, I personally worked on two projects where customers used the F5 as a L2 Transparent DDoS mitigation device after suffering outages.

Project #1: Customer had a pair of PA-7050’s( App ID and Threat Detection enabled) on the perimeter and a 1 Gbps internet circuit. They were hit with a NTP reflection attack that caused an outage and the Palo Alto boxes to fall over. Post mortem, the NTP reflection attack was on 230 Mbps in size but exhausted the CPS on the PA. For those of you wondering, the box only had 478 Mbps running thru it at the time.

We positioned the 5250v box at the perimeter for mitigation. Currently the network and security team sees the F5 as their first line of security for any and all L3/L4 attacks.

Project #2: This customer has a mixture of technologies in their environment which lead to a false sense of security. At the perimeter, they had Cisco 5585-40’s along with 20Gbps internet circuits. Early in the fall, they were hit with a 3Gbps TCP SYN flood (3 million unique client IPs) that took the down the PA’s.

We positioned the VIPRION B2250 as a L2 transparent DDoS firewall. The beautiful part of this solution was, it went directly into production and it did not associate a single helpdesk ticket; it truly went in as a bump on the wire.

If you have any questions or comments, feel free to leave your thoughts below.

Regards,

Category: DDoS

Why a 3Gbps DoS can take out your 40Gbps NGFW and 20Gbps Internet Circuit