Hackers Attacked Our Site: How WAF Saved the Day

1. Strange Data Appearing in the DB

One pattern that shows up repeatedly in security incident case studies goes like this. Shortly after a service launch, data like this starts appearing in an order_note column:

' OR 1=1 --
'; DROP TABLE users; --
UNION SELECT 1, @@version --

A chill runs down your spine just looking at it. SQL Injection. Someone is running an automated scanner to probe the site for vulnerabilities. Even if the ORM blocks the basic attacks, logs often show over 100 requests per second, driving server CPU to 100%.

The instinct might be to block those IPs at the code level—but that still forces the server to spend resources checking each IP. The right answer is to stop them before they even reach the application server.

---

2. Enter the Savior: Web Application Firewall (WAF)

WAF is literally a Firewall for Web Apps. Let's use an airport analogy.

• Network Firewall (L3/L4): Passport Control
  • Checks IP (Nationality) and Port (Destination).
  • "You are from a blocked country (IP)? Denied."
  • But if the passport is valid, it lets you pass even if you have a bomb in your bag.
• WAF (L7): Security Checkpoint (X-ray)
  • Inspects the content of HTTP packets (Body, Header, Cookie).
  • "Your IP is clean, but you have a knife (SQL Injection) in your bag (Body)? Denied."

We urgently attached AWS WAF to our ALB (Load Balancer).

2.1. Defense Phase 1: OWASP Core Rule Set (CRS)

WAF comes with a standard set of rules called OWASP CRS. Just turning this on automatically blocks globally known attack patterns.

• SQL Injection: Blocks patterns like ' OR 1=1.
• XSS: Blocks requests containing <script> tags.
• LFI/RFI: Blocks attempts to access system files like /etc/passwd.
• Known CVEs: Blocks exploits like ShellShock or Java Serialization.

The moment we applied it, the attack logs stopped dead. The overheating server CPU returned to peace.

---

3. Security Models: Positive vs Negative

There are two philosophies when operating a WAF.

1) Negative Security Model (Blacklist)

• Concept: "Everyone is welcome. But I will block Bad Guys (Known Attacks)."
• Pros: Less inconvenience for normal users. Easy to set up.
• Cons: Cannot block new/unknown attacks (Zero-day exploits).
• Example: AWS Managed Rules, OWASP CRS. This is the default.

2) Positive Security Model (Whitelist)

• Concept: "Everyone is banned. Only Authorized Guys (Known Good) can enter."
• Pros: Extremely secure. Blocks Zero-day attacks because they aren't on the whitelist.
• Cons: Very hard to maintain. High risk of blocking normal users (False Positives).
• Example: Defining "API requests must be JSON, and id must be an integer" in WAF.

---

4. The WAF Nightmare: False Positives

A few days after deploying WAF, a new report comes in. A user can't complete registration.

The logs show the blocked user's nickname was "Select". The WAF saw the word Select, thought "Is this a SQL Injection SELECT * FROM?" and blocked it. (True story).

This is a False Positive. Security so strong it treats innocent users as criminals.

Solution:

1. Count Mode (Log Only): Initially, don't block. Just log.
2. Tuning: Analyze logs and add exceptions (e.g., "Disable SQL inspection for nickname field").
3. Enforce: Switch to Block Mode only when you are confident.

---

5. Why Cloud WAF is the Standard

In the old days, we installed open-source WAFs like ModSecurity directly on Nginx. But today, Cloud WAF (AWS, Cloudflare) is the way to go.

1. Performance: WAF inspection consumes CPU. Cloud WAF offloads this burden to the Edge, keeping your origin server fast.
2. Update Speed: When a new vulnerability (like Log4Shell) hits, Cloud vendors update their rules within hours. Your shield upgrades while you sleep.
3. DDoS Protection: Cloud WAFs have massive bandwidth to absorb volumetric attacks that would crush a single server.

---

6. FAQ: Common WAF Questions

Q. Use WAF or implement Rate Limiting in code?

Both.

• WAF Rate Limiting: Blocks high-volume attackers (e.g., >1000 req/min) at the edge. Cheap and fast.
• App Rate Limiting: Handles complex logic (e.g., "10 OTP requests per day per user"). Use WAF to block the gross traffic, and Code to handle the fine-grained logic.

Q. Does WAF decrypt HTTPS?

Yes. To inspect the packet body, WAF must decrypt the HTTPS traffic. This is why you attach WAF to the Load Balancer (ALB) or use a CDN (Cloudflare) where the SSL termination happens.

Q. Is WAF a silver bullet?

No. WAF cannot stop Business Logic Attacks. If a hacker logs in normally and scrapes all your public data, WAF sees it as "normal traffic". You need other defenses (like Captcha or anomaly detection) for that.

---

7. Conclusion: Security is Cheaper when Bought

Startup CTOs often say, "Is AWS WAF expensive?"

Let's do the math. AWS WAF is $5/month + $0.60 per million requests. For most startups, it costs less than two cups of coffee.

But if you get hacked? GDPR fines, loss of customer trust, engineering time for recovery... It costs millions. Your company might die.

Developers, Don't stay up all night writing Regex filters for SQL Injection. Just turn on WAF. Use that time to build features that make money. That is the smartest trade-off you can make.

---

9. Deep Dive: ModSecurity (The Open Source King)

If you cannot afford Cloud WAF, ModSecurity is the industry standard open-source WAF. It plugs into Nginx, Apache, and IIS. It uses the SecRule language, which is powerful but looks like cryptic regex soup.

Example Rule (Block SQL Injection):

SecRule ARGS "DELETE[[:space:]]+FROM" "id:1000,deny,msg:'SQL Injection Attempt'"

In reality, you don't write these. You download the OWASP CRS (Core Rule Set), which contains thousands of pre-written rules. Be warned: ModSecurity adds latency to every request because it has to regex-match the entire payload body. It can slow down your server by 10-20% if not tuned correctly.

10. Educational: How Hackers Bypass WAF (WAF Evasion)

Security is a cat-and-mouse game.

1. Case Toggling: SeLeCt * FrOm users (Bypasses naive regex).
2. Comment Insertion: SELECT/**/username/**/FROM (Bypasses space detection).
3. HPP (HTTP Parameter Pollution): ?id=1&id=2. Some WAFs check the first id, but the backend uses the second id.
4. Chunked Transfer Encoding: Sending the malicious payload in tiny pieces across multiple TCP packets to confuse the WAF stream inspector.

Knowing these techniques helps you confirm why a "Cloud WAF" that constantly updates its engine is superior to a static static regex rule.