Computer Networking › Section H · Security in computer networks › Topic H.9
Operational security: firewalls and intrusion detection
Cryptography protects data in transit, but an organisation also has to decide which traffic may enter or leave its network at all, and notice when something gets through anyway. Firewalls enforce that decision and intrusion detection systems watch for trouble.
Guarding the border
An organisation's network meets the internet at one or a few points, and those points are where operational security begins. A firewall, whether a dedicated appliance or software running on a router or server, stands at that point and separates the internal network from the outside, passing some packets and blocking others. A well-placed firewall has three properties: all traffic between inside and outside passes through it, only traffic allowed by the local security policy is let through, and the firewall itself is hardened against attack. "Cybersecurity: attacks and defences" in Computer Science introduced a gateway firewall; this topic looks at the kinds of firewall and how their rules work.
Stateless packet filters
The simplest firewall is a packet filter, often built into the gateway router, which examines each datagram on its own. It can match on source and destination IP addresses, the protocol field (TCP, UDP or ICMP), source and destination ports, TCP flags such as SYN and ACK, ICMP message types and the interface the packet arrived on. The policy is an ordered list of rules called an access control list. For each packet the filter applies the first rule that matches and ignores the rest, so order matters. A careful policy ends with a rule that denies everything not explicitly allowed, a default-deny stance.
Worked example: applying an ordered rule list
A company uses 203.0.113.0/24, with a web server at 203.0.113.10. Rule 1 allows inbound TCP to 203.0.113.10 port 443. Rule 2 allows outbound TCP from inside to any address on port 443. Rule 3 allows inbound TCP from source port 443 to inside addresses if the ACK bit is set. Rule 4 denies everything. 1. A SYN from the internet to 203.0.113.10:443 matches rule 1: allowed. 2. A SYN to 203.0.113.25:22 matches only rule 4: dropped. 3. A reply from 198.51.100.7:443 to 203.0.113.25 with ACK set matches rule 3: allowed. 4. The same packet with SYN but no ACK matches only rule 4: dropped. Check: no outsider can open a connection except to the web server.
Stateful filters
The rule list above has a weakness: rule 3 trusts the ACK bit, and an outsider can set that bit in a packet that belongs to no real connection. A stateful filter closes the gap by tracking connections. When it sees an inside host open a TCP connection with a SYN, it records the addresses and ports in a connection table; when the connection closes with FIN, or stays idle too long, it removes the entry. An inbound packet is allowed only if it matches a table entry or an explicit rule. UDP has no connection setup, so stateful filters track UDP flows with short timeouts. Most home routers combine this with NAT.
Application gateways
Packet filters see headers, not users or content. An application gateway, or proxy, works at the application layer: internal hosts connect to the gateway, which makes the outside connection on their behalf. Because it understands the protocol, it can require users to log in, allow only certain operations and log everything. Next-generation firewalls add application awareness to filtering, and web application firewalls sit in front of web servers, blocking requests that look like SQL injection. Encryption limits all of them: a gateway that must read HTTPS has to terminate TLS itself, which gives it access to users' data and must be handled with care.
Where firewalls go
One firewall at the border is rarely enough. Servers that must be reached from outside, such as a public web or mail server, usually sit in a demilitarised zone (DMZ), a separate segment with its own rules, so a compromised public server does not expose the internal network. Large networks are divided into segments, with firewalls between departments, data-centre zones and guest Wi-Fi, so that an attacker who gets in at one point cannot move freely. Host firewalls on each server and laptop add another layer. Zero-trust designs push this furthest, authorising every request whatever network it comes from.
Intrusion detection systems
A firewall decides from headers or protocol rules; an intrusion detection system (IDS) looks deeper. It performs deep packet inspection, examining payloads and patterns across many packets, and raises alerts when it sees suspicious activity: a known attack, a host scanning many ports, or a server suddenly sending far more data than usual. An intrusion prevention system (IPS) sits inline and can also drop suspicious traffic. Organisations often place sensors at several points, behind the border firewall and inside important segments. Open-source tools include Snort and Suricata for detection and Zeek for detailed traffic logging.
Signatures and anomalies
A signature-based IDS compares traffic against a database of patterns that describe known attacks, such as particular byte sequences or message sequences. It is precise about what it knows, but it misses brand-new attacks and depends on prompt signature updates. An anomaly-based IDS first learns what normal traffic looks like, then flags statistically unusual behaviour. It can catch attacks no one has seen before, but normal traffic varies, so it raises more false alarms. Both are blind to the content of encrypted traffic, though they can still watch addresses, timing and volumes. Many deployments combine the two methods.
Worked example: the false-alarm problem
An IDS watches 1,000,000 connections a day, of which 100 are attacks. It detects 99% of attacks and wrongly flags 0.1% of harmless connections. 1. True alerts: 99% of 100 = 99. 2. Harmless connections: 999,900. 3. False alerts: 0.1% of 999,900 = 999.9, about 1,000. 4. Total alerts: about 1,099, of which only 99 are real. 5. So only about 9% of alerts are genuine: 99 ÷ 1,099 ≈ 0.09. Check: because attacks are rare, even a small false-positive rate swamps analysts. This is why tuning rules, and ranking alerts by severity, matters as much as detection rate.
Common mistakes to avoid
First, writing rules in the wrong order; the first matching rule wins, so a broad allow early on overrides later denies. Second, ending a policy without a default deny. Third, trusting the ACK bit in a stateless filter to mean a packet belongs to a real connection. Fourth, putting public servers on the internal network instead of in a DMZ. Fifth, assuming an IDS catches every attack; signature systems miss new ones and anomaly systems raise false alarms. Sixth, forgetting that encrypted traffic hides payloads from both firewalls and IDSs.
Key takeaways
A firewall sits where all inside-outside traffic passes, lets through only what policy allows, and is itself hardened. Stateless packet filters apply an ordered access control list to header fields, ending in default deny. Stateful filters track connections, so only replies to real connections get in. Application gateways and web application firewalls understand protocols but need TLS termination to read HTTPS. DMZs, segmentation and host firewalls add layers. An IDS inspects packets deeply, by signatures or anomalies; an IPS can also block. Rare attacks mean false alarms can outnumber true ones, so tuning matters.
Flashcards
Flashcard
Isolates an internal network, passing only allowed traffic
Test your understanding
Question 1 of 5
A filter's rules are: 1, allow inbound TCP to 203.0.113.10 port 443; 2, deny inbound TCP to 203.0.113.0/24; 3, allow inbound TCP to 203.0.113.10 port 22; 4, deny all. What happens to an inbound SYN to 203.0.113.10 port 22?
Question 2 of 5
How does a stateful filter improve on a stateless rule that admits inbound TCP packets with the ACK bit set?
Question 3 of 5
Why are public web servers usually placed in a DMZ rather than on the internal network?
Question 4 of 5
An IDS sees 500,000 connections a day, 50 of them attacks. It detects 98% of attacks and falsely flags 0.2% of harmless connections. About what fraction of its alerts are real?
Question 5 of 5
Which statement best compares signature-based and anomaly-based intrusion detection?
