Category: Web

Following my obituary for Cisco’s WAF, Packet Pushers did a really great WAF-focused podcast with Raven Alder, appropriately named Saving the Web with Dinky Putt Putt Firewalls. If you have more than a fleeting interest in protecting business web applications, you should definitely listen to it. Just as an aside: when they were recording the podcast, I was writing my To WAF or not to WAF post ... and it’s nice to see we’re closely aligned on most points.

There’s just a bit I’d like to add to their ponderings. What Raven describes is the “proper” (arduous, time-consuming and labor-intensive) use of WAF that we’re used to from the layer-3/4 firewalls: learning what your web application does (learning because the design specs were never updated to reflect reality) and then applying the knowledge to filter everything else (what I sometimes call the fascist mode – whatever is not explicitly permitted is dropped).

A few days ago I wrote that you should always strive to understand the technologies beyond the reach of your current job. Stephen Foskett is an amazing example to follow: although he’s a storage guru, he knows way more about HTTP than most web developers and details of the web server architecture that most server administrators are not aware of. Read his High-Performance, Low-Memory Apache/PHP Virtual Private Server; you’ll definitely enjoy the details.

Extremely valid comment made by Pavel Skovajsa in response to my “Rest in peace, my WAF friend” post beautifully illustrates the compartmentalized state some IT organizations face; before going there, let’s start with the basic questions.

Do we need WAF ... as a function, not as a box or a specific product? It’s the same question as “do we need virus scanners” or “do we need firewalls” in a different disguise. In an ideal world where all the developers would be security-conscious and there would be no bugs, the answer is “NO”. As we all know, we’re in a different dimension and getting further away from the heavens every time someone utters “just good enough” phrase or any other such bingo-winning slogan.

It’s popular to bash IT vendors’ lack of security awareness (Microsoft comes to mind immediately), but they’re still far ahead of a typical web application developer. At least they get huge exposure, which forces them to implement security frameworks.

A few years ago, Cisco bought a company that made application-level firewalls, first an XML-focused product (XML Gateway) that was also able to verify your XML data, later a Web Application Firewall (WAF), which was effectively the XML product with half of the brains ripped out.

I was really looking forward to these products. Layer-3 firewalls cannot protect web sites against application-layer problems like SQL injections or cross-site scripting, so we definitely need something on the application layer and the WAF (and XML Gateway) ran as virtual appliance in VMware, making them ideal for my lab environment. I quickly lost interest after the first cursory contact with the XML Gateway as you could only manage both products with a web-based GUI (and I definitely don’t want to publish blog posts full of screenshots).

Many thanks to those of you that responded with Netflix details (special thanks to Volcker for sending me the packet capture). Immediately after someone mentioned firewalls, I knew what the most sensible answer should be: to get across almost anything, use HTTP. No surprise, Netflix chose to use it. However, they’ve managed to deploy streaming video over TCP, which is not a trivial task. So, how did they do it?

The Cisco Subnet RSS feed I’m receiving from Network World contained interesting information a few days ago: Cisco has reissued the HTTP security advisory from 2005. The 2005 bug was “trivial”: they forgot to quote the “<” character in the output HTML stream as “&lt;” and you could thus insert HTML code into the router’s output by sending pings to the router and inspecting the buffers with show buffers assigned dump (I found the original proof-of-concept exploit on the Wayback Machine). However, I’ve checked the behavior on 12.4(15)T1 and all dangerous characters (“<” and quotes) were properly quoted. So, I’m left with two explanations.

Cisco has released security response acknowledging that the IPS software does not properly handle a rarely used Unicode encoding. Reading the security notice you might be left wondering what's going on. Here's the whole story.

Within an URI (web address), the ASCII characters can be encoded in one of three ways:

• Unless they are reserved, they can be included in the URI directly (for example, you can always use the letter a in an URI).
• You can always escape a character using its hexadecimal value. Letter a can thus be written as %61.
• Unicode character set includes full-width form of ASCII characters, where letter a can be encoded as a two-byte value 0xFF61 (or %ff%61 in an URI)

The IPS software (standalone or integrated in Cisco IOS) does not recognize the sequence %ff%61 as letter a. It's thus possible to evade some IPS triggers by replacing ASCII characters with their full-width Unicode encoding.

By default, IOS routers use enable passwords to authenticate incoming HTTP (web) requests. You could also use local usernames as the authentication mechanism, or you could deploy full-blown AAA-based solution.

IOS has (yet another) nice "underdocumented" feature - if you don't have an enable password/secret configured on your router, you can access IOS HTTP(S) server (assuming it's enabled with the ip http server command) without any authentication whatsoever. Of course you'd never do that in a production environment, but it's nice to know you can always configure the router from a web browser if needed (see also the discussion on default passwords with Cisco SDM).

Update 2012-08-27: Stefan de Kooter reported the bug had been fixed in IOS release 15.1(4)M.

If you run multiple web sites on a single physical server, it's highly likely that you rely on the ability of HTTP/1.1 clients to specify the Host: field in the HTTP request to indicate which web site they're trying to access.

Cisco IOS always inserts the web server's IP address (not the hostname) in the Host: field of the HTTP request, regardless of whether you enter IP address or hostname in the URL part of an IOS command that supports HTTP (for example, copy or more command) ... and regardless of whether the hostname is locally configured with the ip host command or resolved by an external DNS server specified in the ip name-server command.