The escape function and international characters in URLs

Over the past few days I’ve been looking at how to send Czech characters in a GET request, and therefore in a URL.
I was solving it in the context of our HTTP server,
Jetty,
but the conclusions I reached apply more generally.

The URL specification doesn’t say which encoding to use for international characters.
That’s because characters outside US-ASCII aren’t supposed to appear there in the first place.
Suppose you’re searching for a place and need to put Kunžak in a URL. JavaScript 1.0 defined the escape function for this. It takes a single argument, a string, and
encodes it as hexadecimal escape sequences. In our example the URL should contain the encoded sequence rather than Kunžak itself. That’s how it should work in theory. In practice it’s a different story.

We encoded the Czech characters with escape but never said which encoding
we used. escape(“ž”) in UTF-8 returns a completely different number from escape(“ž”) in Windows-1250.
So which encoding gets used? You might think it should depend on
the HTML page’s encoding, or charset. Wrong. Internet Explorer (tested on 5.00.3315.1000, SP2;q319182;Q321232;Q323759;Q328970;)
ignores the page encoding and always uses Windows-1250 for escape. Mozilla behaves as we’d expect, with the page encoding determining the resulting escape sequence.

The ECMAScript 3.0 specification marks this function
as obsolete and discourages its use.
It replaces it with encodeURI, which is
specified to always use UTF-8. That’s good in view of
the IETF’s work
on moving from URLs to
IRIs. The proposal
would, among other things, make UTF-8 the standard encoding for escape sequences.

I finally solved the problem by writing my own conversion function to turn Czech characters into UTF-8 escape sequences. It replaces the characters using a hard-coded table of constants containing the UTF-8 escape sequences.