URL Encoder / Decoder

Percent-encoding replaces one byte with three characters: a %and the byte in hex. Not one character. One byte, so an accented letter becomes six characters and an emoji becomes twelve.

é    UTF-8 bytes  c3 a9   ->   %C3%A9
日    UTF-8 bytes  e6 97 a5 ->   %E6%97%A5
space             20      ->   %20

Everything difficult about URLs follows from a second fact: the specification never says which characters must be encoded, only which ones are safe to leave alone. That depends entirely on where in the URL the value is going, and that is the decision the three encoders below disagree about.

Three functions, three answers

Input   a b&c=d/e?f#g+h

component   a%20b%26c%3Dd%2Fe%3Ff%23g%2Bh
whole URL   a%20b&c=d/e?f#g+h
form        a+b%26c%3Dd%2Fe%3Ff%23g%2Bh

The middle one left &, =, /, ? and # alone, because in a whole URL
they are structure. In a value they are not.

Encoding a whole URL with the component rules is the classic mistake: the slashes become %2F, the whole thing turns into one long path segment, and the server returns 404 for a URL that reads correctly to a human.

The plus sign is the one that gets everybody

?note=1+1                  urlencoded reading:  "1 1"
?note=1+1                  path-style reading:  "1+1"

?note=1%2B1                both readings:       "1+1"

+ means space in form encoding and means + anywhere else. Both readings are correct, and which one applies depends on what is parsing the string, not on how it was written. PHP's$_GET and Java's URLDecoder treat query strings as form-encoded and turn + into a space. JavaScript'sURLSearchParams does too. ButdecodeURIComponent does not, and neither does a path segment.

So a phone number written as +44 7700 900000 arrives as44 7700 900000 in one framework and correctly in another. The only value that survives everything is %2B. Encode it and stop thinking about it.

Strict RFC 3986, and when it matters

encodeURIComponent leaves !, ',(, ) and * alone. RFC 3986 lists them as reserved, so strictly they should be encoded, and most of the time nobody notices the difference.

Two places notice immediately. OAuth 1.0 signatures and AWS Signature V4 both build a canonical string by percent-encoding every parameter, and both specify the strict rules. Sign with the browser's version and the signature is computed over a different string than the server computes it over, and the only error you get back is that the signature does not match. The strict switch above is for exactly that.

Double encoding, and the %25 that gives it away

Encode %20 a second time and it becomes %2520, because the % is itself a character that needs encoding. The symptom is a link with visible %20 in the address bar and a 404 behind it, and the tell is always a %25 somewhere in the string. It is flagged above whenever it appears.

It happens most often when a URL is put inside another URL as aredirect_uri or a next= parameter, and some layer in between helpfully encodes what was already encoded. Decode twice to get back to the original.

Two ways to write the same host

Hostnames are ASCII only. Anything else travels as Punycode, somünchen.example.com is sent asxn--mnchen-3ya.example.com, and the breakdown above shows both because they are the same host and only one of them is in the request.

This is also where homograph attacks live. Cyrillic а and Latina are different characters that render identically, soаpple.com is a domain somebody else can register. Registries and browsers both have defences now, and both have gaps, so a label mixing two writing systems is called out above whenever one turns up.

What is not encoded, and never should be

The structural characters are the URL. A ? starts the query, a# starts the fragment, & and =separate parameters, and / separates path segments. Encoding them turns structure into content. That is the point when the value contains one and the whole problem when it does not.

One consequence worth knowing: the fragment never leaves the browser. Whatever is after the # is not sent to the server, ever, in any request. It is why single-page routers used to live there, and why putting a token in a fragment keeps it out of server logs while putting one in a query parameter does not.

How long is too long

There is no limit in the specification. In practice Internet Explorer stopped at 2,083 characters and enough infrastructure was built against that number that it became the safe ceiling: nginx defaults to an 8 KB request line, Apache to 8,190 bytes, and various proxies and CDNs sit somewhere in between. Modern browsers will happily carry tens of thousands, right up until something in the middle truncates one without a word.

Anything over 2,000 characters is flagged above. If a URL is that long, the data in it usually wants to be a POST body instead.