Base64 Decode Errors: Valid Bytes Are Not Always UTF-8 Text

Restoring bytes and reading characters are separate steps

RFC 4648 Base64 represents byte data using characters. It does not establish whether the original was text or an image, or identify its text encoding. These minimal inputs isolate the two steps; they are not sample image files.

w6k= restores C3 A9, which is é in UTF-8. /w== restores one byte, FF, which fails UTF-8 decoding. ww== restores only C3, missing the required following byte. All three passed byte decoding. @@@ was rejected during byte decoding.

RFC 4648: Base64

Replacement characters and strict decoding differ

The WHATWG Encoding Standard defines replacement as the default TextDecoder error mode; fatal: true throws on decoding errors. Our default UTF-8 decoder test with FF produced the U+FFFD replacement character. Moyoutil uses fatal: true and reports an error for that input. Suppressing an exception to get a replacement character does not recover the exact original text.

const bytes = Uint8Array.from(atob('/w=='), c => c.charCodeAt(0));
// bytes: [255] = FF
new TextDecoder('utf-8').decode(bytes); // U+FFFD
new TextDecoder('utf-8', {fatal: true}).decode(bytes); // TypeError

First confirm that the source is UTF-8 text. Images and compressed files are outside this text tool’s scope. For text in another encoding, check the source system’s encoding information and use a tool supporting it. Base64 alone cannot establish the encoding. Moyoutil does not guess other encodings and does not support the Base64URL characters - and _.

WHATWG: TextDecoder

Frequently Asked Questions