Note on UTF-16 encoded files and Claude Code

I've noticed an interesting thing about Claude Code on my Windows 11 desktop: if you ask it to process Unicode UTF-18 encoded files, it struggles with them. It might say, "attempting to read with Bash failed, let me try with PowerShell", or similar. Then, it goes to a very lengthy process of writing complex PowerShell scripts to process the file, all of which just wastes tokens.

Eventually, it will get there, but there's a delay, and you might even bump to a usage limit if working with the Claude Professional subscription. This subscription level isn't unlimited: none of Anthropic's AI subscriptions are, at least at this writing.

So, my advice: if you need to work with UTF-16 encoded files (such as: XML file, SQL scripts, etc.) then save/re-encode them to UTF-8 first. Much easier for Claude Code that way, and less token consumption.