Missing special characters and ASP.NET

It cost me one Friday evening to try to figure out why some special characters in ASP.NET query strings were not passed into my Delphi application.

For example, I had a basic HTML form that forwards data to an .ASPX page using the GET method.

This worked fine until I decided to test some Finnish characters like Ä and Ö. For example, giving the input "Järvinen" and then using the Request.QueryString property to read the data only revealed "Jrvinen".

My first impression was that this was some kind of ASP.NET security setting for I had earlier learned about ASP.NET's way of protecting web applications from scripting attacks, etc. But I was wrong. So, my application misbehaved just the same when I did set the @Page directive's ValidateRequest atrribute to "false".

Next, I have Google a workout, but that didn't solve my problem. I even created a test project in Visual Studio .NET to see if this was a Delphi problem, but it was not.

Finally, I decided to launch up my Mozilla Firefox browser to check to see what data my ASP.NET web application returns.

At this point I noticed how the default character set for my web applications was UTF-8, and luckily that rang a bell in my head. That is, when I create web pages using Dreamweaver, I always use the "iso-8859-1" encoding, which is of course different than "utf-8". Once I changed my HTML page to use UTF-8, the special characters were no longer missing or disappearing.

Given this hindsight, it was easy to find a Microsoft Knowledge Base (KB) article that details this "error". You can find it by using the number 835385.

For a reference, you can change an ASP.NET application's default character set in the web.config file under the "globalization" section. This is an example how to use "iso-8859-1" everywhere:

<configuration>
   <system.web>
      <globalization
requestEncoding="iso-8859-1"
responseEncoding="iso-8859-1"/>
   </system.web>
</configuration>