Writing a HTTP client with Microsoft WinHTTP 5.0
Posted: (EET/GMT+2)
Writing a HTTP client with Microsoft WinHTTP 5.0
Dec 6, 2001
Borland Delphi 6 or later
Microsoft Windows HTTP Services 5.0 (WinHTTP) SDK installed
Microsoft's Windows HTTP Services 5.0 (WinHTTP) is the latest version of the company's HTTP client application programming API. This API allows you to easily create advanced HTTP client applications. Unlike the WinInet API, WinHTTP is designed for server applications that don't require a user interface.
Before you can start to use the WinHTTP API with for example Borland Delphi 6, you need to download the WinHTTP SDK from the Microsoft MSDN site. Once on the site, search for "winhttp" and you will find a page that contains a download link. Installing the SDK is simple, but remember that you have to have a NT based operating system, namely Windows NT 4.0, Windows 2000 or Windows XP.
Once installed, you should find documentation, sample applications and C language header files under the installation directory (or should that be "folder"?). If you are using Delphi for your development, you of course need a way to use the APIs from your environment. This requires that the header file WINHTTP.H to be translated to Object Pascal.
The following is a simple translation of the most common structures. You can also download the file WINHTTP.PAS directly by clicking the link at the bottom of this article.
unit WinHTTP;
{$ALIGN 4 - set record field align to 4 bytes }
interface
Uses Windows;
Type
HINTERNET = Pointer;
PHINTERNET = ^HInternet;
INTERNET_PORT = Word;
PINTERNET_PORT = ^INTERNET_PORT;
PURL_COMPONENTS = ^TURL_COMPONENTS;
TURL_COMPONENTS = Record
dwStructSize : Integer; { size of this structure. Used in version check }
lpszScheme : PWideChar; { pointer to scheme name }
dwSchemeLength : Integer; { length of scheme name }
nScheme : Integer; { enumerated scheme type (if known) }
lpszHostName : PWideChar; { pointer to host name }
dwHostNameLength : Integer; { length of host name }
nPort : INTERNET_PORT; { converted port number }
lpszUserName : PWideChar; { pointer to user name }
dwUserNameLength : Integer; { length of user name }
lpszPassword : PWideChar; { pointer to password }
dwPasswordLength : Integer; { length of password }
lpszUrlPath : PWideChar; { pointer to URL-path }
dwUrlPathLength : Integer; { length of URL-path }
lpszExtraInfo : PWideChar; { pointer to extra information (e.g. ?foo or #foo) }
dwExtraInfoLength : Integer; { length of extra information }
End;
Const
WinHTTP5DLL = 'winhttp5.dll';
INTERNET_DEFAULT_PORT = 0;
INTERNET_DEFAULT_HTTP_PORT = 80;
INTERNET_DEFAULT_HTTPS_PORT = 443;
INTERNET_SCHEME_HTTP = 1;
INTERNET_SCHEME_HTTPS = 2;
WINHTTP_ACCESS_TYPE_DEFAULT_PROXY = 0;
WINHTTP_ACCESS_TYPE_NO_PROXY = 1;
WINHTTP_ACCESS_TYPE_NAMED_PROXY = 3;
WINHTTP_NO_REFERER = nil;
WINHTTP_DEFAULT_ACCEPT_TYPES = nil;
WINHTTP_NO_ADDITIONAL_HEADERS = nil;
WINHTTP_NO_REQUEST_DATA = nil;
Function WinHttpOpen(pwszUserAgent : PWideChar; dwAccessType : Integer;
pwszProxyName : PWideChar; pwszProxyBypass : PWideChar;
dwFlags : Integer) : HINTERNET;
StdCall; External WinHTTP5DLL;
Function WinHttpCloseHandle(Handle : HInternet) : Bool;
StdCall; External WinHTTP5DLL;
Function WinHttpCheckPlatform : Bool;
StdCall; External WinHTTP5DLL;
Function WinHttpConnect(hSession : HINTERNET; pswzServerName : PWideChar;
nServerPort : Word; dwReserved : Integer) : HINTERNET;
StdCall; External WinHTTP5DLL;
Function WinHttpOpenRequest(hConnect : HINTERNET; pwszVerb : PWideChar;
pwszObjectName : PWideChar; pwszVersion : PWideChar;
pwszReferrer : PWideChar; ppwszAcceptTypes : Pointer;
dwFlags : Integer) : HINTERNET;
StdCall; External WinHTTP5DLL;
Function WinHttpSendRequest(hRequest : HINTERNET;
pwszHeaders : PwideChar; dwHeadersLength : Integer;
lpOptional : Pointer; dwOptionalLength : Integer;
dwTotalLength : Integer; dwContext : PInteger) : Bool;
StdCall; External WinHTTP5DLL;
Function WinHttpReceiveResponse(hRequest : HINTERNET;
lpReserved : Pointer) : Bool;
StdCall; External WinHTTP5DLL;
Function WinHttpQueryDataAvailable(hRequest : HINTERNET;
Var lpdwNumberOfBytesAvailable : Integer) : Bool;
StdCall; External WinHTTP5DLL;
Function WinHttpReadData(hRequest : HINTERNET; lpBuffer : Pointer;
dwNumberOfBytesToRead : Integer;
Var lpdwNumberOfBytesRead : Integer) : Bool;
StdCall; External WinHTTP5DLL;
Function WinHttpQueryHeaders(hRequest : HINTERNET; dwInfoLevel : Integer;
pwszName : PWideChar; lpBuffer : Pointer;
Var lpdwBufferLength : Integer; lpdwIndex : PInteger) : Bool;
StdCall; External WinHTTP5DLL;
Function WinHttpCrackUrl(pwszUrl : PWideChar; dwUrlLength : Integer;
dwFlags : Integer; Var lpUrlComponents : TURL_COMPONENTS) : Bool;
StdCall; External WinHTTP5DLL;
implementation
end.
Note how the actual API functions are implemented in WINHTTP5.DLL.
Developing a simple client
When developing web applications, you sometimes need a way to simply see the HTTP headers and body without any formatting. If you are using a web browser, you can of course choose to view the source, but that only displays the HTML code, not the HTTP headers.
The following is part of the code of an application dubbed HTTPGet. This application is a simple one; it simply takes in a URL as a command line parameter, and outputs the given URL along with headers to the console.
Program HTTPGet;
{$APPTYPE CONSOLE}
Uses
SysUtils,
WinHTTP in 'WinHTTP.pas';
Var
Session : HINTERNET;
NoBody : Boolean;
Quiet : Boolean;
Procedure CheckParams;
Begin
If ((ParamCount = 0) Or FindCmdLineSwitch('?')) Then Begin
WriteLn('HTTPGet v1.0 Copyright (C) WhirlWater 2001.');
WriteLn('This program simply "gets" the resource pointed to by the given HTTP URL.');
WriteLn('The purpose of this application is to quickly retrieve HTTP headers and body');
WriteLn('for example debugging purposes. It can also used to test whether an URL could');
WriteLn('pose a security risk, such as a browser-enabled virus and/or script code.');
WriteLn;
WriteLn('This program requires Microsoft Windows HTTP Services (WinHTTP) 5.0 to operate.');
WriteLn;
WriteLn('Usage: HTTPGET [URL [-n] [-q]]');
WriteLn;
WriteLn('Parameters: URL = the URL to fetch, must include "http://"');
WriteLn(' -n = don''t display the resource body, just the headers');
WriteLn(' -q = quiet operation, just display data and error messages');
Halt(1);
End;
NoBody := FindCmdLineSwitch('N');
Quiet := FindCmdLineSwitch('Q');
End;
Procedure FetchURL;
Var
URL : String;
Host,Path : String;
Failed : Boolean;
Begin
Failed := False;
Try
URL := ParamStr(1);
OpenWinHTTPSession;
CrackURL(URL,Host,Path);
DoURLFetch(Host,Path);
WinHttpCloseHandle(Session);
If (Not Quiet) Then WriteLn('---------------'#13#10'Done.');
Except
On E : Exception do Begin
Failed := True;
WriteLn('HTTPGen: '+E.Message);
End;
End;
If Failed Then Halt(2);
End;
Begin
CheckParams;
FetchURL;
End.
Here, the code first reads the given command line parameters and then continues to initialize WinHTTP and then fetch the URL. Describing the WinHTTP API itself is beyond the scope of this article, but it's nice to know that all WinHTTP APIs begin with "WinHttp". This makes it easy to distinguish between custom function and WinHTTP API calls.
Before WinHTTP can be utilized, it needs to be initialized. This can be done with the WinHttpOpen function. The following code illustrates:
Procedure OpenWinHTTPSession;
Begin
Session := WinHttpOpen('WhirlWater HTTPGet 1.0',
WINHTTP_ACCESS_TYPE_DEFAULT_PROXY,nil,nil,0);
If (Session = nil) Then RaiseLastOSError;
End;
Note that if you are using an older version of Delphi such as Delphi 5, you can replace the RaiseLastOSError procedure call with RaiseLastWin32Error procedure. These procedures can be considered equal.
Cracking URLs
WinHTTP requires that URLs are "cracked" or divided to their parts before the given URL can be fetched. For example, when you want to connect a HTTP server with a call to WinHttpConnect, you need to have the host name, not the full URL.
Because writing your custom parsing procedures can be tedious, WinHTTP provides a convenient function named WinHttpCrackUrl. The following code snippet shows how to use this function in real code:
Procedure CrackURL(URL : String; Var Host,Path : String);
Var
Comp : TURL_COMPONENTS;
URLW : WideString;
Begin
FillChar(Comp,SizeOf(Comp),0);
With Comp do Begin
dwStructSize := SizeOf(Comp);
dwHostNameLength := -1;
dwUrlPathLength := -1;
dwExtraInfoLength := -1;
End;
URLW := URL;
If (Not WinHttpCrackUrl(PWideChar(URLW),0,0,Comp)) Then ShowWinHTTPError;
Host := Copy(Comp.lpszHostName,1,Comp.dwHostNameLength);
If (Comp.dwUrlPathLength = 0) Then Path := '/'
Else Path := WideString(Comp.lpszUrlPath);
End;
Note that the strings WinHTTP manipulates are mostly Unicode. Fortunately Delphi makes it easy to convert between Unicode and ANSI strings - you simply need to do an assignment.
Fetching URLs
Once you have an URL cracked, it's time to fetch the given resource and output it to the standard output (stdout). The following code does just this:
Procedure DoURLFetch(Host,Path : String);
Var
Connection : HINTERNET;
Request : HINTERNET;
Size : Integer;
Read,Status : Integer;
Buffer : PChar;
Headers : PWideChar;
Data,Header : String;
Begin
Connection := WinHttpConnect(Session,PWideChar(WideString(Host)),
INTERNET_DEFAULT_HTTP_PORT,0);
If (Connection = nil) Then ShowWinHTTPError;
Request := WinHttpOpenRequest(Connection,'GET',
PWideChar(WideString(Path)),nil,nil,
WINHTTP_DEFAULT_ACCEPT_TYPES,0);
If (Request = nil) Then ShowWinHTTPError;
If (Not Quiet) Then WriteLn('Sending HTTP request...');
If (Not WinHttpSendRequest(Request,WINHTTP_NO_ADDITIONAL_HEADERS,
0,WINHTTP_NO_REQUEST_DATA,0,0,nil)) Then ShowWinHTTPError;
If (Not Quiet) Then WriteLn('HTTP request sent...');
If (Not WinHttpReceiveResponse(Request,nil)) Then ShowWinHTTPError;
If (Not Quiet) Then WriteLn('HTTP response received...');
{ headers }
WinHttpQueryHeaders(Request,WINHTTP_QUERY_RAW_HEADERS_CRLF,
WINHTTP_HEADER_NAME_BY_INDEX,WINHTTP_NO_OUTPUT_BUFFER,
Size,WINHTTP_NO_HEADER_INDEX);
GetMem(Headers,Size*SizeOf(WideChar)+1);
Try
If (Not WinHttpQueryHeaders(Request,WINHTTP_QUERY_RAW_HEADERS_CRLF,
WINHTTP_HEADER_NAME_BY_INDEX,Headers,Size,
WINHTTP_NO_HEADER_INDEX)) Then ShowWinHTTPError
Else Header := WideCharToString(Headers);
Finally
FreeMem(Headers);
End;
If (Not Quiet) Then WriteLn('HTTP response headers reads...');
{ body }
If (Not WinHttpQueryDataAvailable(Request,Size)) Then ShowWinHTTPError
Else Begin
Data := '';
While (Size > 0) do Begin
GetMem(Buffer,Size+1);
Try
If (Not WinHttpReadData(Request,Buffer,Size,Read)) Then ShowWinHTTPError
Else Begin
Buffer[Read] := #0; { add string terminator }
Data := Data+Buffer;
End;
Finally
FreeMem(Buffer);
End;
If (Not WinHttpQueryDataAvailable(Request,Size)) Then ShowWinHTTPError;
End;
{ all data fetched, now get the HTTP status code }
Size := SizeOf(Status);
If (Not WinHttpQueryHeaders(Request,WINHTTP_QUERY_STATUS_CODE Or
WINHTTP_QUERY_FLAG_NUMBER,
WINHTTP_NO_OUTPUT_BUFFER,@Status,Size,
nil)) Then ShowWinHTTPError;
End;
If (Not Quiet) Then WriteLn('All HTTP data read...'#13#10'---------------');
WinHttpCloseHandle(Request);
WinHttpCloseHandle(Connection);
{ finish with the results }
Write(Header);
If (Not NoBody) Then WriteLn(Data);
End;
Here, the most important functions are WinHttpSendRequest, WinHttpReceiveResponse, WinHttpQueryHeaders and WinHttpReadData. Of course, you will need other functions as well, but focusing on these in the above code makes it easier to understand what's going on.
Once a request has been sent out with WinHttpSendRequest, the WinHttpReceiveResponse function can be called. This function blocks until the server responds or a timeout (by default, 60 seconds) occurs. If a response could be successfully read, then the example application enters a loop that fetches all the data from the response and writes it to the console. Also note how the WinHttpQueryHeaders is used to retrieve and display the HTTP headers.
The following picture illustrates a sample run with the -n switch:
Easy, aint it?
Download the example code
Download writingahttpclientwithwinhttp5.zip (33 kB) which contains the sample application HTTPGet. Please note that the sample application will require Delphi 6 or later as well as Microsoft WinHTTP SDK 5.0 installed. Future Microsoft .NET operating systems might contain the necessary DLLs already installed.
* * *
Need help developing cool Internet applications with Delphi? Contact us!