Win32 shared memory access between processes

Sometimes two Windows processes need to exchange data, and using a file, socket or some heavier IPC mechanism feels like unnecessary work. Windows provides another option: shared memory.

The Win32 API implements shared memory using file mapping objects. Despite the name, file mapping does not necessarily have to map an actual file on disk. If we create a mapping backed by the system paging file, two or more processes can simply map the same block of memory into their address spaces.

The basic Windows API functions we need are:

CreateFileMapping()
MapViewOfFile()
OpenFileMapping()
UnmapViewOfFile()
CloseHandle()

Suppose the first process wants to create a small area of shared memory:

HANDLE hMapping;
hMapping = CreateFileMapping(
  INVALID_HANDLE_VALUE,       // use paging file
  NULL,                       // default security
  PAGE_READWRITE, 0, 4096,    // size
  "MySharedMemory");

if (hMapping == NULL)
{
  // error
}

Passing INVALID_HANDLE_VALUE tells Windows that the mapping should be backed by the paging file instead of an ordinary file. The name, "MySharedMemory", allows another process to find the same mapping.

Next, we map the memory block into the current process' address space:

void *pMemory;
pMemory = MapViewOfFile(hMapping, FILE_MAP_ALL_ACCESS,
  0, 0, 0);

if (pMemory == NULL)
{
  // error
}

At this point pMemory behaves much like an ordinary pointer to a 4096-byte memory area.

For example:

strcpy((char *)pMemory, "Hello from process one");

Opening the memory from another process

The second process doesn't create the mapping. Instead, it opens the named mapping:

HANDLE hMapping;
hMapping = OpenFileMapping(FILE_MAP_ALL_ACCESS,
    FALSE, "MySharedMemory");

if (hMapping == NULL)
{
    // error
}

It then calls MapViewOfFile() just as the first process did:

char *pMemory;

pMemory = (char *)MapViewOfFile(hMapping,
    FILE_MAP_ALL_ACCESS, 0, 0, 0);

printf("%s\n", pMemory);

Both processes are now looking at the same underlying memory. One important detail is that the address returned by MapViewOfFile() does not have to be the same in both processes. Each process has its own virtual address space.

For that reason, putting ordinary pointers into shared memory is generally a bad idea.

For example, don't do this:

struct SharedData
{
    char *pszMessage;
};

The pointer may be meaningful in the process that created it, but meaningless in another process. Instead, put the actual data into the shared area, or use offsets relative to the beginning of the mapping.

A simple shared structure might look like this:

struct SharedData
{
    DWORD version;
    DWORD counter;
    char message[256];
};

Then both applications can interpret the shared block using the same structure definition.

Shared memory doesn't provide synchronization

There is an important catch: shared memory gives two processes access to the same bytes, but it doesn't prevent them from accessing those bytes at the same time.

For example:

Process A                   Process B

writes counter
                            reads counter
writes message
                            reads message

Process B could potentially see a mixture of old and new data. So, in most real applications, shared memory is used together with one or more synchronization objects. There are many ways to do this, and for example a named mutex is one possibility:

CreateMutex(NULL, FALSE, "MySharedMemoryMutex");

Because kernel synchronization objects can also be named, another process can open the same mutex and use it to protect access to the mapping.

Events can also be useful. One process might write data into shared memory and then signal an event to tell another process that new data is available.

I gave a general overview of these Win32 mechanisms in my earlier article, "Win32 thread synchronization primitives: a quick overview" from March 2002.

The same primitives are useful between processes as well as between threads.

What about C#?

This gets particularly interesting with .NET. There is no managed MemoryMappedFile class in the current .NET Framework, but we can still use Win32 shared memory through P/Invoke.

For example:

[DllImport("kernel32.dll", SetLastError=true, CharSet=CharSet.Auto)]
static extern IntPtr CreateFileMapping(IntPtr hFile,
    IntPtr lpAttributes, uint flProtect,
    uint dwMaximumSizeHigh, uint dwMaximumSizeLow,
    string lpName);

[DllImport("kernel32.dll", SetLastError=true)]
static extern IntPtr MapViewOfFile(IntPtr hFileMappingObject,
    uint dwDesiredAccess, uint dwFileOffsetHigh,
    uint dwFileOffsetLow, UIntPtr dwNumberOfBytesToMap);

There are corresponding declarations for OpenFileMapping(), UnmapViewOfFile() and CloseHandle().

The C# application can therefore open exactly the same named mapping as a native Win32 application. This is useful when gradually moving an existing application toward .NET. A native application and a managed application can communicate through the same shared-memory structure without requiring either side to be completely rewritten.

There are some details to watch out for, however. If structures are shared between C and C#, their memory layouts must agree. Field sizes, packing and character representation all matter.

For example, a native structure such as:

struct SharedData
{
    DWORD version;
    DWORD counter;
    char message[256];
};

...could be represented approximately like this in C#:

[StructLayout(LayoutKind.Sequential, CharSet=CharSet.Ansi)]
struct SharedData
{
    public uint Version;
    public uint Counter;

    [MarshalAs(UnmanagedType.ByValTStr, SizeConst=256)]
    public string Message;
}

Depending on what is being stored, it can sometimes be simpler to treat the mapping as a block of bytes and explicitly marshal values in and out.

When is shared memory useful?

Shared memory is particularly attractive when processes need to exchange a substantial amount of data frequently. Other IPC mechanisms may be more natural for other situations.

Window messages work well for small notifications between GUI applications. Pipes provide a convenient stream between processes. Sockets are useful when the same communication mechanism may eventually need to cross machine boundaries. COM provides a much higher-level object-oriented interface.

Shared memory is different because the operating system doesn't have to copy every piece of data through a messaging interface. Once both processes have mapped the same memory, they can access it directly. This can make it very fast to operate. The price is that the applications themselves have to agree on the format of the memory and on the synchronization protocol.

Cleaning up

When a process no longer needs the mapped view, it should call:

UnmapViewOfFile(pMemory);

And then:

CloseHandle(hMapping);

The underlying named mapping continues to exist as long as at least one process still has an open handle to it. Once the last handle is closed, Windows can destroy the mapping. This also means that there doesn't necessarily need to be a permanent "server" process. Whichever application starts first can create the mapping, and others can subsequently open it.

Shared memory is one of those Win32 facilities that becomes especially interesting when working with both native and managed code. The mechanism itself is fairly small: create a named mapping, map it into each process, and agree on the format of the bytes.

The more difficult part is usually designing the protocol around it: synchronization, versioning, structure layout and deciding who owns which data.

Happy Win32 hacking!