I am dynamically loading my VxD using CreateFile but when I close my handle to the file the VxD remains in memory. How can I get this to unload correctly?
Automatic Unloading of VxDs
Many applications load VxDs with a call to the Win32 API CreateFile. The system recognizes the special string \\.\ at the start of a pathname as an indication of that the call refers to a device, and loads the corresponding VxD if it is not already loaded. However, developers often find that the VxD fails to unload when the file handle is closed. There are four common mistakes cause this:
The call to CreateFile must specify FILE_FLAG_DELETE_ON_CLOSE in the flags parameter. This tells the operating system that it should attempt to unload the VxD when the file handle is closed.
The name specified in the call to CreateFile must match the name that appears in the VxDs DDB. Occasionally, the file name of a VxD does not match the name given in the macro that declares the DDB (Declare_Virtual_Device). Without this correspondence, the system fails to unload the VxD. Note that the name is case sensitive. It is safest to always use upper case in the DDB declaration and in the call to CreateFile.
All handles to the VxD must be closed. The system maintains reference counts, so the VxD does not unload until all handles are closed.
The VxD must handle control messsage W32_DEVICEIOCONTROL, and must return DEVIOCTL_NOERROR (== 0) when the passed control code is DIOC_CLOSEHANDLE ( == -1). Failure to do this indicates to the operating system that the VxD did not expect to be closed, so it does not proceed with the unload.
If you avoid the above errors, then a VxD that your application loads by calling CreateFile should automatically unload when you call CloseHandle, or when your application terminates.