Not sure exactly when to call IOCancelIRP and if it is safe or not.
On Calling IoCancelIrp
IoCancelIrp must be used with caution in order to avoid attempts to cancel IRPs that have already been completed and freed. The rules for dealing with this condition depend on how the IRP in question was initialized.
Consider the case where a driver allocates and initializes an IRP, sends it to a lower device, and then waits for the IRP to complete. In other words, the driver expects the lower device to complete the IRP synchronously. The driver waits by calling KeWaitForSingleObject, passing an event object associated with the IRP.
The driver may not want to wait indefinitely. Fortunately, KeWaitForSingleObject allows its caller to specify a timeout period. If the event is not signaled before the timeout period expires, the service returns STATUS_TIMEOUT.
Upon return of STATUS_TIMEOUT, the driver needs to notify the lower device that the operation has timed out. The best way to do this is to call IoCancelIrp, which tells the lower device to cancel the IRP. The problem is that the lower device may complete the IRP just as the timer expires, and that could result in the IRP being freed or even reused by another device. It is the responsibility of the calling driver to either prevent the completion of the IRP from proceeding to its final deallocation, or to serialize the cancellation with the completion operation. The choice depends on how the IRP was created and initialized.
IRPs created with IoAllocateIrp
If a driver allocates an IRP with IoAllocateIrp, then the driver must set up a completion routine that returns STATUS_MORE_PROCESSING_REQUIRED. When a completion routine returns this value, the system suspends completion of the IRP at the IRP stack location associated with the completion routine. This is required regardless of whether or not the driver ever tries to cancel the IRP, because an IRP initialized by IoAllocateIrp lacks the information that the system would need to carry out the final stages of IRP completion.
In a driver that enforces a timeout period, the completion routine may set the event on which the calling driver is waiting, but must not call IoFreeIrp. If the completion routine were to free the IRP, the main thread's concurrent call IoCancelIrp could cause a page fault or data corruption. The solution is to require the main thread to regain control of the IRP after the lower device completes or cancels it, by means of a completion routine that prematurely terminates the completion process. Then can the main thread safely free the IRP.
Here is a fragment of DriverWorks code to illustrate how this is done:
KIrp I(KIrp::Allocate()); // uses IoAllocateIrp
TRUE, TRUE, TRUE);
// . . . set up more IRP parameters here . . .
timeout.QuadPart = -(50*1000*1000); // 5 seconds
if ( CompletionEvent.Wait(KernelMode, TRUE, &timeout) == STATUS_TIMEOUT )
status = I.Status();
And here is the completion routine:
NTSTATUS SynchCompletionRoutine(PDEVICE_OBJECT pDev, PIRP pIrp, PVOID Ctx)
KEvent* pEvent = static_cast(Ctx);
Note that the second call to KEvent::Wait inside the if clause for the timeout is necessary for serialization. By convention, the lower device must respond to the cancellation request in a timely manner, assuming that it has not already completed the IRP. If it has completed the IRP, then the IRP is not in a cancelable state, and setting its cancel flag is harmless. Either way, the completion routine will run and set the event on which the main thread is waiting. If the lower device neither responds to the cancel request nor completes the IRP, the thread is hung.
By the way, never call IoCancelIrp while holding the global cancel spin lock. Doing so will cause a deadlock because the system needs to take that lock before calling an IRP's cancel routine.
IRPs created with IoBuildDeviceIoControlRequest or IoBuildSynchronousFsdRequest
IRPs that a driver builds with IoBuildDeviceIoControlRequest or IoBuildSynchronousFsdRequest contain the information that the system needs to carry out the final stage of IRP completion. Specifically, such IRPs are put on a list of IRPs associated with the calling thread. As a result, a driver does not need to set up a completion routine that prematurely terminates completion processing. The system automatically sets the event provided by the driver, and then frees the IRP. The key point is that this final processing is done at IRQL=APC_LEVEL, in the context of the thread that created the IRP.
A driver can take advantage of the fact that the final completion processing is done in the original thread at raised IRQL to serialize completion processing and cancellation. Consider this fragment from the DDK's parallel port class driver:
KeInitializeEvent(&event, NotificationEvent, FALSE);
irp = IoBuildDeviceIoControlRequest(
NULL, 0, NULL, 0, TRUE, &event,
// note that no completion routine has been set up
timeout.QuadPart = -(50*1000*1000); // 5 seconds
status = KeWaitForSingleObject(&event, Executive, KernelMode, FALSE, &timeout);
if (status == STATUS_TIMEOUT)
if (KeReadStateEvent(&event) == 0)
KeWaitForSingleObject(&event, Executive, KernelMode, FALSE, NULL);
If the request times out, the driver raises IRQL to APC_LEVEL. The system cannot run the APC that performs final completion processing while IRQL is raised. Before calling IoCancelIrp, the driver checks the state of the event in order to determine if the APC ran just prior to raising IRQL. If the event is still not signaled, then it's safe to request cancellation because the IRP cannot have been freed. The driver lowers IRQL before entering the second wait. As above, the thread may hang if the lower device neither cancels nor completes the IRP.
It follows that if a driver builds a synchronous IRP with one of the above services, it should wait for the IRP's completion on the same thread. Otherwise, the APC does not ensure serialization on multiprocessor systems.
One final note: IRPs built with IoBuildAsynchronousFsdRequest behave as those built with IoAllocateIrp