Friday, June 23, 2017

Hyper-V sockets and AfdConnect

  When we use sockets and call CONNECT method, it calls mswsock!WSPConnect. Then mswsock!WSPConnect calls NtDeviceIoControlFile. NtDeviceIoControlFile has following structure:
NTSTATUS WINAPI NtDeviceIoControlFile(
 _In_  HANDLE           FileHandle, - point to \Device\Afd
 _In_  HANDLE           Event,
 _In_  PIO_APC_ROUTINE  ApcRoutine, - 0
 _In_  PVOID            ApcContext, - 0
 _Out_ PIO_STATUS_BLOCK IoStatusBlock,- 0036F4C0
 _In_  ULONG            IoControlCode – 00012007 - IOCTL code
 _In_  PVOID            InputBuffer,- 0036F4E8
 _In_  ULONG            InputBufferLength, 30
 _Out_ PVOID            OutputBuffer,- 0
 _In_  ULONG            OutputBufferLength - 0
);
NtDeviceIoControlFile goes to execute of afd!AfdConnect function. It analyze InputBufferLength and, if it above AfdStandardAddressLength constant (eq 0x1C in Windows Server 2016) function, calls nt!ExAllocatePoolWithTagPriority
PVOID ExAllocatePoolWithTagPriority(
 _In_ POOL_TYPE        PoolType,- NonPagedPoolNx
 _In_ SIZE_T           NumberOfBytes,- (InputBufferLength-0xC)
 _In_ ULONG            Tag,- AfdR
 _In_ EX_POOL_PRIORITY Priority - LowPoolPriority
);

If we change InputBufferLength parameter of ntdll!NtDeviceIoControlFile from 0x30 to 0x4FFFFFFF before execution, we get:
kd> k
# Child-SP          RetAddr           Call Site
00 ffff8b80`7489c678 fffff805`53c46d9f nt!ExAllocatePoolWithTagPriority
01 ffff8b80`7489c680 fffff803`57293180 afd!AfdConnect+0x36f
02 ffff8b80`7489c820 fffff803`57292064 nt!IopSynchronousServiceTail+0x1a0
03 ffff8b80`7489c8e0 fffff803`572919e6 nt!IopXxxControlFile+0x674
04 ffff8b80`7489ca20 fffff803`56fd5493 nt!NtDeviceIoControlFile+0x56
05 ffff8b80`7489ca90 00000000`52dd222c nt!KiSystemServiceCopyEnd+0x13

kd> r
rcx=0000000000000200
rdx=000000004ffffff3 – buffer size
r8=0000000052646641  
r9=0000000000000000

kd> !poolused 2 AfdR
Sorting by NonPaged Pool Consumed
              NonPaged                  Paged
Tag     Allocs         Used     Allocs         Used
AfdR         6   1 342 178 160          0            0    Afd remote address buffer , Binary: afd.sys
TOTAL         6   1 342 178 160          0            0

Then user buffer (InputBuffer parameter) will be copy to allocated kernel pool by memove. If you don’t allocate user buffer page fault exception may be generated during of copy, especially if buffer size was set to hundreds of megabytes (exception will be handled by afd!AfdExceptionFilter). This kernel pool is freed by nt!ExFreePoolWithTag at the end of afd!AfdConnect.
It was tested with Hyper-V Sockets, but probably, it will work with standard TCP\IP sockets too.
I believe, that LowPoolPriority parameter of nt!ExAllocatePoolWithTagPriority will not allow operation system to hang because of limited kernel pool size.

Tested on afd version:

WINDBG>lmvm afd
   Image path: \SystemRoot\system32\drivers\afd.sys
   Image name: afd.sys
   Timestamp:        Sat Oct 15 06:53:45 2016 (5801A849)