Hyper-V sockets and AfdConnect

  When we use sockets and call CONNECT method, it calls mswsock!WSPConnect. Then mswsock!WSPConnect calls NtDeviceIoControlFile. NtDeviceIoControlFile has following structure:

NTSTATUS WINAPI NtDeviceIoControlFile(

 _In_  HANDLE           FileHandle, - point to \Device\Afd

 _In_  HANDLE           Event,

 _In_  PIO_APC_ROUTINE  ApcRoutine, - 0

 _In_  PVOID            ApcContext, - 0

 _Out_ PIO_STATUS_BLOCK IoStatusBlock,- 0036F4C0

 _In_  ULONG            IoControlCode – 00012007 - IOCTL code

 _In_  PVOID            InputBuffer,- 0036F4E8

 _In_  ULONG            InputBufferLength, 30

 _Out_ PVOID            OutputBuffer,- 0

 _In_  ULONG            OutputBufferLength - 0

);

NtDeviceIoControlFile goes to execute of afd!AfdConnect function. It analyze InputBufferLength and, if it above AfdStandardAddressLength constant (eq 0x1C in Windows Server 2016) function, calls nt!ExAllocatePoolWithTagPriority

PVOID ExAllocatePoolWithTagPriority(

 _In_ POOL_TYPE        PoolType,- NonPagedPoolNx

 _In_ SIZE_T           NumberOfBytes,- (InputBufferLength-0xC)

 _In_ ULONG            Tag,- AfdR

 _In_ EX_POOL_PRIORITY Priority - LowPoolPriority

);

If we change InputBufferLength parameter of ntdll!NtDeviceIoControlFile from 0x30 to 0x4FFFFFFF before execution, we get:

kd> k

# Child-SP          RetAddr           Call Site

00 ffff8b80`7489c678 fffff805`53c46d9f nt!ExAllocatePoolWithTagPriority

01 ffff8b80`7489c680 fffff803`57293180 afd!AfdConnect+0x36f

02 ffff8b80`7489c820 fffff803`57292064 nt!IopSynchronousServiceTail+0x1a0

03 ffff8b80`7489c8e0 fffff803`572919e6 nt!IopXxxControlFile+0x674

04 ffff8b80`7489ca20 fffff803`56fd5493 nt!NtDeviceIoControlFile+0x56

05 ffff8b80`7489ca90 00000000`52dd222c nt!KiSystemServiceCopyEnd+0x13

kd> r

rcx=0000000000000200

rdx=000000004ffffff3 – buffer size

r8=0000000052646641  

r9=0000000000000000

kd> !poolused 2 AfdR

Sorting by NonPaged Pool Consumed

              NonPaged                  Paged

Tag     Allocs         Used     Allocs         Used

AfdR         6   1 342 178 160          0            0    Afd remote address buffer , Binary: afd.sys

TOTAL         6   1 342 178 160          0            0

Then user buffer (InputBuffer parameter) will be copy to allocated kernel pool by memove. If you don’t allocate user buffer page fault exception may be generated during of copy, especially if buffer size was set to hundreds of megabytes (exception will be handled by afd!AfdExceptionFilter). This kernel pool is freed by nt!ExFreePoolWithTag at the end of afd!AfdConnect.

It was tested with Hyper-V Sockets, but probably, it will work with standard TCP\IP sockets too.

I believe, that LowPoolPriority parameter of nt!ExAllocatePoolWithTagPriority will not allow operation system to hang because of limited kernel pool size.

Tested on afd version:

WINDBG>lmvm afd

   Image path: \SystemRoot\system32\drivers\afd.sys

   Image name: afd.sys

   Timestamp:        Sat Oct 15 06:53:45 2016 (5801A849)

Hyper-V Sockets and PnP

How does Powershell Direct session work internally? When application, which uses Hyper-V socket, makes CONNECT call, root partition sends message to guest OS using winhvr!WinHvPostMessage:

kd> !dc @rdx – before vmcall (part of message body)

#227b36000 00000001 00000000 00000001 000000c4 ................

#227b36010 00000001 00000000 999e53d4 4c3e3d5c .........S..\=>L

#227b36020 d0be7987 e156c06e

If we look at the parameters of vmicvmsession, we find out, that it has trigger for start.

After the message was delivered to guest OS, vmbus driver added new interface:

kd> kcn

# Call Site

00 nt!IoRegisterDeviceInterface

01 Wdf01000!Mx::MxRegisterDeviceInterface

02 Wdf01000!FxDeviceInterface::Register

03 Wdf01000!FxDeviceInterface::Register

04 Wdf01000!imp_WdfDeviceCreateDeviceInterface

05 vmbus!RootStartDeviceInterfaceByContext

06 hvsocket!VmbusTlXPartProcessNewConnection

07 vmbus!RootNotifyDeviceInterfaceArrival

08 Wdf01000!FxWorkItem::WorkItemHandler

09 Wdf01000!FxWorkItem::WorkItemThunk

0a nt!IopProcessWorkItem

0b nt!ExpWorkerThread

0c nt!PspSystemThreadStartup

0d nt!KiStartSystemThread

And PnP mechanism works:

kd> kcn

# Call Site

00 nt!PnpNotifyDeviceClassChange

01 nt!PnpDeviceEventWorker

02 nt!ExpWorkerThread

03 nt!PspSystemThreadStartup

Service trigger works and service starts. After that icsvc.dll configures socket and begins communication with Host OS.

It is interesting that we can change that GUID to GUID of another device. We can get list of services, that have trigger on DEVICE INTERFACE ARRIVAL event:

All Hyper-V guest services

Bluetooth Support Service

Windows Camera Frame Server

Human Interface Device Service

Geolocation Service

Microsoft Passport

Portable Device Enumerator Service

Sensor Service

Sensor Monitoring Service

Storage Service

Touch Keyboard and Handwriting (probably)

And if we point that GUID in CONNECT call

clsid_str = L"{53f56307-b6bf-11d0-94f2-00a0c91efb8b}";

CLSIDFromString(clsid_str, &ServiceID); //when you call CONNECT with that GUID Portable Device Enumerator Service will start in guest OS (even in shielded VM)

guest OS starts this service, because it believes, that triggered device was appeared in system.

One point: before do this you must add GUID to HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Virtualization\GuestCommunicationServices as key in Host OS (need local admin rights)

It works on Shielded VM too (tested in Admin-Trusted mode), besides Hyper-V Powershell Direct Service – it not started

   

               It very interesting to see integration between Hyper-V sockets and PnP subsystem in Windows.

   

Powershell Direct (few internals). Part 2

According https://docs.microsoft.com/en-us/windows-server/virtualization/hyper-v/manage/manage-windows-virtual-machines-with-powershell-direct Powershell Direct session needs Hyper-V administrator privilege for user, which start that session. Yes, it is true, if you use Enter-PSSession cmdlet, but if you will be use Hyper-V sockets natively you don’t need any privileges for it.