Commit Graph

80813 Commits

Author SHA1 Message Date
Marcelo Tosatti
10589a4699 KVM: MMU: Concurrent guest walkers
Do not hold kvm->lock mutex across the entire pagefault code,
only acquire it in places where it is necessary, such as mmu
hash list, active list, rmap and parent pte handling.

Allow concurrent guest walkers by switching walk_addr() to use
mmap_sem in read-mode.

And get rid of the lockless __gfn_to_page.

[avi: move kvm_mmu_pte_write() locking inside the function]
[avi: add locking for real mode]
[avi: fix cmpxchg locking]

Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 18:01:20 +02:00
Avi Kivity
774ead3ad9 KVM: Disable vapic support on Intel machines with FlexPriority
FlexPriority accelerates the tpr without any patching.

Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 18:01:20 +02:00
Avi Kivity
b93463aa59 KVM: Accelerated apic support
This adds a mechanism for exposing the virtual apic tpr to the guest, and a
protocol for letting the guest update the tpr without causing a vmexit if
conditions allow (e.g. there is no interrupt pending with a higher priority
than the new tpr).

Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 18:01:20 +02:00
Avi Kivity
b209749f52 KVM: local APIC TPR access reporting facility
Add a facility to report on accesses to the local apic tpr even if the
local apic is emulated in the kernel.  This is basically a hack that
allows userspace to patch Windows which tends to bang on the tpr a lot.

Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 18:01:20 +02:00
Avi Kivity
565f1fbd9d KVM: Print data for unimplemented wrmsr
This can help diagnosing what the guest is trying to do.  In many cases
we can get away with partial emulation of msrs.

Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 18:01:20 +02:00
Avi Kivity
dfc5aa00cb KVM: MMU: Add cache miss statistic
Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 18:01:19 +02:00
Eddie Dong
caa5b8a5ed KVM: MMU: Coalesce remote tlb flushes
Host side TLB flush can be merged together if multiple
spte need to be write-protected.

Signed-off-by: Yaozu (Eddie) Dong <eddie.dong@intel.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 18:01:19 +02:00
Zhang Xiantao
ec10f4750d KVM: Expose ioapic to ia64 save/restore APIs
IA64 also needs to see ioapic structure in irqchip.

Signed-off-by: xiantao.zhang@intel.com <xiantao.zhang@intel.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 18:01:19 +02:00
Zhang Xiantao
5736199afb KVM: Move kvm_vcpu_kick() to x86.c
Moving kvm_vcpu_kick() to x86.c. Since it should be
common for all archs, put its declarations in <linux/kvm_host.h>

Signed-off-by: Zhang Xiantao <xiantao.zhang@intel.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 18:01:19 +02:00
Zhang Xiantao
0eb8f49848 KVM: Move ioapic code to common directory.
Move ioapic code to common, since IA64 also needs it.

Signed-off-by: Zhang Xiantao <xiantao.zhang@intel.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 18:01:19 +02:00
Zhang Xiantao
82470196fa KVM: Move irqchip declarations into new ioapic.h and lapic.h
This allows reuse of ioapic in ia64.

Signed-off-by: Zhang Xiantao <xiantao.zhang@intel.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 18:01:19 +02:00
Avi Kivity
0fce5623ba KVM: Move drivers/kvm/* to virt/kvm/
Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 18:01:18 +02:00
Avi Kivity
edf884172e KVM: Move arch dependent files to new directory arch/x86/kvm/
This paves the way for multiple architecture support.  Note that while
ioapic.c could potentially be shared with ia64, it is also moved.

Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 18:01:18 +02:00
Ryan Harper
9584bf2c93 KVM: VMX: Add printk_ratelimit in vmx_intr_assist
Add printk_ratelimit check in front of printk.  This prevents spamming
of the message during 32-bit ubuntu 6.06server install.  Previously, it
would hang during the partition formatting stage.

Signed-off-by: Ryan Harper <ryanh@us.ibm.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 17:58:10 +02:00
Zhang Xiantao
0711456c0d KVM: Portability: Move kvm_vm_stat to x86.h
This patch moves kvm_vm_stat to x86.h, and every arch
can define its own kvm_vm_stat in $arch.h

Signed-off-by: Zhang Xiantao <xiantao.zhang@intel.com>
Acked-by: Carsten Otte <cotte@de.ibm.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 17:58:10 +02:00
Zhang Xiantao
bfc6d222bd KVM: Portability: Move round_robin_prev_vcpu and tss_addr to kvm_arch
This patches moves two fields round_robin_prev_vcpu and tss to kvm_arch.

Signed-off-by: Zhang Xiantao <xiantao.zhang@intel.com>
Acked-by: Carsten Otte <cotte@de.ibm.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 17:58:10 +02:00
Zhang Xiantao
d7deeeb02c KVM: Portability: move vpic and vioapic to kvm_arch
This patches moves two fields vpid and vioapic to kvm_arch

Signed-off-by: Zhang Xiantao <xiantao.zhang@intel.com>
Acked-by: Carsten Otte <cotte@de.ibm.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 17:58:10 +02:00
Zhang Xiantao
f05e70ac03 KVM: Portability: Move mmu-related fields to kvm_arch
This patches moves mmu-related fields to kvm_arch.

Signed-off-by: Zhang Xiantao <xiantao.zhang@intel.com>
Acked-by: Carsten Otte <cotte@de.ibm.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 17:58:10 +02:00
Zhang Xiantao
d69fb81f05 KVM: Portability: Move memslot aliases to new struct kvm_arch
This patches create kvm_arch to hold arch-specific kvm fileds
and moves fields naliases and aliases to kvm_arch.

Signed-off-by: Zhang Xiantao <xiantao.zhang@intel.com>
Acked-by: Carsten Otte <cotte@de.ibm.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 17:58:10 +02:00
Zhang Xiantao
77b4c255af KVM: Portability: Move kvm_vcpu_stat to x86.h
This patches moves kvm_vcpu_stat to x86.h, so every
arch can define its own kvm_vcpu_stat structure.

Signed-off-by: Zhang Xiantao <xiantao.zhang@intel.com>
Acked-by: Carsten Otte <cotte@de.ibm.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 17:58:10 +02:00
Zhang Xiantao
d17fbbf738 KVM: Portability: Expand the KVM_VCPU_COMM in kvm_vcpu structure.
This patches removes KVM_COMM macro, original it is hold
kvm_vcpu common fields.

Signed-off-by: Zhang Xiantao <xiantao.zhang@intel.com>
Acked-by: Carsten Otte <cotte@de.ibm.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 17:58:09 +02:00
Zhang Xiantao
d657a98e3c KVM: Portability: Move kvm_vcpu definition back to kvm.h
This patches moves kvm_vcpu definition to kvm.h, and finally
kvm.h includes x86.h.

Signed-off-by: Zhang Xiantao <xiantao.zhang@intel.com>
Acked-by: Carsten Otte <cotte@de.ibm.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 17:58:09 +02:00
Zhang Xiantao
1d737c8a68 KVM: Portability: Split mmu-related static inline functions to mmu.h
Since these functions need to know the details of kvm or kvm_vcpu structure,
it can't be put in x86.h.  Create mmu.h to hold them.

Signed-off-by: Zhang Xiantao <xiantao.zhang@intel.com>
Acked-by: Carsten Otte <cotte@de.ibm.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 17:58:09 +02:00
Zhang Xiantao
ad312c7c79 KVM: Portability: Introduce kvm_vcpu_arch
Move all the architecture-specific fields in kvm_vcpu into a new struct
kvm_vcpu_arch.

Signed-off-by: Zhang Xiantao <xiantao.zhang@intel.com>
Acked-by: Carsten Otte <cotte@de.ibm.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 17:58:09 +02:00
Zhang Xiantao
682c59a3f3 KVM: Portability: Move kvm{pic,ioapic} accesors to x86 specific code
Signed-off-by: Zhang Xiantao <xiantao.zhang@intel.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 17:53:22 +02:00
Marcelo Tosatti
2bacc55c7c KVM: MMU: emulated cmpxchg8b should be atomic on i386
Emulate cmpxchg8b atomically on i386. This is required to avoid a guest
pte walker from seeing a splitted write.

[avi: make it compile]

Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 17:53:22 +02:00
Joerg Roedel
62b9abaaf8 KVM: SVM: support writing 0 to K8 performance counter control registers
This lets SVM ignore writes of the value 0 to the performance counter control
registers.  Thus enabling them will still fail in the guest, but a write of 0
which keeps them disabled is accepted.  This is required to boot Windows
Vista 64bit.

[avi: avoid fall-thru in switch statement]

Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
Signed-off-by: Markus Rechberger <markus.rechberger@amd.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 17:53:22 +02:00
Joerg Roedel
722f6ecbcf KVM: LAPIC: minor debugging compile fix
This patch fixes a compile error of the LAPIC code with APIC debugging enabled.

Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
Signed-off-by: Markus Rechberger <markus.rechberger@amd.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 17:53:21 +02:00
Marcelo Tosatti
7819026eef KVM: MMU: Fix SMP shadow instantiation race
There is a race where VCPU0 is shadowing a pagetable entry while VCPU1
is updating it, which results in a stale shadow copy.

Fix that by comparing the contents of the cached guest pte with the
current guest pte after write-protecting the guest pagetable.

Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 17:53:21 +02:00
Joerg Roedel
1d07543414 KVM: SVM: Exit to userspace if write to cr8 and not using in-kernel apic
With this patch KVM on SVM will exit to userspace if the guest writes to CR8
and the in-kernel APIC is disabled.

Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
Signed-off-by: Markus Rechberger <markus.rechberger@amd.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 17:53:21 +02:00
Avi Kivity
e833240f3c KVM: MMU: Use mmu_set_spte() for real-mode shadows
In addition to removing some duplicated code, this also handles the unlikely
case of real-mode code updating a guest page table.  This can happen when
one vcpu (in real mode) touches a second vcpu's (in protected mode) page
tables, or if a vcpu switches to real mode, touches page tables, and switches
back.

Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 17:53:21 +02:00
Avi Kivity
bc750ba860 KVM: MMU: Adjust mmu_set_spte() debug code for gpte removal
Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 17:53:21 +02:00
Avi Kivity
1c4f1fd6d5 KVM: MMU: Move set_pte() into guest paging mode independent code
As set_pte() no longer references either a gpte or the guest walker, we can
move it out of paging mode dependent code (which compiles twice and is
generally nasty).

Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 17:53:21 +02:00
Avi Kivity
2fbf4cf13f KVM: MMU: Remove walker argument to set_pte()
Unused.

Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 17:53:21 +02:00
Avi Kivity
e3f9550422 KVM: MMU: Pass pte dirty flag to set_pte() instead of calculating it on-site
This allows us to remove its dependency on pt_element_t.

Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 17:53:20 +02:00
Avi Kivity
b4ab019ce7 KVM: MMU: No need to pick up nx bit from guest pte
We already set it according to cumulative access permissions.

Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 17:53:20 +02:00
Avi Kivity
41074d07c7 KVM: MMU: Fix inherited permissions for emulated guest pte updates
When we emulate a guest pte write, we fail to apply the correct inherited
permissions from the parent ptes.  Now that we store inherited permissions
in the shadow page, we can use that to update the pte permissions correctly.

Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 17:53:20 +02:00
Avi Kivity
bedbe4ee86 KVM: MMU: Move pte access calculation into a helper function
Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 17:53:20 +02:00
Avi Kivity
8d87a03aea KVM: MMU: Set nx bit correctly on shadow ptes
While the page table walker correctly generates a guest page fault
if a guest tries to execute a non-executable page, the shadow code does
not mark it non-executable.  This means that if a guest accesses an nx
page first with a read access, then subsequent code fetch accesses will
succeed.

Fix by setting the nx bit on shadow ptes.

Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 17:53:20 +02:00
Avi Kivity
fe135d2ceb KVM: MMU: Simplify calculation of pte access
The nx bit is awkwardly placed in the 63rd bit position; furthermore it
has a reversed meaning compared to the other bits, which means we can't use
a bitwise and to calculate compounded access masks.

So, we simplify things by creating a new 3-bit exec/write/user access word,
and doing all calculations in that.

Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 17:53:20 +02:00
Marcelo Tosatti
b3e4e63fd9 KVM: MMU: Use cmpxchg for pte updates on walk_addr()
In preparation for multi-threaded guest pte walking, use cmpxchg()
when updating guest pte's. This guarantees that the assignment of the
dirty bit can't be lost if two CPU's are faulting the same address
simultaneously.

[avi: fix kunmap_atomic() parameters]

Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 17:53:19 +02:00
Avi Kivity
80a8119ca3 KVM: SVM: Trap access to the cr8 register
Later we may be able to use the virtual tpr feature, but for now,
just trap it.

Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 17:53:19 +02:00
Avi Kivity
6e3d5dfbad KVM: x86 emulator: Fix stack instructions on 64-bit mode
Stack instructions are always 64-bit on 64-bit mode; many of the
emulated stack instructions did not take that into account.  Fix by
adding a 'Stack' bitflag and setting the operand size appropriately
during the decode stage (except for 'push r/m', which is in a group
with a few other instructions, so it gets its own treatment).

This fixes random crashes on Vista x64.

Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 17:53:19 +02:00
Joerg Roedel
152ff9be2e KVM: SVM: Emulate read/write access to cr8
This patch adds code to emulate the access to the cr8 register to the x86
instruction emulator in kvm.  This is needed on svm, where there is no
hardware decode for control register access.

Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
Signed-off-by: Markus Rechberger <markus.rechberger@amd.com>
Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 17:53:19 +02:00
Avi Kivity
e5314067f6 KVM: VMX: Avoid exit when setting cr8 if the local apic is in the kernel
With apic in userspace, we must exit to userspace after a cr8 write in order
to update the tpr.  But if the apic is in the kernel, the exit is unnecessary.

Noticed by Joerg Roedel.

Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 17:53:19 +02:00
Avi Kivity
e934c9c1c8 KVM: x86 emulator: fix eflags preparation for emulation
We prepare eflags for the emulated instruction, then clobber it with an 'andl'.
Fix by popping eflags as the last thing in the sequence.

Patch taken from Xen (16143:959b4b92b6bf)

Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 17:53:19 +02:00
Avi Kivity
7ee5d940f5 KVM: Use generalized exception queue for injecting #UD
Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 17:53:18 +02:00
Avi Kivity
c1a5d4f990 KVM: Replace #GP injection by the generalized exception queue
Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 17:53:18 +02:00
Avi Kivity
c3c91fee51 KVM: Replace page fault injection by the generalized exception queue
Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 17:53:18 +02:00
Avi Kivity
298101da2f KVM: Generalize exception injection mechanism
Instead of each subarch doing its own thing, add an API for queuing an
injection, and manage failed exception injection centerally (i.e., if
an inject failed due to a shadow page fault, we need to requeue it).

Signed-off-by: Avi Kivity <avi@qumranet.com>
2008-01-30 17:53:18 +02:00