kernel-ark/arch/um/kernel/physmem.c
Jeff Dike d67b569f5f [PATCH] uml: skas0 - separate kernel address space on stock hosts
UML has had two modes of operation - an insecure, slow mode (tt mode) in
which the kernel is mapped into every process address space which requires
no host kernel modifications, and a secure, faster mode (skas mode) in
which the UML kernel is in a separate host address space, which requires a
patch to the host kernel.

This patch implements something very close to skas mode for hosts which
don't support skas - I'm calling this skas0.  It provides the security of
the skas host patch, and some of the performance gains.

The two main things that are provided by the skas patch, /proc/mm and
PTRACE_FAULTINFO, are implemented in a way that require no host patch.

For the remote address space changing stuff (mmap, munmap, and mprotect),
we set aside two pages in the process above its stack, one of which
contains a little bit of code which can call mmap et al.

To update the address space, the system call information (system call
number and arguments) are written to the stub page above the code.  The
%esp is set to the beginning of the data, the %eip is set the the start of
the stub, and it repeatedly pops the information into its registers and
makes the system call until it sees a system call number of zero.  This is
to amortize the cost of the context switch across multiple address space
updates.

When the updates are done, it SIGSTOPs itself, and the kernel process
continues what it was doing.

For a PTRACE_FAULTINFO replacement, we set up a SIGSEGV handler in the
child, and let it handle segfaults rather than nullifying them.  The
handler is in the same page as the mmap stub.  The second page is used as
the stack.  The handler reads cr2 and err from the sigcontext, sticks them
at the base of the stack in a faultinfo struct, and SIGSTOPs itself.  The
kernel then reads the faultinfo and handles the fault.

A complication on x86_64 is that this involves resetting the registers to
the segfault values when the process is inside the kill system call.  This
breaks on x86_64 because %rcx will contain %rip because you tell SYSRET
where to return to by putting the value in %rcx.  So, this corrupts $rcx on
return from the segfault.  To work around this, I added an
arch_finish_segv, which on x86 does nothing, but which on x86_64 ptraces
the child back through the sigreturn.  This causes %rcx to be restored by
sigreturn and avoids the corruption.  Ultimately, I think I will replace
this with the trick of having it send itself a blocked signal which will be
unblocked by the sigreturn.  This will allow it to be stopped just after
the sigreturn, and PTRACE_SYSCALLed without all the back-and-forth of
PTRACE_SYSCALLing it through sigreturn.

This runs on a stock host, so theoretically (and hopefully), tt mode isn't
needed any more.  We need to make sure that this is better in every way
than tt mode, though.  I'm concerned about the speed of address space
updates and page fault handling, since they involve extra round-trips to
the child.  We can amortize the round-trip cost for large address space
updates by writing all of the operations to the data page and having the
child execute them all at the same time.  This will help fork and exec, but
not page faults, since they involve only one page.

I can't think of any way to help page faults, except to add something like
PTRACE_FAULTINFO to the host.  There is PTRACE_SIGINFO, but UML doesn't use
siginfo for SIGSEGV (or anything else) because there isn't enough
information in the siginfo struct to handle page faults (the faulting
operation type is missing).  Adding that would make PTRACE_SIGINFO a usable
equivalent to PTRACE_FAULTINFO.

As for the code itself:

- The system call stub is in arch/um/kernel/sys-$(SUBARCH)/stub.S.  It is
  put in its own section of the binary along with stub_segv_handler in
  arch/um/kernel/skas/process.c.  This is manipulated with run_syscall_stub
  in arch/um/kernel/skas/mem_user.c.  syscall_stub will execute any system
  call at all, but it's only used for mmap, munmap, and mprotect.

- The x86_64 stub calls sigreturn by hand rather than allowing the normal
  sigreturn to happen, because the normal sigreturn is a SA_RESTORER in
  UML's address space provided by libc.  Needless to say, this is not
  available in the child's address space.  Also, it does a couple of odd
  pops before that which restore the stack to the state it was in at the
  time the signal handler was called.

- There is a new field in the arch mmu_context, which is now a union.
  This is the pid to be manipulated rather than the /proc/mm file
  descriptor.  Code which deals with this now checks proc_mm to see whether
  it should use the usual skas code or the new code.

- userspace_tramp is now used to create a new host process for every UML
  process, rather than one per UML processor.  It checks proc_mm and
  ptrace_faultinfo to decide whether to map in the pages above its stack.

- start_userspace now makes CLONE_VM conditional on proc_mm since we need
  separate address spaces now.

- switch_mm_skas now just sets userspace_pid[0] to the new pid rather
  than PTRACE_SWITCH_MM.  There is an addition to userspace which updates
  its idea of the pid being manipulated each time around the loop.  This is
  important on exec, when the pid will change underneath userspace().

- The stub page has a pte, but it can't be mapped in using tlb_flush
  because it is part of tlb_flush.  This is why it's required for it to be
  mapped in by userspace_tramp.

Other random things:

- The stub section in uml.lds.S is page aligned.  This page is written
  out to the backing vm file in setup_physmem because it is mapped from
  there into user processes.

- There's some confusion with TASK_SIZE now that there are a couple of
  extra pages that the process can't use.  TASK_SIZE is considered by the
  elf code to be the usable process memory, which is reasonable, so it is
  decreased by two pages.  This confuses the definition of
  USER_PGDS_IN_LAST_PML4, making it too small because of the rounding down
  of the uneven division.  So we round it to the nearest PGDIR_SIZE rather
  than the lower one.

- I added a missing PT_SYSCALL_ARG6_OFFSET macro.

- um_mmu.h was made into a userspace-usable file.

- proc_mm and ptrace_faultinfo are globals which say whether the host
  supports these features.

- There is a bad interaction between the mm.nr_ptes check at the end of
  exit_mmap, stack randomization, and skas0.  exit_mmap will stop freeing
  pages at the PGDIR_SIZE boundary after the last vma.  If the stack isn't
  on the last page table page, the last pte page won't be freed, as it
  should be since the stub ptes are there, and exit_mmap will BUG because
  there is an unfreed page.  To get around this, TASK_SIZE is set to the
  next lowest PGDIR_SIZE boundary and mm->nr_ptes is decremented after the
  calls to init_stub_pte.  This ensures that we know the process stack (and
  all other process mappings) will be below the top page table page, and
  thus we know that mm->nr_ptes will be one too many, and can be
  decremented.

Things that need fixing:

- We may need better assurrences that the stub code is PIC.

- The stub pte is set up in init_new_context_skas.

- alloc_pgdir is probably the right place.

Signed-off-by: Jeff Dike <jdike@addtoit.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-07-07 18:23:44 -07:00

487 lines
10 KiB
C

/*
* Copyright (C) 2000 - 2003 Jeff Dike (jdike@addtoit.com)
* Licensed under the GPL
*/
#include "linux/mm.h"
#include "linux/rbtree.h"
#include "linux/slab.h"
#include "linux/vmalloc.h"
#include "linux/bootmem.h"
#include "linux/module.h"
#include "asm/types.h"
#include "asm/pgtable.h"
#include "kern_util.h"
#include "user_util.h"
#include "mode_kern.h"
#include "mem.h"
#include "mem_user.h"
#include "os.h"
#include "kern.h"
#include "init.h"
struct phys_desc {
struct rb_node rb;
int fd;
__u64 offset;
void *virt;
unsigned long phys;
struct list_head list;
};
static struct rb_root phys_mappings = RB_ROOT;
static struct rb_node **find_rb(void *virt)
{
struct rb_node **n = &phys_mappings.rb_node;
struct phys_desc *d;
while(*n != NULL){
d = rb_entry(*n, struct phys_desc, rb);
if(d->virt == virt)
return(n);
if(d->virt > virt)
n = &(*n)->rb_left;
else
n = &(*n)->rb_right;
}
return(n);
}
static struct phys_desc *find_phys_mapping(void *virt)
{
struct rb_node **n = find_rb(virt);
if(*n == NULL)
return(NULL);
return(rb_entry(*n, struct phys_desc, rb));
}
static void insert_phys_mapping(struct phys_desc *desc)
{
struct rb_node **n = find_rb(desc->virt);
if(*n != NULL)
panic("Physical remapping for %p already present",
desc->virt);
rb_link_node(&desc->rb, (*n)->rb_parent, n);
rb_insert_color(&desc->rb, &phys_mappings);
}
LIST_HEAD(descriptor_mappings);
struct desc_mapping {
int fd;
struct list_head list;
struct list_head pages;
};
static struct desc_mapping *find_mapping(int fd)
{
struct desc_mapping *desc;
struct list_head *ele;
list_for_each(ele, &descriptor_mappings){
desc = list_entry(ele, struct desc_mapping, list);
if(desc->fd == fd)
return(desc);
}
return(NULL);
}
static struct desc_mapping *descriptor_mapping(int fd)
{
struct desc_mapping *desc;
desc = find_mapping(fd);
if(desc != NULL)
return(desc);
desc = kmalloc(sizeof(*desc), GFP_ATOMIC);
if(desc == NULL)
return(NULL);
*desc = ((struct desc_mapping)
{ .fd = fd,
.list = LIST_HEAD_INIT(desc->list),
.pages = LIST_HEAD_INIT(desc->pages) });
list_add(&desc->list, &descriptor_mappings);
return(desc);
}
int physmem_subst_mapping(void *virt, int fd, __u64 offset, int w)
{
struct desc_mapping *fd_maps;
struct phys_desc *desc;
unsigned long phys;
int err;
fd_maps = descriptor_mapping(fd);
if(fd_maps == NULL)
return(-ENOMEM);
phys = __pa(virt);
desc = find_phys_mapping(virt);
if(desc != NULL)
panic("Address 0x%p is already substituted\n", virt);
err = -ENOMEM;
desc = kmalloc(sizeof(*desc), GFP_ATOMIC);
if(desc == NULL)
goto out;
*desc = ((struct phys_desc)
{ .fd = fd,
.offset = offset,
.virt = virt,
.phys = __pa(virt),
.list = LIST_HEAD_INIT(desc->list) });
insert_phys_mapping(desc);
list_add(&desc->list, &fd_maps->pages);
virt = (void *) ((unsigned long) virt & PAGE_MASK);
err = os_map_memory(virt, fd, offset, PAGE_SIZE, 1, w, 0);
if(!err)
goto out;
rb_erase(&desc->rb, &phys_mappings);
kfree(desc);
out:
return(err);
}
static int physmem_fd = -1;
static void remove_mapping(struct phys_desc *desc)
{
void *virt = desc->virt;
int err;
rb_erase(&desc->rb, &phys_mappings);
list_del(&desc->list);
kfree(desc);
err = os_map_memory(virt, physmem_fd, __pa(virt), PAGE_SIZE, 1, 1, 0);
if(err)
panic("Failed to unmap block device page from physical memory, "
"errno = %d", -err);
}
int physmem_remove_mapping(void *virt)
{
struct phys_desc *desc;
virt = (void *) ((unsigned long) virt & PAGE_MASK);
desc = find_phys_mapping(virt);
if(desc == NULL)
return(0);
remove_mapping(desc);
return(1);
}
void physmem_forget_descriptor(int fd)
{
struct desc_mapping *desc;
struct phys_desc *page;
struct list_head *ele, *next;
__u64 offset;
void *addr;
int err;
desc = find_mapping(fd);
if(desc == NULL)
return;
list_for_each_safe(ele, next, &desc->pages){
page = list_entry(ele, struct phys_desc, list);
offset = page->offset;
addr = page->virt;
remove_mapping(page);
err = os_seek_file(fd, offset);
if(err)
panic("physmem_forget_descriptor - failed to seek "
"to %lld in fd %d, error = %d\n",
offset, fd, -err);
err = os_read_file(fd, addr, PAGE_SIZE);
if(err < 0)
panic("physmem_forget_descriptor - failed to read "
"from fd %d to 0x%p, error = %d\n",
fd, addr, -err);
}
list_del(&desc->list);
kfree(desc);
}
EXPORT_SYMBOL(physmem_forget_descriptor);
EXPORT_SYMBOL(physmem_remove_mapping);
EXPORT_SYMBOL(physmem_subst_mapping);
void arch_free_page(struct page *page, int order)
{
void *virt;
int i;
for(i = 0; i < (1 << order); i++){
virt = __va(page_to_phys(page + i));
physmem_remove_mapping(virt);
}
}
int is_remapped(void *virt)
{
struct phys_desc *desc = find_phys_mapping(virt);
return(desc != NULL);
}
/* Changed during early boot */
unsigned long high_physmem;
extern unsigned long physmem_size;
void *to_virt(unsigned long phys)
{
return((void *) uml_physmem + phys);
}
unsigned long to_phys(void *virt)
{
return(((unsigned long) virt) - uml_physmem);
}
int init_maps(unsigned long physmem, unsigned long iomem, unsigned long highmem)
{
struct page *p, *map;
unsigned long phys_len, phys_pages, highmem_len, highmem_pages;
unsigned long iomem_len, iomem_pages, total_len, total_pages;
int i;
phys_pages = physmem >> PAGE_SHIFT;
phys_len = phys_pages * sizeof(struct page);
iomem_pages = iomem >> PAGE_SHIFT;
iomem_len = iomem_pages * sizeof(struct page);
highmem_pages = highmem >> PAGE_SHIFT;
highmem_len = highmem_pages * sizeof(struct page);
total_pages = phys_pages + iomem_pages + highmem_pages;
total_len = phys_len + iomem_pages + highmem_len;
if(kmalloc_ok){
map = kmalloc(total_len, GFP_KERNEL);
if(map == NULL)
map = vmalloc(total_len);
}
else map = alloc_bootmem_low_pages(total_len);
if(map == NULL)
return(-ENOMEM);
for(i = 0; i < total_pages; i++){
p = &map[i];
set_page_count(p, 0);
SetPageReserved(p);
INIT_LIST_HEAD(&p->lru);
}
max_mapnr = total_pages;
return(0);
}
struct page *phys_to_page(const unsigned long phys)
{
return(&mem_map[phys >> PAGE_SHIFT]);
}
struct page *__virt_to_page(const unsigned long virt)
{
return(&mem_map[__pa(virt) >> PAGE_SHIFT]);
}
phys_t page_to_phys(struct page *page)
{
return((page - mem_map) << PAGE_SHIFT);
}
pte_t mk_pte(struct page *page, pgprot_t pgprot)
{
pte_t pte;
pte_set_val(pte, page_to_phys(page), pgprot);
if(pte_present(pte))
pte_mknewprot(pte_mknewpage(pte));
return(pte);
}
/* Changed during early boot */
static unsigned long kmem_top = 0;
unsigned long get_kmem_end(void)
{
if(kmem_top == 0)
kmem_top = CHOOSE_MODE(kmem_end_tt, kmem_end_skas);
return(kmem_top);
}
void map_memory(unsigned long virt, unsigned long phys, unsigned long len,
int r, int w, int x)
{
__u64 offset;
int fd, err;
fd = phys_mapping(phys, &offset);
err = os_map_memory((void *) virt, fd, offset, len, r, w, x);
if(err) {
if(err == -ENOMEM)
printk("try increasing the host's "
"/proc/sys/vm/max_map_count to <physical "
"memory size>/4096\n");
panic("map_memory(0x%lx, %d, 0x%llx, %ld, %d, %d, %d) failed, "
"err = %d\n", virt, fd, offset, len, r, w, x, err);
}
}
#define PFN_UP(x) (((x) + PAGE_SIZE-1) >> PAGE_SHIFT)
extern int __syscall_stub_start, __binary_start;
void setup_physmem(unsigned long start, unsigned long reserve_end,
unsigned long len, unsigned long highmem)
{
unsigned long reserve = reserve_end - start;
int pfn = PFN_UP(__pa(reserve_end));
int delta = (len - reserve) >> PAGE_SHIFT;
int err, offset, bootmap_size;
physmem_fd = create_mem_file(len + highmem);
offset = uml_reserved - uml_physmem;
err = os_map_memory((void *) uml_reserved, physmem_fd, offset,
len - offset, 1, 1, 0);
if(err < 0){
os_print_error(err, "Mapping memory");
exit(1);
}
/* Special kludge - This page will be mapped in to userspace processes
* from physmem_fd, so it needs to be written out there.
*/
os_seek_file(physmem_fd, __pa(&__syscall_stub_start));
os_write_file(physmem_fd, &__syscall_stub_start, PAGE_SIZE);
bootmap_size = init_bootmem(pfn, pfn + delta);
free_bootmem(__pa(reserve_end) + bootmap_size,
len - bootmap_size - reserve);
}
int phys_mapping(unsigned long phys, __u64 *offset_out)
{
struct phys_desc *desc = find_phys_mapping(__va(phys & PAGE_MASK));
int fd = -1;
if(desc != NULL){
fd = desc->fd;
*offset_out = desc->offset;
}
else if(phys < physmem_size){
fd = physmem_fd;
*offset_out = phys;
}
else if(phys < __pa(end_iomem)){
struct iomem_region *region = iomem_regions;
while(region != NULL){
if((phys >= region->phys) &&
(phys < region->phys + region->size)){
fd = region->fd;
*offset_out = phys - region->phys;
break;
}
region = region->next;
}
}
else if(phys < __pa(end_iomem) + highmem){
fd = physmem_fd;
*offset_out = phys - iomem_size;
}
return(fd);
}
static int __init uml_mem_setup(char *line, int *add)
{
char *retptr;
physmem_size = memparse(line,&retptr);
return 0;
}
__uml_setup("mem=", uml_mem_setup,
"mem=<Amount of desired ram>\n"
" This controls how much \"physical\" memory the kernel allocates\n"
" for the system. The size is specified as a number followed by\n"
" one of 'k', 'K', 'm', 'M', which have the obvious meanings.\n"
" This is not related to the amount of memory in the host. It can\n"
" be more, and the excess, if it's ever used, will just be swapped out.\n"
" Example: mem=64M\n\n"
);
unsigned long find_iomem(char *driver, unsigned long *len_out)
{
struct iomem_region *region = iomem_regions;
while(region != NULL){
if(!strcmp(region->driver, driver)){
*len_out = region->size;
return(region->virt);
}
}
return(0);
}
int setup_iomem(void)
{
struct iomem_region *region = iomem_regions;
unsigned long iomem_start = high_physmem + PAGE_SIZE;
int err;
while(region != NULL){
err = os_map_memory((void *) iomem_start, region->fd, 0,
region->size, 1, 1, 0);
if(err)
printk("Mapping iomem region for driver '%s' failed, "
"errno = %d\n", region->driver, -err);
else {
region->virt = iomem_start;
region->phys = __pa(region->virt);
}
iomem_start += region->size + PAGE_SIZE;
region = region->next;
}
return(0);
}
__initcall(setup_iomem);
/*
* Overrides for Emacs so that we follow Linus's tabbing style.
* Emacs will notice this stuff at the end of the file and automatically
* adjust the settings for this buffer only. This must remain at the end
* of the file.
* ---------------------------------------------------------------------------
* Local variables:
* c-file-style: "linux"
* End:
*/