kernel-ark/include/linux
Lee Schermerhorn 480eccf9ae Fix NUMA Memory Policy Reference Counting
This patch proposes fixes to the reference counting of memory policy in the
page allocation paths and in show_numa_map().  Extracted from my "Memory
Policy Cleanups and Enhancements" series as stand-alone.

Shared policy lookup [shmem] has always added a reference to the policy,
but this was never unrefed after page allocation or after formatting the
numa map data.

Default system policy should not require additional ref counting, nor
should the current task's task policy.  However, show_numa_map() calls
get_vma_policy() to examine what may be [likely is] another task's policy.
The latter case needs protection against freeing of the policy.

This patch adds a reference count to a mempolicy returned by
get_vma_policy() when the policy is a vma policy or another task's
mempolicy.  Again, shared policy is already reference counted on lookup.  A
matching "unref" [__mpol_free()] is performed in alloc_page_vma() for
shared and vma policies, and in show_numa_map() for shared and another
task's mempolicy.  We can call __mpol_free() directly, saving an admittedly
inexpensive inline NULL test, because we know we have a non-NULL policy.

Handling policy ref counts for hugepages is a bit trickier.
huge_zonelist() returns a zone list that might come from a shared or vma
'BIND policy.  In this case, we should hold the reference until after the
huge page allocation in dequeue_hugepage().  The patch modifies
huge_zonelist() to return a pointer to the mempolicy if it needs to be
unref'd after allocation.

Kernel Build [16cpu, 32GB, ia64] - average of 10 runs:

		w/o patch	w/ refcount patch
	    Avg	  Std Devn	   Avg	  Std Devn
Real:	 100.59	    0.38	 100.63	    0.43
User:	1209.60	    0.37	1209.91	    0.31
System:   81.52	    0.42	  81.64	    0.34

Signed-off-by:  Lee Schermerhorn <lee.schermerhorn@hp.com>
Acked-by: Andi Kleen <ak@suse.de>
Cc: Christoph Lameter <clameter@sgi.com>
Acked-by: Mel Gorman <mel@csn.ul.ie>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-09-19 11:24:18 -07:00
..
amba
byteorder
dvb V4L/DVB (5893): DVB: fix includes of video.h when __KERNEL__ is undefined 2007-07-30 16:26:27 -03:00
hdlc
isdn
lockd
mlx4
mmc mmc: update kerneldoc 2007-07-26 01:53:39 +02:00
mtd
netfilter [NETFILTER]: Add xt_statistic.h to the header list for usermode programs 2007-08-04 21:18:16 -07:00
netfilter_arp
netfilter_bridge
netfilter_ipv4
netfilter_ipv6
nfsd
raid [BLOCK] Get rid of request_queue_t typedef 2007-07-24 09:28:11 +02:00
rtc
spi spi kerneldoc update 2007-07-31 15:39:44 -07:00
sunrpc
tc_act
tc_ematch
usb
8250_pci.h
a.out.h
ac97_codec.h
acct.h
acpi_pmtmr.h
acpi.h ACPI: autoload modules - Create ACPI alias interface 2007-07-23 13:56:16 -04:00
adb.h
adfs_fs_i.h
adfs_fs_sb.h
adfs_fs.h
aer.h PCI AER: fix warnings when PCIEAER=n 2007-09-11 04:22:16 -07:00
affs_hardblocks.h
agp_backend.h
agpgart.h
aio_abi.h
aio.h
amifd.h
amifdreg.h
amigaffs.h
anon_inodes.h
apm_bios.h [x86 setup] Make struct apm_bios_info cross-architecture 2007-07-25 12:02:21 -07:00
apm-emulation.h
arcdevice.h
arcfb.h
async_tx.h
ata.h ata: add ATA_MWDMA* and ATA_SWDMA* defines 2007-08-31 04:00:19 -04:00
atalk.h
atm_eni.h
atm_he.h
atm_idt77105.h
atm_nicstar.h
atm_suni.h
atm_tcp.h
atm_zatm.h
atm.h
atmapi.h
atmarp.h
atmbr2684.h
atmclip.h
atmdev.h
atmel_pdc.h
atmioc.h
atmlec.h
atmmpc.h
atmppp.h
atmsap.h
atmsvc.h
attribute_container.h
audit.h Renumber AUDIT_TTY_[GS]ET 2007-08-23 21:37:45 -07:00
auto_fs4.h
auto_fs.h
auxvec.h
ax25.h
b1lli.h
b1pcmcia.h
backing-dev.h
backlight.h
baycom.h
bcd.h
bfs_fs.h
binfmts.h
bio.h BLOCK: Hide the contents of linux/bio.h if CONFIG_BLOCK=n 2007-08-11 22:34:49 +02:00
bit_spinlock.h
bitmap.h
bitops.h
bitrev.h
blkdev.h [patch] QUEUE_FLAG_READFULL QUEUE_FLAG_WRITEFULL comment fix 2007-07-27 08:08:24 +02:00
blkpg.h
blktrace_api.h Fix remap handling by blktrace 2007-08-11 22:34:48 +02:00
blockgroup_lock.h
bootmem.h
bottom_half.h
bpqether.h
bsg.h [SCSI] bsg: update sg_io_v4 structure 2007-07-31 10:43:05 -05:00
buffer_head.h
bug.h
cache.h
calc64.h
capability.h
capi.h
cciss_ioctl.h
cd1400.h
cdev.h
cdk.h
cdrom.h
cfag12864b.h
chio.h
circ_buf.h
clk.h
clockchips.h timer: remove clockevents_unregister_notifier 2007-08-11 15:47:42 -07:00
clocksource.h
cm4000_cs.h
cn_proc.h
coda_cache.h
coda_fs_i.h
coda_linux.h
coda_psdev.h
coda.h
coff.h
com20020.h
compat.h
compiler-gcc3.h
compiler-gcc4.h
compiler-gcc.h
compiler-intel.h
compiler.h make __chk_{user,io}_ptr() accept pointers to volatile 2007-07-26 11:11:57 -07:00
completion.h
comstats.h
concap.h
configfs.h
connector.h
console_struct.h
console.h
consolemap.h
const.h
cpu.h PM: Fix dependencies of CONFIG_SUSPEND and CONFIG_HIBERNATION 2007-08-31 01:42:22 -07:00
cpufreq.h
cpumask.h
cpuset.h
cramfs_fs_sb.h
cramfs_fs.h
crash_dump.h
crc7.h
crc16.h
crc32.h
crc32c.h
crc-ccitt.h
crc-itu-t.h
crypto.h
cryptohash.h
ctype.h
cuda.h
cyclades.h
cyclomx.h
cycx_cfm.h
cycx_drv.h
cycx_x25.h
dcache.h
dccp.h
dcookies.h
debug_locks.h
debugfs.h
delay.h
delayacct.h
device-mapper.h
device.h Fix ThinkPad T42 poweroff failure introduced by by "PM: Introduce pm_power_off_prepare" 2007-07-26 12:13:06 -07:00
devpts_fs.h
dio.h
dirent.h
display.h
dlm_device.h
dlm_netlink.h
dlm.h
dm9000.h
dm-ioctl.h
dma-mapping.h
dmaengine.h
dmapool.h
dmi.h
dn.h
dnotify.h
dqblk_v1.h
dqblk_v2.h
dqblk_xfs.h
ds1wm.h
ds1286.h
ds17287rtc.h
dtlk.h
edac.h
edd.h
eeprom_93cx6.h
efi.h
efs_dir.h
efs_fs_i.h
efs_fs_sb.h
efs_fs.h
efs_vh.h
eisa.h
elevator.h [BLOCK] Get rid of request_queue_t typedef 2007-07-24 09:28:11 +02:00
elf-em.h
elf-fdpic.h
elf.h
elfcore.h
elfnote.h
err.h
errno.h
errqueue.h
etherdevice.h
ethtool.h [NET]: ethtool_perm_addr only has one implementation 2007-07-31 14:00:29 -07:00
eventfd.h
eventpoll.h
exportfs.h
ext2_fs_sb.h
ext2_fs.h
ext3_fs_i.h
ext3_fs_sb.h
ext3_fs.h
ext3_jbd.h
ext4_fs_extents.h
ext4_fs_i.h
ext4_fs_sb.h
ext4_fs.h
ext4_jbd2.h
fadvise.h
falloc.h
fault-inject.h
fb.h
fcdevice.h
fcntl.h
fd1772.h
fd.h
fddidevice.h
fdreg.h
fib_rules.h
file.h
filter.h
firewire-cdev.h
firewire-constants.h
firmware.h
flat.h
font.h
freezer.h Introduce CONFIG_SUSPEND for suspend-to-Ram and standby 2007-07-29 16:45:38 -07:00
fs_enet_pd.h
fs_stack.h
fs_struct.h
fs_uart_pd.h
fs.h [PATCH] remove mm/filemap.c:file_send_actor() 2007-08-11 22:34:47 +02:00
fsl_devices.h
fsnotify.h
fuse.h
futex.h
gameport.h
gen_stats.h
genalloc.h
generic_acl.h
generic_serial.h
genetlink.h
genhd.h [PARTITION] MSDOS: Fix Sun num_partitions handling. 2007-07-30 00:27:28 -07:00
getcpu.h
gfp.h
gfs2_ondisk.h
gigaset_dev.h
gpio_keys.h
gpio_mouse.h
hardirq.h
harrier_defs.h
hash.h
hayesesp.h
hdlc.h
hdlcdrv.h
hdpu_features.h
hdreg.h
hdsmart.h
hid-debug.h
hid.h
hiddev.h
highmem.h
highuid.h
hil_mlc.h
hil.h
hippidevice.h
hp_sdc.h
hpet.h
hrtimer.h
htirq.h
hugetlb.h hugepage: fix broken check for offset alignment in hugepage mappings 2007-08-31 01:42:23 -07:00
hw_random.h
hwmon-sysfs.h
hwmon-vid.h
hwmon.h
hysdn_if.h
i2c-algo-bit.h
i2c-algo-pca.h
i2c-algo-pcf.h
i2c-algo-sgi.h
i2c-dev.h
i2c-gpio.h
i2c-id.h
i2c-ocores.h
i2c-pnx.h
i2c-pxa.h
i2c.h i2c.h kernel-doc additions 2007-07-31 15:39:41 -07:00
i2o-dev.h
i2o.h
i8k.h
ibmtr.h
icmp.h
icmpv6.h
ide.h ide: add ide_dev_is_sata() helper (take 2) 2007-09-11 22:28:36 +02:00
idr.h
ieee80211.h
if_addr.h
if_arcnet.h
if_arp.h
if_bonding.h
if_bridge.h
if_cablemodem.h
if_ec.h
if_eql.h
if_ether.h
if_fc.h
if_fddi.h
if_frad.h
if_hippi.h
if_infiniband.h
if_link.h
if_ltalk.h
if_macvlan.h
if_packet.h
if_plip.h
if_ppp.h
if_pppol2tp.h [PPPOL2TP]: Fix endianness annotations. 2007-08-26 18:35:42 -07:00
if_pppox.h
if_shaper.h
if_slip.h
if_strip.h
if_tr.h
if_tun.h
if_tunnel.h
if_vlan.h
if_wanpipe.h
if.h
igmp.h
in6.h
in_route.h
in.h
inet_diag.h
inet.h
inetdevice.h
init_task.h
init.h x86_64: Don't mark __exitcall as __cold 2007-08-11 15:58:13 -07:00
initrd.h
inotify.h
input-polldev.h
input.h Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/dtor/input 2007-09-12 07:57:00 -07:00
interrupt.h rip some includes from linux/interrupt.h 2007-07-28 19:42:22 -07:00
io.h
ioc3.h
ioc4.h
ioctl.h
ioport.h
ioprio.h
ip6_tunnel.h
ip.h
ipc.h
ipmi_msgdefs.h
ipmi_smi.h
ipmi.h
ipsec.h
ipv6_route.h
ipv6.h [IPV6]: Remove circular dependency on if_inet6.h 2007-07-31 02:28:17 -07:00
ipx.h
irda.h
irq_cpustat.h
irq.h irq.h fix kernel-doc 2007-07-31 15:39:41 -07:00
irqflags.h
irqreturn.h
isa.h Fix non-ISA link error in drivers/scsi/advansys.c 2007-09-16 21:13:58 -07:00
isapnp.h
isdn_divertif.h
isdn_ppp.h
isdn.h
isdnif.h
isicom.h
iso_fs.h
istallion.h
ixjuser.h
jbd2.h
jbd.h
jffs2.h
jhash.h
jiffies.h
journal-head.h
joystick.h
kallsyms.h
kbd_diacr.h
kbd_kern.h m68k/mac: Make mac_hid_mouse_emulate_buttons() declaration visible 2007-08-22 19:52:45 -07:00
Kbuild [KBUILD]: Sanitize tc_ematch headers. 2007-08-26 18:35:34 -07:00
kd.h
kdebug.h kdebug.h: forward-declare struct struct notifier_block 2007-07-31 15:39:40 -07:00
kdev_t.h
kernel_stat.h
kernel.h PTR_ALIGN 2007-09-11 17:21:20 -07:00
kernelcapi.h
kexec.h
key-ui.h
key.h
keyboard.h Input: add more Braille keycodes 2007-09-04 23:16:04 -04:00
keyctl.h
kfifo.h
klist.h
kmalloc_sizes.h
kmod.h
kobj_map.h
kobject.h kset: kernel-doc cleanups 2007-07-30 14:25:12 -07:00
kprobes.h
kref.h
ks0108.h
kthread.h
ktime.h
kvm_para.h
kvm.h
lapb.h
latency.h
lcd.h
leds.h leds: Add missing include for leds.h 2007-09-11 22:24:45 +01:00
lguest_bus.h lguest: documentation III: Drivers 2007-07-26 11:35:17 -07:00
lguest_launcher.h lguest: documentation III: Drivers 2007-07-26 11:35:17 -07:00
lguest.h Provide timespec to guests rather than jiffies clock. 2007-07-28 19:54:33 -07:00
libata.h libata: implement BROKEN_HPA horkage and apply it to affected drives 2007-08-31 04:21:13 -04:00
libps2.h
license.h
limits.h
linkage.h
linux_logo.h
list.h
llc.h
lm_interface.h
lock_dlm_plock.h
lockdep.h
log2.h
loop.h loop.h build fix 2007-07-24 12:24:59 -07:00
lp.h
lzo.h
m41t00.h
m48t86.h
magic.h
major.h
matroxfb.h
mbcache.h
mc6821.h
mc146818rtc.h
mca-legacy.h
mca.h
memory_hotplug.h
memory.h
mempolicy.h Fix NUMA Memory Policy Reference Counting 2007-09-19 11:24:18 -07:00
mempool.h
meye.h
migrate.h
mii.h
minix_fs.h
miscdevice.h
mm_inline.h
mm_types.h
mm.h fix NULL pointer dereference in __vm_enough_memory() 2007-08-22 19:52:45 -07:00
mman.h
mmtimer.h
mmzone.h Apply memory policies to top two highest zones when highest zone is ZONE_MOVABLE 2007-08-22 19:52:47 -07:00
mnt_namespace.h
mod_devicetable.h Cross-compilation between e.g. i386 -> 64bit could break -> work around it 2007-08-17 15:16:37 -07:00
module.h
moduleloader.h
moduleparam.h
mount.h
mpage.h
mqueue.h
mroute.h
msdos_fs.h
msg.h
msi.h
mtio.h
mutex-debug.h
mutex.h
mv643xx.h [WATCHDOG] mv64x60_wdt: Add arch/powerpc platform support 2007-07-24 21:16:02 +00:00
n_r3964.h
namei.h
nbd.h
ncp_fs_i.h
ncp_fs_sb.h
ncp_fs.h
ncp_mount.h
ncp_no.h
ncp.h
neighbour.h
net.h
netdevice.h [NET]: Share correct feature code between bridging and bonding 2007-08-13 22:52:14 -07:00
netfilter_arp.h
netfilter_bridge.h
netfilter_decnet.h
netfilter_ipv4.h
netfilter_ipv6.h
netfilter.h [NETFILTER]: Fix/improve deadlock condition on module removal netfilter 2007-09-11 11:28:26 +02:00
netlink.h
netpoll.h
netrom.h
nfs2.h
nfs3.h
nfs4_acl.h
nfs4_mount.h
nfs4.h
nfs_fs_i.h
nfs_fs_sb.h
nfs_fs.h NFS: Fix a write request leak in nfs_invalidate_page() 2007-09-01 10:14:54 -04:00
nfs_idmap.h
nfs_mount.h
nfs_page.h
nfs_xdr.h
nfs.h
nfsacl.h
nfsd_idmap.h
nl80211.h
nls.h
nmi.h
node.h
nodemask.h
notifier.h [NET]: Allow netdev REGISTER/CHANGENAME events to fail 2007-07-31 02:28:15 -07:00
nsc_gpio.h
nsproxy.h
nubus.h
numa.h
nvram.h
of_device.h
of_platform.h [POWERPC] Constify of_platform_driver match_table 2007-07-22 21:30:59 +10:00
of.h
oom.h
oprofile.h
page-flags.h
pagemap.h
pagevec.h
param.h
parport_pc.h
parport.h
parser.h
pata_platform.h
patchkey.h
pci_hotplug.h
pci_ids.h PCI: irq and pci_ids patch for Intel Tolapai 2007-09-11 04:22:17 -07:00
pci_regs.h
pci-acpi.h
pci.h PCI: make pcie_get_readrq visible in pci.h 2007-08-22 14:48:41 -07:00
pcieport_if.h
pda_power.h
percpu_counter.h
percpu.h
personality.h
pfkeyv2.h
pfn.h
pg.h
phantom.h
phonedev.h
phy.h
pid_namespace.h
pid.h
pipe_fs_i.h
pkt_cls.h
pkt_sched.h
pktcdvd.h
platform_device.h
plist.h
pm_legacy.h
pm.h Fallout from "Remove fs.h from mm.h" patch 2007-07-30 12:25:49 -07:00
pmu.h [POWERPC] Fix undefined reference to device_power_up/resume 2007-08-25 16:58:27 +10:00
pnp.h PNP: fix up after Lindent 2007-07-26 11:35:21 -07:00
pnpbios.h PNP: fix up after Lindent 2007-07-26 11:35:21 -07:00
poison.h
poll.h Fix select on /proc files without ->poll 2007-09-11 17:21:20 -07:00
posix_acl_xattr.h
posix_acl.h
posix_types.h
posix-timers.h
power_supply.h
ppdev.h
ppp_channel.h
ppp_defs.h
ppp-comp.h
prctl.h
preempt.h [PATCH] sched: arch preempt notifier mechanism 2007-07-26 13:40:43 +02:00
prefetch.h
prio_tree.h
proc_fs.h Remove unused struct proc_dir_entry::set 2007-08-11 15:47:40 -07:00
profile.h
ps2esdi.h
ptrace.h
qnx4_fs.h
qnxtypes.h
quicklist.h
quota.h
quotaio_v1.h
quotaio_v2.h
quotaops.h
radeonfb.h
radix-tree.h
raid_class.h
ramfs.h
random.h
raw.h
rbtree.h
rcupdate.h RCU: Remove prototype for nonexistent function synchronize_idle() 2007-08-11 15:47:41 -07:00
reboot.h
reciprocal_div.h
reiserfs_acl.h
reiserfs_fs_i.h
reiserfs_fs_sb.h
reiserfs_fs.h fix preprocessor idiocy in reiserfs 2007-07-28 19:42:22 -07:00
reiserfs_xattr.h
relay.h
resource.h
resume-trace.h
rfkill.h
rio_drv.h
rio_ids.h
rio_regs.h
rio.h
rmap.h
romfs_fs.h
root_dev.h
rose.h
route.h
rslib.h
rtc-v3020.h
rtc.h
rtmutex.h
rtnetlink.h [TCP]: Allow minimum RTO to be configurable via routing metrics. 2007-08-30 22:10:28 -07:00
rwsem-spinlock.h
rwsem.h
rxrpc.h
sc26198.h
scatterlist.h
scc.h
sched.h Fix user namespace exiting OOPs 2007-09-19 11:24:18 -07:00
screen_info.h
sctp.h
scx200_gpio.h
scx200.h
sdla.h
seccomp.h
securebits.h
security.h fix NULL pointer dereference in __vm_enough_memory() 2007-08-22 19:52:45 -07:00
selection.h selection.h: add tty_struct forward declaration 2007-08-22 19:52:47 -07:00
selinux_netlink.h
selinux.h
sem.h
seq_file.h
seqlock.h
serial167.h
serial_8250.h Use resource_size_t for serial port IO addresses 2007-07-24 12:24:58 -07:00
serial_core.h Use resource_size_t for serial port IO addresses 2007-07-24 12:24:58 -07:00
serial_pnx8xxx.h
serial_reg.h Serial 8250: handle saving the clear-on-read bits from the LSR and MSR 2007-08-22 19:52:44 -07:00
serial.h
serialP.h
serio.h
shm.h
shmem_fs.h
signal.h x86: i386-show-unhandled-signals-v3 2007-07-22 11:03:37 -07:00
signalfd.h
skbuff.h [NET] skbuff: Add skb_cow_head 2007-09-16 16:21:16 -07:00
slab_def.h
slab.h
slob_def.h
slub_def.h SLUB: Force inlining for functions in slub_def.h 2007-08-31 01:09:21 -07:00
sm501-regs.h
sm501.h
smb_fs_i.h
smb_fs_sb.h
smb_fs.h
smb_mount.h
smb.h
smbno.h
smp_lock.h
smp.h
snmp.h
socket.h
sockios.h
som.h
sonet.h
sony-laptop.h
sonypi.h
sort.h
sound.h
soundcard.h
spinlock_api_smp.h
spinlock_api_up.h
spinlock_types_up.h
spinlock_types.h
spinlock_up.h
spinlock.h
splice.h
srcu.h
stacktrace.h
stallion.h
start_kernel.h
stat.h
statfs.h
stddef.h
stop_machine.h
string.h
stringify.h
superhyway.h
suspend.h Introduce CONFIG_SUSPEND for suspend-to-Ram and standby 2007-07-29 16:45:38 -07:00
svga.h
swap.h
swapops.h
synclink.h
sys.h
syscalls.h
sysctl.h Add CTL_PROC back 2007-07-31 15:39:39 -07:00
sysdev.h
sysfs.h
sysrq.h
sysv_fs.h
task_io_accounting_ops.h
task_io_accounting.h
taskstats_kern.h
taskstats.h
tc.h
tcp.h
telephony.h
termios.h
textsearch_fsm.h
textsearch.h
tfrc.h
thread_info.h
threads.h
ticable.h
tick.h
tifm.h
time.h Cache xtime every call to update_wall_time 2007-07-25 10:17:44 -07:00
timer.h
timerfd.h
times.h
timex.h
tiocl.h
tipc_config.h
tipc.h
topology.h [PATCH] sched: remove cache_hot_time 2007-08-02 17:41:40 +02:00
toshiba.h
transport_class.h
trdevice.h
tsacct_kern.h
tty_driver.h
tty_flip.h
tty_ldisc.h
tty.h fix serial buffer memory leak 2007-08-11 15:47:41 -07:00
types.h
uaccess.h
udf_fs_i.h
udf_fs_sb.h
udf_fs.h
udp.h
ufs_fs_i.h
ufs_fs_sb.h
ufs_fs.h
uinput.h
uio_driver.h
uio.h
ultrasound.h
umem.h
un.h
unistd.h
unwind.h
usb_gadget.h
usb_sl811.h
usb_usual.h
usb.h USB: usb.h kernel-doc additions 2007-07-30 13:27:44 -07:00
usbdevice_fs.h
user_namespace.h Convert uid hash to hlist 2007-09-19 11:24:18 -07:00
user.h
utime.h
uts.h
utsname.h
vermagic.h
vfs.h
via.h
video_decoder.h
video_encoder.h
video_output.h
videodev2.h V4L/DVB (5968): videodev2.h: remove superfluous FBUF GLOBAL_INV_ALPHA support 2007-08-20 12:18:01 -03:00
videodev.h
videotext.h
vmalloc.h
vmstat.h
vt_buffer.h
vt_kern.h
vt.h
wait.h
wanrouter.h
watchdog.h
wireless.h
workqueue.h
writeback.h
x25.h
xattr.h
xfrm.h
yam.h
zconf.h
zlib.h
zorro_ids.h
zorro.h
zutil.h