davidlt/kernel-ark
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
7a0268fa1a
kernel-ark/include/asm-powerpc
History
Anton Blanchard 7a0268fa1a [PATCH] powerpc/64: per cpu data optimisations 
The current ppc64 per cpu data implementation is quite slow. eg:

        lhz 11,18(13)           /* smp_processor_id() */
        ld 9,.LC63-.LCTOC1(30)  /* per_cpu__variable_name */
        ld 8,.LC61-.LCTOC1(30)  /* __per_cpu_offset */
        sldi 11,11,3            /* form index into __per_cpu_offset */
        mr 10,9
        ldx 9,11,8              /* __per_cpu_offset[smp_processor_id()] */
        ldx 0,10,9              /* load per cpu data */

5 loads for something that is supposed to be fast, pretty awful. One
reason for the large number of loads is that we have to synthesize 2
64bit constants (per_cpu__variable_name and __per_cpu_offset).

By putting __per_cpu_offset into the paca we can avoid the 2 loads
associated with it:

        ld 11,56(13)            /* paca->data_offset */
        ld 9,.LC59-.LCTOC1(30)  /* per_cpu__variable_name */
        ldx 0,9,11              /* load per cpu data

Longer term we can should be able to do even better than 3 loads.
If per_cpu__variable_name wasnt a 64bit constant and paca->data_offset
was in a register we could cut it down to one load. A suggestion from
Rusty is to use gcc's __thread extension here. In order to do this we
would need to free up r13 (the __thread register and where the paca
currently is). So far Ive had a few unsuccessful attempts at doing that :)

The patch also allocates per cpu memory node local on NUMA machines.
This patch from Rusty has been sitting in my queue _forever_ but stalled
when I hit the compiler bug. Sorry about that.

Finally I also only allocate per cpu data for possible cpus, which comes
straight out of the x86-64 port. On a pseries kernel (with NR_CPUS == 128)
and 4 possible cpus we see some nice gains:

             total       used       free     shared    buffers cached
Mem:       4012228     212860    3799368          0          0 162424

             total       used       free     shared    buffers cached
Mem:       4016200     212984    3803216          0          0 162424

A saving of 3.75MB. Quite nice for smaller machines. Note: we now have
to be careful of per cpu users that touch data for !possible cpus.

At this stage it might be worth making the NUMA and possible cpu
optimisations generic, but per cpu init is done so early we have to be
careful that all architectures have their possible map setup correctly.

Signed-off-by: Anton Blanchard <anton@samba.org>
Signed-off-by: Paul Mackerras <paulus@samba.org>
2006-01-11 14:49:45 +11:00
..
iseries [PATCH] powerpc: Remove ItLpRegSave area from the paca 2006-01-09 14:50:32 +11:00
8253pit.h [PATCH] powerpc: Standardize on _ASM_POWERPC header symbol prefix 2005-09-09 22:11:34 +10:00
a.out.h [PATCH] powerpc: Merge include/asm-ppc*/a.out.h into include/asm-powerpc 2005-09-21 19:21:08 +10:00
abs_addr.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
agp.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
asm-compat.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
atomic.h [PATCH] mutex subsystem, add atomic_xchg() to all arches 2006-01-09 15:59:17 -08:00
auxvec.h [PATCH] powerpc: Merge vdso's and add vdso support to 32 bits kernel 2005-11-11 22:25:39 +11:00
backlight.h powerpc: Merged asm/backlight.h 2005-10-11 09:59:38 +10:00
bitops.h [FLS64]: generic version 2006-01-03 13:11:06 -08:00
bootx.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
btext.h [PATCH] powerpc: Unify udbg (#2) 2006-01-09 14:49:54 +11:00
bug.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
bugs.h [PATCH] powerpc: Make check_bugs() static inline 2005-09-09 22:11:35 +10:00
byteorder.h [PATCH] powerpc: merge byteorder.h 2005-09-28 15:42:53 +10:00
cache.h [PATCH] Kill L1_CACHE_SHIFT_MAX 2006-01-08 20:13:39 -08:00
cacheflush.h [PATCH] powerpc: Merge cacheflush.h and cache.h 2005-11-10 13:09:22 +11:00
checksum.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
compat.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
cputable.h [PATCH] ppc64: Fix oprofile when compiled as a module 2006-01-09 16:02:52 +11:00
cputime.h [PATCH] Move the identical files from include/asm-ppc{,64} 2005-08-30 13:32:05 +10:00
current.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
dbdma.h powerpc: Merge various powermac-related header files. 2005-10-20 20:53:39 +10:00
delay.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
div64.h [PATCH] Move the identical files from include/asm-ppc{,64} 2005-08-30 13:32:05 +10:00
dma-mapping.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/paulus/powerpc-merge 2006-01-09 10:03:44 -08:00
dma.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
eeh_event.h [PATCH] powerpc: PCI Error Recovery: PPC64 core recovery routines 2006-01-10 15:28:32 +11:00
eeh.h [PATCH] powerpc: PCI Error Recovery: PPC64 core recovery routines 2006-01-10 15:28:32 +11:00
elf.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
emergency-restart.h [PATCH] Move the identical files from include/asm-ppc{,64} 2005-08-30 13:32:05 +10:00
errno.h [PATCH] powerpc: Standardize on _ASM_POWERPC header symbol prefix 2005-09-09 22:11:34 +10:00
fcntl.h [PATCH] Consolidate the asm-ppc*/fcntl.h files into asm-powerpc 2005-09-07 16:57:39 -07:00
firmware.h [PATCH] powerpc: Create a trampoline for the fwnmi vectors 2006-01-09 14:52:17 +11:00
floppy.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
futex.h [PATCH] powerpc: Consolidate asm compatibility macros 2005-11-10 13:10:38 +11:00
grackle.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
hardirq.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
heathrow.h powerpc: Fix some #ifndef __KERNEL__ that should be #ifdef 2006-01-09 15:14:05 +11:00
hvcall.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
hvconsole.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
hvcserver.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
hw_irq.h ppc64: remove ppc_irq_dispatch_handler 2005-11-09 16:19:53 +11:00
i8259.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
ibmebus.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
ide.h powerpc: merge ide.h 2005-11-08 12:20:34 +11:00
io.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
ioctl.h [PATCH] powerpc: Standardize on _ASM_POWERPC header symbol prefix 2005-09-09 22:11:34 +10:00
ioctls.h powerpc: remove duplicate ioctl definitions 2005-11-01 14:36:30 +11:00
iommu.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
ipc.h [PATCH] Move the identical files from include/asm-ppc{,64} 2005-08-30 13:32:05 +10:00
ipcbuf.h [PATCH] powerpc: Keep fixing merged ipcbuf.h 2005-11-03 16:58:17 +11:00
ipic.h [PATCH] powerpc: moved ipic code to arch/powerpc 2006-01-09 14:48:57 +11:00
irq.h [PATCH] ppc32: fix PQ2 PCI DMA interrupt handling 2005-11-11 22:19:56 +11:00
kdebug.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
kdump.h [PATCH] powerpc: Reroute interrupts from 0 + offset to PHYSICAL_START + offset 2006-01-09 14:52:21 +11:00
kexec.h [PATCH] Kdump: powerpc and s390 build failure fix 2006-01-10 08:01:27 -08:00
keylargo.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
kmap_types.h [PATCH] powerpc: Merge kmap_types.h 2005-09-19 09:38:49 +10:00
kprobes.h [PATCH] kprobes: fix build breakage 2006-01-10 08:01:40 -08:00
linkage.h [PATCH] powerpc: Standardize on _ASM_POWERPC header symbol prefix 2005-09-09 22:11:34 +10:00
lmb.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
local.h [PATCH] Move all the very similar files to asm-powerpc 2005-08-30 13:32:06 +10:00
lppaca.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
machdep.h [PATCH] cell: enable pause(0) in cpu_idle 2006-01-09 15:44:32 +11:00
macio.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
mc146818rtc.h [PATCH] powerpc: Standardize on _ASM_POWERPC header symbol prefix 2005-09-09 22:11:34 +10:00
mediabay.h powerpc: Merge various powermac-related header files. 2005-10-20 20:53:39 +10:00
mman.h [PATCH] madvise(MADV_REMOVE): remove pages from tmpfs shm backing store 2006-01-06 08:33:22 -08:00
mmu_context.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
mmu.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
mmzone.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
module.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
mpic.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
msgbuf.h [PATCH] powerpc: Merge a few more include files 2005-09-09 22:11:35 +10:00
mutex.h [PATCH] mutex subsystem, add default include/asm-*/mutex.h files 2006-01-09 15:59:19 -08:00
namei.h [PATCH] powerpc: Standardize on _ASM_POWERPC header symbol prefix 2005-09-09 22:11:34 +10:00
numnodes.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
nvram.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
of_device.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
ohare.h powerpc: Fix some #ifndef __KERNEL__ that should be #ifdef 2006-01-09 15:14:05 +11:00
oprofile_impl.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
paca.h [PATCH] powerpc/64: per cpu data optimisations 2006-01-11 14:49:45 +11:00
page_32.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
page_64.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
page.h [PATCH] powerpc: Add CONFIG_CRASH_DUMP 2006-01-09 14:52:14 +11:00
param.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
parport.h [PATCH] powerpc: parallel port init fix 2006-01-11 14:49:24 +11:00
pci-bridge.h [PATCH] powerpc: Save device BARs much earlier in the boot sequence 2006-01-10 15:30:39 +11:00
pci.h [PATCH] powerpc: make pcibios_claim_one_bus available to other code 2006-01-09 14:51:08 +11:00
percpu.h [PATCH] powerpc/64: per cpu data optimisations 2006-01-11 14:49:45 +11:00
pgalloc.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
pgtable-4k.h powerpc: Trivially merge several headers from asm-ppc64 to asm-powerpc 2005-11-19 20:17:32 +11:00
pgtable-64k.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
pgtable.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
pmac_feature.h [PATCH] 3/5 powerpc: Add platform functions interpreter 2006-01-09 15:47:18 +11:00
pmac_low_i2c.h [PATCH] 3/5 powerpc: Add platform functions interpreter 2006-01-09 15:47:18 +11:00
pmac_pfunc.h [PATCH] 3/5 powerpc: Add platform functions interpreter 2006-01-09 15:47:18 +11:00
pmc.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
poll.h [PATCH] powerpc: Standardize on _ASM_POWERPC header symbol prefix 2005-09-09 22:11:34 +10:00
posix_types.h [PATCH] powerpc: Merge asm-ppc*/posix_types.h 2005-09-21 19:21:07 +10:00
ppc_asm.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
ppc-pci.h [PATCH] powerpc: Save device BARs much earlier in the boot sequence 2006-01-10 15:30:39 +11:00
processor.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
prom.h [PATCH] powerpc: Remove device_node addrs/n_addr 2006-01-09 14:53:55 +11:00
pSeries_reconfig.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
ptrace.h [PATCH] syscall entry/exit revamp 2006-01-09 14:49:01 +11:00
reg_8xx.h [PATCH] powerpc: Merge cacheflush.h and cache.h 2005-11-10 13:09:22 +11:00
reg.h [PATCH] cell: enable pause(0) in cpu_idle 2006-01-09 15:44:32 +11:00
resource.h [PATCH] Move all the very similar files to asm-powerpc 2005-08-30 13:32:06 +10:00
rtas.h [PATCH] powerpc: Make early debugging configurable via Kconfig 2006-01-11 14:48:26 +11:00
rtc.h powerpc: Make set_rtc_time() return error code from lower-level function 2005-10-22 15:57:55 +10:00
rwsem.h Merge ../linux-2.6 by hand 2005-10-31 13:37:12 +11:00
scatterlist.h [PATCH] powerpc: merge scatterlist.h 2005-10-27 16:45:52 +10:00
seccomp.h powerpc: Fix some #ifndef __KERNEL__ that should be #ifdef 2006-01-09 15:14:05 +11:00
sections.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
semaphore.h Merge ../linux-2.6 by hand 2005-10-31 13:37:12 +11:00
sembuf.h [PATCH] powerpc: Standardize on _ASM_POWERPC header symbol prefix 2005-09-09 22:11:34 +10:00
serial.h powerpc: Introduce a new config symbol to control 16550 early debug code 2006-01-10 16:19:05 +11:00
setup.h [PATCH] powerpc: Merge a few more include files 2005-09-09 22:11:35 +10:00
shmbuf.h [PATCH] powerpc: Fix __power64__ typos that should be __powerpc64__ 2005-09-09 22:11:35 +10:00
shmparam.h [PATCH] powerpc: Standardize on _ASM_POWERPC header symbol prefix 2005-09-09 22:11:34 +10:00
sigcontext.h powerpc: merge sigcontext.h 2005-11-03 16:24:25 +11:00
siginfo.h [PATCH] ppc64: Add definitions for new PTRACE calls 2005-09-12 17:19:12 +10:00
signal.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
smp.h powerpc: Various UP build fixes 2005-11-07 13:18:13 +11:00
smu.h spelling: s/retreive/retrieve/ 2006-01-10 00:10:13 +01:00
socket.h [PATCH] powerpc: Standardize on _ASM_POWERPC header symbol prefix 2005-09-09 22:11:34 +10:00
sockios.h [PATCH] powerpc: Standardize on _ASM_POWERPC header symbol prefix 2005-09-09 22:11:34 +10:00
sparsemem.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
spinlock_types.h [PATCH] powerpc: merge include/asm-ppc*/spinlock_types.h into include/asm-powerpc/spinlock_types.h 2005-09-21 19:21:09 +10:00
spinlock.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
spu_csa.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
spu.h [PATCH] spufs: set irq affinity for running threads 2006-01-09 15:44:57 +11:00
sstep.h powerpc: Make single-stepping emulation (mostly) usable on 32-bit 2005-10-28 22:48:08 +10:00
stat.h powerpc: merge stat.h 2005-11-03 16:02:23 +11:00
statfs.h [PATCH] powerpc: merge include/asm-ppc*/statfs.h into include/asm-powerpc/statfs.h 2005-09-21 19:21:10 +10:00
string.h [PATCH] powerpc: Standardize on _ASM_POWERPC header symbol prefix 2005-09-09 22:11:34 +10:00
synch.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
system.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
tce.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
termbits.h [PATCH] powerpc: Standardize on _ASM_POWERPC header symbol prefix 2005-09-09 22:11:34 +10:00
termios.h powerpc: use asm-generic/termios.h 2005-11-01 14:36:55 +11:00
thread_info.h [PATCH] syscall entry/exit revamp 2006-01-09 14:49:01 +11:00
time.h merge filename and modify references to iseries/hv_call.h 2005-11-01 16:59:20 +11:00
timex.h [PATCH] ppc32: Fix timekeeping 2005-10-12 08:24:47 -07:00
tlb.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
tlbflush.h [PATCH] ppc64: support 64k pages 2005-11-06 16:56:47 -08:00
topology.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
types.h [PATCH] ppc64: make dma_addr_t 64 bits 2005-10-27 16:45:50 +10:00
uaccess.h [PATCH] powerpc: Consolidate asm compatibility macros 2005-11-10 13:10:38 +11:00
ucontext.h powerpc: merge ucontext.h 2005-11-03 16:59:17 +11:00
udbg.h [PATCH] powerpc: Make early debugging configurable via Kconfig 2006-01-11 14:48:26 +11:00
unaligned.h [PATCH] powerpc: Standardize on _ASM_POWERPC header symbol prefix 2005-09-09 22:11:34 +10:00
uninorth.h powerpc: Merge various powermac-related header files. 2005-10-20 20:53:39 +10:00
unistd.h powerpc: Update __NR_syscalls to account for SPU syscalls 2006-01-09 14:49:34 +11:00
user.h [PATCH] powerpc: Merge a few more include files 2005-09-09 22:11:35 +10:00
vdso_datapage.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
vdso.h [PATCH] powerpc: Make the vDSO functions set error code (#2) 2005-11-16 14:05:11 +11:00
vga.h [PATCH] powerpc: Merge asm-ppc*/vga.h 2005-09-21 19:21:09 +10:00
vio.h [PATCH] powerpc: sanitize header files for user space includes 2006-01-09 15:13:08 +11:00
xmon.h powerpc: Simplify and clean up the xmon terminal I/O 2005-11-08 22:55:08 +11:00
xor.h [PATCH] Move the identical files from include/asm-ppc{,64} 2005-08-30 13:32:05 +10:00