davidlt/kernel-ark
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
2bed6eeb8c
kernel-ark/lib
History
Dave Hansen 534acc057b lib: flexible array implementation 
Once a structure goes over PAGE_SIZE*2, we see occasional allocation
failures.  Some people have chosen to switch over to things like vmalloc()
that will let them keep array-like access to such a large structures.
But, vmalloc() has plenty of downsides.

Here's an alternative.  I think it's what Andrew was suggesting here:

	http://lkml.org/lkml/2009/7/2/518

I call it a flexible array.  It does all of its work in PAGE_SIZE bits, so
never does an order>0 allocation.  The base level has
PAGE_SIZE-2*sizeof(int) bytes of storage for pointers to the second level.
 So, with a 32-bit arch, you get about 4MB (4183112 bytes) of total
storage when the objects pack nicely into a page.  It is half that on
64-bit because the pointers are twice the size.  There's a table detailing
this in the code.

There are kerneldocs for the functions, but here's an
overview:

flex_array_alloc() - dynamically allocate a base structure
flex_array_free() - free the array and all of the
		    second-level pages
flex_array_free_parts() - free the second-level pages, but
			  not the base (for static bases)
flex_array_put() - copy into the array at the given index
flex_array_get() - copy out of the array at the given index
flex_array_prealloc() - preallocate the second-level pages
			between the given indexes to
			guarantee no allocs will occur at
			put() time.

We could also potentially just pass the "element_size" into each of the
API functions instead of storing it internally.  That would get us one
more base pointer on 32-bit.

I've been testing this by running it in userspace.  The header and patch
that I've been using are here, as well as the little script I'm using to
generate the size table which goes in the kerneldocs.

	http://sr71.net/~dave/linux/flexarray/

[akpm@linux-foundation.org: coding-style fixes]
Signed-off-by: Dave Hansen <dave@linux.vnet.ibm.com>
Reviewed-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2009-07-29 19:10:36 -07:00
..
lzo lzo: use get/put_unaligned_* helpers 2008-07-25 10:53:26 -07:00
reed_solomon lib: Remove unnecessary inclusions of asm/semaphore.h 2008-04-18 22:17:17 -04:00
zlib_deflate lib/: Spelling fixes 2008-02-03 17:48:52 +02:00
zlib_inflate bzip2/lzma: library support for gzip, bzip2 and lzma decompression 2009-01-04 15:53:34 -08:00
.gitignore
argv_split.c LIB: Replace inappropriate include of <linux/bug.h> 2007-10-20 00:26:10 +02:00
atomic64.c lib: export generic atomic64_t functions 2009-07-29 19:10:35 -07:00
audit.c [PATCH] audit signal recipients 2007-05-11 05:38:25 -04:00
bcd.c rtc: BCD codeshrink 2008-07-24 10:47:33 -07:00
bitmap.c bitmap: fix end condition in bitmap_find_free_region 2009-03-12 19:32:51 -07:00
bitrev.c lib: export bitrev16 2008-06-06 11:29:10 -07:00
bug.c allow bug table entries to use relative pointers (and use it on x86-64) 2008-12-16 18:40:32 +01:00
bust_spinlocks.c oops handling: ensure that any oops is flushed to the mtdoops console 2009-01-06 15:59:11 -08:00
check_signature.c uninline check_signature() 2007-07-16 09:05:50 -07:00
checksum.c lib/checksum.c: fix endianess bug 2009-06-19 14:58:13 +02:00
cmdline.c generic, memparse(): constify argument 2008-07-28 15:05:23 +02:00
cpumask.c x86: remove some alloc_bootmem_cpumask_var calling 2009-06-11 19:27:07 +03:00
crc7.c CRC7 support 2007-07-17 10:23:04 -07:00
crc16.c
crc32.c lib/: Spelling fixes 2008-02-03 17:48:52 +02:00
crc32defs.h
crc-ccitt.c
crc-itu-t.c CRC ITU-T V.41 2007-05-10 18:24:13 +02:00
crc-t10dif.c [SCSI] lib: Add support for the T10 (SCSI) Data Integrity Field CRC 2008-07-12 08:22:32 -05:00
ctype.c
debug_locks.c lockdep: warn about lockdep disabling after kernel taint 2009-04-12 16:10:51 +02:00
debugobjects.c debugobjects: delay free of internal objects 2009-03-17 12:28:30 +01:00
dec_and_lock.c atomic: only take lock when the counter drops to zero on UP as well 2009-06-16 19:47:47 -07:00
decompress_bunzip2.c kmemtrace, kbuild: fix slab.h dependency problem in lib/decompress_bunzip2.c 2009-04-03 12:22:45 +02:00
decompress_inflate.c kmemtrace, kbuild: fix slab.h dependency problem in lib/decompress_inflate.c 2009-04-03 12:21:36 +02:00
decompress_unlzma.c kmemtrace, kbuild: fix slab.h dependency problem in lib/decompress_unlzma.c 2009-04-03 12:22:59 +02:00
decompress.c bzip2/lzma: don't stop search at first unconfigured compression 2009-02-19 13:36:55 -08:00
devres.c [POWERPC] devres: Add devm_ioremap_prot() 2008-05-05 16:47:14 +10:00
div64.c add an inlined version of iter_div_u64_rem 2008-06-12 10:47:58 +02:00
dma-debug.c dma-debug: Fix the overlap() function to be correct and readable 2009-07-10 22:18:45 +02:00
dump_stack.c
dynamic_debug.c Dynamic debug: fix typo: -/-> 2009-07-28 13:45:22 -07:00
extable.c module: trim exception table on init free. 2009-06-12 21:47:04 +09:30
fault-inject.c Remove remaining unwinder code 2009-01-06 15:59:11 -08:00
find_last_bit.c bitmap: find_last_bit() 2009-01-01 10:12:19 +10:30
find_next_bit.c bitops: remove "optimizations" 2008-04-29 08:11:16 -07:00
flex_array.c lib: flexible array implementation 2009-07-29 19:10:36 -07:00
gcd.c lib: add lib/gcd.c 2009-06-18 13:04:05 -07:00
gen_crc32table.c
genalloc.c lib/genalloc.c: remove unmatched write_lock() in gen_pool_destroy 2009-06-16 19:47:53 -07:00
halfmd4.c
hexdump.c hexdump: remove the trailing space 2009-06-16 19:47:51 -07:00
hweight.c remove asm/bitops.h includes 2007-10-19 11:53:41 -07:00
idr.c cgroup: CSS ID support 2009-04-02 19:04:53 -07:00
inflate.c inflate: refactor inflate malloc code 2008-07-25 10:53:28 -07:00
int_sqrt.c
iomap_copy.c
iomap.c Use WARN() in lib/ 2008-07-26 12:00:07 -07:00
iommu-helper.c introduce generic iommu_num_pages function 2008-10-16 11:21:33 -07:00
ioremap.c lib/ioremap.c should #include <linux/io.h> 2007-10-17 08:42:50 -07:00
irq_regs.c
is_single_threaded.c CRED: Rename is_single_threaded() to is_wq_single_threaded() 2008-11-14 10:39:21 +11:00
kasprintf.c lib: move kasprintf to a separate file 2007-07-31 15:39:39 -07:00
Kconfig lib: Provide generic atomic64_t implementation 2009-06-15 13:27:38 +10:00
Kconfig.debug kmemleak: Allow the early log buffer to be configurable. 2009-06-25 10:16:13 +01:00
Kconfig.kgdb kgdb: remove the requirement for CONFIG_FRAME_POINTER 2008-08-01 08:39:34 -05:00
Kconfig.kmemcheck kmemcheck: make kconfig accessible for other architectures 2009-06-15 15:49:17 +02:00
kernel_lock.c sched: TIF_NEED_RESCHED -> need_reshed() cleanup 2009-03-06 12:48:55 +01:00
klist.c driver core: Remove completion from struct klist_node 2009-01-06 10:44:30 -08:00
kobject_uevent.c driver core: allow non-root users to listen to uevents 2009-04-16 16:17:09 -07:00
kobject.c kobject: make kset_create check kobject_set_name return value 2009-06-15 21:30:24 -07:00
kref.c kref: add kref_set() 2008-01-24 20:40:05 -08:00
libcrc32c.c libcrc32c: Fix "crc32c undefined" compilation error 2008-12-25 11:01:42 +11:00
list_debug.c list debugging: use WARN() instead of BUG() 2008-07-25 10:53:29 -07:00
lmb.c lmb: Rework lmb_dump_all() output 2009-02-11 13:38:00 +11:00
locking-selftest-hardirq.h
locking-selftest-mutex.h
locking-selftest-rlock-hardirq.h
locking-selftest-rlock-softirq.h
locking-selftest-rlock.h
locking-selftest-rsem.h
locking-selftest-softirq.h
locking-selftest-spin-hardirq.h
locking-selftest-spin-softirq.h
locking-selftest-spin.h
locking-selftest-wlock-hardirq.h
locking-selftest-wlock-softirq.h
locking-selftest-wlock.h
locking-selftest-wsem.h
locking-selftest.c locking: rename trace_softirq_[enter|exit] => lockdep_softirq_[enter|exit] 2009-03-13 01:32:36 +01:00
Makefile lib: flexible array implementation 2009-07-29 19:10:36 -07:00
nlattr.c Merge branch 'master' of git://git.kernel.org/pub/scm/linux/kernel/git/kaber/nf-next-2.6 2009-03-26 22:45:23 -07:00
parser.c vfs: Use const for kernel parser table 2008-10-13 10:10:37 -07:00
percpu_counter.c Merge branch 'core-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2009-01-06 17:10:04 -08:00
plist.c Use WARN() in lib/ 2008-07-26 12:00:07 -07:00
prio_heap.c lib: fix sparse shadowed variable warning 2009-01-06 15:59:11 -08:00
prio_tree.c
proportions.c Merge branch 'core-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2009-01-06 17:10:04 -08:00
radix-tree.c lib: do code optimization for radix_tree_lookup() and radix_tree_lookup_slot() 2009-06-16 19:47:49 -07:00
random32.c random32: seeding improvement 2008-07-30 16:29:19 -07:00
ratelimit.c __ratelimit() cpu flags can't be static 2008-07-28 16:30:20 -07:00
rational.c lib: isolate rational fractions helper function 2009-06-11 08:51:08 -07:00
rbtree.c rb_tree: remove redundant if()-condition in rb_erase() 2009-06-16 19:47:56 -07:00
reciprocal_div.c
rwsem-spinlock.c lib: remove fastcall from lib/* 2008-02-08 09:22:31 -08:00
rwsem.c x86: fix UML and -regparm=3 2008-01-30 13:33:00 +01:00
scatterlist.c scatterlist: make sure sg_miter_next() doesn't return 0 sized mappings 2009-04-22 08:35:09 +02:00
sha1.c [PATCH] Numerous fixes to kernel-doc info in source files. 2007-02-11 10:51:32 -08:00
show_mem.c lib: generic show_mem() 2008-07-26 12:00:10 -07:00
smp_processor_id.c cpumask: convert lib/smp_processor_id to new cpumask ops 2009-01-30 15:47:34 +01:00
sort.c generic swap(): lib/sort.c: rename swap to swap_func 2009-01-08 08:31:14 -08:00
spinlock_debug.c Use helpers to obtain task pid in printks 2007-10-19 11:53:43 -07:00
string_helpers.c [SCSI] lib: string_get_size(): don't hang on zero; no decimals on exact 2008-10-23 11:42:20 -05:00
string.c Add a new sysfs_streq() string comparison function 2008-05-01 08:03:59 -07:00
swiotlb.c swiotlb: change swiotlb_bus_to[phys,virt] prototypes 2009-04-08 16:18:38 +02:00
syscall.c task_current_syscall 2008-07-26 12:00:10 -07:00
textsearch.c remove CONFIG_KMOD from lib 2008-07-22 19:24:31 +10:00
ts_bm.c textsearch: ts_bm: support case insensitive searching in Boyer-Moore algorithm 2008-07-08 02:37:54 -07:00
ts_fsm.c textsearch: ts_fsm: return error on request for case insensitive search 2008-07-08 02:38:27 -07:00
ts_kmp.c textsearch: ts_kmp: support case insensitive searching in Knuth-Morris-Pratt algorithm 2008-07-08 02:38:09 -07:00
vsprintf.c vsprintf: introduce %pf format specifier 2009-04-29 20:55:55 +02:00