davidlt/kernel-ark
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
2a82b8be8a
kernel-ark/fs/xfs
History
David Chinner 2a82b8be8a [XFS] Concurrent Multi-File Data Streams 
In media spaces, video is often stored in a frame-per-file format. When
dealing with uncompressed realtime HD video streams in this format, it is
crucial that files do not get fragmented and that multiple files a placed
contiguously on disk.

When multiple streams are being ingested and played out at the same time,
it is critical that the filesystem does not cross the streams and
interleave them together as this creates seek and readahead cache miss
latency and prevents both ingest and playout from meeting frame rate
targets.

This patch set creates a "stream of files" concept into the allocator to
place all the data from a single stream contiguously on disk so that RAID
array readahead can be used effectively. Each additional stream gets
placed in different allocation groups within the filesystem, thereby
ensuring that we don't cross any streams. When an AG fills up, we select a
new AG for the stream that is not in use.

The core of the functionality is the stream tracking - each inode that we
create in a directory needs to be associated with the directories' stream.
Hence every time we create a file, we look up the directories' stream
object and associate the new file with that object.

Once we have a stream object for a file, we use the AG that the stream
object point to for allocations. If we can't allocate in that AG (e.g. it
is full) we move the entire stream to another AG. Other inodes in the same
stream are moved to the new AG on their next allocation (i.e. lazy
update).

Stream objects are kept in a cache and hold a reference on the inode.
Hence the inode cannot be reclaimed while there is an outstanding stream
reference. This means that on unlink we need to remove the stream
association and we also need to flush all the associations on certain
events that want to reclaim all unreferenced inodes (e.g. filesystem
freeze).

SGI-PV: 964469
SGI-Modid: xfs-linux-melb:xfs-kern:29096a

Signed-off-by: David Chinner <dgc@sgi.com>
Signed-off-by: Barry Naujok <bnaujok@sgi.com>
Signed-off-by: Donald Douwsma <donaldd@sgi.com>
Signed-off-by: Christoph Hellwig <hch@infradead.org>
Signed-off-by: Tim Shimmin <tes@sgi.com>
Signed-off-by: Vlad Apostolov <vapo@sgi.com>
2007-07-14 15:40:53 +10:00
..
linux-2.6 [XFS] Concurrent Multi-File Data Streams 2007-07-14 15:40:53 +10:00
quota [XFS] Use generic shrinker interfaces in XFS. 2007-07-14 15:23:53 +10:00
support [XFS] reducing the number of random number functions. 2007-05-08 13:49:03 +10:00
Kbuild kbuild/xfs: introduce fs/xfs/Kbuild 2006-01-09 20:48:03 +01:00
Kconfig [PATCH] BLOCK: Make it possible to disable the block layer [try #6] 2006-09-30 20:52:31 +02:00
Makefile [XFS] Sort out some cosmetic differences between XFS trees. 2005-09-05 11:47:01 +10:00
Makefile-linux-2.6 [XFS] Concurrent Multi-File Data Streams 2007-07-14 15:40:53 +10:00
xfs_acl.c [XFS] Remove unused header files for MAC and CAP checking functionality. 2007-02-10 18:37:28 +11:00
xfs_acl.h [XFS] Resolve a namespace collision on remaining vtypes for FreeBSD 2006-06-09 17:07:12 +10:00
xfs_ag.h [XFS] Concurrent Multi-File Data Streams 2007-07-14 15:40:53 +10:00
xfs_alloc_btree.c [XFS] Lazy Superblock Counters 2007-07-14 15:28:50 +10:00
xfs_alloc_btree.h [XFS] Remove unused arguments from the XFS_BTREE_*_ADDR macros. 2007-02-10 18:37:33 +11:00
xfs_alloc.c [XFS] Lazy Superblock Counters 2007-07-14 15:28:50 +10:00
xfs_alloc.h [XFS] Lazy Superblock Counters 2007-07-14 15:28:50 +10:00
xfs_arch.h [XFS] Merge in trivial changes, sync up headers with userspace 2006-01-12 10:29:53 +11:00
xfs_attr_leaf.c [XFS] The last argument "lsn" of xfs_trans_commit() is always called with 2007-05-08 13:48:42 +10:00
xfs_attr_leaf.h [XFS] Add EA list callbacks for xfs kernel use. Cleanup some namespace 2006-09-28 11:01:37 +10:00
xfs_attr_sf.h [XFS] endianess annotations for xfs_attr_shortform_t 2006-03-17 17:29:25 +11:00
xfs_attr.c [XFS] The last argument "lsn" of xfs_trans_commit() is always called with 2007-05-08 13:48:42 +10:00
xfs_attr.h [XFS] Add EA list callbacks for xfs kernel use. Cleanup some namespace 2006-09-28 11:01:37 +10:00
xfs_behavior.c [XFS] remove bhv_lookup, _range version works aswell and has more useful 2006-09-28 10:58:52 +10:00
xfs_behavior.h [XFS] remove bhv_lookup, _range version works aswell and has more useful 2006-09-28 10:58:52 +10:00
xfs_bit.c [XFS] Kill off xfs_count_bits 2007-07-14 15:36:43 +10:00
xfs_bit.h [XFS] Kill off xfs_count_bits 2007-07-14 15:36:43 +10:00
xfs_bmap_btree.c [XFS] Remove a bunch of unused functions from XFS. 2007-02-10 18:37:40 +11:00
xfs_bmap_btree.h [XFS] Remove a bunch of unused functions from XFS. 2007-02-10 18:37:40 +11:00
xfs_bmap.c [XFS] Concurrent Multi-File Data Streams 2007-07-14 15:40:53 +10:00
xfs_bmap.h [XFS] Remove unused argument to xfs_bmap_finish 2007-02-10 18:37:16 +11:00
xfs_btree.c [XFS] endianess annotations for xfs_bmbt_key Trivial as there are no 2006-09-28 10:58:17 +10:00
xfs_btree.h [XFS] Simplify XFS min/max macros. 2007-07-14 15:36:53 +10:00
xfs_buf_item.c [XFS] Kill off xfs_count_bits 2007-07-14 15:36:43 +10:00
xfs_buf_item.h [XFS] Get rid of old 5.3/6.1 v1 log items. Cleanup patch sent in by Eric 2007-02-10 18:35:02 +11:00
xfs_clnt.h [XFS] Concurrent Multi-File Data Streams 2007-07-14 15:40:53 +10:00
xfs_da_btree.c [XFS] Remove a bunch of unused functions from XFS. 2007-02-10 18:37:40 +11:00
xfs_da_btree.h [XFS] Remove a bunch of unused functions from XFS. 2007-02-10 18:37:40 +11:00
xfs_dfrag.c [XFS] propogate return codes from flush routines 2007-05-08 13:49:27 +10:00
xfs_dfrag.h [XFS] Add parameters to xfs_bmapi() and xfs_bunmapi() to have them report 2006-06-09 14:48:12 +10:00
xfs_dinode.h [XFS] Concurrent Multi-File Data Streams 2007-07-14 15:40:53 +10:00
xfs_dir2_block.c [XFS] Reduce shouting by removing unnecessary macros from dir2 code. 2007-07-14 15:37:02 +10:00
xfs_dir2_block.h [XFS] Reduce shouting by removing unnecessary macros from dir2 code. 2007-07-14 15:37:02 +10:00
xfs_dir2_data.c [XFS] Reduce shouting by removing unnecessary macros from dir2 code. 2007-07-14 15:37:02 +10:00
xfs_dir2_data.h [XFS] Reduce shouting by removing unnecessary macros from dir2 code. 2007-07-14 15:37:02 +10:00
xfs_dir2_leaf.c [XFS] Reduce shouting by removing unnecessary macros from dir2 code. 2007-07-14 15:37:02 +10:00
xfs_dir2_leaf.h [XFS] Reduce shouting by removing unnecessary macros from dir2 code. 2007-07-14 15:37:02 +10:00
xfs_dir2_node.c [XFS] Reduce shouting by removing unnecessary macros from dir2 code. 2007-07-14 15:37:02 +10:00
xfs_dir2_node.h [XFS] Reduce shouting by removing unnecessary macros from dir2 code. 2007-07-14 15:37:02 +10:00
xfs_dir2_sf.c [XFS] Reduce shouting by removing unnecessary macros from dir2 code. 2007-07-14 15:37:02 +10:00
xfs_dir2_sf.h [XFS] Reduce shouting by removing unnecessary macros from dir2 code. 2007-07-14 15:37:02 +10:00
xfs_dir2_trace.c [XFS] Remove version 1 directory code. Never functioned on Linux, just 2006-06-20 13:04:51 +10:00
xfs_dir2_trace.h [XFS] Update license/copyright notices to match the prefered SGI 2005-11-02 14:58:39 +11:00
xfs_dir2.c [XFS] Reduce shouting by removing unnecessary macros from dir2 code. 2007-07-14 15:37:02 +10:00
xfs_dir2.h [XFS] Remove version 1 directory code. Never functioned on Linux, just 2006-06-20 13:04:51 +10:00
xfs_dmapi.h [XFS] Remove KERNEL_VERSION macros from xfs_dmapi.h 2006-11-11 18:05:06 +11:00
xfs_dmops.c [XFS] Remove version 1 directory code. Never functioned on Linux, just 2006-06-20 13:04:51 +10:00
xfs_error.c [XFS] reducing the number of random number functions. 2007-05-08 13:49:03 +10:00
xfs_error.h [XFS] Remove a bunch of unused functions from XFS. 2007-02-10 18:37:40 +11:00
xfs_extfree_item.c [XFS] Keep stack usage down for 4k stacks by using noinline. 2007-02-10 18:34:56 +11:00
xfs_extfree_item.h [XFS] cleanup the field types of some item format structures 2006-09-28 10:55:43 +10:00
xfs_filestream.c [XFS] Concurrent Multi-File Data Streams 2007-07-14 15:40:53 +10:00
xfs_filestream.h [XFS] Concurrent Multi-File Data Streams 2007-07-14 15:40:53 +10:00
xfs_fs.h [XFS] Concurrent Multi-File Data Streams 2007-07-14 15:40:53 +10:00
xfs_fsops.c [XFS] Concurrent Multi-File Data Streams 2007-07-14 15:40:53 +10:00
xfs_fsops.h [XFS] Write log dummy record when freezing filesystem 2006-01-11 15:30:08 +11:00
xfs_ialloc_btree.c [XFS] use NULL for pointer initialisation instead of zero-cast-to-ptr 2006-09-28 10:58:40 +10:00
xfs_ialloc_btree.h [XFS] Remove unused arguments from the XFS_BTREE_*_ADDR macros. 2007-02-10 18:37:33 +11:00
xfs_ialloc.c [XFS] Lazy Superblock Counters 2007-07-14 15:28:50 +10:00
xfs_ialloc.h [XFS] Lazy Superblock Counters 2007-07-14 15:28:50 +10:00
xfs_iget.c [XFS] Add lockdep support for XFS 2007-05-08 13:50:19 +10:00
xfs_imap.h [XFS] Update license/copyright notices to match the prefered SGI 2005-11-02 14:58:39 +11:00
xfs_inode_item.c [XFS] Keep stack usage down for 4k stacks by using noinline. 2007-02-10 18:34:56 +11:00
xfs_inode_item.h [XFS] cleanup the field types of some item format structures 2006-09-28 10:55:43 +10:00
xfs_inode.c [XFS] Concurrent Multi-File Data Streams 2007-07-14 15:40:53 +10:00
xfs_inode.h [XFS] Concurrent Multi-File Data Streams 2007-07-14 15:40:53 +10:00
xfs_inum.h [XFS] Update license/copyright notices to match the prefered SGI 2005-11-02 14:58:39 +11:00
xfs_iocore.c [XFS] Fix to prevent the notorious 'NULL files' problem after a crash. 2007-05-08 13:49:46 +10:00
xfs_iomap.c [XFS] Cleanup inode extent size hint extraction 2007-07-14 15:35:36 +10:00
xfs_iomap.h [XFS] Fix to prevent the notorious 'NULL files' problem after a crash. 2007-05-08 13:49:46 +10:00
xfs_itable.c Fix occurrences of "the the " 2007-05-09 08:57:56 +02:00
xfs_itable.h [XFS] pv 956241, author: nathans, rv: vapo - make ino validation checks 2006-09-28 11:06:15 +10:00
xfs_log_priv.h [XFS] Fixes the leak in reservation space because we weren't ungranting 2006-09-28 11:04:16 +10:00
xfs_log_recover.c [XFS] Lazy Superblock Counters 2007-07-14 15:28:50 +10:00
xfs_log_recover.h [XFS] Update license/copyright notices to match the prefered SGI 2005-11-02 14:58:39 +11:00
xfs_log.c [XFS] Lazy Superblock Counters 2007-07-14 15:28:50 +10:00
xfs_log.h [XFS] Remove several macros that are no longer used anywhere 2006-09-28 11:02:57 +10:00
xfs_mount.c [XFS] Use do_div() on 64 bit types. 2007-07-14 15:36:08 +10:00
xfs_mount.h [XFS] Concurrent Multi-File Data Streams 2007-07-14 15:40:53 +10:00
xfs_mru_cache.c [XFS] Concurrent Multi-File Data Streams 2007-07-14 15:40:53 +10:00
xfs_mru_cache.h [XFS] Concurrent Multi-File Data Streams 2007-07-14 15:40:53 +10:00
xfs_qmops.c [XFS] The last argument "lsn" of xfs_trans_commit() is always called with 2007-05-08 13:48:42 +10:00
xfs_quota.h [XFS] Fix uquota and oquota enforcement problems. 2007-05-08 13:49:33 +10:00
xfs_refcache.h [XFS] Cleanup cosmetic differences between source trees. 2005-11-03 16:14:31 +11:00
xfs_rename.c [XFS] The last argument "lsn" of xfs_trans_commit() is always called with 2007-05-08 13:48:42 +10:00
xfs_rtalloc.c [XFS] Don't grow filesystems past the size they can index. 2007-07-14 15:21:29 +10:00
xfs_rtalloc.h [XFS] Remove a bunch of unused functions from XFS. 2007-02-10 18:37:40 +11:00
xfs_rw.c [XFS] The last argument "lsn" of xfs_trans_commit() is always called with 2007-05-08 13:48:42 +10:00
xfs_rw.h [XFS] Cleanup inode extent size hint extraction 2007-07-14 15:35:36 +10:00
xfs_sb.h [XFS] Lazy Superblock Counters 2007-07-14 15:28:50 +10:00
xfs_trans_ail.c [XFS] Workaround log space issue by increasing XFS_TRANS_PUSH_AIL_RESTARTS 2007-02-10 18:35:52 +11:00
xfs_trans_buf.c [XFS] Remove version 1 directory code. Never functioned on Linux, just 2006-06-20 13:04:51 +10:00
xfs_trans_extfree.c [XFS] Remove version 1 directory code. Never functioned on Linux, just 2006-06-20 13:04:51 +10:00
xfs_trans_inode.c [XFS] Remove version 1 directory code. Never functioned on Linux, just 2006-06-20 13:04:51 +10:00
xfs_trans_item.c [XFS] Portability changes: remove prdev, stick to one diagnostic 2006-06-09 15:29:40 +10:00
xfs_trans_priv.h [XFS] Add lock annotations to xfs_trans_update_ail and 2006-09-28 11:04:07 +10:00
xfs_trans_space.h [XFS] Remove version 1 directory code. Never functioned on Linux, just 2006-06-20 13:04:51 +10:00
xfs_trans.c [XFS] Apply transaction delta counts atomically to incore counters 2007-07-14 15:32:09 +10:00
xfs_trans.h [XFS] Lazy Superblock Counters 2007-07-14 15:28:50 +10:00
xfs_types.h [XFS] Update license/copyright notices to match the prefered SGI 2005-11-02 14:58:39 +11:00
xfs_utils.c [XFS] propogate return codes from flush routines 2007-05-08 13:49:27 +10:00
xfs_utils.h [XFS] Resolve a namespace collision on remaining vtypes for FreeBSD 2006-06-09 17:07:12 +10:00
xfs_vfsops.c [XFS] Concurrent Multi-File Data Streams 2007-07-14 15:40:53 +10:00
xfs_vnodeops.c [XFS] Concurrent Multi-File Data Streams 2007-07-14 15:40:53 +10:00
xfs.h [XFS] Concurrent Multi-File Data Streams 2007-07-14 15:40:53 +10:00