davidlt/kernel-ark
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
7d3579e8e6
kernel-ark/fs
History
Michael Rubin f629d1c9bd mm: add account_page_writeback() 
To help developers and applications gain visibility into writeback
behaviour this patch adds two counters to /proc/vmstat.

  # grep nr_dirtied /proc/vmstat
  nr_dirtied 3747
  # grep nr_written /proc/vmstat
  nr_written 3618

These entries allow user apps to understand writeback behaviour over time
and learn how it is impacting their performance.  Currently there is no
way to inspect dirty and writeback speed over time.  It's not possible for
nr_dirty/nr_writeback.

These entries are necessary to give visibility into writeback behaviour.
We have /proc/diskstats which lets us understand the io in the block
layer.  We have blktrace for more in depth understanding.  We have
e2fsprogs and debugsfs to give insight into the file systems behaviour,
but we don't offer our users the ability understand what writeback is
doing.  There is no way to know how active it is over the whole system, if
it's falling behind or to quantify it's efforts.  With these values
exported users can easily see how much data applications are sending
through writeback and also at what rates writeback is processing this
data.  Comparing the rates of change between the two allow developers to
see when writeback is not able to keep up with incoming traffic and the
rate of dirty memory being sent to the IO back end.  This allows folks to
understand their io workloads and track kernel issues.  Non kernel
engineers at Google often use these counters to solve puzzling performance
problems.

Patch #4 adds a pernode vmstat file with nr_dirtied and nr_written

Patch #5 add writeback thresholds to /proc/vmstat

Currently these values are in debugfs. But they should be promoted to
/proc since they are useful for developers who are writing databases
and file servers and are not debugging the kernel.

The output is as below:

 # grep threshold /proc/vmstat
 nr_pages_dirty_threshold 409111
 nr_pages_dirty_background_threshold 818223

This patch:

This allows code outside of the mm core to safely manipulate page
writeback state and not worry about the other accounting.  Not using these
routines means that some code will lose track of the accounting and we get
bugs.

Modify nilfs2 to use interface.

Signed-off-by: Michael Rubin <mrubin@google.com>
Reviewed-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Reviewed-by: Wu Fengguang <fengguang.wu@intel.com>
Cc: KONISHI Ryusuke <konishi.ryusuke@lab.ntt.co.jp>
Cc: Jiro SEKIBA <jir@unicus.jp>
Cc: Dave Chinner <david@fromorbit.com>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Cc: Nick Piggin <nickpiggin@yahoo.com.au>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2010-10-26 16:52:06 -07:00
..
9p fs/9p: Don't use dotl version of mknod for dotu inode operations 2010-09-13 08:13:03 -05:00
adfs Merge branch 'vfs' of git://git.kernel.org/pub/scm/linux/kernel/git/arnd/bkl 2010-10-22 10:52:01 -07:00
affs Merge branch 'vfs' of git://git.kernel.org/pub/scm/linux/kernel/git/arnd/bkl 2010-10-22 10:52:01 -07:00
afs writeback: remove nonblocking/encountered_congestion references 2010-10-26 16:52:05 -07:00
autofs Merge branch 'vfs' of git://git.kernel.org/pub/scm/linux/kernel/git/arnd/bkl 2010-10-22 10:52:01 -07:00
autofs4 Merge branch 'llseek' of git://git.kernel.org/pub/scm/linux/kernel/git/arnd/bkl 2010-10-22 10:52:56 -07:00
befs fix typos concerning "initiali[zs]e" 2010-06-16 18:05:05 +02:00
bfs BKL: Remove BKL from BFS 2010-10-04 21:10:35 +02:00
btrfs Merge branch 'for-2.6.37/barrier' of git://git.kernel.dk/linux-2.6-block 2010-10-22 17:07:18 -07:00
cachefiles llseek: automatically add .llseek fop 2010-10-15 15:53:27 +02:00
ceph writeback: remove nonblocking/encountered_congestion references 2010-10-26 16:52:05 -07:00
cifs writeback: remove nonblocking/encountered_congestion references 2010-10-26 16:52:05 -07:00
coda Coda: replace BKL with mutex 2010-10-25 08:02:40 -07:00
configfs fix setattr error handling in sysfs, configfs 2010-06-04 17:16:29 -04:00
cramfs cramfs: only unlock new inodes 2010-08-18 01:01:33 -04:00
debugfs llseek: automatically add .llseek fop 2010-10-15 15:53:27 +02:00
devpts Simplify devpts_get_sb() failure exits 2010-05-21 18:31:12 -04:00
dlm Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/teigland/dlm 2010-10-22 17:33:16 -07:00
ecryptfs Merge branch 'for-next' of git://git.kernel.org/pub/scm/linux/kernel/git/jikos/trivial 2010-10-24 13:41:39 -07:00
efs
exofs exofs: Remove inode->i_count manipulation in exofs_new_inode 2010-10-25 18:03:07 +02:00
exportfs
ext2 Merge branch 'for-next' of git://git.kernel.org/pub/scm/linux/kernel/git/jikos/trivial 2010-10-24 13:41:39 -07:00
ext3 Merge branch 'for-2.6.37/barrier' of git://git.kernel.dk/linux-2.6-block 2010-10-22 17:07:18 -07:00
ext4 Merge branch 'for-2.6.37/barrier' of git://git.kernel.dk/linux-2.6-block 2010-10-22 17:07:18 -07:00
fat Merge branch 'for-2.6.37/barrier' of git://git.kernel.dk/linux-2.6-block 2010-10-22 17:07:18 -07:00
freevxfs BKL: remove BKL from freevxfs 2010-10-21 18:48:09 +02:00
fscache Add a dummy printk function for the maintenance of unused printks 2010-08-12 09:51:35 -07:00
fuse Merge branch 'llseek' of git://git.kernel.org/pub/scm/linux/kernel/git/arnd/bkl 2010-10-22 10:52:56 -07:00
gfs2 writeback: remove nonblocking/encountered_congestion references 2010-10-26 16:52:05 -07:00
hfs Merge branch 'vfs' of git://git.kernel.org/pub/scm/linux/kernel/git/arnd/bkl 2010-10-22 10:52:01 -07:00
hfsplus hfsplus: fix getxattr return value 2010-10-15 05:45:00 -07:00
hostfs hostfs: fix UML crash: remove f_spare from hostfs 2010-10-26 16:52:04 -07:00
hpfs Merge branch 'vfs' of git://git.kernel.org/pub/scm/linux/kernel/git/arnd/bkl 2010-10-22 10:52:01 -07:00
hppfs llseek: automatically add .llseek fop 2010-10-15 15:53:27 +02:00
hugetlbfs Merge branch 'hwpoison' of git://git.kernel.org/pub/scm/linux/kernel/git/ak/linux-mce-2.6 2010-10-26 10:13:10 -07:00
isofs BKL: Remove BKL from isofs 2010-10-04 21:10:45 +02:00
jbd Merge branch 'for-2.6.37/barrier' of git://git.kernel.dk/linux-2.6-block 2010-10-22 17:07:18 -07:00
jbd2 Merge branch 'for-2.6.37/barrier' of git://git.kernel.dk/linux-2.6-block 2010-10-22 17:07:18 -07:00
jffs2 BKL: Remove BKL from jffs2 2010-10-04 21:10:50 +02:00
jfs Merge branch 'for-next' of git://git.kernel.org/pub/scm/linux/kernel/git/jikos/trivial 2010-10-24 13:41:39 -07:00
lockd Merge branch 'for-2.6.37' of git://linux-nfs.org/~bfields/linux 2010-10-26 09:55:25 -07:00
logfs llseek: automatically add .llseek fop 2010-10-15 15:53:27 +02:00
minix minix: fix regression in minix_mkdir() 2010-09-09 18:57:25 -07:00
ncpfs ncpfs: Lock socket in ncpfs while setting its callbacks 2010-10-05 11:02:14 +02:00
nfs writeback: remove nonblocking/encountered_congestion references 2010-10-26 16:52:05 -07:00
nfs_common
nfsd Merge branch 'for-2.6.37' of git://linux-nfs.org/~bfields/linux 2010-10-26 09:55:25 -07:00
nilfs2 mm: add account_page_writeback() 2010-10-26 16:52:06 -07:00
nls
notify Merge branch 'llseek' of git://git.kernel.org/pub/scm/linux/kernel/git/arnd/bkl 2010-10-22 10:52:56 -07:00
ntfs BKL: Remove BKL from NTFS 2010-10-04 21:10:41 +02:00
ocfs2 Merge branch 'for-next' of git://git.kernel.org/pub/scm/linux/kernel/git/jikos/trivial 2010-10-24 13:41:39 -07:00
omfs Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/bcopeland/omfs 2010-08-10 11:47:36 -07:00
openpromfs
partitions Merge branch 'for-linus' of git://git.kernel.dk/linux-2.6-block 2010-10-25 07:45:10 -07:00
proc oom: fix locking for oom_adj and oom_score_adj 2010-10-26 16:52:05 -07:00
qnx4 BKL: remove BKL from qnx4 2010-10-21 18:48:04 +02:00
quota Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs-2.6 2010-08-10 11:26:52 -07:00
ramfs check ATTR_SIZE contraints in inode_change_ok 2010-08-09 16:47:39 -04:00
reiserfs writeback: remove nonblocking/encountered_congestion references 2010-10-26 16:52:05 -07:00
romfs llseek: automatically add .llseek fop 2010-10-15 15:53:27 +02:00
smbfs Merge branch 'vfs' of git://git.kernel.org/pub/scm/linux/kernel/git/arnd/bkl 2010-10-22 10:52:01 -07:00
squashfs Merge branch 'llseek' of git://git.kernel.org/pub/scm/linux/kernel/git/arnd/bkl 2010-10-22 10:52:56 -07:00
sysfs Merge git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/driver-core-2.6 2010-10-22 19:36:42 -07:00
sysv fs/sysv/super.c: add support for non-PDP11 v7 filesystems 2010-08-11 08:59:23 -07:00
ubifs Merge branch 'linux-next' of git://git.infradead.org/ubifs-2.6 2010-10-22 16:34:03 -07:00
udf Merge branch 'vfs' of git://git.kernel.org/pub/scm/linux/kernel/git/arnd/bkl 2010-10-22 10:52:01 -07:00
ufs Merge branch 'vfs' of git://git.kernel.org/pub/scm/linux/kernel/git/arnd/bkl 2010-10-22 10:52:01 -07:00
xfs writeback: remove nonblocking/encountered_congestion references 2010-10-26 16:52:05 -07:00
aio.c aio: do not return ERESTARTSYS as a result of AIO 2010-09-22 17:22:39 -07:00
anon_inodes.c Revert "anon_inode: set S_IFREG on the anon_inode" 2010-05-27 22:03:05 -04:00
attr.c check ATTR_SIZE contraints in inode_change_ok 2010-08-09 16:47:39 -04:00
bad_inode.c bkl: Remove locked .ioctl file operation 2010-08-14 00:24:24 +02:00
binfmt_aout.c Don't dump task struct in a.out core-dumps 2010-10-14 10:57:40 -07:00
binfmt_elf_fdpic.c binfmt_elf_fdpic: Fix clear_user() error handling 2010-06-01 08:11:06 -07:00
binfmt_elf.c ARM: 6342/1: fix ASLR of PIE executables 2010-10-08 10:02:53 +01:00
binfmt_em86.c
binfmt_flat.c flat: tweak default stack alignment 2010-06-29 15:29:31 -07:00
binfmt_misc.c llseek: automatically add .llseek fop 2010-10-15 15:53:27 +02:00
binfmt_script.c Make do_execve() take a const filename pointer 2010-08-17 18:07:43 -07:00
binfmt_som.c
bio-integrity.c fs/bio-integrity.c: return -ENOMEM on kmalloc failure 2010-08-23 13:36:59 +02:00
bio.c block: unify flags for struct bio and struct request 2010-08-07 18:20:39 +02:00
block_dev.c block: remove BLKDEV_IFL_WAIT 2010-09-16 20:52:58 +02:00
buffer.c writeback: remove nonblocking/encountered_congestion references 2010-10-26 16:52:05 -07:00
char_dev.c Merge branch 'llseek' of git://git.kernel.org/pub/scm/linux/kernel/git/arnd/bkl 2010-10-22 10:52:56 -07:00
compat_binfmt_elf.c
compat_ioctl.c fix rawctl compat ioctls breakage on amd64 and itanic 2010-10-19 11:29:54 +02:00
compat.c Merge branch 'for-2.6.37' of git://linux-nfs.org/~bfields/linux 2010-10-26 09:55:25 -07:00
dcache.c fs: brlock vfsmount_lock 2010-08-18 08:35:48 -04:00
dcookies.c
direct-io.c O_DIRECT: fix the splitting up of contiguous I/O 2010-09-09 18:57:22 -07:00
drop_caches.c simplify checks for I_CLEAR/I_FREEING 2010-08-09 16:47:44 -04:00
eventfd.c llseek: automatically add .llseek fop 2010-10-15 15:53:27 +02:00
eventpoll.c llseek: automatically add .llseek fop 2010-10-15 15:53:27 +02:00
exec.c oom: add per-mm oom disable count 2010-10-26 16:52:05 -07:00
fcntl.c vfs: take O_NONBLOCK out of the O_* uniqueness test 2010-09-09 18:57:25 -07:00
fifo.c llseek: automatically add .llseek fop 2010-10-15 15:53:27 +02:00
file_table.c fs: scale files_lock 2010-08-18 08:35:48 -04:00
file.c vfs: use kmalloc() to allocate fdmem if possible 2010-08-11 08:59:02 -07:00
filesystems.c
fs_struct.c fs: fs_struct rwlock to spinlock 2010-08-18 08:35:46 -04:00
fs-writeback.c writeback: always use sb->s_bdi for writeback purposes 2010-10-04 14:25:33 +02:00
generic_acl.c vfs: update ctime when changing the file's permission by setfacl 2010-08-18 01:04:22 -04:00
inode.c Merge branch 'for-linus' of git://git.infradead.org/users/eparis/notify 2010-08-10 11:39:13 -07:00
internal.h fs: brlock vfsmount_lock 2010-08-18 08:35:48 -04:00
ioctl.c bkl: Remove locked .ioctl file operation 2010-08-14 00:24:24 +02:00
ioprio.c
Kconfig BKL: introduce CONFIG_BKL. 2010-10-21 15:44:13 +02:00
Kconfig.binfmt
libfs.c libfs: Fix shift bug in generic_check_addressable() 2010-09-10 08:42:48 -07:00
locks.c fs/locks.c: prepare for BKL removal 2010-10-05 11:02:04 +02:00
Makefile nfsd: allow deprecated interface to be compiled out. 2010-09-22 15:33:14 -04:00
mbcache.c mbcache: Limit the maximum number of cache entries 2010-08-18 06:24:41 -04:00
mpage.c
namei.c fs: brlock vfsmount_lock 2010-08-18 08:35:48 -04:00
namespace.c BKL: Remove BKL from do_new_mount() 2010-10-04 21:10:43 +02:00
nfsctl.c
no-block.c llseek: automatically add .llseek fop 2010-10-15 15:53:27 +02:00
open.c fs: cleanup files_lock locking 2010-08-18 08:35:47 -04:00
pipe.c pipe: fix failure to return error code on ->confirm() 2010-10-21 14:56:33 +02:00
pnode.c fs: brlock vfsmount_lock 2010-08-18 08:35:48 -04:00
pnode.h
posix_acl.c
read_write.c vfs: make no_llseek the default 2010-10-15 15:53:46 +02:00
read_write.h
readdir.c vfs: fix warning: 'dirent' is used uninitialized in this function 2010-08-09 20:45:05 -07:00
select.c
seq_file.c fs/seq_file.c: Remove unnecessary casts of private_data 2010-09-23 13:28:23 +02:00
signalfd.c Merge branch 'hwpoison' of git://git.kernel.org/pub/scm/linux/kernel/git/ak/linux-mce-2.6 2010-10-26 10:13:10 -07:00
splice.c splice: fix misuse of SPLICE_F_NONBLOCK 2010-08-07 18:52:56 +02:00
stack.c
stat.c Mark arguments to certain syscalls as being const 2010-08-13 16:53:13 -07:00
statfs.c add f_flags to struct statfs(64) 2010-08-09 16:48:44 -04:00
super.c fs: scale files_lock 2010-08-18 08:35:48 -04:00
sync.c get rid of file_fsync() 2010-08-09 16:47:43 -04:00
timerfd.c llseek: automatically add .llseek fop 2010-10-15 15:53:27 +02:00
utimes.c Mark arguments to certain syscalls as being const 2010-08-13 16:53:13 -07:00
xattr_acl.c
xattr.c fs: xattr_handler table should be const 2010-05-21 18:31:18 -04:00