Commit Graph

31 Commits

Author SHA1 Message Date
Sage Weil
bc4fdca857 ceph: fix pg_mapping leak on pg_temp updates
Free the ceph_pg_mapping structs when they are removed from the pg_temp
rbtree.  Also fix a leak in the __insert_pg_mapping() error path.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-07-23 10:02:06 -07:00
Dan Carpenter
b0bbb0be8f ceph: add kfree() to error path
We leak a "pi" on this error path.

Signed-off-by: Dan Carpenter <error27@gmail.com>
Signed-off-by: Sage Weil <sage@newdream.net>
2010-07-08 08:03:24 -07:00
Sage Weil
cebc5be6b6 ceph: fix crush map update decoding
If the incremental osdmap has a new crush map, advance the position after
decoding so that we can parse the rest of the osdmap properly.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-06-17 10:22:48 -07:00
Julia Lawall
7e34bc524e fs/ceph: Use ERR_CAST
Use ERR_CAST(x) rather than ERR_PTR(PTR_ERR(x)).  The former makes more
clear what is the purpose of the operation, which otherwise looks like a
no-op.

In the case of fs/ceph/inode.c, ERR_CAST is not needed, because the type of
the returned value is the same as the type of the enclosing function.

The semantic patch that makes this change is as follows:
(http://coccinelle.lip6.fr/)

// <smpl>
@@
type T;
T x;
identifier f;
@@

T f (...) { <+...
- ERR_PTR(PTR_ERR(x))
+ x
 ...+> }

@@
expression x;
@@

- ERR_PTR(PTR_ERR(x))
+ ERR_CAST(x)
// </smpl>

Signed-off-by: Julia Lawall <julia@diku.dk>
Signed-off-by: Sage Weil <sage@newdream.net>
2010-05-29 09:12:41 -07:00
Sage Weil
d85b705663 ceph: resubmit requests on pg mapping change (not just primary change)
OSD requests need to be resubmitted on any pg mapping change, not just when
the pg primary changes.  Resending only when the primary changes results in
occasional 'hung' requests during osd cluster recovery or rebalancing.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-05-11 09:53:56 -07:00
Linus Torvalds
96e35b40c0 Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/sage/ceph-client
* 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/sage/ceph-client:
  ceph: use separate class for ceph sockets' sk_lock
  ceph: reserve one more caps space when doing readdir
  ceph: queue_cap_snap should always queue dirty context
  ceph: fix dentry reference leak in dcache readdir
  ceph: decode v5 of osdmap (pool names) [protocol change]
  ceph: fix ack counter reset on connection reset
  ceph: fix leaked inode ref due to snap metadata writeback race
  ceph: fix snap context reference leaks
  ceph: allow writeback of snapped pages older than 'oldest' snapc
  ceph: fix dentry rehashing on virtual .snap dir
2010-04-14 18:45:31 -07:00
Sage Weil
2844a76a25 ceph: decode v5 of osdmap (pool names) [protocol change]
Teach the client to decode an updated format for the osdmap.  The new
format includes pool names, which will be useful shortly.  Get this change
in earlier rather than later.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-04-09 15:50:58 -07:00
Tejun Heo
5a0e3ad6af include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h
percpu.h is included by sched.h and module.h and thus ends up being
included when building most .c files.  percpu.h includes slab.h which
in turn includes gfp.h making everything defined by the two files
universally available and complicating inclusion dependencies.

percpu.h -> slab.h dependency is about to be removed.  Prepare for
this change by updating users of gfp and slab facilities include those
headers directly instead of assuming availability.  As this conversion
needs to touch large number of source files, the following script is
used as the basis of conversion.

  http://userweb.kernel.org/~tj/misc/slabh-sweep.py

The script does the followings.

* Scan files for gfp and slab usages and update includes such that
  only the necessary includes are there.  ie. if only gfp is used,
  gfp.h, if slab is used, slab.h.

* When the script inserts a new include, it looks at the include
  blocks and try to put the new include such that its order conforms
  to its surrounding.  It's put in the include block which contains
  core kernel includes, in the same order that the rest are ordered -
  alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
  doesn't seem to be any matching order.

* If the script can't find a place to put a new include (mostly
  because the file doesn't have fitting include block), it prints out
  an error message indicating which .h file needs to be added to the
  file.

The conversion was done in the following steps.

1. The initial automatic conversion of all .c files updated slightly
   over 4000 files, deleting around 700 includes and adding ~480 gfp.h
   and ~3000 slab.h inclusions.  The script emitted errors for ~400
   files.

2. Each error was manually checked.  Some didn't need the inclusion,
   some needed manual addition while adding it to implementation .h or
   embedding .c file was more appropriate for others.  This step added
   inclusions to around 150 files.

3. The script was run again and the output was compared to the edits
   from #2 to make sure no file was left behind.

4. Several build tests were done and a couple of problems were fixed.
   e.g. lib/decompress_*.c used malloc/free() wrappers around slab
   APIs requiring slab.h to be added manually.

5. The script was run on all .h files but without automatically
   editing them as sprinkling gfp.h and slab.h inclusions around .h
   files could easily lead to inclusion dependency hell.  Most gfp.h
   inclusion directives were ignored as stuff from gfp.h was usually
   wildly available and often used in preprocessor macros.  Each
   slab.h inclusion directive was examined and added manually as
   necessary.

6. percpu.h was updated not to include slab.h.

7. Build test were done on the following configurations and failures
   were fixed.  CONFIG_GCOV_KERNEL was turned off for all tests (as my
   distributed build env didn't work with gcov compiles) and a few
   more options had to be turned off depending on archs to make things
   build (like ipr on powerpc/64 which failed due to missing writeq).

   * x86 and x86_64 UP and SMP allmodconfig and a custom test config.
   * powerpc and powerpc64 SMP allmodconfig
   * sparc and sparc64 SMP allmodconfig
   * ia64 SMP allmodconfig
   * s390 SMP allmodconfig
   * alpha SMP allmodconfig
   * um on x86_64 SMP allmodconfig

8. percpu.h modifications were reverted so that it could be applied as
   a separate patch and serve as bisection point.

Given the fact that I had only a couple of failures from tests on step
6, I'm fairly confident about the coverage of this conversion patch.
If there is a breakage, it's likely to be something in one of the arch
headers which should be easily discoverable easily on most builds of
the specific arch.

Signed-off-by: Tejun Heo <tj@kernel.org>
Guess-its-ok-by: Christoph Lameter <cl@linux-foundation.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
2010-03-30 22:02:32 +09:00
Sage Weil
efd7576b23 ceph: fix pg pool decoding from incremental osdmap update
The incremental map decoding of pg pool updates wasn't skipping
the snaps and removed_snaps vectors.  This caused osd requests
to stall when pool snapshots were created or fs snapshots were
deleted.  Use a common helper for full and incremental map
decoders that decodes pools properly.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-03-23 07:46:57 -07:00
Sage Weil
e53a8fd773 ceph: fix osdmap decoding when pools include (removed) snaps
Add missing pointer dereference (p is a void **).

Signed-off-by: Sage Weil <sage@newdream.net>
2010-03-01 15:28:00 -08:00
Sage Weil
4fc51be8fa ceph: use rbtree for pg pools; decode new osdmap format
Since we can now create and destroy pg pools, the pool ids will be sparse,
and an array no longer makes sense for looking up by pool id.  Use an
rbtree instead.

The OSDMap encoding also no longer has a max pool count (previously used to
allocate the array).  There is a new pool_max, that is the largest pool id
we've ever used, although we don't actually need it in the client.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-02-17 10:02:49 -08:00
Sage Weil
9794b146fa ceph: fix memory leak when destroying osdmap with pg_temp mappings
Also move _lookup_pg_mapping into a helper.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-02-17 10:02:48 -08:00
Sage Weil
02f90c6109 ceph: add uid field to ceph_pg_pool
Also verify encoding version as we go.

Signed-off-by: Sage Weil <sage@newdream.net>
2010-02-11 11:48:49 -08:00
Sage Weil
361be8601d ceph: precede encoded ceph_pg_pool struct with version
Signed-off-by: Sage Weil <sage@newdream.net>
2010-01-25 16:03:02 -08:00
Sage Weil
7067f797b8 ceph: fix incremental osdmap pg_temp decoding bug
An incremental pg_temp wasn't being decoded properly (wrong bound on
for loop).

Also remove unused local variable, while we're at it.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-12-21 16:40:00 -08:00
Sage Weil
30dc6381bb ceph: fix error paths for corrupt osdmap messages
Both osdmap_decode() and osdmap_apply_incremental() should never return
NULL.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-12-21 16:39:59 -08:00
Sage Weil
9ec7cab14e ceph: hex dump corrupt server data to KERN_DEBUG
Also, print fsid using standard format, NOT hex dump.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-12-21 16:39:52 -08:00
Sage Weil
767ea5c33a ceph: do not feed bad device ids to crush
Do not feed bad (large) device ids to CRUSH.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-12-09 15:57:16 -08:00
Sage Weil
fb690390e3 ceph: make CRUSH hash function a bucket property
Make the integer hash function a property of the bucket it is used on.  This
allows us to gracefully add support for new hash functions without starting
from scatch.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-11-07 20:18:22 -08:00
Sage Weil
1654dd0cf5 ceph: make object hash a pg_pool property
The object will be hashed to a placement seed (ps) based on the pg_pool's
hash function.  This allows new hashes to be introduced into an existing
object store, or selection of a hash appropriate to the objects that
will be stored in a particular pool.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-11-06 21:55:25 -08:00
Sage Weil
1bdb70e590 ceph: clean up 'osd%d down' console msg
No ceph prefix.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-11-06 13:57:49 -08:00
Sage Weil
51042122d4 ceph: fix endian conversions for ceph_pg
The endian conversions don't quite work with the old union ceph_pg.  Just
make it a regular struct, and make each field __le.  This is simpler and it
has the added bonus of actually working.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-11-04 16:36:07 -08:00
Sage Weil
63f2d21195 ceph: use fixed endian encoding for ceph_entity_addr
We exchange struct ceph_entity_addr over the wire and store it on disk.
The sockaddr_storage.ss_family field, however, is host endianness.  So,
fix ss_family endianness to big endian when sending/receiving over the
wire.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-11-03 15:17:56 -08:00
Noah Watkins
ff1d1f7179 ceph: fix intra strip unit length calculation
Commit 645a102581 fixes calculation of object
offset for layouts with multiple stripes per object. This updates the
calculation of the length written to take into account multiple stripes per
object.

Signed-off-by: Noah Watkins <noah@noahdesu.com>
Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-30 13:56:14 -07:00
Sage Weil
645a102581 ceph: fix object striping calculation for non-default striping schemes
We were incorrectly calculationing of object offset.  If we have multiple
stripe units per object, we need to shift to the start of the current
su in addition to the offset within the su.

Also rename bno to ono (object number) to avoid some variable naming
confusion.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-28 17:45:41 -07:00
Sage Weil
5600f5ebd3 ceph: correct comment to match striping calculation
The object extent offset is the file offset _modulo_ the stripe unit.
The code was correct, the comment was wrong.

Reported-by: Noah Watkins <jayhawk@soe.ucsc.edu>
Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-28 17:45:37 -07:00
Noah Watkins
35e054a66e ceph: remove redundant use of le32_to_cpu
Using stripe unit size calculated and saved on the stack to avoid
a redundant call to le32_to_cpu.

Signed-off-by: Noah Watkins <noah@noahdesu.com>
Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-28 17:44:37 -07:00
Sage Weil
ee7fdfaff7 ceph: include preferred osd in placement seed
Mix the preferred osd (if any) into the placement seed that is fed into
the CRUSH object placement calculation.  This prevents all the placement
pgs from peering with the same osds.

Rev the osd client protocol with this change.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-19 11:42:41 -07:00
Sage Weil
c89136ea42 ceph: convert encode/decode macros to inlines
This avoids the fugly pass by reference and makes the code a bit easier
to read.

Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-14 09:59:09 -07:00
Sage Weil
991abb6ecf ceph: fail gracefully on corrupt osdmap (bad pg_temp mapping)
Return an error and report a corrupt map instead of crying BUG().

Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-09 11:58:11 -07:00
Sage Weil
f24e9980eb ceph: OSD client
The OSD client is responsible for reading and writing data from/to the
object storage pool.  This includes determining where objects are
stored in the cluster, and ensuring that requests are retried or
redirected in the event of a node failure or data migration.

If an OSD does not respond before a timeout expires, keepalive
messages are sent across the lossless, ordered communications channel
to ensure that any break in the TCP is discovered.  If the session
does reset, a reconnection is attempted and affected requests are
resent (by the message transport layer).

Signed-off-by: Sage Weil <sage@newdream.net>
2009-10-06 11:31:10 -07:00