Commit Graph

70 Commits

Author SHA1 Message Date
Arnaldo Carvalho de Melo
b44308f540 perf scripting: Shut up 'perf record' final status
We want just the script output, not internal details about the record phase.

Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Cc: Tom Zanussi <tzanussi@gmail.com>
LKML-Reference: <new-submission>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-10-26 15:20:09 -02:00
Ben Hutchings
d1e95bb530 perf trace: Fix detection of script extension
The extension starts with the last dot in the name, not the first.

Cc: Ingo Molnar <mingo@elte.hu>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
LKML-Reference: <1286723462.2955.206.camel@localhost>
Signed-off-by: Ben Hutchings <ben@decadent.org.uk>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-10-23 15:31:20 -02:00
Ben Hutchings
44e668c6fa perf trace: Use $PERF_EXEC_PATH in canned report scripts
Set $PERF_EXEC_PATH before starting the record and report scripts, and
make them use it where necessary.

Cc: Ingo Molnar <mingo@elte.hu>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
LKML-Reference: <1286723403.2955.205.camel@localhost>
Signed-off-by: Ben Hutchings <ben@decadent.org.uk>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-10-23 15:31:20 -02:00
Andrea Gelmini
b7eead86d2 perf trace: Clean up #includes
Removed duplicated #includes util/trace-event.h and
util/exec_cmd.h.
Grouped and sorted all the #includes.

Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Tom Zanussi <tzanussi@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
LKML-Reference: <1281016299-23958-14-git-send-email-andrea.gelmini@gelma.net>
Signed-off-by: Andrea Gelmini <andrea.gelmini@gelma.net>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-08-05 19:38:02 -03:00
Frederic Weisbecker
ffabd99e05 perf: Report lost events in perf trace debug mode
Account and report lost events in perf trace debugging mode,
useful to check the reliability of the traces.

Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Masami Hiramatsu <mhiramat@redhat.com>
Cc: Tom Zanussi <tzanussi@gmail.com>
2010-06-24 23:36:23 +02:00
Frederic Weisbecker
6fcf7ddbb7 perf: Don't print traces when debugging ordering
Errors due to ordering bugs are easily lost in the middle
of traces.

When we are in this mode, don't print the traces so that
we don't miss the debugging messages.
But display a comforting message if we didn't encounter any
ordering problem.

Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Paul Mackerras <paulus@samba.org>
2010-06-24 23:36:05 +02:00
Arnaldo Carvalho de Melo
cee75ac7ec perf hist: Clarify events_stats fields usage
The events_stats.total field is too generic, rename it to .total_period,
and also add a comment explaining that it is the sum of all the .period
fields in samples, that is needed because we use auto-freq to avoid
sampling artifacts.

Ditto for events_stats.lost, that is the sum of all lost_event.lost
fields, i.e. the number of events the kernel dropped.

Looking at the users, builtin-sched.c can make use of these fields and
stop doing it again.

Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Tom Zanussi <tzanussi@gmail.com>
LKML-Reference: <new-submission>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-05-14 13:16:55 -03:00
Tom Zanussi
3824a4e8da perf/trace/scripting: don't show script start/stop messages by default
Only print the script start/stop messages in verbose mode - users
normally don't care and it just clutters up the output.

Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
LKML-Reference: <1273466820-9330-7-git-send-email-tzanussi@gmail.com>
Signed-off-by: Tom Zanussi <tzanussi@gmail.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-05-10 19:50:59 -03:00
Arnaldo Carvalho de Melo
1c02c4d2e9 perf hist: Introduce hists class and move lots of methods to it
In cbbc79a we introduced support for multiple events by introducing a
new "event_stat_id" struct and then made several perf_session methods
receive a point to it instead of a pointer to perf_session, and kept the
event_stats and hists rb_tree in perf_session.

While working on the new newt based browser, I realised that it would be
better to introduce a new class, "hists" (short for "histograms"),
renaming the "event_stat_id" struct and the perf_session methods that
were really "hists" methods, as they manipulate only struct hists
members, not touching anything in the other perf_session members.

Other optimizations, such as calculating the maximum lenght of a symbol
name present in an hists instance will be possible as we add them,
avoiding a re-traversal just for finding that information.

The rationale for the name "hists" to replace "event_stat_id" is that we
may have multiple sets of hists for the same event_stat id, as, for
instance, the 'perf diff' tool has, so event stat id is not what
characterizes what this struct and the functions that manipulate it do.

Cc: Eric B Munson <ebmunson@us.ibm.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Tom Zanussi <tzanussi@gmail.com>
LKML-Reference: <new-submission>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-05-10 13:13:49 -03:00
Tom Zanussi
454c407ec1 perf: add perf-inject builtin
Currently, perf 'live mode' writes build-ids at the end of the
session, which isn't actually useful for processing live mode events.

What would be better would be to have the build-ids sent before any of
the samples that reference them, which can be done by processing the
event stream and retrieving the build-ids on the first hit.  Doing
that in perf-record itself, however, is off-limits.

This patch introduces perf-inject, which does the same job while
leaving perf-record untouched.  Normal mode perf still records the
build-ids at the end of the session as it should, but for live mode,
perf-inject can be injected in between the record and report steps
e.g.:

perf record -o - ./hackbench 10 | perf inject -v -b | perf report -v -i -

perf-inject reads a perf-record event stream and repipes it to stdout.
At any point the processing code can inject other events into the
event stream - in this case build-ids (-b option) are read and
injected as needed into the event stream.

Build-ids are just the first user of perf-inject - potentially
anything that needs userspace processing to augment the trace stream
with additional information could make use of this facility.

Cc: Ingo Molnar <mingo@elte.hu>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
LKML-Reference: <1272696080-16435-3-git-send-email-tzanussi@gmail.com>
Signed-off-by: Tom Zanussi <tzanussi@gmail.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2010-05-02 13:36:56 -03:00
Frederic Weisbecker
e1889d75af perf: Add a perf trace option to check samples ordering reliability
To ensure sample events time reordering is reliable, add a -d option
to perf trace to check that automatically.

Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Hitoshi Mitake <mitake@dcl.info.waseda.ac.jp>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Masami Hiramatsu <mhiramat@redhat.com>
Cc: Tom Zanussi <tzanussi@gmail.com>
2010-04-24 03:50:48 +02:00
Frederic Weisbecker
e0a808c65c perf: Use generic sample reordering in perf trace
Use the new generic sample events reordering from perf trace.
Before that, the displayed traces were ordered as they were
in the input as recorded by perf record (not time ordered).

This makes eventually perf trace displaying the events as beeing
time ordered.

Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Hitoshi Mitake <mitake@dcl.info.waseda.ac.jp>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Masami Hiramatsu <mhiramat@redhat.com>
Cc: Tom Zanussi <tzanussi@gmail.com>
2010-04-24 03:50:45 +02:00
Tom Zanussi
a0cccc2e8e perf trace: Invoke live mode automatically if record/report not specified
Currently, live mode is invoked by explicitly invoking the
record and report sides and connecting them with a pipe e.g.

 $ perf trace record rwtop -o - | perf trace report rwtop 5 -i -

In terms of usability, it's not that bad, but it does require
the user to type and remember more than necessary.

This patch allows the user to accomplish the same thing without
specifying the separate record/report steps or the pipe.  So the
same command as above can be accomplished more simply as:

 $ perf trace rwtop 5

Notice that the '-i -' and '-o -' aren't required in this case -
they're added internally, and that any extra arguments are
passed along to the report script (but not to the record
script).

The overall effect is that any of the scripts listed in 'perf
trace -l' can now be used directly in live mode, with the
expected arguments, by simply specifying the script and args to
'perf trace'.

Signed-off-by: Tom Zanussi <tzanussi@gmail.com>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Cc: fweisbec@gmail.com
Cc: rostedt@goodmis.org
Cc: k-keiichi@bx.jp.nec.com
Cc: acme@ghostprotocols.net
LKML-Reference: <1270184365-8281-12-git-send-email-tzanussi@gmail.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2010-04-14 11:56:09 +02:00
Tom Zanussi
c7929e4727 perf: Convert perf header build_ids into build_id events
Bypasses the build_id perf header code and replaces it with a
synthesized event and processing function that accomplishes the
same thing, used when reading/writing perf data to/from a pipe.

Signed-off-by: Tom Zanussi <tzanussi@gmail.com>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Cc: fweisbec@gmail.com
Cc: rostedt@goodmis.org
Cc: k-keiichi@bx.jp.nec.com
Cc: acme@ghostprotocols.net
LKML-Reference: <1270184365-8281-9-git-send-email-tzanussi@gmail.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2010-04-14 11:56:08 +02:00
Tom Zanussi
9215545e99 perf: Convert perf tracing data into a tracing_data event
Bypasses the tracing_data perf header code and replaces it with
a synthesized event and processing function that accomplishes
the same thing, used when reading/writing perf data to/from a
pipe.

The tracing data is pretty large, and this patch doesn't attempt
to break it down into component events.  The tracing_data event
itself doesn't actually contain the tracing data, rather it
arranges for the event processing code to skip over it after
it's read, using the skip return value added to the event
processing loop in a previous patch.

Signed-off-by: Tom Zanussi <tzanussi@gmail.com>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Cc: fweisbec@gmail.com
Cc: rostedt@goodmis.org
Cc: k-keiichi@bx.jp.nec.com
Cc: acme@ghostprotocols.net
LKML-Reference: <1270184365-8281-8-git-send-email-tzanussi@gmail.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2010-04-14 11:56:07 +02:00
Tom Zanussi
cd19a035f3 perf: Convert perf event types into event type events
Bypasses the event type perf header code and replaces it with a
synthesized event and processing function that accomplishes the
same thing, used when reading/writing perf data to/from a pipe.

Signed-off-by: Tom Zanussi <tzanussi@gmail.com>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Cc: fweisbec@gmail.com
Cc: rostedt@goodmis.org
Cc: k-keiichi@bx.jp.nec.com
Cc: acme@ghostprotocols.net
LKML-Reference: <1270184365-8281-7-git-send-email-tzanussi@gmail.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2010-04-14 11:56:07 +02:00
Tom Zanussi
2c46dbb517 perf: Convert perf header attrs into attr events
Bypasses the attr perf header code and replaces it with a
synthesized event and processing function that accomplishes the
same thing, used when reading/writing perf data to/from a pipe.

Making the attrs into events allows them to be streamed over a
pipe along with the rest of the header data (in later patches).
It also paves the way to allowing events to be added and removed
from perf sessions dynamically.

Signed-off-by: Tom Zanussi <tzanussi@gmail.com>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Cc: fweisbec@gmail.com
Cc: rostedt@goodmis.org
Cc: k-keiichi@bx.jp.nec.com
Cc: acme@ghostprotocols.net
LKML-Reference: <1270184365-8281-6-git-send-email-tzanussi@gmail.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2010-04-14 11:56:07 +02:00
Tom Zanussi
c239da3b4b perf trace: Introduce special handling for pipe input
Adds special treatment for stdin - if the user specifies '-i -'
to perf trace, the intent is that the event stream be read from
stdin rather than from a disk file.

The actual handling of the '-' filename is done by the session;
this just adds a signal handler to stop reporting, and turns off
interference by the pager.

Signed-off-by: Tom Zanussi <tzanussi@gmail.com>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Cc: fweisbec@gmail.com
Cc: rostedt@goodmis.org
Cc: k-keiichi@bx.jp.nec.com
Cc: acme@ghostprotocols.net
LKML-Reference: <1270184365-8281-5-git-send-email-tzanussi@gmail.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2010-04-14 11:56:06 +02:00
Ian Munsie
c055564217 perf: Fix endianness argument compatibility with OPT_BOOLEAN() and introduce OPT_INCR()
Parsing an option from the command line with OPT_BOOLEAN on a
bool data type would not work on a big-endian machine due to the
manner in which the boolean was being cast into an int and
incremented. For example, running 'perf probe --list' on a
PowerPC machine would fail to properly set the list_events bool
and would therefore print out the usage information and
terminate.

This patch makes OPT_BOOLEAN work as expected with a bool
datatype. For cases where the original OPT_BOOLEAN was
intentionally being used to increment an int each time it was
passed in on the command line, this patch introduces OPT_INCR
with the old behaviour of OPT_BOOLEAN (the verbose variable is
currently the only such example of this).

I have reviewed every use of OPT_BOOLEAN to verify that a true
C99 bool was passed. Where integers were used, I verified that
they were only being used for boolean logic and changed them to
bools to ensure that they would not be mistakenly used as ints.
The major exception was the verbose variable which now uses
OPT_INCR instead of OPT_BOOLEAN.

Signed-off-by: Ian Munsie <imunsie@au.ibm.com>
Acked-by: David S. Miller <davem@davemloft.net>
Cc: <stable@kernel.org> # NOTE: wont apply to .3[34].x cleanly, please backport
Cc: Git development list <git@vger.kernel.org>
Cc: Ian Munsie <imunsie@au1.ibm.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Cc: Hitoshi Mitake <mitake@dcl.info.waseda.ac.jp>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Eric B Munson <ebmunson@us.ibm.com>
Cc: Valdis.Kletnieks@vt.edu
Cc: WANG Cong <amwang@redhat.com>
Cc: Thiago Farina <tfransosi@gmail.com>
Cc: Masami Hiramatsu <mhiramat@redhat.com>
Cc: Xiao Guangrong <xiaoguangrong@cn.fujitsu.com>
Cc: Jaswinder Singh Rajput <jaswinderrajput@gmail.com>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Tom Zanussi <tzanussi@gmail.com>
Cc: Anton Blanchard <anton@samba.org>
Cc: John Kacur <jkacur@redhat.com>
Cc: Li Zefan <lizf@cn.fujitsu.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
LKML-Reference: <1271147857-11604-1-git-send-email-imunsie@au.ibm.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2010-04-14 11:26:44 +02:00
Tom Zanussi
cf4fee5028 perf trace: Don't use pager if scripting
It's useful for paging through raw traces, but just gets in the
way when scripting.

Signed-off-by: Tom Zanussi <tzanussi@gmail.com>
Cc: fweisbec@gmail.com
Cc: rostedt@goodmis.org
LKML-Reference: <1267599873-8193-3-git-send-email-tzanussi@gmail.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2010-03-04 12:19:55 +01:00
Tom Zanussi
10c95f4f41 perf trace/scripting: Remove extraneous header read
perf_header__read() is already done in perf_session__open(), so
remove it from the script gen case.

Signed-off-by: Tom Zanussi <tzanussi@gmail.com>
Cc: fweisbec@gmail.com
Cc: rostedt@goodmis.org
LKML-Reference: <1267599873-8193-2-git-send-email-tzanussi@gmail.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2010-03-04 12:19:54 +01:00
Tom Zanussi
7e4b21b84c perf/scripts: Add Python scripting engine
Add base support for Python scripting to perf trace.

Signed-off-by: Tom Zanussi <tzanussi@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Keiichi KII <k-keiichi@bx.jp.nec.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
LKML-Reference: <1264580883-15324-6-git-send-email-tzanussi@gmail.com>
Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com>
2010-02-25 04:07:29 +01:00
Tom Zanussi
f526d68b6c perf/scripts: Fix supported language listing option
'perf trace -s list' prints a list of the supported scripting
languages.  One problem with it is that it falls through and prints
the trace as well.  The use of 'list' for this also makes it easy to
confuse with 'perf trace -l', used for listing available scripts.  So
change 'perf trace -s list' to 'perf trace -s lang' and fixes the
fall-through problem.

Signed-off-by: Tom Zanussi <tzanussi@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Keiichi KII <k-keiichi@bx.jp.nec.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
LKML-Reference: <1264580883-15324-2-git-send-email-tzanussi@gmail.com>
Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com>
2010-02-23 20:34:42 +01:00
Hitoshi Mitake
408f0d18ba perf trace: Add -i option for choosing input file
perf trace lacks -i option for choosing input file.
This patch adds it to perf trace.

Signed-off-by: Hitoshi Mitake <mitake@dcl.info.waseda.ac.jp>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
LKML-Reference: <1264167929-6741-1-git-send-email-mitake@dcl.info.waseda.ac.jp>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2010-01-27 13:01:34 +01:00
Arnaldo Carvalho de Melo
0d755034db perf tools: Don't cast RIP to pointers
Since they can come from another architecture with bigger
pointers, i.e. processing a 64-bit perf.data on a 32-bit arch.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1263478990-8200-1-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2010-01-16 10:58:45 +01:00
Arnaldo Carvalho de Melo
55aa640f54 perf session: Remove redundant prefix & suffix from perf_event_ops
Since now all that we have are perf event handlers, leave just
the name of the event.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1261957026-15580-9-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-12-28 09:03:35 +01:00
Arnaldo Carvalho de Melo
d549c76901 perf session: Remove sample_type_check from event_ops
This is really something tools need to do before asking for the
events to be processed, leaving perf_session__process_events to
do just that, process events.

Also add a msg parameter to perf_session__has_traces() so that
the right message can be printed, fixing a regression added by
me in the previous cset (right timechart message) and also
fixing 'perf kmem', that was not asking if 'perf kmem record'
was ran.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1261957026-15580-6-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-12-28 09:03:34 +01:00
Arnaldo Carvalho de Melo
27295592c2 perf session: Share the common trace sample_check routine as perf_session__has_traces
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1261957026-15580-5-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-12-28 09:03:33 +01:00
Arnaldo Carvalho de Melo
0422a4fc2c perf diff: Fix usage array, it must end with a NULL entry
Fixing this:

 [acme@doppio linux-2.6-tip]$ perf diff --hell
   Error: unknown option `hell'

  usage: perf diff [<options>] [old_file] [new_file]
 Segmentation fault
 [acme@doppio linux-2.6-tip]$

Also go over the other such arrays to check if they all were OK,
they are, but there were some minor changes to do like making
one static and renaming another to match the command it refers
to.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1261161358-23959-1-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-12-18 20:01:52 +01:00
Arnaldo Carvalho de Melo
655000e7c7 perf symbols: Adopt the strlists for dso, comm
Will be used in perf diff too.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1260914682-29652-2-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-12-16 08:53:49 +01:00
Arnaldo Carvalho de Melo
75be6cf487 perf symbols: Make symbol_conf global
This simplifies a lot of functions, less stuff to be done by
tool writers.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1260914682-29652-1-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-12-16 08:53:48 +01:00
Tom Zanussi
3875294f5c perf trace/scripting: Add 'record' and 'report' options
Allow scripts to be recorded/executed by simply specifying the
script root name (the script name minus extension) along with
'record' or 'report' to 'perf trace'.

The script names shown by 'perf trace -l' can be directly used
to run the command-line contained within the corresponding
'-record' and '-report' versions of scripts in the scripts/*/bin
directories.

For example, to record the trace data needed to run the
wakeup-latency.pl script, the user can easily find the name of
the corresponding script from the script list and invoke it
using 'perf trace record', without having to remember the
details of how to do the same thing using the lower-level perf
trace command-line options:

root@tropicana:~# perf trace -l
List of available trace scripts:
  workqueue-stats                      workqueue stats (ins/exe/create/destroy)
  wakeup-latency                       system-wide min/max/avg wakeup latency
  rw-by-file <comm>                    r/w activity for a program, by file
  check-perf-trace                     useless but exhaustive test script
  rw-by-pid                            system-wide r/w activity

root@tropicana:~# perf trace record wakeup-latency
^C[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.296 MB perf.data (~12931
samples) ]

To run the wakeup-latency.pl script using the captured data,
change 'record' to 'report' in the command-line:

root@tropicana:~# perf trace report wakeup-latency

wakeup_latency stats:

total_wakeups: 65
avg_wakeup_latency (ns): 22417
min_wakeup_latency (ns): 3470
max_wakeup_latency (ns): 223311

perf trace Perl script stopped

If the script takes options, thay can be simply added to the end
of the 'report' invocation:

root@tropicana:~# perf trace record rw-by-file
^C[ perf record: Woken up 2 times to write data ]
[ perf record: Captured and wrote 0.782 MB perf.data (~34171
samples) ]

root@tropicana:~# perf trace report rw-by-file perf

file read counts for perf:

    fd     # reads  bytes_requested
------  ----------  -----------
   122        1934     1980416
   120           1          32

file write counts for perf:

    fd    # writes  bytes_written
------  ----------  -----------
     3        4006      280568

perf trace Perl script stopped

Signed-off-by: Tom Zanussi <tzanussi@gmail.com>
Cc: fweisbec@gmail.com
Cc: rostedt@goodmis.org
LKML-Reference: <1260867220-15699-6-git-send-email-tzanussi@gmail.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-12-15 10:31:33 +01:00
Tom Zanussi
4b9c0c596e perf trace/scripting: List available scripts
Lists the available perf trace scripts, one per line e.g.:

root@tropicana:~# perf trace -l
List of available trace scripts:
  workqueue-stats                      workqueue stats (ins/exe/create/destroy)
  wakeup-latency                       system-wide min/max/avg wakeup latency
  rw-by-file <comm>                    r/w activity for a program, by file
  check-perf-trace                     useless but exhaustive test script
  rw-by-pid                            system-wide r/w activity

To be consistent with the other listing options in perf, the
current latency trace option was changed to '-L', and '-l' is
now used to access the script listing as:

To create the list, it searches each scripts/*/bin directory for
files ending with "-report" and reads information found in
certain comment lines contained in those shell scripts:

  - if the comment line starts with "description:", the rest of the
    line is used as a 'half-line' description.  To keep each line in
    the list to a single line, the description should be limited to 40
    characters (the rest of the line contains the script name and
    args)

  - if the comment line starts with "args:", the rest of the line
    names the args the script supports.  Required args should be
    surrounded by <> brackets, optional args by [] brackets.

The current scripts in scripts/perl/bin have also been updated
with description: and args: comments.

Signed-off-by: Tom Zanussi <tzanussi@gmail.com>
Cc: fweisbec@gmail.com
Cc: rostedt@goodmis.org
LKML-Reference: <1260867220-15699-5-git-send-email-tzanussi@gmail.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-12-15 10:31:32 +01:00
Tom Zanussi
586bc5cce8 perf trace/scripting: Add support for script args
One oversight of the original scripting_ops patch was a lack of
support for passing args to handler scripts.  This adds
argc/argv to the start_script() scripting_op, and changes the
rw-by-file script to take 'comm' arg rather than the 'perf'
value currently hard-coded.  It also takes the opportunity to do
some related minor cleanup.

Signed-off-by: Tom Zanussi <tzanussi@gmail.com>
Cc: fweisbec@gmail.com
Cc: rostedt@goodmis.org
LKML-Reference: <1260867220-15699-2-git-send-email-tzanussi@gmail.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-12-15 10:31:31 +01:00
Arnaldo Carvalho de Melo
f823e441ab perf session: Event statistics also are per session
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1260810361-22828-1-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-12-15 08:50:28 +01:00
Arnaldo Carvalho de Melo
c019879bcc perf session: Adopt the sample_type variable
All tools had copies, and perf diff would have to specify a
sample_type_check method just for copying it.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1260807780-19377-2-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-12-14 17:34:56 +01:00
Arnaldo Carvalho de Melo
4e4f06e4c8 perf session: Move the hist_entries rb tree to perf_session
As we'll need to sort multiple times for multiple perf sessions,
so that we can then do a diff.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1260803439-16783-1-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-12-14 16:57:18 +01:00
Arnaldo Carvalho de Melo
4aa6563641 perf session: Move kmaps to perf_session
There is still some more work to do to disentangle map creation
from DSO loading, but this happens only for the kernel, and for
the early adopters of perf diff, where this disentanglement
matters most, we'll be testing different kernels, so no problem
here.

Further clarification: right now we create the kernel maps for
the various modules and discontiguous kernel text maps when
loading the DSO, we should do it as a two step process, first
creating the maps, for multiple mappings with the same DSO
store, then doing the dso load just once, for the first hit on
one of the maps sharing this DSO backing store.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1260741029-4430-6-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-12-14 16:57:17 +01:00
Arnaldo Carvalho de Melo
b3165f4144 perf session: Move the global threads list to perf_session
So that we can process two perf.data files.

We still need to add a O_MMAP mode for perf_session so that we
can do all the mmap stuff in it.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1260741029-4430-5-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-12-14 16:57:16 +01:00
Arnaldo Carvalho de Melo
ec91336973 perf session: Reduce the number of parms to perf_session__process_events
By having the cwd/cwdlen in the perf_session struct and
full_paths in perf_event_ops.

Now its just a matter of passing the ops.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1260741029-4430-4-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-12-14 16:57:16 +01:00
Arnaldo Carvalho de Melo
13df45ca1c perf session: Register the idle thread in perf_session__process_events
No need for all tools to register it and then immediately call
perf_session__process_events.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1260741029-4430-3-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-12-14 16:57:15 +01:00
Arnaldo Carvalho de Melo
301a0b0202 perf session: Ditch register_perf_file_handler
Pass the event_ops to perf_session__process_events instead.

Also move the event_ops definition to session.h, starting to
move things around to their right place, trimming the many
unneeded headers we have.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1260741029-4430-2-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-12-14 16:57:15 +01:00
Arnaldo Carvalho de Melo
d8f66248d6 perf session: Pass the perf_session to the event handling operations
They will need it to get the right threads list, etc.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1260741029-4430-1-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-12-14 16:57:13 +01:00
Arnaldo Carvalho de Melo
94c744b6c0 perf tools: Introduce perf_session class
That does all the initialization boilerplate, opening the file,
reading the header, checking if it is valid, etc.

And that will as well have the threads list, kmap (now) global
variable, etc, so that we can handle two (or more) perf.data files
describing sessions to compare.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1260573842-19720-1-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-12-12 07:42:12 +01:00
OGAWA Hirofumi
180f95e29a perf: Make common SAMPLE_EVENT parser
Currently, sample event data is parsed for each commands, and it
is assuming that the data is not including other data. (E.g.
timechart, trace, etc. can't parse the event if it has
PERF_SAMPLE_CALLCHAIN)

So, even if we record the superset data for multiple commands at
a time, commands can't parse. etc.

To fix it, this makes common sample event parser, and use it to
parse sample event correctly. (PERF_SAMPLE_READ is unsupported
for now though, it seems to be not using.)

Signed-off-by: OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
LKML-Reference: <87hbs48imv.fsf@devron.myhome.or.jp>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-12-06 18:15:01 +01:00
Ingo Molnar
cf72344d1a perf scripting: Fix build
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
LKML-Reference: <new-submission>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-11-28 10:11:00 +01:00
Tom Zanussi
16c632de64 perf trace: Add Perl scripting support
Implement trace_scripting_ops to make Perl a supported perf
trace scripting language.

Additionally adds code that allows Perl trace scripts to access
the 'flag' and 'symbolic' (__print_flags(), __print_symbolic())
field information parsed from the trace format files.

Also adds the Perl implementation of the generate_script()
trace_scripting_op, which creates a ready-to-run perf trace Perl
script based on existing trace data.  Scripts generated by this
implementation print out all the fields for each event mentioned
in perf.data (and will detect and generate the proper scripting
code for 'flag' and 'symbolic' fields), and will additionally
generate handlers for the special 'trace_unhandled',
'trace_begin' and 'trace_end' handlers.  Script authors can
simply remove the printing code to implement their own custom
event handling.

Signed-off-by: Tom Zanussi <tzanussi@gmail.com>
Cc: fweisbec@gmail.com
Cc: rostedt@goodmis.org
Cc: anton@samba.org
Cc: hch@infradead.org
LKML-Reference: <1259133352-23685-4-git-send-email-tzanussi@gmail.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-11-28 10:04:26 +01:00
Tom Zanussi
956ffd027b perf trace: Add scripting ops
Adds an interface, scripting_ops, that when implemented for a
particular scripting language enables built-in support for trace
stream processing using that language.

The interface is designed to enable full-fledged language
interpreters to be embedded inside the perf executable and
thereby make the full capabilities of the supported languages
available for trace processing.

See below for details on the interface.

This patch also adds a couple command-line options to 'perf
trace':

The -s option option is used to specify the script to be run.
Script names that can be used with -s take the form:

[language spec:]scriptname[.ext]

Scripting languages register a set of 'language specs' that can
be used to specify scripts for the registered languages.  The
specs can be used either as prefixes or extensions.

If [language spec:] is used, the script is taken as a script of
the matching language regardless of any extension it might have.
 If [language spec:] is not used, [.ext] is used to look up the
language it corresponds to.  Language specs are case
insensitive.

e.g. Perl scripts can be specified in the following ways:

Perl:scriptname
pl:scriptname.py # extension ignored
PL:scriptname
scriptname.pl
scriptname.perl

The -g [language spec] option gives users an easy starting point
for writing scripts in the specified language.  Scripting
support for a particular language can implement a
generate_script() scripting op that outputs an empty (or
near-empty) set of handlers for all the events contained in a
given perf.data trace file - this option gives users a direct
way to access that.

Adding support for a scripting language
---------------------------------------

The main thing that needs to be done do add support for a new
language is to implement the scripting_ops interface:

It consists of the following four functions:

    start_script()
    stop_script()
    process_event()
    generate_script()

start_script() is called before any events are processed, and is
meant to give the scripting language support an opportunity to
set things up to receive events e.g. create and initialize an
instance of a language interpreter.

stop_script() is called after all events are processed, and is
meant to give the scripting language support an opportunity to
clean up e.g. destroy the interpreter instance, etc.

process_event() is called once for each event and takes as its
main parameter a pointer to the binary trace event record to be
processed. The implementation is responsible for picking out the
binary fields from the event record and sending them to the
script handler function associated with that event e.g. a
function derived from the event name it's meant to handle e.g.
'sched::sched_switch()'.  The 'format' information for trace
events can be used to parse the binary data and map it into a
form usable by a given scripting language; see the Perl
implemention in subsequent patches for one possible way to
leverage the existing trace format parsing code in perf and map
that info into specific scripting language types.

generate_script() should generate a ready-to-run script for the
current set of events in the trace, preferably with bodies that
print out every field for each event.  Again, look at the Perl
implementation for clues as to how that can be done.  This is an
optional, but very useful op.

Support for a given language should also add a language-specific
setup function and call it from setup_scripting().  The
language-specific setup function associates the the scripting
ops for that language with one or more 'language specifiers'
(see below) using script_spec_register().  When a script name is
specified on the command line, the scripting ops associated with
the specified language are used to instantiate and use the
appropriate interpreter to process the trace stream.

In general, it should be relatively easy to add support for a
new language, especially if the language implementation supports
an interface allowing an interpreter to be 'embedded' inside
another program (in this case the containing program will be
'perf trace'). If so, it should be relatively straightforward to
translate trace events into invocations of user-defined script
functions where e.g. the function name corresponds to the event
type and the function parameters correspond to the event fields.
 The event and field type information exported by the event
tracing infrastructure (via the event 'format' files) should be
enough to parse and send any piece of trace data to the user
script.  The easiest way to see how this can be done would be to
look at the Perl implementation contained in
perf/util/trace-event-perl.c/.h.

There are a couple of other things that aren't covered by the
scripting_ops or setup interface and are technically optional,
but should be implemented if possible.  One of these is support
for 'flag' and 'symbolic' fields e.g. being able to use more
human-readable values such as 'GFP_KERNEL' or
HI/BLOCK_IOPOLL/TASKLET in place of raw flag values.  See the
Perl implementation to see how this can be done. The other thing
is support for 'calling back' into the perf executable to access
e.g. uncommon fields not passed by default into handler
functions, or any metadata the implementation might want to make
available to users via the language interface.  Again, see the
Perl implementation for examples.

Signed-off-by: Tom Zanussi <tzanussi@gmail.com>
Cc: fweisbec@gmail.com
Cc: rostedt@goodmis.org
Cc: anton@samba.org
Cc: hch@infradead.org
LKML-Reference: <1259133352-23685-2-git-send-email-tzanussi@gmail.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-11-28 10:04:24 +01:00
Arnaldo Carvalho de Melo
62daacb51a perf tools: Reorganize event processing routines, lotsa dups killed
While implementing event__preprocess_sample, that will do all of
the symbol lookup in one convenient function, I noticed that
util/process_event.[ch] were not being used at all, then started
looking if there were other functions that could be shared
and...

All those functions really don't need to receive offset + head,
the only thing they did was common to all of them, so do it at
one place instead.

Stats about number of each type of event processed now is done
in a central place.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: John Kacur <jkacur@redhat.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1259346563-12568-11-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-11-27 20:22:01 +01:00
Arnaldo Carvalho de Melo
b32d133aec perf symbols: Simplify symbol machinery setup
And also express its configuration toggles via a struct.

Now all one has to do is to call symbol__init(NULL) if the
defaults are OK, or pass a struct symbol_conf pointer with the
desired configuration.

If a tool uses kernel_maps__find_symbol() to look at the kernel
and modules mappings for a symbol but didn't call symbol__init()
first, that will generate a one time warning too, alerting the
subcommand developer that symbol__init() must be called.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frédéric Weisbecker <fweisbec@gmail.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Paul Mackerras <paulus@samba.org>
LKML-Reference: <1259071517-3242-2-git-send-email-acme@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-11-24 16:37:02 +01:00