Граф коммитов

195 Коммитов

Автор SHA1 Сообщение Дата
Adrian Hunter febce6dc1f perf scripts python: export-to-postgresql.py: Export calls parent_id
Export to the 'calls' table the newly created 'parent_id' and create an
index for it.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: https://lkml.kernel.org/n/tip-eybd6fnk6j9r7g643lsideoo@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-03-01 14:53:33 -03:00
Adrian Hunter 07c5ebead8 perf scripts python: export-to-postgresql.py: Fix invalid input syntax for integer error
Fix SQL query error "invalid input syntax for integer":

  Traceback (most recent call last):
    File "tools/perf/scripts/python/export-to-postgresql.py", line 465, in <module>
      do_query(query, 'CREATE VIEW calls_view AS '
    File "tools/perf/scripts/python/export-to-postgresql.py", line 274, in do_query
      raise Exception("Query failed: " + q.lastError().text())
  Exception: Query failed: ERROR:  invalid input syntax for integer: ""
  LINE 1: ...ch_count,call_id,return_id,CASE WHEN flags=0 THEN '' WHEN fl...
                                                               ^
  (22P02) QPSQL: Unable to create query
  Error running python script tools/perf/scripts/python/export-to-postgresql.py

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Fixes: f08046cb30 ("perf thread-stack: Represent jmps to the start of a different symbol")
Link: https://lkml.kernel.org/n/tip-strfpdozrvg7bi1xzrivxzqt@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-03-01 14:53:12 -03:00
Adrian Hunter 8ce9a7251d perf scripts python: export-to-sqlite.py: Export calls parent_id
Export to the 'calls' table the newly created 'parent_id'.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: https://lkml.kernel.org/n/tip-b09oukl48rsl9azkp2wmh0bl@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-03-01 14:52:29 -03:00
Tony Jones de667cce7f perf script python: Add Python3 support to syscall-counts-by-pid.py
Support both Python2 and Python3 in the syscall-counts-by-pid.py script

There may be differences in the ordering of output lines due to
differences in dictionary ordering etc.  However the format within lines
should be unchanged.

The use of 'from __future__' implies the minimum supported Python2 version
is now v2.6

Signed-off-by: Tony Jones <tonyj@suse.de>
Link: http://lkml.kernel.org/r/20190222230619.17887-15-tonyj@suse.de
Signed-off-by: Seeteena Thoufeek <s1seetee@linux.vnet.ibm.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-02-25 17:17:13 -03:00
Tony Jones 1d1b0dbb85 perf script python: Add Python3 support to syscall-counts.py
Support both Python2 and Python3 in the syscall-counts.py script

There may be differences in the ordering of output lines due to
differences in dictionary ordering etc.  However the format within lines
should be unchanged.

The use of 'from __future__' implies the minimum supported Python2 version
is now v2.6

Signed-off-by: Tony Jones <tonyj@suse.de>
Link: http://lkml.kernel.org/r/20190222230619.17887-14-tonyj@suse.de
Signed-off-by: Seeteena Thoufeek <s1seetee@linux.vnet.ibm.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-02-25 17:17:10 -03:00
Tony Jones e985bf761d perf script python: Add Python3 support to stat-cpi.py
Support both Python2 and Python3 in the stat-cpi.py script

There may be differences in the ordering of output lines due to
differences in dictionary ordering etc.  However the format within lines
should be unchanged.

The use of 'from __future__' implies the minimum supported Python2 version
is now v2.6

Signed-off-by: Seeteena Thoufeek <s1seetee@linux.vnet.ibm.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Link: http://lkml.kernel.org/r/20190222230619.17887-13-tonyj@suse.de
Signed-off-by: Tony Jones <tonyj@suse.de>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-02-25 17:17:07 -03:00
Tony Jones 6d22d9991c perf script python: Add Python3 support to stackcollapse.py
Support both Python2 and Python3 in the stackcollapse.py script

There may be differences in the ordering of output lines due to
differences in dictionary ordering etc.  However the format within lines
should be unchanged.

The use of 'from __future__' implies the minimum supported Python2 version
is now v2.6

Signed-off-by: Tony Jones <tonyj@suse.de>
Cc: Paolo Bonzini <pbonzini@redhat.com> <pbonzini@redhat.com>
Link: http://lkml.kernel.org/r/20190222230619.17887-12-tonyj@suse.de
Signed-off-by: Seeteena Thoufeek <s1seetee@linux.vnet.ibm.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-02-25 17:17:05 -03:00
Tony Jones ee75a896ae perf script python: Add Python3 support to sctop.py
Support both Python2 and Python3 in the sctop.py script

There may be differences in the ordering of output lines due to
differences in dictionary ordering etc.  However the format within lines
should be unchanged.

The use of 'from __future__' implies the minimum supported Python2 version
is now v2.6

Signed-off-by: Tony Jones <tonyj@suse.de>
Cc: Tom Zanussi <tzanussi@gmail.com>
Link: http://lkml.kernel.org/r/20190222230619.17887-11-tonyj@suse.de
Signed-off-by: Seeteena Thoufeek <s1seetee@linux.vnet.ibm.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-02-25 17:17:03 -03:00
Tony Jones 118af5bf79 perf script python: Add Python3 support to powerpc-hcalls.py
Support both Python2 and Python3 in the powerpc-hcalls.py script

There may be differences in the ordering of output lines due to
differences in dictionary ordering etc.  However the format within lines
should be unchanged.

The use of 'from __future__' implies the minimum supported Python2 version
is now v2.6

Signed-off-by: Tony Jones <tonyj@suse.de>
Cc: Ravi Bangoria <ravi.bangoria@linux.ibm.com>
Link: http://lkml.kernel.org/r/20190222230619.17887-10-tonyj@suse.de
Signed-off-by: Seeteena Thoufeek <s1seetee@linux.vnet.ibm.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-02-25 17:16:57 -03:00
Tony Jones 8c42b9600e perf script python: Add Python3 support to net_dropmonitor.py
Support both Python2 and Python3 in the net_dropmonitor.py script

There may be differences in the ordering of output lines due to
differences in dictionary ordering etc.  However the format within lines
should be unchanged.

The use of 'from __future__' implies the minimum supported Python2 version
is now v2.6

Signed-off-by: Tony Jones <tonyj@suse.de>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Link: http://lkml.kernel.org/r/20190222230619.17887-9-tonyj@suse.de
Signed-off-by: Seeteena Thoufeek <s1seetee@linux.vnet.ibm.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-02-25 17:16:55 -03:00
Tony Jones e4d053ddb4 perf script python: Add Python3 support to mem-phys-addr.py
Support both Python2 and Python3 in the mem-phys-addr.py script

There may be differences in the ordering of output lines due to
differences in dictionary ordering etc.  However the format within lines
should be unchanged.

The use of 'from __future__' implies the minimum supported Python2 version
is now v2.6

Signed-off-by: Tony Jones <tonyj@suse.de>
Link: http://lkml.kernel.org/r/20190222230619.17887-8-tonyj@suse.de
Signed-off-by: Seeteena Thoufeek <s1seetee@linux.vnet.ibm.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-02-25 17:16:51 -03:00
Tony Jones 9b2700efc5 perf script python: Add Python3 support to failed-syscalls-by-pid.py
Support both Python2 and Python3 in the failed-syscalls-by-pid.py script

There may be differences in the ordering of output lines due to
differences in dictionary ordering etc.  However the format within lines
should be unchanged.

The use of 'from __future__' implies the minimum supported Python2 version
is now v2.6

Signed-off-by: Tony Jones <tonyj@suse.de>
Cc: Tom Zanussi <tzanussi@gmail.com>
Link: http://lkml.kernel.org/r/20190222230619.17887-5-tonyj@suse.de
Signed-off-by: Seeteena Thoufeek <s1seetee@linux.vnet.ibm.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-02-25 17:16:48 -03:00
Tony Jones 02b03ec383 perf script python: Add Python3 support to netdev-times.py
Support both Python2 and Python3 in the netdev-times.py script

There may be differences in the ordering of output lines due to
differences in dictionary ordering etc.  However the format within lines
should be unchanged.

The use of 'from __future__' implies the minimum supported Python2
version is now v2.6.

Signed-off-by: Tony Jones <tonyj@suse.de>
Cc: Sanagi Koki <sanagi.koki@jp.fujitsu.com>
Link: http://lkml.kernel.org/r/20190222230619.17887-2-tonyj@suse.de
Signed-off-by: Seeteena Thoufeek <s1seetee@linux.vnet.ibm.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-02-25 17:16:42 -03:00
Adrian Hunter cd358012ba perf scripts python: exported-sql-viewer.py: Add top calls report
Add a new report to display top calls by elapsed time. It displays calls
in descending order of time elapsed between when the function was called
and when it returned.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-02-22 16:52:07 -03:00
Adrian Hunter fc2c77aa84 perf scripts python: exported-sql-viewer.py: Remove no selection error
If no selection is made on the 'Selected branches' dialog, then the
output is the same as the 'All branches' report. That is not really an
error, and is not desirable for future reports, so remove it.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-02-22 16:52:07 -03:00
Adrian Hunter 0d5f8f230c perf scripts python: exported-sql-viewer.py: Remove SQLTableDialogDataItem
Remove SQLTableDialogDataItem as it is no longer used.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-02-22 16:52:07 -03:00
Adrian Hunter 1c3ca1b3ae perf scripts python: exported-sql-viewer.py: Create new dialog data item classes
Create new dialog data item classes to replace SQLTableDialogDataItem.
This separates out different dialog data items and makes it easier to
add new ones. SQLTableDialogDataItem is removed in a separate patch
because it makes the diff more readable.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-02-22 16:52:07 -03:00
Adrian Hunter 947cc38d47 perf scripts python: exported-sql-viewer.py: Move report name into ReportVars
The report name is a report variable so move it into into ReportVars.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-02-22 16:52:07 -03:00
Adrian Hunter 0bf0947a95 perf scripts python: exported-sql-viewer.py: Factor out ReportVars
Factor out ReportVars to provide a single container for information from
report dialogs.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-02-22 16:52:07 -03:00
Adrian Hunter 0924cd687f perf scripts python: exported-sql-viewer.py: Factor out ReportDialogBase
Factor out ReportDialogBase so it can be re-used.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-02-22 16:52:07 -03:00
Adrian Hunter 8c90fef9a8 perf scripts python: exported-sql-viewer.py: Move column headers
Move column headers from SQLAutoTableModel into SQLTableModel so that
they can be used for other models based on SQLTableModel.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-02-22 16:52:07 -03:00
Adrian Hunter 655cb952de perf scripts python: exported-sql-viewer.py: Hide Call Graph option if no calls table
The Call Graph depends on the calls table which is optional when exporting
data, so hide the Call Graph option if there is no calls table.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-02-22 16:52:07 -03:00
Adrian Hunter df8794fe68 perf scripts python: exported-sql-viewer.py: Remove leftover debugging prints
Remove leftover debugging prints.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-02-22 16:52:07 -03:00
Adrian Hunter b3a67546fd perf scripts python: exported-sql-viewer.py: Fix missing shebang
exported-sql-viewer.py is a standalone python script and requires a
shebang. Also only python2 is supported at present. Restore the shebang
but use the more flexible 'env' form.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: stable@vger.kernel.org
Fixes: a38352de44 ("perf script python: Remove explicit shebang from Python script")
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-02-22 16:52:07 -03:00
Jiri Olsa 5ff328836d perf tools: Rename build libperf to perf
Rename build libperf to perf, because it's used to build perf.

The libperf build object name will be used for libperf library.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/20190213123246.4015-4-jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-02-14 15:18:08 -03:00
Adrian Hunter f08046cb30 perf thread-stack: Represent jmps to the start of a different symbol
The compiler might optimize a call/ret combination by making it a jmp.
However the thread-stack does not presently cater for that, so that such
control flow is not visible in the call graph. Make it visible by
recording on the stack a branch to the start of a different symbol.
Note, that means when a ret pops the stack, all jmps must be popped off
first.

Example:

  $ cat jmp-to-fn.c
  __attribute__((noinline)) int bar(void)
  {
          return -1;
  }

  __attribute__((noinline)) int foo(void)
  {
          return bar() + 1;
  }

  int main()
  {
          return foo();
  }
  $ gcc -ggdb3 -Wall -Wextra -O2 -o jmp-to-fn jmp-to-fn.c
  $ objdump -d jmp-to-fn
  <SNIP>
  0000000000001040 <main>:
      1040:       31 c0                   xor    %eax,%eax
      1042:       e9 09 01 00 00          jmpq   1150 <foo>
  <SNIP>
  0000000000001140 <bar>:
      1140:       b8 ff ff ff ff          mov    $0xffffffff,%eax
      1145:       c3                      retq
  <SNIP>
  0000000000001150 <foo>:
      1150:       31 c0                   xor    %eax,%eax
      1152:       e8 e9 ff ff ff          callq  1140 <bar>
      1157:       83 c0 01                add    $0x1,%eax
      115a:       c3                      retq
  <SNIP>
  $ perf record -o jmp-to-fn.perf.data -e intel_pt/cyc/u ./jmp-to-fn
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0,017 MB jmp-to-fn.perf.data ]
  $ perf script -i jmp-to-fn.perf.data --itrace=be -s ~/libexec/perf-core/scripts/python/export-to-sqlite.py jmp-to-fn.db branches calls
  2019-01-08 13:24:58.783069 Creating database...
  2019-01-08 13:24:58.794650 Writing records...
  2019-01-08 13:24:59.008050 Adding indexes
  2019-01-08 13:24:59.015802 Done
  $  ~/libexec/perf-core/scripts/python/exported-sql-viewer.py jmp-to-fn.db

Before:

    main
        -> bar

After:

    main
        -> foo
            -> bar

Committer testing:

Install the python2-pyside package, then select these menu options
on the GUI:

   "Reports"
      "Context sensitive callgraphs"

Then go on expanding the symbols, to get, full picture when doing this
on a fedora:29 with gcc version 8.2.1 20181215 (Red Hat 8.2.1-6) (GCC):

jmp-to-fn
  PID:TID
    _start                (ld-2.28.so)
      __libc_start_main
        main
          foo
            bar

To verify that indeed, this fixes the problem.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Link: http://lkml.kernel.org/r/20190109091835.5570-5-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-02-06 10:00:40 -03:00
Tony Jones a38352de44 perf script python: Remove explicit shebang from Python scripts
The scripts in scripts/python are intended to be run from 'perf script'
and the Python version used is dictated by how perf was built (PYTHON=).

Also most distros follow pep-0394 which recommends that /usr/bin/python
refer to Python2 and so may not exist on the system (if PYTHON=python3).

- Remove the explicit shebang
- Install the scripts as mode 644

Signed-off-by: Tony Jones <tonyj@suse.de>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Ravi Bangoria <ravi.bangoria@linux.ibm.com>
Cc: Seeteena Thoufeek <s1seetee@linux.vnet.ibm.com>
Link: http://lkml.kernel.org/r/20190124005229.16146-6-tonyj@suse.de
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-01-25 15:12:10 +01:00
Adrian Hunter 35fa1cee21 perf scripts python: exported-sql-viewer.py: Fix table find when table re-ordered
Table rows can be re-ordered by selecting a column to sort by. After
re-ordering, the "find" operation was highlighting the wrong row, fix
it.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/r/20181104151238.15947-5-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-11-05 14:53:00 -03:00
Adrian Hunter 65b24292e8 perf scripts python: exported-sql-viewer.py: Add help window
Add a window to display help. It is also possible to display the help
only, by using the option "--help-only" instead of a database name.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/r/20181104151238.15947-4-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-11-05 14:52:45 -03:00
Adrian Hunter 210cf1f961 perf scripts python: exported-sql-viewer.py: Add Selected branches report
Fetching data from the database can be slow. Add a report that provides
the ability to select a subset of branches.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/r/20181104151238.15947-3-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-11-05 14:51:55 -03:00
Adrian Hunter 5ed4419d47 perf scripts python: exported-sql-viewer.py: Fall back to /usr/local/lib/libxed.so
Fall back to /usr/local/lib/libxed.so to cater for distributions that do
not have /usr/local/lib in the library path by default.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/r/20181104151238.15947-2-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-11-05 14:51:31 -03:00
Adrian Hunter 76099f98ae perf scripts python: exported-sql-viewer.py: Add All branches report
Add a report to display branches in a similar fashion to perf script. The
main purpose of this report is to display disassembly, however, presently,
the only supported disassembler is Intel XED, and additionally the object
code must be present in perf build ID cache.

To use Intel XED, libxed.so must be present. To build and install
libxed.so:
	git clone https://github.com/intelxed/mbuild.git mbuild
	git clone https://github.com/intelxed/xed
	cd xed
	./mfile.py --share
	sudo ./mfile.py --prefix=/usr/local install
	sudo ldconfig

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/r/20181023075949.18920-1-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-10-23 14:47:14 -03:00
Adrian Hunter 8392b74b57 perf scripts python: exported-sql-viewer.py: Add ability to display all the database tables
Displaying all the database tables can help make the database easier to
understand.

Committer testing:

Opened all the tables, even the sqlite master table, which I selected
everything and used control+C, lets see if it works...

  CREATE VIEW threads_view AS SELECT id,machine_id,(SELECT host_or_guest FROM machines_view WHERE id = machine_id) AS host_or_guest,process_id,pid,tid FROM threads

Humm, nope, just one of the cells got copied, even with everything selected :-)

Anyway, works as advertised, useful for perusing the data.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/r/20181001062853.28285-17-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-10-23 14:39:18 -03:00
Adrian Hunter 82f68e2898 perf scripts python: exported-sql-viewer.py: Add ability to shrink / enlarge font
Shrinking the font allows more information to display.

Committer testing:

Works, tested with the convenient Control+Shift+'+' and Control+'-' as
well with the more cumbersome top menu "Edit" + "Enlarge/Shrink font"
options.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/r/20181001062853.28285-16-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-10-23 14:34:40 -03:00
Adrian Hunter ebd70c7dc2 perf scripts python: exported-sql-viewer.py: Add ability to find symbols in the call-graph
Add a Find bar that appears at the bottom of the call-graph window.

Committer testing:

Using:

  python tools/perf/scripts/python/exported-sql-viewer.py pt_example branches calls

Using the database built in the first "Committer Testing" section in
this patch series I was able to:

  "Reports"
      "Context-Sensitive Call Graphs"
           Control+F or select "Edit" in the top menu then "Find"
                __poll<ENTER>

and find the first place where the "__poll" function appears, then
press the down arrow in the lower right corner and go to the next, etc.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/r/20181001062853.28285-15-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-10-23 14:30:33 -03:00
Adrian Hunter 1beb5c7b07 perf scripts python: exported-sql-viewer.py: Add support for multiple sub-windows
Use Qt MDI (multiple document interface) to support multiple sub-windows.
Put the data model in a cache so that each sub-window can share the same
data. This allows mutiple views of the call-graph at the same time and
paves the way to add more reports.

Committer testing:

Starts with a "File  Reports  Windows" main menu, from the "Reports" I
can get what was available up to now, the "Context-Sensitivi Call Graph"
option.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/r/20181001062853.28285-14-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-10-23 14:27:30 -03:00
Adrian Hunter 031c2a004b perf scripts python: call-graph-from-sql.py: Rename to exported-sql-viewer.py
Additional reports will be added to the script so rename to reflect the
more general purpose.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/r/20181001062853.28285-13-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-10-23 14:26:44 -03:00
Adrian Hunter 341e73cbd3 perf scripts python: call-graph-from-sql.py: Refactor TreeItem class
class TreeItem represents items at all levels of the call-graph tree.
However, not all the levels represent the same data i.e. the top-level is
comms, the next level is threads, and subsequent levels are functions.
Consequently it is simpler to have separate classes for different levels
with commonality in a base class. Refactor TreeItem class accordingly.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/r/20181001062853.28285-12-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-10-23 14:26:06 -03:00
Adrian Hunter 4be9ace7e1 perf scripts python: call-graph-from-sql.py: Add data helper functions
Add helper functions for a few common cases.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/r/20181001062853.28285-11-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-10-23 14:25:42 -03:00
Adrian Hunter 70d831e85c perf scripts python: call-graph-from-sql.py: Factor out CallGraphModel from TreeModel
Factor out CallGraphModel from TreeModel, which paves the way to reuse
TreeModel in future reports.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/r/20181001062853.28285-10-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-10-23 14:23:52 -03:00
Adrian Hunter e99ef8141a perf scripts python: call-graph-from-sql.py: Remove use of setObjectName()
The object name is never used, so don't bother setting it.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/r/20181001062853.28285-9-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-10-23 14:23:41 -03:00
Adrian Hunter 5f9dfef1bb perf scripts python: call-graph-from-sql.py: Add a class for global data
Keep global data in a single object that is easy to pass around as
needed, without polluting the global namespace.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/r/20181001062853.28285-8-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-10-23 14:23:31 -03:00
Adrian Hunter b2556c46a6 perf scripts python: call-graph-from-sql.py: Separate the database details into a class
Separate the database details into a class that can provide different
connections using the same connection information.  That paves the way
for sub-processes that require their own connection.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/r/20181001062853.28285-7-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-10-23 14:23:13 -03:00
Adrian Hunter 7e4fc93e2a perf scripts python: call-graph-from-sql.py: Make a "Main" function
Make a "Main" function so that the variables used do not pollute the global
namespace.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/r/20181001062853.28285-6-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-10-23 14:22:54 -03:00
Adrian Hunter 99a097c987 perf scripts python: call-graph-from-sql.py: Change icon
There are not many standard icons, but the computer icon looks slightly
better than the information icon.

Committer testing:

Noticed the change on the icon on the gnome menu right next to the
"Activities" menu, looks nicer indeed.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/r/20181001062853.28285-5-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-10-23 14:21:43 -03:00
Adrian Hunter 3c4ef45150 perf scripts python: call-graph-from-sql.py: Set a minimum window size
Prevent weirdly small window size.

Committer testing:

Seems to work, but even before this patch, on my system, it always
started with:

xwininfo: Window id: 0x1e00002 "Call Graph: pt_example"
<SNIP>
  Width: 800
  Height: 600
<SNIP>

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/r/20181001062853.28285-4-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-10-23 14:19:30 -03:00
Adrian Hunter 1d865c06f5 perf scripts python: call-graph-from-sql.py: Provide better default column sizes
Set initial column sizes to improve initial display.

Committer testing:

Extended instructions on testing this, using the sqlite variant:

Make sure you have the SQLite glue for python+Qt installed, on fedora 27
I used:

  # dnf install python-pyside

Collect some PT samples, say 5-secs worth, system wide:

  # perf record -r 10 -e intel_pt//u -a sleep 5
  [ perf record: Woken up 49 times to write data ]
  [ perf record: Captured and wrote 96.131 MB perf.data ]

This results in this perf.data file:

  # ls -larth perf.data
  -rw-------. 1 root root 97M Oct 23 10:11 perf.data

With the following attributes:

  # perf evlist -v
  intel_pt//u: type: 8, size: 112, config: 0x300e601, { sample_period, sample_freq }: 1, sample_type: IP|TID|TIME|CPU|IDENTIFIER, read_format: ID, disabled: 1, inherit: 1, exclude_kernel: 1, exclude_hv: 1, sample_id_all: 1
  dummy:u: type: 1, size: 112, config: 0x9, { sample_period, sample_freq }: 1, sample_type: IP|TID|TIME|CPU|IDENTIFIER, read_format: ID, inherit: 1, exclude_kernel: 1, exclude_hv: 1, mmap: 1, comm: 1, task: 1, sample_id_all: 1, mmap2: 1, comm_exec: 1, context_switch: 1
  #

Then generate the "pt_example" tables using:

  # perf script -s ~/libexec/perf-core/scripts/python/export-to-sqlite.py pt_example branches calls
  2018-10-23 10:56:59.177711 Creating database...
  2018-10-23 10:56:59.195842 Writing records...
   instruction trace error type 1 cpu 2 pid 1644 tid 1644 ip 0x263984516750 code 5: Failed to get instruction
   instruction trace error type 1 cpu 2 pid 1644 tid 1644 ip 0x7f26e116fd20 code 6: Trace doesn't match instruction
   instruction trace error type 1 cpu 2 pid 1644 tid 1644 ip 0x7f26e162c9ee code 6: Trace doesn't match instruction
   instruction trace error type 1 cpu 2 pid 1644 tid 1644 ip 0x7f26e9ce831a code 6: Trace doesn't match instruction
  <SNIP>
   instruction trace error type 1 cpu 0 pid 1644 tid 1644 ip 0x7f26e13d07b4 code 6: Trace doesn't match instruction
  Warning:
  132 instruction trace errors
  2018-10-23 11:25:25.015717 Adding indexes
  2018-10-23 11:25:28.788061 Done
  #

In my example, that perf.data file generated this db:

  # file pt_example
  pt_example: SQLite 3.x database, last written using SQLite version 3020001
  [root@seventh perf]# ls -lah pt_example
  -rw-r--r--. 1 root root 6.6G Oct 23 11:25 pt_example
  #

Then use this python script to use that db and provide a GUI:

  $ python tools/perf/scripts/python/call-graph-from-sql.py pt_example branches calls

I compared the column widths before this patch and after applying it,
the visual results match the patch intent.

The following patches will refer to this set of instructions in the "Committer
Testing" section.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/r/20181001062853.28285-3-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-10-23 14:15:30 -03:00
Adrian Hunter 3e71c70c94 perf scripts python: call-graph-from-sql.py: Use SPDX license identifier
Use SPDX license identifier in call-graph-from-sql.py.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/r/20181001062853.28285-2-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-10-22 14:28:04 -03:00
Adrian Hunter d005efe18d perf script python: Fix export-to-sqlite.py sample columns
With the "branches" export option, not all sample columns are exported.
However the unwanted columns are not at the end of the tuple, as assumed
by the code. Fix by taking the first 15 and last 3 values, instead of
the first 18.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: stable@vger.kernel.org
Link: http://lkml.kernel.org/r/20180911114504.28516-3-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-09-25 11:37:05 -03:00
Adrian Hunter 25e11700b5 perf script python: Fix export-to-postgresql.py occasional failure
Occasional export failures were found to be caused by truncating 64-bit
pointers to 32-bits. Fix by explicitly setting types for all ctype
arguments and results.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: stable@vger.kernel.org
Link: http://lkml.kernel.org/r/20180911114504.28516-2-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-09-25 11:33:06 -03:00
Jeremy Cline 12aa6c7389 perf scripts python: Add Python 3 support to EventClass.py
Support both Python 2 and Python 3 in EventClass.py. ``print`` is now a
function rather than a statement. This should have no functional change.

Signed-off-by: Jeremy Cline <jeremy@jcline.org>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Herton Krzesinski <herton@redhat.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/0100016341a73aac-e0734bdc-dcab-4c61-8333-d8be97524aa0-000000@email.amazonses.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-07-11 10:01:50 -03:00
Jeremy Cline 8c1c1ab2d2 perf scripts python: Add Python 3 support to sched-migration.py
Support both Python 2 and Python 3 in the sched-migration.py script.
This should have no functional change.

Signed-off-by: Jeremy Cline <jeremy@jcline.org>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Herton Krzesinski <herton@redhat.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/0100016341a737a5-44ec436f-3440-4cac-a03f-ddfa589bf308-000000@email.amazonses.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-07-11 10:01:50 -03:00
Jeremy Cline c45b168eff perf scripts python: Add Python 3 support to Util.py
Support both Python 2 and Python 3 in Util.py. The dict class no longer
has a ``has_key`` method and print is now a function rather than a
statement. This should have no functional change.

Signed-off-by: Jeremy Cline <jeremy@jcline.org>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Herton Krzesinski <herton@redhat.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/0100016341a730c6-8db8b9b1-da2d-4ee3-96bf-47e0ae9796bd-000000@email.amazonses.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-07-11 10:01:50 -03:00
Jeremy Cline 2ab89262ff perf scripts python: Add Python 3 support to SchedGui.py
Fix a single syntax error in SchedGui.py to support both Python 2 and
Python 3. This should have no functional change.

Signed-off-by: Jeremy Cline <jeremy@jcline.org>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Herton Krzesinski <herton@redhat.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/0100016341a72d26-75729663-fe55-4309-8c9b-302e065ed2f1-000000@email.amazonses.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-07-11 10:01:50 -03:00
Jeremy Cline 770d2f86c0 perf scripts python: Add Python 3 support to Core.py
Support both Python 2 and Python 3 in Core.py. This should have no
functional change.

Signed-off-by: Jeremy Cline <jeremy@jcline.org>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Herton Krzesinski <herton@redhat.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/0100016341a72ebe-e572899e-f445-4765-98f0-c314935727f9-000000@email.amazonses.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-07-11 10:01:50 -03:00
Ravi Bangoria ec1e6e6a68 perf script powerpc: Python script for hypervisor call statistics
Add python script to show hypervisor call statistics. Ex,

  # perf record -a -e "{powerpc:hcall_entry,powerpc:hcall_exit}"
  # perf script -s scripts/python/powerpc-hcalls.py
    hcall                            count   min(ns)   max(ns)   avg(ns)
    --------------------------------------------------------------------
    H_RANDOM                            82       838      1164       904
    H_PUT_TCE                           47      1078      5928      2003
    H_EOI                              266      1336      3546      1654
    H_ENTER                             28      1646      4038      1952
    H_PUT_TCE_INDIRECT                 230      2166     18168      6109
    H_IPI                              238      1072      3232      1688
    H_SEND_LOGICAL_LAN                  42      5488     21366      7694
    H_STUFF_TCE                        294       986      6210      3591
    H_XIRR                             266      2286      6990      3783
    H_PROTECT                           10      2196      3556      2555
    H_VIO_SIGNAL                       294      1028      2784      1311
    H_ADD_LOGICAL_LAN_BUFFER            53      1978      3450      2600
    H_SEND_CRQ                          77      1762      7240      2447

Signed-off-by: Ravi Bangoria <ravi.bangoria@linux.ibm.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
Link: http://lkml.kernel.org/r/20180605124801.17210-1-ravi.bangoria@linux.ibm.com
[ Fixup typo: table_loockup -> table_lookup ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-06-06 12:52:06 -03:00
Jaroslav Škarvada 66dfdff03d perf tools: Add Python 3 support
Added Python 3 support while keeping Python 2.7 compatibility.

Committer notes:

This doesn't make it to auto detect python 3, one has to explicitely ask
it to build with python 3 devel files, here are the instructions
provided by Jaroslav:

 ---
  $ cp -a tools/perf tools/python3-perf
  $ make V=1 prefix=/usr -C tools/perf PYTHON=/usr/bin/python2 all
  $ make V=1 prefix=/usr -C tools/python3-perf PYTHON=/usr/bin/python3 all
  $ make V=1 prefix=/usr -C tools/python3-perf PYTHON=/usr/bin/python3 DESTDIR=%{buildroot} install-python_ext
  $ make V=1 prefix=/usr -C tools/perf PYTHON=/usr/bin/python2 DESTDIR=%{buildroot} install-python_ext
 ---

We need to make this automatic, just like the existing tests for checking if
the python2 devel files are in place, allowing the build with python3 if
available, fallbacking to python2 and then just disabling it if none are
available.

So, using the PYTHON variable to build it using O= we get:

Before this patch:

  $ rpm -q python3 python3-devel
  python3-3.6.4-7.fc27.x86_64
  python3-devel-3.6.4-7.fc27.x86_64
  $ rm -rf /tmp/build/perf/ ; mkdir -p /tmp/build/perf ; make O=/tmp/build/perf PYTHON=/usr/bin/python3 -C tools/perf install-bin
  make: Entering directory '/home/acme/git/linux/tools/perf'
  <SNIP>
  Makefile.config:670: Python 3 is not yet supported; please set
  Makefile.config:671: PYTHON and/or PYTHON_CONFIG appropriately.
  Makefile.config:672: If you also have Python 2 installed, then
  Makefile.config:673: try something like:
  Makefile.config:674:
  Makefile.config:675:   make PYTHON=python2
  Makefile.config:676:
  Makefile.config:677: Otherwise, disable Python support entirely:
  Makefile.config:678:
  Makefile.config:679:   make NO_LIBPYTHON=1
  Makefile.config:680:
  Makefile.config:681: *** .  Stop.
  make[1]: *** [Makefile.perf:212: sub-make] Error 2
  make: *** [Makefile:110: install-bin] Error 2
  make: Leaving directory '/home/acme/git/linux/tools/perf'
  $

After:

  $ make O=/tmp/build/perf PYTHON=python3 -C tools/perf install-bin
  $ ldd ~/bin/perf | grep python
	libpython3.6m.so.1.0 => /lib64/libpython3.6m.so.1.0 (0x00007f58a31e8000)
  $ rpm -qf /lib64/libpython3.6m.so.1.0
  python3-libs-3.6.4-7.fc27.x86_64
  $

Now verify that when using the binding the right ELF file is loaded,
using perf trace:

  $ perf trace -e open* perf test python
     0.051 ( 0.016 ms): perf/3927 openat(dfd: CWD, filename: /etc/ld.so.cache, flags: CLOEXEC           ) = 3
<SNIP>
  18: 'import perf' in python                               :
     8.849 ( 0.013 ms): sh/3929 openat(dfd: CWD, filename: /etc/ld.so.cache, flags: CLOEXEC           ) = 3
<SNIP>
    25.572 ( 0.008 ms): python3/3931 openat(dfd: CWD, filename: /tmp/build/perf/python/perf.cpython-36m-x86_64-linux-gnu.so, flags: CLOEXEC) = 3
<SNIP>
 Ok
<SNIP>
  $

And using tools/perf/python/twatch.py, to show PERF_RECORD_ metaevents:

  $ python3 tools/perf/python/twatch.py
  cpu: 3, pid: 16060, tid: 16060 { type: fork, pid: 5207, ppid: 16060, tid: 5207, ptid: 16060, time: 10798513015459}
  cpu: 3, pid: 16060, tid: 16060 { type: fork, pid: 5208, ppid: 16060, tid: 5208, ptid: 16060, time: 10798513562503}
  cpu: 0, pid: 5208, tid: 5208 { type: comm, pid: 5208, tid: 5208, comm: grep }
  cpu: 2, pid: 5207, tid: 5207 { type: comm, pid: 5207, tid: 5207, comm: ps }
  cpu: 2, pid: 5207, tid: 5207 { type: exit, pid: 5207, ppid: 5207, tid: 5207, ptid: 5207, time: 10798551337484}
  cpu: 3, pid: 5208, tid: 5208 { type: exit, pid: 5208, ppid: 5208, tid: 5208, ptid: 5208, time: 10798551292153}
  cpu: 3, pid: 601, tid: 601 { type: fork, pid: 5209, ppid: 601, tid: 5209, ptid: 601, time: 10801779977324}
  ^CTraceback (most recent call last):
    File "tools/perf/python/twatch.py", line 68, in <module>
      main()
    File "tools/perf/python/twatch.py", line 40, in main
      evlist.poll(timeout = -1)
  KeyboardInterrupt
  $

  # ps ax|grep twatch
 5197 pts/8    S+     0:00 python3 tools/perf/python/twatch.py
  # ls -la /proc/5197/smaps
  -r--r--r--. 1 acme acme 0 Feb 19 13:14 /proc/5197/smaps
  # grep python /proc/5197/smaps
  558111307000-558111309000 r-xp 00000000 fd:00 3151710  /usr/bin/python3.6
  558111508000-558111509000 r--p 00001000 fd:00 3151710  /usr/bin/python3.6
  558111509000-55811150a000 rw-p 00002000 fd:00 3151710  /usr/bin/python3.6
  7ffad6fc1000-7ffad7008000 r-xp 00000000 00:2d 220196   /tmp/build/perf/python/perf.cpython-36m-x86_64-linux-gnu.so
  7ffad7008000-7ffad7207000 ---p 00047000 00:2d 220196   /tmp/build/perf/python/perf.cpython-36m-x86_64-linux-gnu.so
  7ffad7207000-7ffad7208000 r--p 00046000 00:2d 220196   /tmp/build/perf/python/perf.cpython-36m-x86_64-linux-gnu.so
  7ffad7208000-7ffad7215000 rw-p 00047000 00:2d 220196   /tmp/build/perf/python/perf.cpython-36m-x86_64-linux-gnu.so
  7ffadea77000-7ffaded3d000 r-xp 00000000 fd:00 3151795  /usr/lib64/libpython3.6m.so.1.0
  7ffaded3d000-7ffadef3c000 ---p 002c6000 fd:00 3151795  /usr/lib64/libpython3.6m.so.1.0
  7ffadef3c000-7ffadef42000 r--p 002c5000 fd:00 3151795  /usr/lib64/libpython3.6m.so.1.0
  7ffadef42000-7ffadefa5000 rw-p 002cb000 fd:00 3151795  /usr/lib64/libpython3.6m.so.1.0
  #

And with this patch, but building normally, without specifying the
PYTHON=python3 part, which will make it use python2 if its devel files are
available, like in this test:

  $ make O=/tmp/build/perf -C tools/perf install-bin
  $ ldd ~/bin/perf | grep python
	libpython2.7.so.1.0 => /lib64/libpython2.7.so.1.0 (0x00007f6a44410000)
  $ ldd /tmp/build/perf/python_ext_build/lib/perf.so  | grep python
	libpython2.7.so.1.0 => /lib64/libpython2.7.so.1.0 (0x00007fed28a2c000)
  $

  [acme@jouet perf]$ tools/perf/python/twatch.py
  cpu: 0, pid: 2817, tid: 2817 { type: fork, pid: 2817, ppid: 2817, tid: 8910, ptid: 2817, time: 11126454335306}
  cpu: 0, pid: 2817, tid: 2817 { type: comm, pid: 2817, tid: 8910, comm: worker }
  $ ps ax | grep twatch.py
   8909 pts/8    S+     0:00 /usr/bin/python tools/perf/python/twatch.py
  $ grep python /proc/8909/smaps
  5579de658000-5579de659000 r-xp 00000000 fd:00 3156044  /usr/bin/python2.7
  5579de858000-5579de859000 r--p 00000000 fd:00 3156044  /usr/bin/python2.7
  5579de859000-5579de85a000 rw-p 00001000 fd:00 3156044  /usr/bin/python2.7
  7f0de01f7000-7f0de023e000 r-xp 00000000 00:2d 230695   /tmp/build/perf/python/perf.so
  7f0de023e000-7f0de043d000 ---p 00047000 00:2d 230695   /tmp/build/perf/python/perf.so
  7f0de043d000-7f0de043e000 r--p 00046000 00:2d 230695   /tmp/build/perf/python/perf.so
  7f0de043e000-7f0de044b000 rw-p 00047000 00:2d 230695   /tmp/build/perf/python/perf.so
  7f0de6f0f000-7f0de6f13000 r-xp 00000000 fd:00 134975   /usr/lib64/python2.7/lib-dynload/_localemodule.so
  7f0de6f13000-7f0de7113000 ---p 00004000 fd:00 134975   /usr/lib64/python2.7/lib-dynload/_localemodule.so
  7f0de7113000-7f0de7114000 r--p 00004000 fd:00 134975   /usr/lib64/python2.7/lib-dynload/_localemodule.so
  7f0de7114000-7f0de7115000 rw-p 00005000 fd:00 134975   /usr/lib64/python2.7/lib-dynload/_localemodule.so
  7f0de7e73000-7f0de8052000 r-xp 00000000 fd:00 3173292  /usr/lib64/libpython2.7.so.1.0
  7f0de8052000-7f0de8251000 ---p 001df000 fd:00 3173292  /usr/lib64/libpython2.7.so.1.0
  7f0de8251000-7f0de8255000 r--p 001de000 fd:00 3173292  /usr/lib64/libpython2.7.so.1.0
  7f0de8255000-7f0de8291000 rw-p 001e2000 fd:00 3173292  /usr/lib64/libpython2.7.so.1.0
  $

Signed-off-by: Jaroslav Škarvada <jskarvad@redhat.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Wang Nan <wangnan0@huawei.com>
LPU-Reference: 20180119205641.24242-1-jskarvad@redhat.com
Link: https://lkml.kernel.org/n/tip-8d7dt9kqp83vsz25hagug8fu@git.kernel.org
[ Removed explicit check for python version, allowing it to really build with python3 ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-02-19 12:28:23 -03:00
Kan Liang 41013f0c09 perf script python: Add script to profile and resolve physical mem type
There could be different types of memory in the system. E.g normal
System Memory, Persistent Memory. To understand how the workload maps to
those memories, it's important to know the I/O statistics of them.  Perf
can collect physical addresses, but those are raw data.  It still needs
extra work to resolve the physical addresses.  Provide a script to
facilitate the physical addresses resolving and I/O statistics.

Profile with MEM_INST_RETIRED.ALL_LOADS or MEM_UOPS_RETIRED.ALL_LOADS
event if any of them is available.

Look up the /proc/iomem and resolve the physical address.  Provide
memory type summary.

Here is an example output:

  # perf script report mem-phys-addr
  Event: mem_inst_retired.all_loads:P
  Memory type                                    count   percentage
  ----------------------------------------  -----------  -----------
  System RAM                                        74        53.2%
  Persistent Memory                                 55        39.6%
  N/A

  ---

Changes since V2:
 - Apply the new license rules.
 - Add comments for globals

Changes since V1:
 - Do not mix DLA and Load Latency. Do not compare the loads and stores.
   Only profile the loads.
 - Use event name to replace the RAW event

Signed-off-by: Kan Liang <Kan.liang@intel.com>
Reviewed-by: Andi Kleen <ak@linux.intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Philippe Ombredanne <pombredanne@nexb.com>
Cc: Stephane Eranian <eranian@google.com>
Link: https://lkml.kernel.org/r/1515099595-34770-1-git-send-email-kan.liang@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-01-12 11:06:57 -03:00
Greg Kroah-Hartman b24413180f License cleanup: add SPDX GPL-2.0 license identifier to files with no license
Many source files in the tree are missing licensing information, which
makes it harder for compliance tools to determine the correct license.

By default all files without license information are under the default
license of the kernel, which is GPL version 2.

Update the files which contain no license information with the 'GPL-2.0'
SPDX license identifier.  The SPDX identifier is a legally binding
shorthand, which can be used instead of the full boiler plate text.

This patch is based on work done by Thomas Gleixner and Kate Stewart and
Philippe Ombredanne.

How this work was done:

Patches were generated and checked against linux-4.14-rc6 for a subset of
the use cases:
 - file had no licensing information it it.
 - file was a */uapi/* one with no licensing information in it,
 - file was a */uapi/* one with existing licensing information,

Further patches will be generated in subsequent months to fix up cases
where non-standard license headers were used, and references to license
had to be inferred by heuristics based on keywords.

The analysis to determine which SPDX License Identifier to be applied to
a file was done in a spreadsheet of side by side results from of the
output of two independent scanners (ScanCode & Windriver) producing SPDX
tag:value files created by Philippe Ombredanne.  Philippe prepared the
base worksheet, and did an initial spot review of a few 1000 files.

The 4.13 kernel was the starting point of the analysis with 60,537 files
assessed.  Kate Stewart did a file by file comparison of the scanner
results in the spreadsheet to determine which SPDX license identifier(s)
to be applied to the file. She confirmed any determination that was not
immediately clear with lawyers working with the Linux Foundation.

Criteria used to select files for SPDX license identifier tagging was:
 - Files considered eligible had to be source code files.
 - Make and config files were included as candidates if they contained >5
   lines of source
 - File already had some variant of a license header in it (even if <5
   lines).

All documentation files were explicitly excluded.

The following heuristics were used to determine which SPDX license
identifiers to apply.

 - when both scanners couldn't find any license traces, file was
   considered to have no license information in it, and the top level
   COPYING file license applied.

   For non */uapi/* files that summary was:

   SPDX license identifier                            # files
   ---------------------------------------------------|-------
   GPL-2.0                                              11139

   and resulted in the first patch in this series.

   If that file was a */uapi/* path one, it was "GPL-2.0 WITH
   Linux-syscall-note" otherwise it was "GPL-2.0".  Results of that was:

   SPDX license identifier                            # files
   ---------------------------------------------------|-------
   GPL-2.0 WITH Linux-syscall-note                        930

   and resulted in the second patch in this series.

 - if a file had some form of licensing information in it, and was one
   of the */uapi/* ones, it was denoted with the Linux-syscall-note if
   any GPL family license was found in the file or had no licensing in
   it (per prior point).  Results summary:

   SPDX license identifier                            # files
   ---------------------------------------------------|------
   GPL-2.0 WITH Linux-syscall-note                       270
   GPL-2.0+ WITH Linux-syscall-note                      169
   ((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause)    21
   ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause)    17
   LGPL-2.1+ WITH Linux-syscall-note                      15
   GPL-1.0+ WITH Linux-syscall-note                       14
   ((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause)    5
   LGPL-2.0+ WITH Linux-syscall-note                       4
   LGPL-2.1 WITH Linux-syscall-note                        3
   ((GPL-2.0 WITH Linux-syscall-note) OR MIT)              3
   ((GPL-2.0 WITH Linux-syscall-note) AND MIT)             1

   and that resulted in the third patch in this series.

 - when the two scanners agreed on the detected license(s), that became
   the concluded license(s).

 - when there was disagreement between the two scanners (one detected a
   license but the other didn't, or they both detected different
   licenses) a manual inspection of the file occurred.

 - In most cases a manual inspection of the information in the file
   resulted in a clear resolution of the license that should apply (and
   which scanner probably needed to revisit its heuristics).

 - When it was not immediately clear, the license identifier was
   confirmed with lawyers working with the Linux Foundation.

 - If there was any question as to the appropriate license identifier,
   the file was flagged for further research and to be revisited later
   in time.

In total, over 70 hours of logged manual review was done on the
spreadsheet to determine the SPDX license identifiers to apply to the
source files by Kate, Philippe, Thomas and, in some cases, confirmation
by lawyers working with the Linux Foundation.

Kate also obtained a third independent scan of the 4.13 code base from
FOSSology, and compared selected files where the other two scanners
disagreed against that SPDX file, to see if there was new insights.  The
Windriver scanner is based on an older version of FOSSology in part, so
they are related.

Thomas did random spot checks in about 500 files from the spreadsheets
for the uapi headers and agreed with SPDX license identifier in the
files he inspected. For the non-uapi files Thomas did random spot checks
in about 15000 files.

In initial set of patches against 4.14-rc6, 3 files were found to have
copy/paste license identifier errors, and have been fixed to reflect the
correct identifier.

Additionally Philippe spent 10 hours this week doing a detailed manual
inspection and review of the 12,461 patched files from the initial patch
version early this week with:
 - a full scancode scan run, collecting the matched texts, detected
   license ids and scores
 - reviewing anything where there was a license detected (about 500+
   files) to ensure that the applied SPDX license was correct
 - reviewing anything where there was no detection but the patch license
   was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied
   SPDX license was correct

This produced a worksheet with 20 files needing minor correction.  This
worksheet was then exported into 3 different .csv files for the
different types of files to be modified.

These .csv files were then reviewed by Greg.  Thomas wrote a script to
parse the csv files and add the proper SPDX tag to the file, in the
format that the file expected.  This script was further refined by Greg
based on the output to detect more types of files automatically and to
distinguish between header and source .c files (which need different
comment types.)  Finally Greg ran the script using the .csv files to
generate the patches.

Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2017-11-02 11:10:55 +01:00
Adrian Hunter 1fe03b5f2d perf script python: Add support for sqlite3 to call-graph-from-sql.py
Add support for SQLite 3 to the call-graph-from-sql.py script. The SQL
statements work as is, so just detect the database type by checking if the
SQLite 3 file exists.

Committer notes:

Tested collecting the PT data on a RHEL7.4, generating the SQLite3
database there and then moving it to a Fedora 26 system where the
call-graph-from-sql.py script was run, using python-pyside version
1.2.2-7fc26 to see the callgraphs using Qt4.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Link: http://lkml.kernel.org/r/1501749090-20357-6-git-send-email-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-08-15 17:03:38 -03:00
Adrian Hunter 69e6e410f1 perf script python: Rename call-graph-from-postgresql.py to call-graph-from-sql.py
Rename call-graph-from-postgresql.py to call-graph-from-sql.py in
preparation for adding support to it for SQLite 3.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Link: http://lkml.kernel.org/r/1501749090-20357-5-git-send-email-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-08-15 16:38:06 -03:00
Adrian Hunter 564b9527d1 perf script python: Add support for exporting to sqlite3
Add support for exporting to SQLite 3 the same data as the PostgreSQL
export.

Committer note:

Tested on RHEL 7.4 using the 1.2.2-4el python-pyside packages from EPEL.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Link: http://lkml.kernel.org/r/1501749090-20357-4-git-send-email-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-08-15 16:37:55 -03:00
Adrian Hunter 2295e9f850 perf scripts python: Fix query in call-graph-from-postgresql.py
Add a missing space which seemed not to affect PostgreSQL but upsets
SQLite.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Link: http://lkml.kernel.org/r/1501749090-20357-3-git-send-email-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-08-15 16:06:20 -03:00
Adrian Hunter c8a827285c perf scripts python: Fix missing call_path_id in export-to-postgresql script
The export does not work if only branches are exported because of a
missing column in the samples table.  Fix by adding the missing
call_path_id.

Fixes: 3521f3bc9d ("perf script: Update export-to-postgresql to support callchain export")
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Link: http://lkml.kernel.org/r/1501749090-20357-2-git-send-email-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-08-15 16:05:36 -03:00
Naveen N. Rao 6fae8663c9 perf scripting python: Add ppc64le to audit uname list
Before patch:

  $ uname -m
  ppc64le
  $ ./perf script -s ./scripts/python/syscall-counts.py
  Install the audit-libs-python package to get syscall names.
  For example:
    # apt-get install python-audit (Ubuntu)
    # yum install audit-libs-python (Fedora)
    etc.

  Press control+C to stop and show the summary
  ^CWarning:
  4 out of order events recorded.

  syscall events:

  event                                          count
  ----------------------------------------  -----------
  4                                             504638
  54                                              1206
  221                                               42
  55                                                21
  3                                                 12
  167                                               10
  11                                                 8
  6                                                  7
  125                                                6
  5                                                  6
  108                                                5
  162                                                4
  90                                                 4
  45                                                 3
  33                                                 3
  311                                                1
  246                                                1
  238                                                1
  93                                                 1
  91                                                 1

After patch:
  ./perf script -s ./scripts/python/syscall-counts.py
  Press control+C to stop and show the summary
  ^CWarning:
  5 out of order events recorded.

  syscall events:

  event                                          count
  ----------------------------------------  -----------
  write                                         643411
  ioctl                                           1206
  futex                                             54
  fcntl                                             27
  poll                                              14
  read                                              12
  execve                                             8
  close                                              7
  mprotect                                           6
  open                                               6
  nanosleep                                          5
  fstat                                              5
  mmap                                               4
  inotify_add_watch                                  3
  brk                                                3
  access                                             3
  timerfd_settime                                    1
  clock_gettime                                      1
  epoll_wait                                         1
  ftruncate                                          1
  munmap                                             1

Signed-off-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
Acked-by: Paul Clarke <pc@us.ibm.com>
Cc: linuxppc-dev@lists.ozlabs.org
Link: http://lkml.kernel.org/n/tip-bnl67p1alkvx97pn9moxz3qp@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-08-11 10:42:52 -03:00
Adrian Hunter cc892720d8 perf intel-pt: Add example script for power events and PTWRITE
Add script intel-pt-events.py that provides an example of how to unpack the
raw data for power events and PTWRITE.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Link: http://lkml.kernel.org/r/1495786658-18063-35-git-send-email-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-06-30 11:50:53 -03:00
Ingo Molnar f282f7a0ec perf/core improvements and fixes:
New features:
 
 - Add --sample-cpu to 'perf record', to explicitely ask for sampling
   the CPU (Jiri Olsa)
 
 Fixes:
 
 - Fix processing of multi byte chunks in objdump output, fixing
   disassemble processing for annotation on at least ARM64 (Jan Stancek)
 
 - Use SyS_epoll_wait in a BPF 'perf test' entry instead of sys_epoll_wait, that
   is not present in the DWARF info in vmlinux files (Arnaldo Carvalho de Melo)
 
 - Add -wno-shadow when processing files using perl headers, fixing
   the build on Fedora Rawhide and Arch Linux (Namhyung Kim)
 
 Infrastructure:
 
 - Annotate prep work to better catch and report errors related to
   using objdump to disassemble DSOs (Arnaldo Carvalho de Melo)
 
 - Add 'alloc', 'scnprintf' and 'and' methods for bitmap processing (Jiri Olsa)
 
 - Add nested output resorting callback in hists processing (Jiri Olsa)
 
 Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2
 
 iQIcBAABCAAGBQJXopDWAAoJENZQFvNTUqpAowMP/RYzsngLAfx2c1OGdXSYdesD
 0bdkTwTA/ob+uQdSvksR6Y9zf+I5nD6dwB6NbBbeXgs8ZwWNNjLObz1hFtYZYQ6j
 qBlf1iJ6k0cHm8EaF5fs0J6/RyU+TasqBrgDiqiMTlHJs5gsyD892vVqz0SxiKwP
 i1nbyG8VrgBKTAv5j7pMZSn12IsSdGzymGzb/sfGmqz38t97Jp3hUj9MDb8I/wMJ
 iEorX0wUNJRu1jfvjiev9gtLvPbmKQ8Nnj05Qy+aT4Lf0iNa4kLz/jqXXeCHs57n
 0uoJoRn/vAqYoBFtyLkYBppuygubc7neuk4AaaOu8CQ6Y2sKgX9WTyZ8a0PxOOQZ
 aDIU/GraJ5mzOJCVq/4QRQPx7OSw0hJ33kNa03+cGxU5uQQdUOLJCrSOL8WOcH8C
 izRwomVLpUvNA1bsWeRl9C01/c/qGKYl7Mytptkt8xbA4LyUAWD7DZhIAIUOV2qY
 ekP8Xsc/qeaHCM80XaYJWhgcAd5SaxfL3aIUalDk6G+4KVMoDlyU3fxa977wEj30
 R2yOZdG8sp3c2KvrdXQASZbcgsLlDq8Bqt7LbtPbQOoa8NEfAl/6e/LIF2CAwgLc
 8pL5j6tPcetEiIpUHoNpuwGEYGCkWwIntPczGK2j3+ppj4r7pYaLO3XxwRu/8tnH
 RG7QV1U68YcFM47awIT5
 =uPmA
 -----END PGP SIGNATURE-----

Merge tag 'perf-core-for-mingo-20160803' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux into perf/urgent

Pull perf/core improvements and fixes from Arnaldo Carvalho de Melo:

New features:

- Add --sample-cpu to 'perf record', to explicitely ask for sampling
  the CPU (Jiri Olsa)

Fixes:

- Fix processing of multi byte chunks in objdump output, fixing
  disassemble processing for annotation on at least ARM64 (Jan Stancek)

- Use SyS_epoll_wait in a BPF 'perf test' entry instead of sys_epoll_wait, that
  is not present in the DWARF info in vmlinux files (Arnaldo Carvalho de Melo)

- Add -wno-shadow when processing files using perl headers, fixing
  the build on Fedora Rawhide and Arch Linux (Namhyung Kim)

Infrastructure changes:

- Annotate prep work to better catch and report errors related to
  using objdump to disassemble DSOs (Arnaldo Carvalho de Melo)

- Add 'alloc', 'scnprintf' and 'and' methods for bitmap processing (Jiri Olsa)

- Add nested output resorting callback in hists processing (Jiri Olsa)

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2016-08-04 11:02:38 +02:00
Namhyung Kim b581c01fff perf tools: Fix build failure on perl script context
On my Archlinux machine, perf faild to build like below:

    CC       scripts/perl/Perf-Trace-Util/Context.o
  In file included from /usr/lib/perl5/core/perl/CORE/perl.h:3905:0,
                   from Context.xs:23:
  /usr/lib/perl5/core/perl/CORE/inline.h: In function :
  /usr/lib/perl5/core/perl/CORE/cop.h:612:13: warning: declaration of 'av'
                                  shadows a previous local [-Werror-shadow]
             AV *av =3D GvAV(PL_defgv);
                 ^
  /usr/lib/perl5/core/perl/CORE/inline.h:526:5: note: in expansion of
                                  macro 'CX_POP_SAVEARRAY'
         CX_POP_SAVEARRAY(cx);
         ^~~~~~~~~~~~~~~~
  In file included from /usr/lib/perl5/core/perl/CORE/perl.h:5853:0,
                   from Context.xs:23:
  /usr/lib/perl5/core/perl/CORE/inline.h:518:9: note:
                                  shadowed declaration is here
         AV *av;
             ^~

What I did to fix is adding '-Wno-shadow' as the error message said it's
the cause of the failure.  Since it's from the perl (not perf) code
base, we don't have the control so I just wanted to ignore the warning
when compiling perl scripting code.

Committer note:

This also fixes the build on Fedora Rawhide.

Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/20160802024317.31725-1-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2016-08-02 12:11:06 -03:00
Linus Torvalds 468fc7ed55 Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next
Pull networking updates from David Miller:

 1) Unified UDP encapsulation offload methods for drivers, from
    Alexander Duyck.

 2) Make DSA binding more sane, from Andrew Lunn.

 3) Support QCA9888 chips in ath10k, from Anilkumar Kolli.

 4) Several workqueue usage cleanups, from Bhaktipriya Shridhar.

 5) Add XDP (eXpress Data Path), essentially running BPF programs on RX
    packets as soon as the device sees them, with the option to mirror
    the packet on TX via the same interface.  From Brenden Blanco and
    others.

 6) Allow qdisc/class stats dumps to run lockless, from Eric Dumazet.

 7) Add VLAN support to b53 and bcm_sf2, from Florian Fainelli.

 8) Simplify netlink conntrack entry layout, from Florian Westphal.

 9) Add ipv4 forwarding support to mlxsw spectrum driver, from Ido
    Schimmel, Yotam Gigi, and Jiri Pirko.

10) Add SKB array infrastructure and convert tun and macvtap over to it.
    From Michael S Tsirkin and Jason Wang.

11) Support qdisc packet injection in pktgen, from John Fastabend.

12) Add neighbour monitoring framework to TIPC, from Jon Paul Maloy.

13) Add NV congestion control support to TCP, from Lawrence Brakmo.

14) Add GSO support to SCTP, from Marcelo Ricardo Leitner.

15) Allow GRO and RPS to function on macsec devices, from Paolo Abeni.

16) Support MPLS over IPV4, from Simon Horman.

* git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next: (1622 commits)
  xgene: Fix build warning with ACPI disabled.
  be2net: perform temperature query in adapter regardless of its interface state
  l2tp: Correctly return -EBADF from pppol2tp_getname.
  net/mlx5_core/health: Remove deprecated create_singlethread_workqueue
  net: ipmr/ip6mr: update lastuse on entry change
  macsec: ensure rx_sa is set when validation is disabled
  tipc: dump monitor attributes
  tipc: add a function to get the bearer name
  tipc: get monitor threshold for the cluster
  tipc: make cluster size threshold for monitoring configurable
  tipc: introduce constants for tipc address validation
  net: neigh: disallow transition to NUD_STALE if lladdr is unchanged in neigh_update()
  MAINTAINERS: xgene: Add driver and documentation path
  Documentation: dtb: xgene: Add MDIO node
  dtb: xgene: Add MDIO node
  drivers: net: xgene: ethtool: Use phy_ethtool_gset and sset
  drivers: net: xgene: Use exported functions
  drivers: net: xgene: Enable MDIO driver
  drivers: net: xgene: Add backward compatibility
  drivers: net: phy: xgene: Add MDIO driver
  ...
2016-07-27 12:03:20 -07:00
Jesper Dangaard Brouer 1db19db7f5 net: tracepoint napi:napi_poll add work and budget
An important information for the napi_poll tracepoint is knowing
the work done (packets processed) by the napi_poll() call. Add
both the work done and budget, as they are related.

Handle trace_napi_poll() param change in dropwatch/drop_monitor
and in python perf script netdev-times.py in backward compat way,
as python fortunately supports optional parameter handling.

Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-07-09 18:05:02 -04:00
Arnaldo Carvalho de Melo dd4629d46c perf script stackcollapse: Remove reference to the perl interpreter
It is ignored and this is actually a python script, not a perl one.

Reported-by: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Milian Wolff <milian.wolff@kdab.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Link: http://lkml.kernel.org/n/tip-0w4bpbqd79v3sl34jvpr11v0@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2016-06-22 09:56:34 -03:00
Paolo Bonzini 6745d8ea82 perf script: Add stackcollapse.py script
Add stackcollapse.py script as an example of parsing call chains, and
also of using optparse to access command line options.

The flame graph tools include a set of scripts that parse output from
various tools (including "perf script"), remove the offsets in the
function and collapse each stack to a single line.  The website also
says "perf report could have a report style [...] that output folded
stacks directly, obviating the need for stackcollapse-perf.pl", so here
it is.

This script is a Python rewrite of stackcollapse-perf.pl, using the perf
scripting interface to access the perf data directly from Python.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Cc: Brendan Gregg <bgregg@netflix.com>
Link: http://lkml.kernel.org/r/1460467573-22989-1-git-send-email-pbonzini@redhat.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2016-06-21 13:18:35 -03:00
Chris Phlipot 3521f3bc9d perf script: Update export-to-postgresql to support callchain export
Update the export-to-postgresql.py to support the newly introduced
callchain export.

callchains are added into the existing call_paths table and can now
be associated with samples when the "callpaths" commandline option
is used with the script.

Ex.:

  $ perf script -s export-to-postgresql.py example_db all callchains

Includes the following changes to enable callchain export via the python export
APIs:

- Add the "callchains" commandline option, which is used to enable
  callchain export by setting the perf_db_export_callchains global
- Add perf_db_export_callchains checks for call_path table creation
  and population.
- Add call_path_id to samples_table to conform with the new API

example usage and output using a small test app:

  test_app.c:

	volatile int x = 0;
	void inc_x_loop()
	{
		int i;
		for(i=0; i<100000000; i++)
			x++;
	}

	void a()
	{
		inc_x_loop();
	}

	void b()
	{
		inc_x_loop();
	}

	int main()
	{
		a();
		b();
		return 0;
	}

example usage:

  $ gcc -g -O0 test_app.c
  $ perf record --call-graph=dwarf ./a.out
  [ perf record: Woken up 77 times to write data ]
  [ perf record: Captured and wrote 19.373 MB perf.data (2404 samples) ]

  $ perf script -s scripts/python/export-to-postgresql.py
	example_db all callchains

  $ psql example_db

  example_db=#
  SELECT
  (SELECT name FROM symbols WHERE id = cps.symbol_id) as symbol,
  (SELECT name FROM symbols WHERE id =
	(SELECT symbol_id from call_paths where id = cps.parent_id))
	as parent_symbol,
  sum(period) as event_count
  FROM samples join call_paths as cps on call_path_id = cps.id
  GROUP BY cps.id,evsel_id
  ORDER BY event_count DESC
  LIMIT 5;

        symbol      |      parent_symbol       | event_count
  ------------------+--------------------------+-------------
   inc_x_loop       | a                        |   734250982
   inc_x_loop       | b                        |   731028057
   unknown          | unknown                  |     1335858
   task_tick_fair   | scheduler_tick           |     1238842
   update_wall_time | tick_do_update_jiffies64 |      650373
  (5 rows)

The above data shows total "self time" in cycles for each call path that was
sampled. It is intended to demonstrate how it accounts separately for the two
ways to reach the "inc_x_loop" function(via "a" and "b").  Recursive common
table expressions can be used as well to get cumulative time spent in a
function as well, but that is beyond the scope of this basic example.

Signed-off-by: Chris Phlipot <cphlipot0@gmail.com>
Acked-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/1461831551-12213-7-git-send-email-cphlipot0@gmail.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2016-05-06 13:00:55 -03:00
Chris Phlipot d6632dd59b perf script: Fix postgresql ubuntu install instructions
The current instructions for setting up an Ubuntu system for using the
export-to-postgresql.py script are incorrect.

The instructions in the script have been updated to work on newer
versions of ubuntu.

-Add missing dependencies to apt-get command:
    python-pyside.qtsql, libqt4-sql-psql
-Add '-s' option to createuser command to force the user to be a
    superuser since the command doesn't prompt as indicated in the
    current instructions.

Tested on: Ubuntu 14.04, Ubuntu 16.04(beta)

Signed-off-by: Chris Phlipot <cphlipot0@gmail.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/1461056164-14914-3-git-send-email-cphlipot0@gmail.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2016-04-19 12:36:54 -03:00
Taeung Song b416e204f8 perf python scripting: Append examples to err msg about audit-libs-python
To print syscall names, the audit-libs-python package is required.. If
not installed, it prints this error string:

    # perf script syscall-counts
    Install the audit-libs-python package to get syscall names.

But the package name is different in Ubuntu, mention that in the error
message, similar to a error message of util/trace-event-scripting.c:

    # perf script syscall-counts
    Install the audit-libs-python package to get syscall names.
    For example:
      # apt-get install python-audit (Ubuntu)
      # yum install audit-libs-python (Fedora)
      etc.

Signed-off-by: Taeung Song <treeze.taeung@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Link: http://lkml.kernel.org/r/1455018790-13425-1-git-send-email-treeze.taeung@gmail.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2016-02-12 11:30:27 -03:00
Jiri Olsa b8a1962d17 perf script: Add stat-cpi.py script
Adding stat-cpi.py as an example of how to do stat scripting.

It computes the CPI metrics from cycles and instructions events.

The CPI is based performance metric showing the Cycles Per Instructions
ratio, which helps to identify cycles-hungry code.

Following stat record/report/script combinations could be used:

- get CPI for given workload

    $ perf stat -e cycles,instructions record ls

    SNIP

     Performance counter stats for 'ls':

             2,904,431      cycles
             3,346,878      instructions              #    1.15  insns per cycle

           0.001782686 seconds time elapsed

    $ perf script -s ./scripts/python/stat-cpi.py
           0.001783: cpu -1, thread -1 -> cpi 0.867803 (2904431/3346878)

    $ perf stat -e cycles,instructions record ls | perf script -s ./scripts/python/stat-cpi.py

    SNIP

           0.001730: cpu -1, thread -1 -> cpi 0.869026 (2928292/3369627)

- get CPI systemwide:

    $ perf stat -e cycles,instructions -a -I 1000 record sleep 3
    #           time             counts unit events
         1.000158618        594,274,711      cycles                     (100.00%)
         1.000158618        441,898,250      instructions
         2.000350973        567,649,705      cycles                     (100.00%)
         2.000350973        432,669,206      instructions
         3.000559210        561,940,430      cycles                     (100.00%)
         3.000559210        420,403,465      instructions
         3.000670798            780,105      cycles                     (100.00%)
         3.000670798            326,516      instructions

    $ perf script -s ./scripts/python/stat-cpi.py
           1.000159: cpu -1, thread -1 -> cpi 1.344823 (594274711/441898250)
           2.000351: cpu -1, thread -1 -> cpi 1.311972 (567649705/432669206)
           3.000559: cpu -1, thread -1 -> cpi 1.336669 (561940430/420403465)
           3.000671: cpu -1, thread -1 -> cpi 2.389178 (780105/326516)

    $ perf stat -e cycles,instructions -a -I 1000 record sleep 3 | perf script -s ./scripts/python/stat-cpi.py
           1.000202: cpu -1, thread -1 -> cpi 1.035091 (940778881/908885530)
           2.000392: cpu -1, thread -1 -> cpi 1.442600 (627493992/434974455)
           3.000545: cpu -1, thread -1 -> cpi 1.353612 (741463930/547766890)
           3.000622: cpu -1, thread -1 -> cpi 2.642110 (784083/296764)

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Tested-by: Kan Liang <kan.liang@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1452077397-31958-4-git-send-email-jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2016-01-06 20:11:16 -03:00
Adrian Hunter 35ca01c117 perf tools: Add more documentation to export-to-postgresql.py script
Add some comments to the script and some 'views' to the created database
that better illustrate the database structure and how it can be used.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/r/1443186956-18718-8-git-send-email-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-09-28 16:53:07 -03:00
Tony Jones 84e5d89a77 perf scripts python: Add new compaction-times script
This patch creates a new script (compaction-times) to report time
spent in mm compaction. It is possible to report times in nanoseconds
(default) or microseconds (-u).

The option -p will break down results by process id, -pv will further
decompose by each compaction entry/exit.

For each compaction entry/exit what is reported is controlled by the
options:

  -t   report only timing
  -m   report migration stats
  -ms  report migration scanner stats
  -fs  report free scanner stats

The default is to report all.

Entries may be further filtered by pid, pid-range or comm (regex).

The script is useful when analysing workloads that compact memory. The
most common example will be THP allocations on systems with a lot of
uptime that has fragmented memory.

This is an example of using the script to analyse a thpscale from
mmtests which deliberately fragments memory and allocates THP in 4
separate threads

  # Recording step, one of the following;
  $ perf record -e 'compaction:mm_compaction_*' ./workload
  # or:
  $ perf script record compaction-times

  # Reporting: basic
  total: 2444505743ns migration: moved=357738 failed=39275
  free_scanner: scanned=2705578 isolated=387875
  migration_scanner: scanned=414426 isolated=397013

  # Reporting: Per task stall times
  $ perf script report compaction-times -- -t -p
  total: 2444505743ns
  6384[thpscale]: 740800017ns
  6385[thpscale]: 274119512ns
  6386[thpscale]: 832961337ns
  6383[thpscale]: 596624877ns

  # Reporting: Per-compaction attempts for task 6385
  $ perf script report compaction-times -- -m -pv 6385
  total: 274119512ns migration: moved=14893 failed=24285
  6385[thpscale]: 274119512ns migration: moved=14893 failed=24285
  6385[thpscale].1: 3033277ns migration: moved=511 failed=1
  6385[thpscale].2: 9592094ns migration: moved=1524 failed=12
  6385[thpscale].3: 2495587ns migration: moved=512 failed=0
  6385[thpscale].4: 2561766ns migration: moved=512 failed=0
  6385[thpscale].5: 2523521ns migration: moved=512 failed=0
  ..... output continues ...

Changes since v1:
- report stats for isolate_migratepages and isolate_freepages
  (Vlastimil Babka)
- refactor code to achieve above
- add help text
- output to stdout/stderr explicitly

Signed-off-by: Tony Jones <tonyj@suse.com>
Cc: Mel Gorman <mgorman@suse.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Link: http://lkml.kernel.org/r/1439840932-8933-1-git-send-email-tonyj@suse.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-08-28 14:53:40 -03:00
Adrian Hunter 4b715d24f4 perf tools: Add example call-graph script
Add a script to produce a call-graph from data exported to a postgresql
database and derived from a processor trace event like intel_pt or intel_bts.

Refer to comments in the scripts call-graph-from-postgresql.py and
export-to-postgresql.py for more details on how to set up the environment,
install the required packages, etc.

Committer note:

From the scripts, for convenience while reading 'git log':

  An example of using this script with Intel PT:

  $ perf record -e intel_pt//u ls
  $ perf script -s ~/libexec/perf-core/scripts/python/export-to-postgresql.py pt_example branches calls
  2015-05-29 12:49:23.464364 Creating database...
  2015-05-29 12:49:26.281717 Writing to intermediate files...
  2015-05-29 12:49:27.190383 Copying to database...
  2015-05-29 12:49:28.140451 Removing intermediate files...
  2015-05-29 12:49:28.147451 Adding primary keys
  2015-05-29 12:49:28.655683 Adding foreign keys
  2015-05-29 12:49:29.365350 Done
  $ python tools/perf/scripts/python/call-graph-from-postgresql.py pt_example
  # The result is a GUI window with a tree representing a context-sensitive
  # call-graph.  Expanding a couple of levels of the tree and adjusting column
  # widths to suit will display something like:

                                         Call Graph: pt_example
  Call Path                        |Object     |Count|Time(ns)|Time(%)|Branch Count|Branch Count(%)
  v- ls
     v- 2638:2638
         v- _start                  ld-2.19.so    1   10074071  100.0        211135          100.0
           |- unknown               unknown       1      13198    0.1             1            0.0
           >- _dl_start             ld-2.19.so    1    1400980   13.9         19637            9.3
           >- _d_linit_internal     ld-2.19.so    1     448152    4.4         11094            5.3
           v-__libc_start_main@plt  ls            1    8211741   81.5        180397           85.4
              >- _dl_fixup          ld-2.19.so    1       7607    0.1           108            0.1
              >- __cxa_atexit       libc-2.19.so  1      11737    0.1            10            0.0
              >- __libc_csu_init    ls            1      10354    0.1            10            0.0
              |- _setjmp            libc-2.19.so  1          0    0.0             4            0.0
              v- main               ls            1    8182043   99.6        180254           99.9

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Link: http://lkml.kernel.org/r/1437150840-31811-11-git-send-email-adrian.hunter@intel.com
[ Added 'python-pyside qt-postgresql' to the yum cmdline installing required packages ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-08-21 12:32:40 -03:00
Jiri Olsa c7355f842b perf build: Add scripts objects building
Move the scripts objects building under build framework to be included
in the libperf build object.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Tested-by: Sukadev Bhattiprolu <sukadev@linux.vnet.ibm.com>
Tested-by: Will Deacon <will.deacon@arm.com>
Cc: Alexis Berlemont <alexis.berlemont@gmail.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Link: http://lkml.kernel.org/n/tip-ry8pd41ahwpq9h46i8te33c7@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-02-12 11:49:53 -03:00
Wang Nan 75e0b5f010 perf scripting perl: Force to use stdbool
When building perf for arm64 I hit a warning (and be treated as an
error) like below:

 aarch64-oe-linux-gcc -o .../scripts/perl/Perf-Trace-Util/Context.o -c -Wbad-function-cast \
         ... scripts/perl/Perf-Trace-Util/Context.c

 In file included from .../usr/lib64/perl/5.14.3/CORE/perl.h:2464:0,
                  from Context.xs:23:
 /.../usr/lib64/perl/5.14.3/CORE/handy.h:108:0: error: "bool" redefined [-Werror]
  #  define bool char
  ^
 In file included from /.../usr/src/kernel/tools/include/linux/types.h:4:0,
                  from /.../usr/src/kernel/arch/arm64/include/uapi/asm/sigcontext.h:19,
		  from /.../usr/include/bits/sigcontext.h:27,
		  from /.../usr/include/signal.h:340,
		  from /.../usr/include/sys/param.h:28,
		  from /.../usr/lib64/perl/5.14.3/CORE/perl.h:678,
		  from Context.xs:23:
  /.../usr/lib/aarch64-oe-linux/gcc/aarch64-oe-linux/4.9.2/include/stdbool.h:33:0: note: this is the location of the previous definition
    #define bool _Bool

Looks like the failure is caused by arm64 uapi/asm/sigcontext.h, which
includes linux/types.h while other archs not.

Current perl consider this problem:

http://perl5.git.perl.org/perl.git/commit/bd31be4baa3ee68abdb92c0db3200efe0fad903b

However there are users which use old version of perl.

This patch includes stdbool.h before Context.xs and define HAS_BOOL to
prevent perl'e headers define its own 'bool'. Code is learn from perl's
git tree.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Acked-by: Jiri Olsa <jolsa@redhat.com>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: Li Zefan <lizefan@huawei.com>
Link: http://lkml.kernel.org/r/1421671397-4659-1-git-send-email-wangnan0@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-01-21 10:05:00 -03:00
Adrian Hunter 6a70307ddc perf tools: Add call information to Python export
Add the ability to export detailed information about paired calls and
returns to Python db export and the export-to-postgresql.py script.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@gmail.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Link: http://lkml.kernel.org/r/1414678188-14946-7-git-send-email-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-11-03 18:10:06 -03:00
Adrian Hunter c29414f5cf perf tools: Add branch_type and in_tx to Python export
Add branch_type and in_tx to Python db export and the
export-to-postgresql.py script.

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@gmail.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Link: http://lkml.kernel.org/r/1414678188-14946-4-git-send-email-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-11-03 18:07:34 -03:00
Adrian Hunter 2987e32f75 perf script: Add Python script to export to postgresql
Add a Python script to export to a postgresql database.

The script requires the Python pyside module and the Qt PostgreSQL
driver.  The packages needed are probably named "python-pyside" and
"libqt4-sql-psql"

The caller of the script must be able to create postgresql databases.

The script takes the database name as a parameter.  The database and
database tables are created.  Data is written to flat files which are
then imported using SQL COPY FROM.

Example:

  $ perf record ls
  ...
  $ perf script report export-to-postgresql lsdb
  2014-02-14 10:55:38.631431 Creating database...
  2014-02-14 10:55:39.291958 Writing to intermediate files...
  2014-02-14 10:55:39.350280 Copying to database...
  2014-02-14 10:55:39.358536 Removing intermediate files...
  2014-02-14 10:55:39.358665 Adding primary keys
  2014-02-14 10:55:39.658697 Adding foreign keys
  2014-02-14 10:55:39.667412 Done
  $ psql lsdb
  lsdb-# \d
              List of relations
   Schema |      Name       | Type  | Owner
  --------+-----------------+-------+-------
   public | comm_threads    | table | acme
   public | comms           | table | acme
   public | dsos            | table | acme
   public | machines        | table | acme
   public | samples         | table | acme
   public | samples_view    | view  | acme
   public | selected_events | table | acme
   public | symbols         | table | acme
   public | threads         | table | acme
  (9 rows)
  lsdb-# \d samples
         Table "public.samples"
      Column     |  Type   | Modifiers
  ---------------+---------+-----------
   id            | bigint  | not null
   evsel_id      | bigint  |
   machine_id    | bigint  |
   thread_id     | bigint  |
   comm_id       | bigint  |
   dso_id        | bigint  |
   symbol_id     | bigint  |
   sym_offset    | bigint  |
   ip            | bigint  |
   time          | bigint  |
   cpu           | integer |
   to_dso_id     | bigint  |
   to_symbol_id  | bigint  |
   to_sym_offset | bigint  |
   to_ip         | bigint  |
   period        | bigint  |
   weight        | bigint  |
   transaction   | bigint  |
   data_src      | bigint  |
  Indexes:
      "samples_pkey" PRIMARY KEY, btree (id)
  Foreign-key constraints:
      "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id)
      "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id)
      "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id)
      "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id)
      "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id)
      "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id)
      "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id)
      "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id)

  lsdb-# \d samples_view
                 View "public.samples_view"
        Column       |          Type           | Modifiers
  -------------------+-------------------------+-----------
   id                | bigint                  |
   time              | bigint                  |
   cpu               | integer                 |
   pid               | integer                 |
   tid               | integer                 |
   command           | character varying(16)   |
   event             | character varying(80)   |
   ip_hex            | text                    |
   symbol            | character varying(2048) |
   sym_offset        | bigint                  |
   dso_short_name    | character varying(256)  |
   to_ip_hex         | text                    |
   to_symbol         | character varying(2048) |
   to_sym_offset     | bigint                  |
   to_dso_short_name | character varying(256)  |

    lsdb=# select * from samples_view;

   id| time       |cpu | pid  | tid  |command| event  |   ip_hex      |           symbol    |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name
   --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+----------
   1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe|    10 | [kernel]| 0       | unknown |         0| unknown
   2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe|    10 | [kernel]| 0       | unknown |         0| unknown
   3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe|    10 | [kernel]| 0       | unknown |         0| unknown

My notes (which may be out-of-date) on setting up postgresql so you can
create databases:

fedora:

        $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql
        $ sudo su - postgres -c initdb
        $ sudo service postgresql start
        $ sudo su - postgres
        $ createuser -s <your username>

I used the the unix user name in createuser.

If it fails, try createuser without -s and answer the following question
to allow your user to create tables:

        Shall the new role be a superuser? (y/n) y

ubuntu:

        $ sudo apt-get install postgresql
        $ sudo su - postgres
        $ createuser <your username>
        Shall the new role be a superuser? (y/n) y

You may want to disable automatic startup.  One way is to edit
/etc/postgresql/9.3/main/start.conf.  Another is to disable the init
script e.g. sudo update-rc.d postgresql disable

Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@gmail.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-29 10:32:49 -02:00
Joseph Schuchart 0f5f5bcd11 perf script: Add callchain to generic and tracepoint events
This provides valuable information for tracing performance problems.

Since this change alters the interface for the python scripts, also
adjust the script generation and the provided scripts.

Signed-off-by: Joseph Schuchart <joseph.schuchart@tu-dresden.de>
Acked-by: Thomas Ilsche <thomas.ilsche@tu-dresden.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@gmail.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Thomas Ilsche <thomas.ilsche@tu-dresden.de>
Link: http://lkml.kernel.org/r/53BE7E1B.10503@tu-dresden.de
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-07-16 17:57:33 -03:00
Daniel Bristot de Oliveira 07100877ea perf scripts: Fallback to syscalls:* when raw_syscalls:* is not available
Older kernels (e.g., RHEL6) do system call tracing via the
syscalls:sys_{enter,exit} tracepoints rather than using raw_syscalls:*.

Update perf python and perl scripts to fallback to syscalls:* when
raw_syscalls:* isn't available.

Signed-off-by: Daniel Bristot de Oliveira <bristot@redhat.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Luis Claudio R. Goncalves <lgoncalv@redhat.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/5a6c64081a3375bc3bc66351b14559678ef4d71e.1402507908.git.bristot@redhat.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-06-25 12:26:56 -03:00
Arnaldo Carvalho de Melo 316c7136f8 perf tools: Finish the removal of 'self' arguments
They convey no information, perhaps I was bitten by some snake at some
point, complete the detox by naming the last of those arguments more
sensibly.

Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Link: http://lkml.kernel.org/n/tip-u1r0dnjoro08dgztiy2g3t2q@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-11-05 15:32:36 -03:00
Ramkumar Ramachandra 750ade7e82 perf script: Fix broken include in Context.xs
765532c8 (perf script: Finish the rename from trace to script,
2010-12-23) made a mistake during find-and-replace replacing
"../../../util/trace-event.h" with "../../../util/script-event.h", a
non-existent file.  Fix this include.

Signed-off-by: Ramkumar Ramachandra <artagnon@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Link: http://lkml.kernel.org/r/1373364033-7918-3-git-send-email-artagnon@gmail.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-07-10 13:47:00 -03:00
Ben Hutchings 6b75c7357c perf: net_dropmonitor: Remove progress indicator
We can read /proc/kallsyms in a fraction of a second, so why waste
a further fraction of a second showing progress?

Signed-off-by: Ben Hutchings <ben@decadent.org.uk>
Signed-off-by: David S. Miller <davem@davemloft.net>
2013-05-22 15:10:11 -07:00
Ben Hutchings 0ce58bae85 perf: net_dropmonitor: Use bisection in symbol lookup
Signed-off-by: Ben Hutchings <ben@decadent.org.uk>
Signed-off-by: David S. Miller <davem@davemloft.net>
2013-05-22 15:10:10 -07:00
Ben Hutchings 326017c757 perf: net_dropmonitor: Do not assume ordering of dictionaries
The sort order of dictionaries in Python is undocumented.  Use
tuples instead, which are documented to be lexically ordered.

Signed-off-by: Ben Hutchings <ben@decadent.org.uk>
Signed-off-by: David S. Miller <davem@davemloft.net>
2013-05-22 15:10:10 -07:00
Ben Hutchings 5a1e99dd20 perf: net_dropmonitor: Fix symbol-relative addresses
The comparison between traced and symbol addresses is backwards: if
the traced address doesn't exactly match a symbol (which we don't
expect it to), we'll show the next symbol and the offset to it,
whereas we should show the previous symbol and the offset from it.

Cc: stable@vger.kernel.org
Signed-off-by: Ben Hutchings <ben@decadent.org.uk>
Signed-off-by: David S. Miller <davem@davemloft.net>
2013-05-22 15:10:10 -07:00
Ben Hutchings 140c3c6a2b perf: net_dropmonitor: Fix trace parameter order
This works much better if we don't treat protocol numbers as addresses.

Cc: stable@vger.kernel.org
Signed-off-by: Ben Hutchings <ben@decadent.org.uk>
Signed-off-by: David S. Miller <davem@davemloft.net>
2013-05-22 15:10:10 -07:00
Jiri Olsa b22e79395c perf perl scripts: Fix SIGALRM and pipe read race for rwtop
Fixing rwtop script race. The issue is caused by rwtop script triggering
SIGALRM and underneath pipe reading layer reporting error when
interrupted.

Fixing this by setting SA_RESTART for rwtop SIGALRM handler, which
avoids interruption of the pipe reading layer.

The discussion for this issue & fix is here:
https://lkml.org/lkml/2012/9/18/123

Signed-off-by: Jiri Olsa <jolsa@redhat.com>
Original-patch-by: Andrew Jones <drjones@redhat.com>
Cc: Andrew Jones <drjones@redhat.com>
Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1360080351-3246-2-git-send-email-jolsa@redhat.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-02-06 18:09:27 -03:00
Tom Zanussi 1de7b7e89d perf script: Remove workqueue-stats script
The tracepoints used by the workqueue-stats script no longer exist so
trying to run the script results in:

  # perf script record workqueue-stats
  invalid or unsupported event: 'workqueue:workqueue_creation'
  Run 'perf list' for a list of valid events

So remove the script until it can be reworked using the new workqueue
tracepoints.

Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Link: http://lkml.kernel.org/r/e7a7637d5df9df86887c3bff7683574665ec5360.1358527965.git.tom.zanussi@linux.intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-01-24 16:40:53 -03:00
Feng Tang 59cbea2294 perf scripts: Add event_analyzing_sample-record/report
So that event_analyzing_sample.py can be shown by "perf script -l"

Signed-off-by: Feng Tang <feng.tang@intel.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1347007349-3102-4-git-send-email-feng.tang@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-17 13:11:15 -03:00
Feng Tang 87b6a3ad40 perf script python: Correct handler check and spelling errors
Correct the checking for handler returned by PyDict_GetItemString(),
also fix some spelling error and remove some data code in
event_analyzing_sample.py, as suggested by Namhyung Kim.

v2: restore back the wrongly removed trace_unhandled() func

Signed-off-by: Feng Tang <feng.tang@intel.com>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Robert Richter <robert.richter@amd.com>
Cc: Stephane Eranian <eranian@google.com>
Link: http://lkml.kernel.org/r/20120809134613.067104c4@feng-i7
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-08-09 13:26:19 -03:00
Feng Tang 0076d546b4 perf scripts python: Add event_analyzing_sample.py as a sample for general event handling
Currently only trace point events are supported in perf/python script,
the first 3 patches of this serie add the support for all types of
events. This script is just a simple sample to show how to gather the
basic information of the events and analyze them.

This script will create one object for each event sample and insert them
into a table in a database, then leverage the simple SQL commands to
sort/group them. User can modify or write their brand new functions
according to their specific requirment.

Here is the sample of how to use the script:

 $ perf record -a tree
 $ perf script -s process_event.py

There is 100 records in gen_events table
Statistics about the general events grouped by thread/symbol/dso:

            comm   number         histgram
==========================================
         swapper       56     ######
            tree       20     #####
            perf       10     ####
            sshd        8     ####
     kworker/7:2        4     ###
     ksoftirqd/7        1     #
 plugin-containe        1     #

                          symbol   number         histgram
==========================================================
           native_write_msr_safe       40     ######
                  __lock_acquire        8     ####
             ftrace_graph_caller        4     ###
           prepare_ftrace_return        4     ###
                      intel_idle        3     ##
              native_sched_clock        3     ##
                  Unknown_symbol        2     ##
                      do_softirq        2     ##
                    lock_release        2     ##
           lock_release_holdtime        2     ##
               trace_graph_entry        2     ##
                        _IO_putc        1     #
                  __d_lookup_rcu        1     #
                      __do_fault        1     #
                      __schedule        1     #
                  _raw_spin_lock        1     #
                       delay_tsc        1     #
             generic_exec_single        1     #
                generic_fillattr        1     #

                                     dso   number         histgram
==================================================================
                       [kernel.kallsyms]       95     #######
                     /lib/libc-2.12.1.so        5     ###

Signed-off-by: Feng Tang <feng.tang@intel.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Robert Richter <robert.richter@amd.com>
Cc: Stephane Eranian <eranian@google.com>
Link: http://lkml.kernel.org/r/1344419875-21665-6-git-send-email-feng.tang@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-08-08 12:55:38 -03:00
Feng Tang 02f1c33f7d perf scripts python: Add a python library EventClass.py
This library defines several class types for perf events which could
help to better analyze the event samples. Currently there are just a few
classes, PerfEvent is the base class for all perf events,  PebsEvent is
a HW base Intel x86 PEBS event, and user could add more SW/HW event
classes based on requriements.

Signed-off-by: Feng Tang <feng.tang@intel.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Robert Richter <robert.richter@amd.com>
Cc: Stephane Eranian <eranian@google.com>
Link: http://lkml.kernel.org/r/1344419875-21665-5-git-send-email-feng.tang@intel.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-08-08 12:53:08 -03:00
Neil Horman 63e03724b5 perf script: Add drop monitor script
A while back I created the dropmonitor protocol, which allowed users to get
reports of dropped frames communicated to them via a netlink socket.

While useful, several people have now asked that I integrate the ability
to do drop monitoring with perf, so they don't have to run additional
tools.

This patch adds a drop monitor script to the perf suite, and provides
the same output that the netlink socket does.

Cc: Ingo Molnar <mingo@elte.hu>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1309801217-22450-1-git-send-email-nhorman@tuxdriver.com
Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2011-09-29 16:41:37 -03:00