[Ezbench-dev] Couple of patches

Submitted by Eero Tamminen on April 28, 2017, 2:12 p.m.


Message ID 3e775242-ad60-9c44-c3b2-4c8af4a72e09@intel.com
State New
Series "Couple of patches"
Headers show

Commit Message

Eero Tamminen April 28, 2017, 2:12 p.m.

Attached are couple of patches improving README and parameter examples.

	- Eero

Patch hide | download patch | download mbox

Misc README improvements

* Fix dependency lists
* Add examples for listing tests
* Fix ezbenchd.py path & add HTTP server example
* Add extra new line before section headings for readability

diff --git a/README b/README
index 3c1691f..bb49662 100644
--- a/README
+++ b/README
@@ -3,18 +3,18 @@ 
 This repo contains a collection of tools to benchmark graphics-related
 == Installation ==
 == Runner.sh / Core.sh ==
 WARNING: This tool can be used directly from the CLI but it is recommenced that
 you use ezbench in conjunction with ezbenchd which together support testing
 kernels and which can recover upon errors.
-WARNING 2: core.sh is now considered deprecated
 This tool is responsible for collecting the data and generating logs that will
 be used by another tool to generate a visual report.
@@ -35,6 +35,7 @@  You may specify whatever name you want by adding -N <name> to the command line.
 This is very useful when testing kernel-related stuff as we need to reboot on
 a new kernel to test a new commit.
 === Dependencies ===
 The real core of ezbench requires:
@@ -61,9 +62,10 @@  To get visual feedback when compiling, you will need:
 If you want to use specify which compositor should be run, you will need to
 install the following applications:
- - unbuffer
+ - unbuffer (part of expect package)
  - wmctrl
 === Configuration ===
 The tests configuration file is named user_parameters.sh. A sample file called
@@ -84,8 +86,29 @@  for example allows users to specify which compositor should be used.
 If core.sh is currently running and you want to stop its execution after the next
 test, you can create a file named 'requestExit' in the log folder.
 === Core.sh examples ===
+You can use following steps to validate that ezbench is correctly
+configured and working; it finds tests, it can run them, it can build
+the software stack and it can do bisection.
+WARNING: running core.sh directly is deprecated. You can skip to
+ezbench section.
+==== Listing tests ====
+This lists all supported tests and their configurations:
+    ./core.sh -l | tr ' ' '\n'
+==== Running tests ====
+This runs a single test without building anything:
+    WIP
 ==== Testing every patchset of a series ====
 The following command will test all the GLB27:Egypt cases but the ones
@@ -111,6 +134,7 @@  profile you are using. To do so, use -c like in the following example:
     ./core.sh -P mesa -b synmark:Gl21Batch2$ -r 3 -c confs.d/x11_comp/kwin HEAD
 ==== Retrospectives ====
 Here is an example of how to generate a retrospective. The interesting part is
@@ -123,6 +147,7 @@  the call to utils/get_commit_list.py which generates a list of commits
                  -m "./recompile-release.sh" \
                  `utils/get_commit_list.py -p ~/repos/mesa -s 2014-12-01 -i "1 week"`
 ==== Giving a large list of tests to core.sh ====
 The command line argument's maximum length is limited. If you ever feel the need
@@ -136,6 +161,7 @@  Here is an example:
     EOF (Ctrl + D)
 == ezbench ==
 This tool is meant to make the usage of core.sh easy and support testing
@@ -148,16 +174,25 @@  the execution of this report.
 This tool uses core.sh as a backend for checking that the commits SHA1 and tests
 do exist so you are sure that the work can be executed when the time comes.
 === Dependencies ===
- - python3
- - dateutils
+Python3 modules:
+ - dateutil
  - numpy
  - scipy
  - pygit2
 === Examples ===
+==== Listing available tests ====
+The ezbench program allows you to list the available tests by running:
+    ./ezbench -l
 ==== Creating a report ====
 The ezbench command allows you to create a new performance report. To create
@@ -166,6 +201,7 @@  profile 'mesa', you need to run the following command:
     ./ezbench -p mesa mesa-tracking-pub-benchmarks
 ==== Adding benchmarks runs ====
 Adding the 2 rounds of benchmark GLB27:Egypt:offscreen to the report
@@ -183,6 +219,7 @@  made that it would also work across reboots which is good when testing kernels:
               -c "`utils/get_commit_list.py -p ~/repos/mesa -s 2014-12-01 -i "1 week"`"
 ==== Checking the status of a report ====
 You can check the status of the 'mesa-tracking-pub-benchmarks' report by calling
@@ -190,6 +227,7 @@  the following command:
     ./ezbench mesa-tracking-pub-benchmarks status
 ==== Changing the execution status of the report ====
 When creating a report, the default state of the report is "initial" which means
@@ -204,6 +242,7 @@  need to run the following command:
  executing any benchmarks from this report. The difference between the "pause"
  and "abort" states is mostly for humans, to convey the actual intent.
 ==== Starting collecting data without ezbenchd.py ====
 If you are not using ezbenchd.py, you may simply run the following command to
@@ -213,6 +252,7 @@  start collecting data:
 This command will automatically change the state of the report to "run".
 == utils/owatch/owatch ==
 This tool monitors the output (on stdout and stderr) of a program and
@@ -225,12 +265,14 @@  This tool is only used by the tests that explicitly invoke it, it
 won't be used automatically for anything. An example is
 === Dependencies ===
  - gcc
  - make
-== utils/ezbenchd.py ==
+== ezbenchd.py ==
 This tool monitors the ezbench reports created using the "ezbench" command and
 runs them if their state is right (state == RUN). It also schedules improvements
@@ -238,19 +280,26 @@  when the run is over before moving on to the next report. It thus allow for
 collaborative sharing of the machine by different reports, as long as they
 make sure not to change the global state.
 === Dependencies ===
  - ezbench.py
+Python3 modules:
+ - mako
 === Examples ===
-Simply running utils/ezbenchd.py is enough to run all reports which are
-currently in the state "RUN".
+Simply running ezbenchd.py is enough to run all reports which are
+currently in the state "RUN", and set up an HTTP server to allow
+interacting with ezbenchd and see results.
     ./ezbench -p mesa mesa-tracking-pub-benchmarks
     ./ezbench -r 2 -b GLB27:Egypt:offscreen -c HEAD mesa-tracking-pub-benchmarks
     ./ezbench mesa-tracking-pub-benchmarks run
-    utils/ezbenchd.py
+    ./ezbenchd.py  --http_server=
 === Systemd integration ===
@@ -272,6 +321,7 @@  an http server listening on all interfaces on the port 8080.
 === Nightly tests ===
 There is currently no real support for nightly tests on ezbench. However, it is


Eero Tamminen July 3, 2017, 8:09 a.m.


On 28.04.2017 17:12, Eero Tamminen wrote:
> Attached are couple of patches improving README and parameter examples.

Please apply at least the Mesa configure options example fix.

	- Eero