README.md: update test sets documentation

Submitted by apinheiro on June 5, 2019, 9:06 a.m.

Details

Message ID 20190605090606.19186-1-apinheiro@igalia.com
State New
Headers show
Series "README.md: update test sets documentation" ( rev: 1 ) in Piglit

Not browsing as part of any series.

Commit Message

apinheiro June 5, 2019, 9:06 a.m.
Since commit "tests: use meta profiles"
(400dcd5cf18ac51ab8e9681d8ecab4132986efbd) some test sets are not
represented anymore as python scripts. So first, it would be good to
mention it, and second it would be good to mention that it would be
better to run the test sets using the set name, not directly the
python script.
---
 README.md | 48 ++++++++++++++++++++++++++++++++----------------
 1 file changed, 32 insertions(+), 16 deletions(-)

Thanks Dylan for the feedback for the RFC one.

I have just included your feedback, and removed the RFC. Im plenty
sure that there are some grammar mistakes, but I think that this is
more or less the info that we need.

Patch hide | download patch | download mbox

diff --git a/README.md b/README.md
index 7366893f8..e4b3c651b 100644
--- a/README.md
+++ b/README.md
@@ -313,28 +313,44 @@  available, then this completion file will be installed system-wide.
 
 ## 4. Available test sets
 
-Test sets are specified as Python scripts in the tests directory.
-The following test sets are currently available:
+Test sets are specified as Python scripts or as xml files in the tests
+directory. As there are two different formats, the easier way to run a
+test set is using the name. So for example, although the following
+would work:
 
+   ./piglit run tests/opengl.py opengl_results -c
+
+It is more convenient to do the following:
+
+   ./piglit run opengl opengl_results -c
+
+As for running the all test suite you need to do this:
+
+   ./piglit run all all_results -c
 
 ### 4.1 OpenGL Tests
 
-  - **sanity.py** This suite contains minimal OpenGL sanity tests. These tests
+  - **sanity** This suite contains minimal OpenGL sanity tests. These tests
     must pass, otherwise the other tests will not generate reliable results.
-  - **all.py** This suite contains all OpenGL tests.
-  - **quick.py** Run all tests, but cut down significantly on their runtime
-    (and thus on the number of problems they can find).
-  - **gpu.py** A further reduced set of tests from quick.py, this runs tests
-    only for hardware functionality and not tests for the software stack.
-  - **llvmpipe.py** A reduced set of tests from gpu.py removing tests that are
-    problematic using llvmpipe
-  - **cpu.py** This profile runs tests that don't touch the gpu, in other words
-    all of the tests in quick.py that are not run by gpu.py
-  - **glslparser.py** A subset of all.py which runs only glslparser tests
-  - **shader.py** A subset of all.py which runs only shader tests
-  - **no_error.py** A modified version of the test list run as khr_no_error
+  - **glslparser** test set that includes the tests run by glslparser.
+  - **shader** all shader_test tests, being run by shader_runner.
+  - **no_error** A modified version of the test list run as khr_no_error
     variants
-
+  - **quick_gl**: This profile filters out a number of very slow
+      opengl tests, and tests that are very exhaustively tested, since
+      they add a good deal of runtime to piglit.
+  - **quick_shader**: Equivalent to quick_gl, but for shader_test (run
+      by shader_runner) tests.
+  - **llvmpipe_gl** A filtered test set, removing tests that are
+      problematic using llvmpipe
+  - **llvmpipe** llvmpipe + quick_shader test set
+  - **gpu**: Profile that includes the "quick" test sets that are
+      efectively executed (not just compiled), so it includes quick_gl
+      + quick_shader
+  - **quick** Profile with including all the "quick" test sets. So
+      it includes quick_gl + quick_shader + glslparser
+  - **all** This suite contains all OpenGL tests. So it is opengl +
+      shader + glslparser
 
 ### 4.2 OpenCL Tests