[PATCH rendercheck] Report results on a per-test basis
Martin Peres
martin.peres at linux.intel.com
Mon Oct 24 12:26:36 UTC 2016
On 21/10/16 19:18, Eric Anholt wrote:
> Martin Peres <martin.peres at linux.intel.com> writes:
>
>> This allows a runner such as EzBench to track each test individually
>> and not limit the resolution to groups.
>>
>> This feature can be triggered by using the -r parameter.
>
> I don't really see the point of this -- you need an external runner to
> be choosing a specific test subset to run per rendercheck call, since
> the full matrix of composite tests is too long. Once you have an
> external runner calling rendercheck per test group, all you would be
> able to do using this patch would be save a few spawns of the process,
> which doesn't seem worth it.
Just to be sure, are you suggesting I do something like this instead?
for format in "a8r8g8b8 x8r8g8b8 ... more formats"; do
./rendercheck -f $format
# parse the successful groups and store them, prefixed by $format so as
I can identify regressions per format?
done
if so, then I agree that this will work for some groups of tests but not
the ones trying the different combinations of formats, such as :
## ConjointXor linear gradient dst=a8r8g8b8, src=(1x1R a8r8g8b8,
1.00:1.00:1.00:1.00): pass
## ConjointXor linear gradient dst=a8r8g8b8, src=(1x1R x8r8g8b8,
1.00:1.00:1.00:1.00): pass
or the ones that do multiple sub-tests, like:
## Src Triangles a8r8g8b8: pass
## Src TriStrip a8r8g8b8: pass
## Src TriFan a8r8g8b8: pass
Indeed, I would like to know with the smallest granularity possible what
are the results for all the sub-tests so as I can bisect as many changes
possible instead of bisecting to the first change that broke the group
of tests, having a double for-loop to go through all the combination of
composite tests is orthogonal to this problem.
This was useful for me to bisect a kernel change that broke rendering
with glamor. It would have had had less chances of succeeding without
this patch because multiple changes might have happened.
Anyway, don't you think this is also nice from a developer perspective
to know what test(s) actually broke in a group, instead of having to
download rendercheck's source, patch it to add debug printfs, recompile
and re-run? Given how piglit works and how devs love sub-tests, I think
I am not the only one here.
Martin
More information about the xorg-devel
mailing list