From: Thiago Jung Bauermann <bauerman@linux•ibm.com>
To: Breno Leitao <leitao@debian•org>
Cc: linuxppc-dev@lists•ozlabs.org,
Tyrel Datwyler <tyreld@linux•vnet.ibm.com>
Subject: Re: [PATCH 2/2] selftests/powerpc: Skip test instead of failing
Date: Mon, 29 Oct 2018 19:08:44 -0300 [thread overview]
Message-ID: <87efc8be37.fsf@morokweng.localdomain> (raw)
In-Reply-To: <132d7e4f-153f-8ba4-95f2-f0d9c1a66f28@debian.org>
Breno Leitao <leitao@debian•org> writes:
> Hi Tyrel,
>
> On 10/23/2018 05:41 PM, Tyrel Datwyler wrote:
>>> --- a/tools/testing/selftests/powerpc/ptrace/core-pkey.c
>>> +++ b/tools/testing/selftests/powerpc/ptrace/core-pkey.c
>>> @@ -352,17 +352,11 @@ static int write_core_pattern(const char *core_pattern)
>>> FILE *f;
>>>
>>> f = fopen(core_pattern_file, "w");
>>> - if (!f) {
>>> - perror("Error writing to core_pattern file");
>>> - return TEST_FAIL;
>>> - }
>>> + SKIP_IF(!f);
>>>
>>> ret = fwrite(core_pattern, 1, len, f);
>>> fclose(f);
>>> - if (ret != len) {
>>> - perror("Error writing to core_pattern file");
>>> - return TEST_FAIL;
>>> - }
>>> + SKIP_IF(ret != len);
>
>> If we don't have proper privileges we should fail on the open, right?
>> So wouldn't we still want to fail on the write if something goes
>> wrong?
>
> That is a good point. Should the test fail or skip if it is not possible
> to create the infrastructure to run the core test?
>
> Trying to find the answer in the current test sets, I find tests where
> the self test skips if the test environment is not able to be set up, as
> for example, when a memory allocation fails.
>
> File: tools/testing/selftests/powerpc/alignment/alignment_handler.c
>
> ci1 = mmap(NULL, bufsize, PROT_WRITE, MAP_SHARED,
> fd, bufsize);
> if ((ci0 == MAP_FAILED) || (ci1 == MAP_FAILED)) {
> printf("\n");
> perror("mmap failed");
> SKIP_IF(1);
> }
I think TEST_FAIL means the test was able to exercise the feature
and found a problem with it. In this case, the test wasn't able to
exercise the feature so it's not appropriate.
Ideally, there should be a TEST_ERROR result for a case like this where
an unexpected problem prevented the testcase from exercising the
feature.
If we're to use the an existing result then I vote for SKIP_IF.
For reference, here are the test results that DejaGnu supports (it is
the test harness used by some GNU projects):
https://www.gnu.org/software/dejagnu/manual/Output-States.html
I would say that SKIP_IF corresponds to UNSUPPORTED in DejaGnu.
--
Thiago Jung Bauermann
IBM Linux Technology Center
next prev parent reply other threads:[~2018-10-29 22:12 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-10-23 20:23 [PATCH 1/2] selftests/powerpc: Allocate base registers Breno Leitao
2018-10-23 20:23 ` [PATCH 2/2] selftests/powerpc: Skip test instead of failing Breno Leitao
2018-10-23 20:41 ` Tyrel Datwyler
2018-10-24 14:11 ` Breno Leitao
2018-10-29 22:08 ` Thiago Jung Bauermann [this message]
2018-10-30 15:16 ` Michael Ellerman
2018-10-31 0:09 ` Tyrel Datwyler
2018-10-31 9:42 ` Michael Ellerman
2018-10-31 9:41 ` Michael Ellerman
2018-10-23 21:39 ` [PATCH 1/2] selftests/powerpc: Allocate base registers Segher Boessenkool
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87efc8be37.fsf@morokweng.localdomain \
--to=bauerman@linux$(echo .)ibm.com \
--cc=leitao@debian$(echo .)org \
--cc=linuxppc-dev@lists$(echo .)ozlabs.org \
--cc=tyreld@linux$(echo .)vnet.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox