Make --test and --include non-cumulative again due to backwards incompatibility problems
#5023
Comments
|
Could you please clarify? |
|
Hi, I have encounter same problem as well, but I don't have idea to collect stack trace within robot framework, so I am sharing the screenshot about the command and console outputs. Hope it help to elaborate clearer. RF6: RF7: |
|
Could someone explain what happens and what you expected to happen? An example is fine, but please use text, not screenshots. |
|
Example What was expected:(and what used to work with robot 6) robot test.robot Runs 20 tests. 17 passed 3 failed robot --rerunfailed test.robot What happens? (with robot 7) robot test.robot Runs 20 tests. 17 passed 3 failed robot --rerunfailed test.robot |
|
I tried reproducing this issue with the following data: *** Test Cases ***
T1
log 1
T2
log 2
T3
log 3
T4
log 4
T5
fail 5
T6
log 6
T7
log 7
T8
log 8
T9
fail 9
T10
log 10When I executed it with two tests fail as expected. When I then use only the two failed tests are run again as expected. There can be something different in your case, but the basic stuff certainly works. If you want us to investigate this, you need to provide a concrete example that we can run. Related to that, |
|
Hi, I am able to reproduce when I filter tests with tag in robot command |
|
Addition: I have added a tag in |
|
I can reproduce the change in behavior when using The behavior change with |
|
I ran into the same issue when migrating from RF 6.1.1 to RF 7.0. If the |
|
I agree that the old behavior when using I'll ask from other users on our Slack how do they feel about this. |
|
developing new features and ideas are always welcomed, but not considering legacy support is to be considered too. |
|
The change with |
--rerunfailed and --include/--exclude/--test are used together has changed in RF 7.0
|
Thanks for your quick response. I agree with your proposal. |
|
We need to make a decision about this. Adding to RF 7.0.1 scope so we don't forget about it. |
|
Just chiming in that we are experiencing this issue as well and would like to see the previous behavior reinstated. Regardless of --include/--exclude the tests are not failed and should not be rerun. |
|
The old behavior will be restored. There weren't that much benefits with #4721, so it's better to revert that than try to fix |
--rerunfailed and --include/--exclude/--test are used together has changed in RF 7.0--test and --include non-cumulative again due to backwards incompatibility problems (i.e. revert #4721)
|
I hopefully get other open issues in RF 7.0.1 done soon and will then create a release candidate with all the fixes. I hope you can test it when it's available to make sure the behavior with |
--test and --include non-cumulative again due to backwards incompatibility problems (i.e. revert #4721)--test and --include non-cumulative again due to backwards incompatibility problems
|
RF 7.0.1 rc 1 with this fix has just been released. It would be great if you could test does the fix work properly before the final release that's targeted for Monday June 10. |
|
Tested scenario with --rerunfailed and --include (and also --exclude and --test ) with RF 7.0.1 rc1 and works fine for me. Thanks for fixing this issue! |
|
@pekkaklarck our test with RF 7.0.1 rc1 was successful as well. |
robot --rerunfailed output.xmluntil robot version 6 picks up tests that had been failed in previous run. However, robot 7 just reruns all suite, both failed and passed testsThe text was updated successfully, but these errors were encountered: