You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/testing-guidelines/testing-guidelines.md
+22-3Lines changed: 22 additions & 3 deletions
Original file line number
Diff line number
Diff line change
@@ -49,7 +49,9 @@ These green check boxes and red crosses are **clickable** as well.
49
49
They will bring you to the corresponding page with details.
50
50
51
51
## Test Frameworks
52
+
52
53
### Pester
54
+
53
55
Our script-based test framework is [Pester](https://github.com/Pester/Pester).
54
56
This is the framework which we are using internally at Microsoft for new script-based tests,
55
57
and a large number of the tests which are part of the PowerShell project have been migrated from that test base.
@@ -59,45 +61,60 @@ Substantial changes were required, to get Pester executing on non-Windows system
59
61
These changes are not yet in the o
8000
fficial Pester code base.
60
62
Some features of Pester may not be available or may have incorrect behavior.
61
63
Please make sure to create issues in [PowerShell/PowerShell](https://github.com/PowerShell/PowerShell/issues) (not Pester) for anything that you find.
64
+
62
65
#### Test Tags
66
+
63
67
The Pester framework allows `Describe` blocks to be tagged, and our CI system relies on those tags to invoke our tests.
68
+
64
69
One of the following tags must be used:
70
+
65
71
*`CI` - this tag indicates that the tests in the `Describe` block will be executed as part of the CI/PR process
66
-
*`Feature` - tests with this tag will not be executed as part of the CI/PR process, but they will be executed on a daily basis as part of a `cron` driven build
67
-
They indicate that the test will be validating more behavior, or will be using remote network resources (ex: package management tests)
72
+
*`Feature` - tests with this tag will not be executed as part of the CI/PR process, but they will be executed on a daily basis as part of a `cron` driven build. They indicate that the test will be validating more behavior, or will be using remote network resources (ex: package management tests)
68
73
*`Scenario` - this tag indicates a larger scale test interacting with multiple areas of functionality and/or remote resources, these tests are also run daily.
69
74
70
75
Additionally, the tag:
76
+
71
77
*`SLOW` indicates that the test takes a somewhat longer time to execute (97% of our `CI` tests take 100ms or less), a test which takes longer than 1 second should be considered as a candidate for being tagged `Slow`
78
+
72
79
### xUnit
80
+
73
81
For those tests which are not easily run via Pester, we have decided to use [xUnit](https://xunit.github.io/) as the test framework.
74
82
Currently, we have a minuscule number of tests which are run by using xUnit.
75
83
76
84
## Running tests outside of CI
85
+
77
86
When working on new features or fixes, it is natural to want to run those tests locally before making a PR.
78
87
Two helper functions are part of the build.psm1 module to help with that:
88
+
79
89
*`Start-PSPester` will execute all Pester tests which are run by the CI system
80
90
*`Start-PSxUnit` will execute the available xUnit tests run by the CI system
91
+
81
92
Our CI system runs these as well; there should be no difference between running these on your dev system, versus in CI.
82
93
83
94
When running tests in this way, be sure that you have started PowerShell with `-noprofile` as some tests will fail if the
84
95
environment is not the default or has any customization.
85
96
86
97
For example, to run all the Pester tests for CI (assuming you are at the root of the PowerShell repo):
98
+
87
99
```
88
100
Import-Module ./build.psm1
89
101
Start-PSPester
90
102
```
103
+
91
104
If you wish to run specific tests, that is possible as well:
0 commit comments