Tests: migrate testing infrastructure to minimal dependencies

This is a complete rework of our testing infrastructure. The main goal is to modernize and drop deprecated or undermaintained dependencies (specifically, grunt, karma, and testswarm). We've achieved that by limiting our dependency list to ones that are unlikely to drop support any time soon. The new dependency list includes: - `qunit` (our trusty unit testing library) - `selenium-webdriver` (for spinning up local browsers) - `express` (for starting a test server and adding middleware) - express middleware includes uses of `body-parser` and `raw-body` - `yargs` (for constructing a CLI with pretty help text) - BrowserStack (for running each of our QUnit modules separately in all of our supported browsers) - `browserstack-local` (for opening a local tunnel. This is the same package still currently used in the new Browserstack SDK) - We are not using any other BrowserStack library. The newest BrowserStack SDK does not fit our needs (and isn't open source). Existing libraries, such as `node-browserstack` or `browserstack-runner`, either do not quite fit our needs, are under-maintained and out-of-date, or are not robust enough to meet all of our requirements. We instead call the [BrowserStack REST API](https://github.com/browserstack/api) directly. - automatically retries individual modules in case of test failure(s) - automatically attempts to re-establish broken tunnels - automatically refreshes the page in case a test run has stalled - runs all browsers concurrently and uses as many sessions as are available under the BrowserStack plan. It will wait for available sessions if there are none. - supports filtering the available list of browsers by browser name, browser version, device, OS, and OS version (see `npm run test:unit -- --list-browsers` for more info). It will retrieve the latest matching browser available if any of those parameters are not specified. - cleans up after itself (closes the local tunnel, stops the test server, etc.) - Requires `BROWSERSTACK_USERNAME` and `BROWSERSTACK_ACCESS_KEY` environment variables. - supports running any local browser as long as the driver is installed, including support for headless mode in Chrome, FF, and Edge - supports running `basic` tests on the latest [jsdom](https://github.com/jsdom/jsdom#readme), which can be seen in action in this PR (see `test:browserless`) - Node tests will run as before in PRs and all non-dependabot branches, but now includes tests on real Safari in a GH actions macos image instead of playwright-webkit. - can run multiple browsers and multiple modules concurrently Other notes: - Stale dependencies have been removed and all remaining dependencies have been upgraded with a few exceptions: - `sinon`: stopped supporting IE in version 10. But, `sinon` has been updated to 9.x. - `husky`: latest does not support Node 10 and runs on `npm install`. Needed for now until git builds are migrated to GitHub Actions. - `rollup`: latest does not support Node 10. Needed for now until git builds are migrated to GitHub Actions. - BrowserStack tests are set to run on each `main` branch commit - `debug` mode leaves Selenium browsers open whether they pass or fail and leaves browsers with test failures open on BrowserStack. The latter is to avoid leaving open too many sessions. - This PR includes a workflow to dispatch BrowserStack runs on-demand - The Node version used for most workflow tests has been upgraded to 20.x - updated supportjQuery to 3.7.1 Run `npm run test:unit -- --help` for CLI documentation Close gh-5418
jquery · timmywil · Mar 5, 2024 · Feb 26, 2024 · Feb 26, 2024 · Feb 28, 2024
commit 2996f4f0c1d76104e94a04748e76f88d5bf26abf
diff --git a/.github/workflows/browserstack-dispatch.yml b/.github/workflows/browserstack-dispatch.yml
@@ -0,0 +1,62 @@
+name: Browserstack (Manual Dispatch)
+
+on:
+  workflow_dispatch:
+    inputs:
+      module:
+        description: 'Module to test'
+        required: true
+        type: choice
+        options:
+          - 'basic'
+          - 'ajax'
+          - 'animation'
+          - 'attributes'
+          - 'callbacks'
+          - 'core'
+          - 'css'
+          - 'data'
+          - 'deferred'
+          - 'deprecated'
+          - 'dimensions'
+          - 'effects'
+          - 'event'
+          - 'manipulation'
+          - 'offset'
+          - 'queue'
+          - 'selector'
+          - 'serialize'
+          - 'support'
+          - 'traversing'
+          - 'tween'
+      browser:
+        description: 'Browser to test, in form of \"browser_[browserVersion | :device]_os_osVersion\"'
+        required: false
+        type: string
+        default: 'chrome__windows_11'
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    environment: browserstack
+    env:
+      BROWSERSTACK_USERNAME: ${{ secrets.BROWSERSTACK_USERNAME }}
+      BROWSERSTACK_ACCESS_KEY: ${{ secrets.BROWSERSTACK_ACCESS_KEY }}
+    steps:
+      - uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
+
+      - uses: actions/setup-node@60edb5dd545a775178f52524783378180af0d1f8 # v4.0.2
+        with:
+          node-version: 20
+
+      - name: Install dependencies
+        run: npm ci
+
+      - name: Build jQuery
+        run: npm run build:all
+
+      - name: Pretest script
+        run: npm run pretest
+
+      - name: Run tests
+        run: npm run test:unit -- -v --browserstack ${{ inputs.browser }} -m ${{ inputs.module }}
diff --git a/.github/workflows/browserstack.yml b/.github/workflows/browserstack.yml
@@ -0,0 +1,64 @@
+name: Browserstack
+
+on:
+  push:
+    branches:
+      - main
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    environment: browserstack
+    env:
+      BROWSERSTACK_USERNAME: ${{ secrets.BROWSERSTACK_USERNAME }}
+      BROWSERSTACK_ACCESS_KEY: ${{ secrets.BROWSERSTACK_ACCESS_KEY }}
+      NODE_VERSION: 20.x
+    name: ${{ matrix.BROWSER }}
+    concurrency:
+      group: ${{ github.workflow }} ${{ matrix.BROWSER }}
+    strategy:
+      fail-fast: false
+      matrix:
+        BROWSER:
+          - 'IE_11'
+          - 'Safari_17'
+          - 'Safari_16'
+          - 'Chrome_120'
+          - 'Chrome_119'
+          - 'Edge_120'
+          - 'Edge_119'
+          - 'Firefox_121'
+          - 'Firefox_120'
+          - 'Firefox_115'
+          - '__iOS_17'
+          - '__iOS_16'
+          - '__iOS_15'
+          - 'Opera_106'
+    steps:
+      - name: Checkout
+        uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
+
+      - name: Use Node.js ${{ env.NODE_VERSION }}
+        uses: actions/setup-node@60edb5dd545a775178f52524783378180af0d1f8 # v4.0.2
+        with:
+          node-version: ${{ env.NODE_VERSION }}
+
+      - name: Cache
+        uses: actions/cache@13aacd865c20de90d75de3b17ebe84f7a17d57d2 # v4.0.0
+        with:
+          path: ~/.npm
+          key: ${{ runner.os }}-node-${{ env.NODE_VERSION }}-npm-lock-${{ hashFiles('**/package-lock.json') }}
+          restore-keys: |
+            ${{ runner.os }}-node-${{ env.NODE_VERSION }}-npm-lock-
+
+      - name: Install dependencies
+        run: npm install
+
+      - name: Build jQuery
+        run: npm run build:all
+
+      - name: Pretest script
+        run: npm run pretest
+
+      - name: Run tests
+        run: npm run test:unit -- -v --browserstack "${{ matrix.BROWSER }}" --retries 3
diff --git a/.github/workflows/node.js.yml b/.github/workflows/node.js.yml
@@ -1,4 +1,4 @@
-name: CI
+name: Node
 
 on:
   pull_request:
@@ -9,45 +9,49 @@ permissions:
   contents: read # to fetch code (actions/checkout)
 
 jobs:
-  build:
+  build-and-test:
     runs-on: ubuntu-latest
+    name: ${{ matrix.NPM_SCRIPT }} - ${{ matrix.NAME }} (${{ matrix.NODE_VERSION }})
     strategy:
       fail-fast: false
       matrix:
-        # Node.js 10 is required by jQuery infra
-        # Do not remove 16.x until jsdom tests are re-enabled on newer Node.js versions.
-        NODE_VERSION: [10.x, 16.x, 18.x, 20.x]
+        NAME: ["Node"]
+        NODE_VERSION: [18.x, 20.x]
         NPM_SCRIPT: ["test:browserless"]
         include:
-          - NAME: "Browser tests: full build, Chrome, Firefox & WebKit"
-            NODE_VERSION: "18.x"
+          - NAME: "Node"
+            NODE_VERSION: "20.x"
+            NPM_SCRIPT: "lint"
+          - NAME: "Chrome/Firefox"
+            NODE_VERSION: "20.x"
             NPM_SCRIPT: "test:browser"
-            BROWSERS: "ChromeHeadless,FirefoxHeadless,WebkitHeadless"
-          - NAME: "Browser tests: slim build, Chrome"
-            NODE_VERSION: "18.x"
+          - NAME: "Chrome"
+            NODE_VERSION: "20.x"
             NPM_SCRIPT: "test:slim"
-            BROWSERS: "ChromeHeadless"
-          - NAME: "Browser tests: no-deprecated build, Chrome"
-            NODE_VERSION: "18.x"
+          - NAME: "Chrome"
+            NODE_VERSION: "20.x"
             NPM_SCRIPT: "test:no-deprecated"
-            BROWSERS: "ChromeHeadless"
-          - NAME: "Browser tests: selector-native build, Chrome"
-            NODE_VERSION: "18.x"
+          - NAME: "Chrome"
+            NODE_VERSION: "20.x"
             NPM_SCRIPT: "test:selector-native"
-            BROWSERS: "ChromeHeadless"
-          - NAME: "Browser tests: AMD build, Chrome stable"
-            NODE_VERSION: "18.x"
+          - NAME: "Chrome"
+            NODE_VERSION: "20.x"
             NPM_SCRIPT: "test:amd"
-            BROWSERS: "ChromeHeadless"
-          - NAME: "Browser tests: full build, Firefox ESR"
-            NODE_VERSION: "18.x"
-            NPM_SCRIPT: "test:browser"
-            BROWSERS: "FirefoxHeadless"
-
+          - NAME: "Firefox ESR"
+            NODE_VERSION: "20.x"
+            NPM_SCRIPT: "test:firefox"
+          - NAME: "Node 10 Build"
+            NODE_VERSION: "10.x"
+            NPM_SCRIPT: "build:main"
     steps:
       - name: Checkout
         uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
 
+      - name: Use Node.js ${{ matrix.NODE_VERSION }}
+        uses: actions/setup-node@60edb5dd545a775178f52524783378180af0d1f8 # v4.0.2
+        with:
+          node-version: ${{ matrix.NODE_VERSION }}
+
       - name: Cache
         uses: actions/cache@13aacd865c20de90d75de3b17ebe84f7a17d57d2 # v4.0.0
         with:
@@ -56,31 +60,49 @@ jobs:
           restore-keys: |
             ${{ runner.os }}-node-${{ matrix.NODE_VERSION }}-npm-lock-
 
-      - name: Use Node.js ${{ matrix.NODE_VERSION }}
-        uses: actions/setup-node@8f152de45cc393bb48ce5d89d36b731f54556e65 # v4.0.0
-        with:
-          node-version: ${{ matrix.NODE_VERSION }}
-
       - name: Install firefox ESR
         run: |
-          export FIREFOX_SOURCE_URL='https://download.mozilla.org/?product=firefox-esr-latest&lang=en-US&os=linux64'
+          export FIREFOX_SOURCE_URL='https://download.mozilla.org/?product=firefox-esr-latest-ssl&lang=en-US&os=linux64'
           wget --no-verbose $FIREFOX_SOURCE_URL -O - | tar -jx -C ${HOME}
         if: contains(matrix.NAME, 'Firefox ESR')
 
       - name: Install dependencies
         run: npm install
 
-      - name: Install Playwright dependencies
-        run: npx playwright-webkit install-deps
-        if: matrix.NPM_SCRIPT == 'test:browser' && contains(matrix.BROWSERS, 'WebkitHeadless')
-
-      - name: Lint code
-        run: npm run build && npm run lint
-        if: matrix.NODE_VERSION == '18.x'
+      - name: Build All for Linting
+    <
10000
span class='blob-code-inner blob-code-marker ' data-code-marker="+">        run: npm run build:all
+        if: contains(matrix.NPM_SCRIPT, 'lint')
 
       - name: Run tests
-        env:
-          BROWSERS: ${{ matrix.BROWSERS }}
         run: |
           export PATH=${HOME}/firefox:$PATH
+          export FIREFOX_BIN=${HOME}/firefox/firefox
           npm run ${{ matrix.NPM_SCRIPT }}
+
+  safari:
+    runs-on: macos-latest
+    env:
+      NODE_VERSION: 20.x
+    name: test:safari - Safari
+    steps:
+      - name: Checkout
+        uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
+
+      - name: Use Node.js ${{ env.NODE_VERSION }}
+        uses: actions/setup-node@60edb5dd545a775178f52524783378180af0d1f8 # v4.0.2
+        with:
+          node-version: ${{ env.NODE_VERSION }}
+
+      - name: Cache
+        uses: actions/cache@13aacd865c20de90d75de3b17ebe84f7a17d57d2 # v4.0.0
+        with:
+          path: ~/.npm
+          key: ${{ runner.os }}-node-${{ env.NODE_VERSION }}-npm-lock-${{ hashFiles('**/package-lock.json') }}
+          restore-keys: |
+            ${{ runner.os }}-node-${{ env.NODE_VERSION }}-npm-lock-
+
+      - name: Install dependencies
+        run: npm install
+
+      - name: Run tests
+        run: npm run test:safari
diff --git a/.gitignore b/.gitignore
@@ -19,3 +19,7 @@ npm-debug.log*
 
 /test/data/core/jquery-iterability-transpiled.js
 /test/data/qunit-fixture.js
+
+# Ignore BrowserStack files
+local.log
+browserstack.err
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -68,12 +68,6 @@ We *love* when people contribute back to the project by patching the bugs they f
 
 Create a fork of the jQuery repo on github at https://github.com/jquery/jquery
 
-Change directory to your web root directory, whatever that might be:
-
-```bash
-$ cd /path/to/your/www/root/
-```
-
 Clone your jQuery fork to work locally
 
 ```bash
@@ -98,29 +92,47 @@ Get in the habit of pulling in the "upstream" main to stay up to date as jQuery
 $ git pull upstream main
 ```
 
-Run the build script
+Install the necessary dependencies
 
 ```bash
-$ npm run build
+$ npm install
 ```
 
-Now open the jQuery test suite in a browser at http://localhost/test. If there is a port, be sure to include it.
+Build all jQuery files
 
-Success! You just built and tested jQuery!
+```bash
+$ npm run build:all
+```
+
+Start a test server
 
+```bash
+$ npm run test:server
+```
+
+Now open the jQuery test suite in a browser at http://localhost:3000/test/.
+
+Success! You just built and tested jQuery!
 
 ### Test Suite Tips...
 
 During the process of writing your patch, you will run the test suite MANY times. You can speed up the process by narrowing the running test suite down to the module you are testing by either double clicking the title of the test or appending it to the url. The following examples assume you're working on a local repo, hosted on your localhost server.
 
 Example:
 
-http://localhost/test/?module=css
+http://localhost:3000/test/?module=css
 
 This will only run the "css" module tests. This will significantly speed up your development and debugging.
 
 **ALWAYS RUN THE FULL SUITE BEFORE COMMITTING AND PUSHING A PATCH!**
 
+#### Change the test server port
+
+The default port for the test server is 3000. You can change the port by setting the `PORT` environment variable.
+
+```bash
+$ PORT=3001 npm run test:server
+```
 
 #### Loading changes on the test page
 
@@ -134,6 +146,29 @@ Alternatively, you can **load tests in AMD** to avoid the need for rebuilding al
 
 Click "Load with AMD" after loading the test page.
 
+#### Running the test suite from the command line
+
+You can also run the test suite from the command line.
+
+First, prepare the tests:
+
+```bash
+$ npm run pretest
+```
+
+Make sure jQuery is built (`npm run build:all`) and run the tests:
+
+```bash
+$ npm run test:unit
+```
+
+This will run each module in its own browser instance and report the results in the terminal.
+
+View the full help for the test suite for more info on running the tests from the command line:
+
+```bash
+$ npm run test:unit -- --help
+```
 
 ### Repo organization