MPIC: Frontend web testing
Open, LowPublic3 Estimated Story Points
Actions

Assigned To

Authored By

	Sfaci
	Jun 27 2024, 12:16 PM

Description

T360731

Description

Frontend is still consolidating and a lot of big changes are being done. It seems it's no the best time to prepare any test here. But, when consolidation finishes, we should create a solid web test suite that allows us to test all the functionality of MPIC while we keep working on the frontend side.

I have been exploring cypress as a framework to create end-to-end web tests. It seems that creating these tests with this library is not a lot of work and the time we can save is huge. There would be a bit of extra work if we want to prepare the frontend to be more testable ("tagging" properly web componentes somehow to identify them while coding the test cases, for example) but it's worth it.

There are already a couple of test cases working at https://gitlab.wikimedia.org/repos/data-engineering/mpic/-/tree/integration-tests?ref_type=heads (integration-tests branch that is WIP). Web tests can be run with `npx cypress run --spec cypress/e2e/mpic.cy.js'

Acceptance Criteria

~~We agree 'cypress' is the proper framework to use for web testing (of course this discussion is open to other frameworks)~~ << Per T368612#10069663 and discussions with QTE, we opted to go with WebdriverIO
We have, at least, test cases for all the positive scenarios:
- Instruments are listed properly
- Register a new instrument
- Modify an existing instrument
- View an instrument
- Delete an instrument

Notes

Cypress library: https://github.com/cypress-io/cypress
Sample e2e test case already working for MPIC (it modifies the description of an instrument): https://gitlab.wikimedia.org/repos/data-engineering/mpic/-/blob/integration-tests/cypress/e2e/mpic.cy.js?ref_type=heads

Related Objects

Mentioned In: T373180: MPIC: Improve and automate Frontend web testing
T371922: MPIC: Automate integration tests
T368801: [EPIC] MPIC: Fix outstanding issues
Mentioned Here: T139740: Port Selenium tests from Ruby to Node.js
T247835: Evaluate WebdriverIO replacements for our browser automation framework
T247844: Upgrade WebdriverIO to v6 in all repositories
T256626: Refactor WebdriverIO tests from sync to async mode
T266841: Outreachy 2020 - 2021 Proposal: Evaluate Microsoft Playwright as replacement for our browser automation
T274579: Upgrade WebdriverIO to v7 in all repositories
T302489: Review Cypress tests
T324685: Upgrade WebdriverIO to v8 in all repositories
T363704: Upgrade WebdriverIO to v9 in all repositories
T360731: [Epic] MP Instrument Configurator Frontend

Event Timeline

Sfaci created this task.Jun 27 2024, 12:16 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 27 2024, 12:16 PM

Sfaci updated the task description. (Show Details)Jun 27 2024, 12:19 PM

Sfaci set the point value for this task to 3.

cjming mentioned this in T368801: [EPIC] MPIC: Fix outstanding issues.Jun 28 2024, 10:21 PM

WDoranWMF moved this task from Sprint Backlog to Wormhole To Sprint 16 on the Data Products (Data Products Sprint 15) board.Jul 3 2024, 4:51 PM

WDoranWMF edited projects, added Data Products (Data Products Sprint 16); removed Data Products (Data Products Sprint 15).

cjming updated the task description. (Show Details)Jul 3 2024, 9:06 PM

@Jrbranaa we are ready to test this work. Is this something you are ready to start helping us with as early as this week?

cjming claimed this task.Jul 23 2024, 4:11 AM

cjming moved this task from Sprint Backlog to In Process on the Data Products (Data Products Sprint 16) board.

cjming moved this task from In Process to Sprint Backlog on the Data Products (Data Products Sprint 16) board.

WDoranWMF moved this task from Sprint Backlog to Wormhole To Sprint 17 on the Data Products (Data Products Sprint 16) board.Jul 24 2024, 4:29 PM

WDoranWMF edited projects, added Data Products (Data Products Sprint 17); removed Data Products (Data Products Sprint 16).

Goal is to prevent regressions when developing additional features and enhancements. @Sfaci what frameworks have you been investigating?

FYI @Jrbranaa

VirginiaPoundstone triaged this task as Low priority.Jul 29 2024, 7:24 PM

@Sfaci I noticed discussion about using Cypress, is there a reason the Selenium isn't being used? I only ask because that's the tooling/framework that we (QTE) support and that would be the quickest path forward for us to engage on it.

We are using mocha and jest to build unit and integration tests for the backend and I was investigating cypress to build web test cases.

There is no real reason to use cypress. I was investigating how to speed up web testing to avoid running manual steps everytime we add a new feature, I found it and I didn't know that Selenium was the supported framework. The work done with cypress was just a proof of concept to see if that framework could be used for MPIC and, as you can see at the ticket description, we left the discussion open to see if other framework could be better or more appropriate. So it seems that Selenium should be the one to use. I'm totally ok with that.

Sfaci mentioned this in T371922: MPIC: Automate integration tests.Aug 6 2024, 6:13 PM

cjming moved this task from Sprint Backlog to In Process on the Data Products (Data Products Sprint 17) board.Tue, Aug 13, 5:54 AM

Fwiw I just picked up this task and i'm looking at VueJS recommendations for E2E testing solutions. Per VueJS docs, Cypress is their framework of choice. They do mention other options including one called Nightwatch which is based on Selenium Webdriver - @Jrbranaa would this suffice for QTE?

Or should we go the route of a straight up vanilla Selenium integration?

@cjming let me check with Željko to see if/how Nightwatch would integrate with our current framework.

Jrbranaa added a project: Quality-and-Test-Engineering-Team.Thu, Aug 15, 11:44 PM

zeljkofilipin added a project: User-zeljkofilipin.Fri, Aug 16, 8:23 AM

zeljkofilipin moved this task from Backlog 🪒 to Deep work 🌊 on the User-zeljkofilipin board.

Hi! I'm working on end-to-end testing here for more than a decade and I'm glad to help! Beware, I have opinions! 😄

QTE developed and is maintaining Selenium-based end-to-end testing framework (documentation). Deep down, the framework is using WebdriverIO. (It's one of the testing tools mentioned in Vue testing docs.)

By maintaining the framework, I mean projects like

I investigated Nightwatch back 2017 when we were switching our end-to-end testing from ruby to javascript (T139740). (Those were the times.) According to the blog post I wrote back then I wasn't impressed with Nightwatch. But, that was many years ago.

Last time I looked at Cypress in Wikimedia (T302489) it was used in several repositories.

We also evaluate Selenium/WebdriverIO alternatives from time to time.

After a few evaluations, the conclusion was that every tool we tried worked fine. (Including our current WebdriverIO.) The only reason we didn't switch to any of the alternatives is that there's an investment in webdriverio (about 30 repositories) and it would take time to rewrite the tests to another tool.

Good news is that our continuous integration should support any tool you choose (we have tested a few). But if you choose another tool, QTE can't help you. We only have experience with WebdriverIO.

QTE has office hours, Monday-Thursday 13:00-15:00 UTC. Let me know if anybody from the team can join the meeting any day next week. That would speed things up, if you need help. (The meeting is in my Google calendar. I can send invitations if that's more convenient.)