[go: up one dir, main page]

Page MenuHomePhabricator

Instrumentation for Community Wishlist
Open, Stalled, LowPublic

Description

From request submitted to RDS via Asana:

We are launching the new Community Wishlist and would like to set up instrumentation to measure its impact. Broadly, we want to know the range of participants, traffic, and funnel metrics of our intake form.

The Wishlist reopens July 15 and we'd love to have some basic, high-level metrics in place.

Provide details about your request here:
The Community Wishlist is re-opening on July 15. We will measure success along two vectors:

We want to answer the following:

  • How much traffic does the wishlist generate?
  • How many people click to open the intake form?
    • how are they submitting wishes? Is it the form, or wikitext, or something else?
  • What is the form completion rate?
  • Which wishes are seeing the most engagement? (pageviews/comments)
  • Which focus areas see the most engagement? (pageviews, comments, votes)

Lastly, we want to measure the impact, however this is something we cna do manually:

  • # of focus areas adopted
  • # of wishes fulfilled by foundation.

Acceptance criteria

  • Connie w/ Jack & Karolin: Measurement plan defining metrics
  • Connie: Instrumentation specification (based on Metrics Platform data contract)
  • Connie: Log this in the data collection activity form (per Data Collection Guidelines)
  • Engineer (TBD): Implementation of the specification T370170
  • Engineer (TBD): Instrumentation QA (pre-deployment and post-deployment) T370170
  • Analyst (TBD): post-deployment data QA T370096

Document

Measurement Plan
Instrumentation Spec

Event Timeline

mpopov triaged this task as High priority.Jun 27 2024, 8:49 PM
mpopov created this task.

Just taking some notes here on the ones I know of, so we can continue discussing and planning:

How much traffic does the wishlist generate?

I assume we mean all visits to:
https://meta.wikimedia.org/wiki/Community_Wishlist and its subpages. (after they are deployed)

How many people click to open the intake form?

Count if visits to: https://meta.wikimedia.org/wiki/Community_Wishlist/Intake

How are they submitting wishes? Is it the form, or wikitext, or something else?

The form us accessed through the above URL that runs gadget that opens the form vs. click on "Edit Source"

What is the form completion rate?

Assuming validation succeeds and submit button (copy might change) in the bottom left is not greyed out and button can therefore be clicked.

Which wishes are seeing the most engagement? (pageviews/comments)

All wishes from this list, have a wish page, each one has it's own talk page
https://meta.wikimedia.org/wiki/Community_Wishlist/Wishes

Which focus areas see the most engagement? (pageviews, comments, votes)

The complete list of focus areas can be found here:
https://meta.wikimedia.org/wiki/Community_Wishlist/wiki/Community_Wishlist/Focus_areas
Each one will have a talk page and votes visibly displayed.

Open question:

  • are we also tracking editing existing wishes?

After conversation with Connie we know that to be able to track comments and votes on wishes and focus areas are hard to track in our case. The best way we can do it is to add specific edit tags for these events. e.g add edit tag "focus-area-voting" to edits that add {{Support}} template to focus area talk pages; and add edit tag "wish-comments" on edits that comment on wish pages

mpopov updated the task description. (Show Details)
mpopov changed the task status from Open to Stalled.Thu, Sep 12, 6:26 PM
mpopov removed cchen as the assignee of this task.
mpopov lowered the priority of this task from High to Low.
mpopov updated the task description. (Show Details)
mpopov moved this task from Next 2 weeks to Blocked on the Product-Analytics (Kanban) board.
mpopov added a subscriber: cchen.

Blocked because instrumentation implementation work (T370170) has been deprioritized while team focuses on addressing issues with core features.