SLEP019: Governance Update - Recognizing Contributions Beyond Code #74
Conversation
Co-authored-by: Gael Varoquaux <gael.varoquaux@normalesup.org>
|
Note that our governance is not very clear on whether we need a SLEP for changes to the governance itself, it only says the same decision making process is required. For changes to the governance doc in the past, we haven't done a SLEP, we have only called a vote on the proposed changes. |
|
I understand the Decision Making Process as indicating coming up with a SLEP and a vote on the internal mailing list for this SLEP prior to Governance changes:
|
Co-authored-by: Andreas Mueller <amueller@microsoft.com>
Co-authored-by: Juan Martin Loyola <jmloyola@outlook.com>
|
some notes: adding some references: b) NumPy Newcomer's Hour: an Experiment on Community Building: Melissa Weber Mendonça |
I think she's also given a talk about contribution metrics (probably similar content to the podcast): |
|
Something to consider, as I sit in on a session on language translation at
this meeting. And our recent attention and evaluation to the various types
of (in/visible) contributions to the project.
To consider that Mariangela Petrizzo's initiative of translation of the
scikit-learn docs is impactful.
And to consider that type of contribution to the project...... that we
would consider nominating her to the team.
scikit-learn/scikit-learn#20763
Reshama
…On Tue, Sep 20, 2022 at 11:06 PM Lucy Liu ***@***.***> wrote:
a) Amanda Casari on ACROSS and Measuring Contributions in OSS
I think she's also given a talk about contribution metrics (probably
similar content to the podcast):
https://www.youtube.com/watch?v=yrBqXfGzx-U
—
Reply to this email directly, view it on GitHub
<#74 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AATEDYBKTLZ3IW4N6BJXEQDV7J3TVANCNFSM6AAAAAAQLOMBV4>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Co-authored-by: Reshama Shaikh <reshama.stat@gmail.com>
|
@reshamas: please, let me know if the resources you suggested have been properly added and cited in 8ca1922. As for translations of the documentation, I think we also need to recognize those contributions. Yet, I think this is a currently a bit orthogonal to the goal of this PR and I think recognising people contributions might better be done after we settle on the Governance update. What do you think? @lucyleeow: thank you for suggesting this resource. The content covered is a bit different: it emphasis defining the community goals then coming up setting metrics accordingly. As this SLEP aims at recognising all contributions and team and mostly is conceptual, I think it might be preferable to include and use this resource at a latter stage, e.g. for the implementation of this SLEP. What do you think? In both cases, I am glad people give their opinion and discuss the community structure and dynamic. This show that we probably have not been having room, nor time, nor place set to discuss those topics. |
No problem, that makes sense - sorry it has been a while since I watched it and forgot! |
It's fine. With the lack of place for people to spontaneously discuss, I personally would rather have spontaneous suggestions and interjections in discussion on GitHub than people keep their views and ideas for themselves. The only place scikit-learn has for people to discuss is the monthly "Core-Devs' meeting". Other projects like SciPy and NumPy have "Community Meeting". I think this situation motives this SLEPs, as in the case we also probably need to change this semantic and come up with another organisation (e.g. have a general "Community Meeting" and than more scoped/focused "Teams' meetings"). |
I think the community meetings are new and the result of work funded by: https://chanzuckerberg.com/eoss/proposals/advancing-an-inclusive-culture-in-the-scientific-python-ecosystem/ |
slep019/proposal.rst
Outdated
| A Contributor is added to the "Recurring Contributor" team after championing by | ||
| three Core contributors (notion of "Core contributors" as defined below). | ||
|
|
||
| **"Create "Experts" Team** |
There was a problem hiding this comment.
Do we really need a separate team here? Should we rather allow Recurring Contributors to be delegated responsibility for review?
There was a problem hiding this comment.
I do not remember how we converged on defining "Experts Team" at the last monthly meeting (do you, @GaelVaroquaux?). to me there's an overlap between concept that we want to introduce at the moment, which brings confusion.
If I were to chose, I would not introduce "Experts Team".
I would rather have:
- Contributions be defined explicitly (see https://github.com/scikit-learn/scikit-learn/pull/24444/files#diff-1bc88ee27d61654009a440dafc16e35fc3ccb64d78db4a07316afd7348793befR35-R56)
- Roles be defined to qualify individual involvement and Contributions in the project, and to give recognition and responsibilities. I believe we could have 3 Roles; following the proposal in DOC SLEP019 Governance Update scikit-learn#24444:
- Contributors
- Recurrent Contributors
- Core Contributions
- Teams be defined as groups of Contributors having similar interest, and the autonomy (and authority) of some aspects of the projects. I believe Teams could be:
- Documentation Team
- Contributor Experience Team
- CI/CD Team
- Topical Teams
- etc.
(In this organisation structure, we could imagine Core Contributors be the Expert we originally intended (e.g. they could lead their team projects, conduct their team discussions etc.) without introducing the notion of "Expert", nor expecting them to behave as such if they do not want to.)
To me, this organisation would fit the needs of the community better, but this would be a notable update which is larger than originally planned.
What do you think? Do you have any proposal?
There was a problem hiding this comment.
I think the point @GaelVaroquaux made with which I agree, was that there is a different between recurring contributors and the experts we trust for reviews. We'd like to acknowledge recurring contributors, but we don't necessarily want that to mean we trust their reviews.
I was more on the side of having several teams for each subject, but seems like we'd like to start with a single team, which basically means "we trust your reviews, but not on everything". I'm happy to start with that, but it doesn't make it easy to tag them. Ideally, we could do something like: @scikit-learn/tree-experts please review
There was a problem hiding this comment.
I wonder if we have enough experts to warrant such fine-grained management (e.g. tree-experts vs manifold-experts etc.). I like the idea of having a group of members whose reviews are trusted locally but not globally. For such contributors/experts, I would prefer their expertise be recorded person-by-person, rather than attributed to a particular team (at least until there's enough to justify such teams). It could be tracked on a document such as our current list of reviewers.
There was a problem hiding this comment.
There was a problem hiding this comment.
I like the idea of setting up code-owners, but I think this feature probably better be used to indicate maintainers for each part of the project code-base rather than experts?
There was a problem hiding this comment.
Looking for a solution that has high inclusivity, high quality, low bureaucracy, easy for core devs to use...
Explicitly introducing the idea of teams for certain parts of the codebase has the advantage that even a core dev that might not know who to delegate to will get some clues from the membership of a team. It also makes it clear that a member might nominate themselves as a reviewer for a specific pr, subject to core dev approval.
However, do we want this formal ownership structure to disallow ad hoc review delegation, such as in the case that there is a very specific piece of code for which someone is expert (too fine grained to create a team)?
As such, I don't think we should encode the specific mechanism of cataloging and proscribing individual contributors' expertise in the governance doc. Once we describe the general power of delegation and introduce the notion of non-committer org members, the team can organically develop management practices around these expert resources.
slep019/proposal.rst
Outdated
| project's website "About Us" webpage [7]_. | ||
|
|
||
| A Contributor is added to the "Recurring Contributor" team after championing by | ||
| three Core contributors (notion of "Core contributors" as defined below). |
There was a problem hiding this comment.
I think this is more burdensome and bureaucratic than necessary, certainly if this team remains distinguished from "experts".
There was a problem hiding this comment.
I entirely agree (see my other comment on lines you commented bellow).
slep019/proposal.rst
Outdated
| PR can be merged with one approval from a Core Developer and one approval from | ||
| the delegated expert. |
There was a problem hiding this comment.
To be sure, can the same core dev approve a PR and delegate to an expert? Or do they need to be two separate core devs involved?
There was a problem hiding this comment.
Is it that they delegate, or that we change our PR merge policy to: two approvals, at least one from a core dev?
There was a problem hiding this comment.
I understood this to mean: merge with one core developer approval and one other approval from either a core developer or a delegate of a core developer.
It's unclear to me whether there is a risk of abuse when one core developer is both approver and delegator.
There was a problem hiding this comment.
Is it that they delegate, or that we change our PR merge policy to: two approvals, at least one from a core dev?
This is not specified yet.
Does it really make sense to introduce an Experts' Team yet?
slep019/proposal.rst
Outdated
|
|
||
| **"Create "Experts" Team** | ||
|
|
||
| Create a "Experts" team, also without specific GitHub permissions. Core |
There was a problem hiding this comment.
No clarity on how someone gets to become an expert. It seems strange that one needs 3 approvals to become a recurring contributor, but no spec is given for approving this more risky role.
There was a problem hiding this comment.
It seems strange that one needs 3 approvals to become a recurring contributor.
This was proposed by @GaelVaroquaux. I would propose something more in line with what exists today, that is having someone be proposed by another person of the role of promotion and have them get this role after a period of one month if no-ones disagrees. This is proposed in the associated PR:
Contributors are promoted as Recurrent Contributors after the proposal of at least another Recurrent Contributor and after a vote on the private mailing list.
There was a problem hiding this comment.
Thank you for sharing your thoughts, @jnothman.
I also think there's confusion or a lack of clarity with the notion of "Experts Team" as its overlap with other concepts.
This observation is also something that was originally mentioned by @adrinjalali and @lorentzenchr on the internal mailing list.
I propose pursuing discussion in threads and converging on notions that make sense for everyone.
slep019/proposal.rst
Outdated
| A Contributor is added to the "Recurring Contributor" team after championing by | ||
| three Core contributors (notion of "Core contributors" as defined below). | ||
|
|
||
| **"Create "Experts" Team** |
There was a problem hiding this comment.
I do not remember how we converged on defining "Experts Team" at the last monthly meeting (do you, @GaelVaroquaux?). to me there's an overlap between concept that we want to introduce at the moment, which brings confusion.
If I were to chose, I would not introduce "Experts Team".
I would rather have:
- Contributions be defined explicitly (see https://github.com/scikit-learn/scikit-learn/pull/24444/files#diff-1bc88ee27d61654009a440dafc16e35fc3ccb64d78db4a07316afd7348793befR35-R56)
- Roles be defined to qualify individual involvement and Contributions in the project, and to give recognition and responsibilities. I believe we could have 3 Roles; following the proposal in DOC SLEP019 Governance Update scikit-learn#24444:
- Contributors
- Recurrent Contributors
- Core Contributions
- Teams be defined as groups of Contributors having similar interest, and the autonomy (and authority) of some aspects of the projects. I believe Teams could be:
- Documentation Team
- Contributor Experience Team
- CI/CD Team
- Topical Teams
- etc.
(In this organisation structure, we could imagine Core Contributors be the Expert we originally intended (e.g. they could lead their team projects, conduct their team discussions etc.) without introducing the notion of "Expert", nor expecting them to behave as such if they do not want to.)
To me, this organisation would fit the needs of the community better, but this would be a notable update which is larger than originally planned.
What do you think? Do you have any proposal?
slep019/proposal.rst
Outdated
| project's website "About Us" webpage [7]_. | ||
|
|
||
| A Contributor is added to the "Recurring Contributor" team after championing by | ||
| three Core contributors (notion of "Core contributors" as defined below). |
There was a problem hiding this comment.
I entirely agree (see my other comment on lines you commented bellow).
slep019/proposal.rst
Outdated
| PR can be merged with one approval from a Core Developer and one approval from | ||
| the delegated expert. |
There was a problem hiding this comment.
Is it that they delegate, or that we change our PR merge policy to: two approvals, at least one from a core dev?
Co-authored-by: Adrin Jalali <adrin.jalali@gmail.com>
There was a problem hiding this comment.
Do we need to vote to include people on any of the teams? For example, if someone gets nominated to be on the Recurring Contributors Team, do we hold a vote?
For me, I am +1 on the "Core Triaging" team and the "Expert Team". I have my reservations about the "Recurring Contributor Team".
slep019/proposal.rst
Outdated
|
|
||
| **"Recurring "Contributors" Team** | ||
|
|
||
| Create a "Recurring Contributors" team on GitHub, which has no specific GitHub |
There was a problem hiding this comment.
If someone is organizing events, helping with sprints, reviewing PRs, or helping other contributors, then I would want them directly in the Contributor Experience Team. These individuals have demonstrated a willingness to help with the project.
From my experience with other non-technical volunteer-lead organizations, named roles are given to individuals by giving them more responsibilities. This turns into a sense of ownership for the group's goals.
Is the Recurring Contributors Team, only for acknowledgement without any responsibility? In that case, I do not think it benefits the project or the contributor. For instance, if someone puts "scikit-learn's Recurring Contributor Team" on a CV, how would they describe that role to an interviewer?
There was a problem hiding this comment.
@thomasjpfan
For PyMC, they use a private Slack for the team. Recurring contributors (+ GSOC students + project contractors) are invited to join the Slack and join the discussions. Otherwise, they are not listed on the website or anything else.
I believe the intention is to recognize and engage recurring contributors who may potentially become team members.
For example, the team in 2020 planned an online conference PyMCon 2020 and recurring contributors may be interested in being more involved.
There was a problem hiding this comment.
From memory, the secrets discussion was a separate one for giving core developers "Admin" permissions, which has access to the secrets.
For Core triagers, "Write" permissions are fine. We have branch protection rules in place, which only allows core-devs to push to main and release branches:
@thomasjpfan here the thread where you summarize that giving 'write' access to triaging people is not a possible solution.
There was a problem hiding this comment.
Ah I see. I mixed up that mailing list discussion with the recent discussion about triage permissions during the 09-2022 meeting.
Looking at the github docs the secrets require "admin" permissions, which is a few levels above "write". With that in mind, I think it is okay to give "write" permissions to triage members now.
There was a problem hiding this comment.
Is the Recurring Contributors Team, only for acknowledgement without any responsibility?
To me, Recurring Contributors would be a Team where Contributors would also be given more responsibilities (as proposed in the associated PRs).
There was a problem hiding this comment.
As for #74 (comment), @adrinjalali mentioned that he wanted to keep the governance short on purpose, but that is a separate topic.
As for recurring contributors it's only responsibility is:
- Willing to attend events and sessions organized by their team.
I think organizing events require leadership from the core team. For me, attending events is not much of a responsibility because it can be done passively. (I'm guessing PyMC's sprint was lead by their core team and called for volunteers from their recurring contributors list)
There was a problem hiding this comment.
@thomasjpfan PyMC's sprint was led by me (at Data Umbrella with contributions from my team at DU) and one of the core team members at PyMC was involved in helping set up the website.
There was a problem hiding this comment.
If someone is organizing events, helping with sprints, reviewing PRs, or helping other contributors, then I would want them directly in the Contributor Experience Team. These individuals have demonstrated a willingness to help with the project.
From my experience with other non-technical volunteer-lead organizations, named roles are given to individuals by giving them more responsibilities. This turns into a sense of ownership for the group's goals.
Is the Recurring Contributors Team, only for acknowledgement without any responsibility? In that case, I do not think it benefits the project or the contributor. For instance, if someone puts "scikit-learn's Recurring Contributor Team" on a CV, how would they describe that role to an interviewer?
@thomasjpfan Some of my thoughts on this are below:
a) I'm not a big fan of the term "contributor experience team" because it centers around the contributor, but we have more users than contributors and much of what we do is for the user. But, that's a side point. I didn't vote for this name change, though it did receive the majority of votes.
b) Can you give examples of "other non-technical volunteer-lead organizations"?
c) > Is the Recurring Contributors Team, only for acknowledgement without any responsibility? In that case, I do not think it benefits the project or the contributor. For instance, if someone puts "scikit-learn's Recurring Contributor Team" on a CV, how would they describe that role to an interviewer?
The recurring contributor is one who has made a/some notable contributions, is being acknowledged and may have a path for 8DEB ward to becoming a core contributor. This role is not solely to build a resume, but to build contributorship towards the project, if they wish.
d) >If someone is organizing events, helping with sprints, reviewing PRs, or helping other contributors, then I would want them directly in the Contributor Experience Team. These individuals have demonstrated a willingness to help with the project.
This seems to be a chicken/egg issue. Someone needs to help with sprints and more before they can be on the "Contributor Experience Team". If they've never done it before, then they can't be on the Team first, but they can help and contribute first, and the "recurring role" provides this opportunity.
There was a problem hiding this comment.
The recurring contributor is one who has made a/some notable contributions, is being acknowledged and may have a path forward to becoming a core contributor. This role is not solely to build a resume, but to build contributorship towards the project, if they wish.
To this end, I am very interested in @adrinjalali's comment regarding "scikit-learn fellows" (with names being discounted for now). In general, I think that a more granular path towards long-term contributions and stewardship, in any capacity, is a good thing for the health of the project. However I think there is much to be gained by making it more explicit, as several have mentioned so far. To that end, I like the idea of having two paths:
People may be nominated for a "recurring contributor" position by any core contributor, and if an internal vote is successful, then the contributor's acceptance results in an immediate inclusion to the role.
OR
People may nominate themselves, in which case relevant core contributors may either support their nomination (for governance sake this would be equivalent to nominating the contributor themselves), or if there exists a gap, provide an explicit path towards acceptance. Of course this isn't so much a quantitative decision boundary, but a heuristic that offers a direction for growth. Something like "contribute more to discussion regarding..." or "try your hand at reviewing some PRs such as..."
Either way, to maintain their position in this role, they must meet expectations regarding activity/participation. In this way, it offers recognition for efforts and provides explicit direction towards elevated rights vis-a-vis core positions.
This way recurring (even if not very frequent) contributors may be recognized and encouraged, while new/aspiring contributors may be inspired and supported.
There was a problem hiding this comment.
I want to acknowledge that scikit-learn and PyMC are different projects of different breadths, in terms of user base, contributors, etc. And I don't believe there needs to be a 1-to-1 replication of their workflow. There are some things I think scikit-learn does very well, and there are some things I observe other open source projects do well that could benefit scikit-learn.
I would like to share some observations on the "recurring contributor" role that I have observed in the PyMC project (https://github.com/pymc-devs). Whenever a community contributor has made a range of contributions that are notable (Example: significant pull requests, reviewing pull requests, specific expertise (conda feedback), answering user questions on platforms (GH, Discourse), creating educational material, organizing sprints, organizing their annual conference PyMCon, GSOC students, etc), they are invited to the PyMC Slack team where the "core team" shares, communicates, discusses, shares fun tweets, conferences and more. It's a very laid-back, friendly and welcoming community atmosphere. They have:
- 48 people on the GitHub team (https://github.com/orgs/pymc-devs/people)
- 70 people in Slack
The benefits to this are many:
- a way for the recurring community member to connect to the core team
- an opportunity to see what areas need more help and who might be interested in contributing (particular issue with a PR, organizing an event, help with a particular issue, etc)
- an opportunity for GSOC students to remain connected and continue to be a contributor to the project, if they wish. (and many GSOC students do continue to stay involved in the project)
I currently observe these routes for the community members to join the scikit-learn team:
- high level researcher at an institution, as a profession
- employee at INRIA (some folks here)
- someone who organized sprints for 3-4 years and created educational resources (me)
- referred by a current team member (Lauren)
- mentored by a current team member (some folks here)
The recurring role provides an avenue aside from one of the above to become a core contributor. Its name can be discussed (recurring, fellow (although that leans towards men, semantically speaking, but then so much of our society does)).
It used to be that only high level researchers were able to become core team members (and this SLEP is here to extend that ;) ).
Co-authored-by: Thomas J. Fan <thomasjpfan@gmail.com>
Co-authored-by: Chiara Marmo <cmarmo@users.noreply.github.com> Co-authored-by: Thomas J. Fan <thomasjpfan@gmail.com>
The American English grammar has its reasons which reason knows nothing of. 🙃 Co-authored-by: Adrin Jalali <adrin.jalali@gmail.com>
| Simplify subsequent changes to the Governance | ||
| ============================================= |
There was a problem hiding this comment.
As discussed during the drafting meeting, this section does not yet specify the duration of the votes instituted through PRs. We ought to make this explicit -- perhaps 4 weeks as is standard for SLEPs would still be appropriate for governance changes?
There was a problem hiding this comment.
I am voting -1, so we can have another voting period. The document has changed quite a bit since the initial vote was called.
There are two possible options:
- A smaller SLEP that focuses on "Simplify subsequent changes to the Governance"
- This document as is, which includes the Triage Team, renaming of the technical committee, extending voting rights and simplify governance changes.
We go for option 1 if we want an easier SLEP to vote on and option 2 if we are confident that the current document has enough support.
There was a problem hiding this comment.
-1
As I already expressed and in line with Thomas’ vote, I vote against accepting in the current form. I agree, however, with the overall goals and hope that a refined version will get accepted. In this regard, the called vote served a purpose in attracting attention.
I would like to suggest a third path.
Could this be an option? |
|
I agree with what @cmarmo is proposing but I see no difference nor extra proposals with the 2. item @thomasjpfan has mentioned i.e. the current SLEP's scope with an explicit (long) exact wording; is this the case, @cmarmo? We need to have actionable while not loosing everyone's time and energy: to me 1. is better for long term while 2. is better for what we aimed at initially but complex to be agreed upon in the current state. Proposal for action: Assuming people who want to participate in those decisions are reactive and proactive enough in the phrasing of terms1, we can get 1. done so as to ease the implementation of 2. using independent, atomic and small modifications for each item which would be discussed and agreed upon in individual threads' (e.g. PRs). What do you think? Please do 👍-react to this message if you want to proceed with this proposal for action. Please do suggest an alternative pathway if you have one. Footnotes
|
Nope. I'm removing all the definitions of new roles from the SLEP. Accepting the definition of new roles outside SLEP proposals makes unnecessary defining them here. Enlarging the scope of the Technical Committee, which becomes a Steering Committee, opens the decision process to new roles already. |
|
I'm happy with @cmarmo suggestion as well. |
|
Dear all, |
|
The vote period has ended about 2 hours ago. More than ⅔ of the votes are negative votes so the SLEP is not accepted, at least in its current status. |
|
@cmarmo the 3rd option sounds good. IIUC, it will make the text smaller and more precise. But again: The induced discussion alone was worth this vote. |
|
What do people think of merging this PR as a "Draft" and iterating to rephrase it in another PR? |
We usually merge before the vote anyway ;) |
|
I do not merge PRs I authored, hence I am looking forward to someone that would merge this first draft. 🙏 |
|
You need to change the status to |
|
@jjerphan could you please add it also to |
Co-authored-by: Adrin Jalali <adrin.jalali@gmail.com>
Description
As discussed on the mailing list, this SLEP proposes updating the Governance to broaden the notion of contribution in scikit-learn.
The HTML output of the SLEP is rendered here: https://scikit-learn-enhancement-proposals--74.org.readthedocs.build/en/74/slep019/proposal.html
Associated implementation PRs: scikit-learn/scikit-learn#24444
🗳️ The vote on this SLEP ended: the SLEP is not accepted as is.
The vote for SLEP019 has been cast on the internal mailing list on Friday, 14 October 2022 around 16:00:00 UTC+0
As specified in SLEP000: