I released mwapi 0.6.1 by backporting just this fix. Thanks for reporting!

For some reason multipart::Part::stream wasn't working, when I switch it to:

I spent a few minutes looking at this today, I started by getting:

In T374683#10297761, @daniel wrote:

The responses were also not returning an etag header, as previously expected.

That shouldn't have been the case... you are seeing the etags now, right?

A limit of 5 edits per minute is incredibly slow (considering RESTBase let us run at 200 pages per second).

In T374683#10295353, @daniel wrote:

Bill correctly identified the root cause: the core endpoitn expects redirect=no (for consistency with index.php), while RESTbase expected redirect=false. We can just accept both, see https://gerrit.wikimedia.org/r/1087568.

In T354037#10295304, @daniel wrote:

When fetching HTML content for editing, use stash=true. That should return an ETag that can then be used with if-modified. I just tested it, it seems to work. Conversion of HTML that was fetched without stashing is unreliable, and always was. RESTbase was just more agressive about practive caching.

In T374683#10294780, @Legoktm wrote:

I'll give the revert ~20 more minutes to roll out fully and report back to see if things are fixed.

In T374683#10294694, @HCoplin-WMF wrote:

@Legoktm -- this was 100% an unintended breakage that we are working to resolve as we speak.

Most of my bots and tools that used RESTBase are now broken, and I suspect this is the cause based on timing. I expect there are probably 20-25 tools that are broken, and a similar number of bots across a number of wikis.

Upload does not go through, and Some("TODO".to_string()) is returned instead

Should be fixed now, I bumped up the memory: https://wikitech.wikimedia.org/w/index.php?title=Tool%3ARustup&diff=2240867&oldid=2236350

Sorry Magnus! I thought I had fixed this but guess not.

If you use the job framework (see https://wikitech.wikimedia.org/wiki/Help:Toolforge/Jobs_framework), there's a mariadb image which contains mysqldump.

In T378886#10286594, @Pppery wrote:

To be clear, I'm not blaming either of you - just highlighting a process failure that should be addressed.

The new ccorg interwiki has now been added, so I've submitted two patches:

• https://gerrit.wikimedia.org/r/1082873 <-- updates license text right away
• https://gerrit.wikimedia.org/r/1082874 <-- adds a new message; once deployed we can migrate https://commons.wikimedia.org/wiki/Template:Cc-pd-mark-footer (10M transclusions)

In T347982#9825643, @Reedy wrote:

The general lack of output on the command isn't helpful either:

our open source virtual event platform for Wikimania (eventyay.com)

In T377497#10255164, @Pppery wrote:

I'm not 100% sure what the purpose of allowlisting is here.

I think the point is to allow people to upload only from sites known to have free licenses to prevent upload-by-url from being misused as a vector for copyvios.

The Phabricator part of this was deployed, so we should be all set here!

with linter_cat = 25

Thanks, just deployed: https://gitlab.wikimedia.org/toolforge-repos/ldap/-/commit/7b5baa2b2f25f2ba2e88070bff8319fb20a062f4

I believe I filed this ticket because certain developers were strongly pushing the use of query builders in MediaWiki, so it seemed like a good general pattern for tools too. Maybe the tool author can check in with one of those MediaWiki developers for help? :)

In T376769#10223261, @matmarex wrote:

Hmm, they’re not even 404s, but 500s for me. Both the space and the ‘ń’ seem to cause that.

Deployed https://gitlab.wikimedia.org/toolforge-repos/ldap/-/commit/99594edfa46508acc8d3f286472e6f2dbd8c08e9

Ack, thanks for the explanation. I'll create a new /cn/ route then and will send a MR to update Phabricator's links too.

This is a silly question - is it possible to register "BryanDavis" as a uid? (probably not, because it has to be lower case?)

https://gitlab.wikimedia.org/toolforge-repos/ldap/-/commit/336f2308cc0d6bd3c2131d3b32653643d6f77c53

In T376726#10211294, @Dreamy_Jazz wrote:

@Legoktm do you still have a use case for this dump? If so, can this be resolved through doing T218592 or just using the API directly?

Hmm, I can't find where in LDAP that is...do you have an example account that's disabled?

Deployed. Also I gave you access to the tool but also happy to review patches in the future.

In T343131#10060220, @mdaniels5757 wrote:

I assume that changing that target URL is not feasible for compatibility reasons. Would it be desirable to add, like, a :ccorg: interwiki that targets //creativecommons.org/$1?

In T297995#9695199, @Sj wrote:

Usage seems surprisingly steady at 2.5-3 q/s, is there any information about how it's currently being used?

+1 to the general idea of making LinkBatch functionality available, I think there are a number of places where module authors can batch to optimize templates, just like we do in MediaWiki.

I believe @jeremyb is going to take a shot at implementing this.

@F1rst_LastNam3 awesome, let us know if you have any questions! (We also chat in Matrix/IRC if that's more convenient)

Awesome! I just switched over my bot and in my limited testing it works the same. I'll look into updating documentation tomorrow.

In T334238#10112457, @MSantos wrote:

I agree with you, it hasn't been properly communicated yet and that's why a formal communication with guidelines for migration hasn't been published, that's my fault and I apologize for the lack of movement here.

What's the replacement API module for this? Is there documentation?

There's now T373716: Reroute RESTbase Parsoid endpoints to core's REST endpoints, which is another attempt at solving this problem and hints that the progress of having callers move isn't meeting the necessary timeframes. I'll echo what @Chlod said and that IMO the biggest issue has been a lack of communication to clients/library authors on what we should be doing, as well as a lack of replies to questions, e.g. T354037.

In T367625#9965960, @daniel wrote:

Right... But that's not what the code actually does. I I'll try to come up with a better explanation to put into the docs.

In T370203#10095575, @BTullis wrote:

In T370203#10092682, @Legoktm wrote:

...I'm genuinely shocked that it's doing so for the purpose of better tracking of users.

I would just like to add a small note here, for the purposes of clarity in how Matomo is used at the Foundation.
This plugin does not change the user tracking behaviour of Matomo at all. It just enables creating reports based on filtering our existing data by different parameters.

Yeah, that's reasonable, I think those 3 platforms will cover like 99% of issues. I think if we could run the more obscure platforms on a tag push instead of every commit, that would be ideal. We could also drop MSRV testing for said obscure platforms and not really lose anything.

In T370203#10074258, @odimitrijevic wrote:

Hi @elukey , @joanna_borun, the reason the plugin was chosen is because there are no other equivalent open source options. The plugin underwent an internal security review and signoff as part of the provisioning process which should provide sufficient assurances wrt to the security concerns above.

There was just one attempt? I think this is still a relatively straightforward task.

@WDoranWMF the linked "initial decision brief covering licensing issues" doc is private - can it be made public?

This was reverted as part of T371376: Linter related error on PCS tests: Cannot use object of type stdClass as array, the Linter extension was expecting it to be an array so need to rethink how to implement this.

In T370624#10027452, @Ladsgroup wrote:

Yeah, grep for partitioned_jobs_config in helm deployment charts.

Sorry, my bad. I've proposed a revert and I can redo it properly when I have a bit more time.

Should this be done in the job queue? Or is there something we can do inside RecordLintJob? Is there an example of other jobs that are sharded by section?

I'm not sure what the best solution is here. The number is wrong and is probably going to be wrong going forwards. We can remove/hide it along the lines of T209899#6656593, but removing it is also an API breaking change and this seems a little trivial to trigger that.

It is useful to see which entries on this list are automated by bots and user scripts, and which entries are done manually.

At least for me, trying to kick off 64 processes all at once just causes my development VM to lock up. Is it possible to automatically determine and scale the value based on the number of CPUs/threads available?

Related to / duplicate of T330725: Merge Linter extension into MediaWiki core?

The problem I ran into with a LintError class is that you'd expect it to look something like (abbreviated):

php

I started writing this (congrats on the successful nerd snipe), but I think we might get just as much value by just changing the signature of recordLint to take in the various elements and does the construction of the lint error internally. I'll upload a patch for consideration shortly.

@Jdforrester-WMF flagged that this is very widely used in production (c.f. https://codesearch.wmcloud.org/search/?q=tideways&files=&excludeFiles=&repos=).

tidways_xhprof was removed in Debian today so I noticed this as well and submitted a patch. I don't know how actively people are using tideways_xhprof, and whether it's premature to remove support if it still works on e.g. Debian stable.

(This isn't under the control of the Docker-Hub-MediaWiki maintainers, sorry!)

Unless it's too slow performance wise, I do think using Shellbox is the easiest path to production in that it's already compliant with the SRE requirements and integrated into MediaWiki. And I think it would be relatively straightforward to switch to a proper service later on if needed.

https://gitlab.wikimedia.org/repos/mwbot-rs/delinter/-/commit/17305d6a6a584c26d3d114c257a8613789750b7f is the workaround I've applied for now.

However, the Chart extension's use case would involve shelling out to a Node.js script, which would need to install dependencies from NPM (we could keep the set of dependencies pretty limited, we probably just need echarts and commander). We're not aware of any existing Node.js-based use cases for Shellbox, so we thought we would raise this issue early.

I just want to flag that this is an issue on talk pages, and maybe should be in Parsoid-Read-Views Phase 1 for DiscussionTools because it causes links to break.

Given multiple configuration sources, ImageMagick always uses the lowest specified limit. So setting a higher limit by adding a configuration file, setting an environment variable, or calling the relevant API has no effect.

Merged both MRs and switched the checkouts in the tool over to GitLab.

Yeah, let's do it, and the source code too.

Feel free to remove the CI jobs, I'll take care of moving the two repos to GitLab soon enough.

Fixed in https://gitlab.wikimedia.org/toolforge-repos/logo-test/-/commit/210df3f7cff6173b1d2e22505a4a044bec8defc3 (sorry I didn't see this task until just now).

So far I've integrated this into https://streaks.toolforge.org/ and wrote a blog post: https://blog.legoktm.com/2024/07/12/making-it-easier-for-toolforge-tools-to-surface-replag.html

Just to clarify, despite being disclosed and announced today, Gadgets is a bundled extension, so the fix was released as part of MediaWiki 1.39.8 / 1.40.4 / 1.41.2 / 1.42.1.

In T363314#9945859, @Legoktm wrote:

We could also download the rasterized PNG version via the thumbnails 🤔

We could also download the rasterized PNG version via the thumbnails 🤔

Yeah, https://github.com/mastodon/mastodon/issues/6569 is the upstream ticket about not supporting SVG. We can handle this better in the tool though to display a better error.

I finally got around to doing this after the high s1 replag earlier this week. Unfortunately the problem with iframes is that they don't dynamically size based on the contents (as far as I know - would love to be wrong!). So when there's no lag (the majority of the time), you have to have empty space where the banner would go.

In T367988#9910314, @thiemowmde wrote:

I know this is probably to much to ask for. But personally I'm not a fan of such inverted "say yes when you meant to say no" checkboxes. I find these unnecessarily confusing. It's almost a dark pattern in some situations. Do you think we can do something about this?

In T365298#9886304, @Tgr wrote:

The user will see something like https://sso.wikimedia.org/en.wikipedia.org/wiki/Special:Userlogin which is potentially confusing or suspicious ("wikimedia.org" has limited brand recognition compared to "wikipedia.org", technically sophisticated users will maybe think the presence of en.wikipedia.org (used to identify from which wiki the user started the login process) is a phishing trick).