Thanks @EAkinloose, that's really helpful. It looks like the Citoid service might be returning 404 on beta cluster. I'm seeing a 404 error from the REST API:

This looks like an issue with Gboard in English (and many other languages, but not all).

https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Citoid/+/1084841

Thanks. I think various stakeholders are highlighting that we need to determine the appropriate scope for the CSS border-radius. I'll merge the one-line revert for the time being, as it fixes a recent regression, but I don't mean to imply this is the correct permanent answer.

Citoid service patch https://gerrit.wikimedia.org/r/c/mediawiki/services/zotero/+/1054354

Here is a video of the same issue affecting a VisualEditor session:

The heart of the new technical functionality required is demoed in the diffAsChange method in https://gerrit.wikimedia.org/r/c/mediawiki/extensions/VisualEditor/+/1039728/6/modules/ve-mw/preinit/ve.init.mw.DiffLoader.js . This still needs packaging somewhere better and testing extensively.

@zoe and I managed to write code where you can merge a remote revision into your open edit session 🎉

@Mvolz 's merged change fixes this:

This happens when the transaction swaps out the DOM node (from <td> to <th>), but only changes an attribute on the DM tableCell node (from style=data to style=header).

This could be implemented by:

• importing the remote revision and displaying it as a ve.dm.VisualDiff;
• then, if the user chooses to proceed, converting the remote revision into a single ve.dm.Change that applies to the common base document (=the document at the start of the edit session);
• then rebasing that change against the current document's completeHistory (=the transactions that correspond to the changes during the edit session), before applying it;
• then applying the rebased change.

We have decided the Editing Team will not use sentencex for the time being as we are not ready to commit to a sentence boundary API. We can revisit this in future as circumstances change.

Issue in the sentencex-js repo: https://github.com/wikimedia/sentencex-js/issues/1

I've taken a look at sentencex-js specifically, which I understand has been ported from the sentencex original written in python. It looks good! There is certainly significant overlap with the Editing Team's UnicodeJS sentencebreak code.

Examining the code's behaviour on the citation placement examples above, we can see that positions ⓿❶❷❸ would be sufficient for all those examples, except the Japanese close bracket, for which the citation placement comes before the Unicode TR29 definition of a sentence boundary would begin.

Patchdemo instance of the above patch set (thanks @DLynch) https://patchdemo.wmflabs.org/wikis/4b31a139ad/w/index.php?title=Douglas_Adams&veaction=edit

Proof-of-concept patch set to surface sentence segmentation in VisualEditor: https://gerrit.wikimedia.org/r/c/VisualEditor/VisualEditor/+/961095

The proof of concept patch set above is probably no good for CI and messy even for local development. It may be better just to fix the upstream bug.

Upstream issue: https://github.com/karma-runner/karma-firefox-launcher/issues/183 . They suggest a workaround of using a custom launcher FirefoxCustom that passes profile: <some-temp-dir>. This works but you have to create the tempdir yourself. Contrary to the discussion there, just setting TMPDIR=/var/tmp in the environment does not seem to work.

The above patch set fixes the case described above. More generally, is it correct that the paste code should pass arbitrary unbalanced content to the TransactionProcessor, and rely on an error being thrown to switch to "safer" processing? Or is that something we wish to change?

After investigating, I think the "Remove does not match insert" errors are caused by actual invalid transactions: see T334677 .

The patch set above fixes the "Unexpected prior attribute value" errors.

This update better detects inserted text, by disregarding HTML tree structure (which isn't very relevant for these purposes).

I think these are good suggestions. I'd also suggest adding an Indic script.

Oh, probably I'm not being clear. I was thinking:

• we want to tag edits that should contain a reference (according to our heuristic)
• regardless whether or not they do contain one
• and regardless whether the reference check UI is deployed

On reflection, I suggest we consider removing requirement 1B ("the new paragraph does not include a reference"), because:

We're currently envisaging two uses for sentence segmentation in Edit Check:

Based on investigations, I believe Change 881705 fixes the likeliest occurrences of this Gboard bug. I don't think it breaks any case that works at the moment, even though it allows the user to experience T330284 (which needs a separate fix).

Technical notes copied from T325129:

Yes, here is a viable fixup for the above case:

1. On an input event of inputType 'deleteContentBackward', save the current cursor position (but only if the selection is collapsed).
2. If a subsequent keyup event occurs during the same tick, and lands inside a contentEditable=false node, then restore the saved cursor position.
3. In any case, clear the saved cursor position at the end of the tick (i.e. using setTimeout).

The fixup has to occur in the keyup event: waiting for the more logical selectionchange event means the soft keyboard has already closed and it's too late.

This appears to happen because English Gboard + Android Chrome lets the cursor land inside a contenteditable=false span. It does not happen with certain other Gboard languages (e.g. Cantonese). Below is a minimal test case.

Change 812093 Patch set 5 handles the <div><br></div> but fails in other cases (e.g. when the native action adds a list item).

Ok, change 812093 triggers the VE Enter handler — which is a big improvement! As it stands, it still leaves a spurious <div><br></div>, into which we can drag the cursor and type text which then doesn't get put into the model (see screenshot). We should check for this spurious div and clean it up — and as a bonus, we can condition triggering the VE Enter handler on finding this div, so that if another IME behaves as we want it to already, then it won't be affected.

The video below shows the bookmarklet used to run undeployed code against a live mediawiki instance:

On a quick test I see a ~5x speedup, testing on 809052 and 909093 above.