rebuildtextindex.php fails due to "Error 1406: Data too long for column 'si_title' at row 1"
Closed, DuplicatePublic
Actions

Assigned To

None

Authored By

	Kghbln
	Feb 13 2021, 5:07 PM

Description

Setup

MediaWiki 1.35.1 (3ff1919) 01:03, 7 February 2021
PHP 7.3.19-1~deb10u1 (apache2handler)
MariaDB 10.3.27-MariaDB-0+deb10u1
ICU 63.1

Issue
When tying to rebuild the full search index php maintenance/rebuildtextindex.php fails with:

php maintenance/rebuildtextindex.php
Clearing searchindex table...Done
Rebuilding index fields for 63073 pages...
500
1000
1500
2000
2500
3000
Wikimedia\Rdbms\DBQueryError from line 1699 of /../w/includes/libs/rdbms/database/Database.php: Error 1406: Data too long for column 'si_title' at row 1 (localhost:6363)
Function: SearchMySQL::update
Query: REPLACE INTO `searchindex` (si_page,si_title,si_text) VALUES ('3380','u8d0bcu8d0b0u8d180u8d0b8u8d0bdu8d0b0 u8d186u8d0b2u8d0b5u8d182u8d0b0u8d0b5u8d0b2u8d0b0 - u8d0bcu8d0bdu8d0b5 u8d0bdu8d180u8d0b0u8d0b2u8d0b8u8d182u8d181u8d18f u8d187u8d182u8d0be u8d0b2u8d18b u8d0b1u8d0beu8d0bbu8d18cu8d0bdu8d18b u8d0bdu8d0b5 u8d0bcu8d0bdu8d0beu8d0b9...',' pu800 tlu800 russian t1u800 u8d0bcu8d0bdu8d0b5 u8d0bdu8d180u8d0b0u8d0b2u8d0b8u8d182u8d181u8d18f u8d187u8d182u8d0be u8d0b2u8d18b u8d0b1u8d0beu8d0bbu8d18cu8d0bdu8d18b u8d0bdu8d0b5 u8d0bcu8d0bdu8d0beu8d0b9... t2u800 itu800 pleases meu800 that iu800 amu800 notu800 your hurt... uu800 itpleasesmethatiamnotyourhurt ru800 u8d0b8u8d180u8d0beu8d0bdu8d0b8u8d18f u8d181u8d183u8d0b4u8d18cu8d0b1u8d18b tag1 songs tag2 poems tag3 film songs tag4 tag5 tag6 lu800 4u800 vu800 lvmfy-ryhas mu800 u8d0bcu8d0bdu8d0b5 u8d0bdu8d180u8d0b0u8d0b2u8d0b8u8d182u8d181u8d18f u8d187u8d182u8d0be u8d0b2u8d18b u8d0b1u8d0beu8d0bbu8d18cu8d0bdu8d18b u8d0bdu8d0b5 u8d0bcu8d0bdu8d0beu8d0b9 u8d0bcu8d0bdu8d0b5 u8d0bdu8d180u8d0b0u8d0b2u8d0b8u8d182u8d181u8d18f u8d187u8d182u8d0be u8d18f u8d0b1u8d0beu8d0bbu8d18cu8d0bdu8d0b0 u8d0bdu8d0b5 u8d0b2u8d0b0u8d0bcu8d0b8 u8d187u8d182u8d0be u8d0bdu8d0b8u8d0bau8d0beu8d0b3u8d0b4u8d0b0 u8d182u8d18fu8d0b6u8d0b5u8d0bbu8d18bu8d0b9 u8d188u8d0b0u8d180 u8d0b7u8d0b5u8d0bcu8d0bdu8d0beu8d0b9 u8d0bdu8d0b5 u8d183u8d0bfu8d0bbu8d18bu8d0b2u8d0b5u8d182 u8d0bfu8d0beu8d0b4 u8d0bdu8d0b0u8d188u8d0b8u8d0bcu8d0b8 u8d0bdu8d0beu8d0b3u8d0b0u8d0bcu8d0b8. u8d0bcu8d0bdu8d0b5 u8d0bdu8d180u8d0b0u8d0b2u8d0b8u8d182u8d181u8d18f u8d187u8d182u8d0be u8d0bcu8d0beu8d0b6u8d0bdu8d0be u8d0b1u8d18bu8d182u8d18c u8d181u8d0bcu8d0b5u8d188u8d0bdu8d0beu8d0b9 u8e28093 u8d180u8d0b0u8d181u8d0bfu8d183u8d189u8d0b5u8d0bdu8d0bdu8d0beu8d0b9 - u8d0b8 u8d0bdu8d0b5 u8d0b8u8d0b3u8d180u8d0b0u8d182u8d18c u8d181u8d0bbu8d0beu8d0b2u8d0b0u8d0bcu8d0b8 u8d0b8 u8d0bdu8d0b5 u8d0bau8d180u8d0b0u8d181u8d0bdu8d0b5u8d182u8d18c u8d183u8d0b4u8d183u8d188u8d0bbu8d0b8u8d0b2u8d0beu8d0b9 u8d0b2u8d0beu8d0bbu8d0bdu8d0beu8d0b9 u8d181u8d0bbu8d0b5u8d0b3u8d0bau8d0b0 u8d181u8d0beu8d0bfu8d180u8d0b8u8d0bau8d0beu8d181u8d0bdu8d183u8d0b2u8d188u8d0b8u8d181u8d18c u8d180u8d183u8d0bau8d0b0u8d0b2u8d0b0u8d0bcu8d0b8. u8d0bcu8d0bdu8d0b5 u8d0bdu8d180u8d0b0u8d0b2u8d0b8u8d182u8d181u8d18f u8d0b5u8d189u8d0b5 u8d187u8d182u8d0be u8d0b2u8d18b u8d0bfu8d180u8d0b8 u8d0bcu8d0bdu8d0b5 u8d181u8d0bfu8d0beu8d0bau8d0beu8d0b9u8d0bdu8d0be u8d0beu8d0b1u8d0bdu8d0b8u8d0bcu8d0b0u8d0b5u8d182u8d0b5 u8d0b4u8d180u8d183u8d0b3u8d183u8d18e u8d0bdu8d0b5 u8d0bfu8d180u8d0beu8d187u8d0b8u8d182u8d0b5 u8d0bcu8d0bdu8d0b5 u8d0b2 u8d0b0u8d0b4u8d0beu8d0b2u8d0beu8d0bc u8d0beu8d0b3u8d0bdu8d0b5 u8d0b3u8d0beu8d180u8d0b5u8d182u8d18c u8d0b7u8d0b0 u8d182u8d0be u8d187u8d182u8d0be u8d18f u8d0bdu8d0b5 u8d0b2u8d0b0u8d181 u8d186u8d0b5u8d0bbu8d183u8d18e. u8d187u8d182u8d0be u8d0b8u8d0bcu8d18f u8d0bdu8d0b5u8d0b6u8d0bdu8d0beu8d0b5 u8d0bcu8d0beu8d0b5 u8d0bcu8d0beu8d0b9 u8d0bdu8d0b5u8d0b6u8d0bdu8d18bu8d0b9 u8d0bdu8d0b5 u8d183u8d0bfu8d0beu8d0bcu8d0b8u8d0bdu8d0b0u8d0b5u8d182u8d0b5 u8d0bdu8d0b8 u8d0b4u8d0bdu8d0b5u8d0bc u8d0bdu8d0b8 u8d0bdu8d0beu8d187u8d18cu8d18e - u8d0b2u8d181u8d183u8d0b5u8e280a6 u8d187u8d182u8d0be u8d0bdu8d0b8u8d0bau8d0beu8d0b3u8d0b4u8d0b0 u8d0b2 u8d186u8d0b5u8d180u8d0bau8d0beu8d0b2u8d0bdu8d0beu8d0b9 u8d182u8d0b8u8d188u8d0b8u8d0bdu8d0b5 u8d0bdu8d0b5 u8d0bfu8d180u8d0beu8d0bfu8d0beu8d18eu8d182 u8d0bdu8d0b0u8d0b4 u8d0bdu8d0b0u8d0bcu8d0b8 u8d0b0u8d0bbu8d0bbu8d0b8u8d0bbu8d183u8d0b9u8d18f u8d181u8d0bfu8d0b0u8d181u8d0b8u8d0b1u8d0be u8d0b2u8d0b0u8d0bc u8d0b8 u8d181u8d0b5u8d180u8d0b4u8d186u8d0b5u8d0bc u8d0b8 u8d180u8d183u8d0bau8d0beu8d0b9 u8d0b7u8d0b0 u8d182u8d0be u8d187u8d182u8d0be u8d0b2u8d18b u8d0bcu8d0b5u8d0bdu8d18f - u8d0bdu8d0b5 u8d0b7u8d0bdu8d0b0u8d18f u8d181u8d0b0u8d0bcu8d0b8 u8e28093 u8d182u8d0b0u8d0ba u8d0bbu8d18eu8d0b1u8d0b8u8d182u8d0b5 u8d0b7u8d0b0 u8d0bcu8d0beu8d0b9 u8d0bdu8d0beu8d187u8d0bdu8d0beu8d0b9 u8d0bfu8d0beu8d0bau8d0beu8d0b9 u8d0b7u8d0b0 u8d180u8d0b5u8d0b4u8d0bau8d0beu8d181u8d182u8d18c u8d0b2u8d181u8d182u8d180u8d0b5u8d187 u8d0b7u8d0b0u8d0bau8d0b0u8d182u8d0bdu8d18bu8d0bcu8d0b8 u8d187u8d0b0u8d181u8d0b0u8d0bcu8d0b8 u8d0b7u8d0b0 u8d0bdu8d0b0u8d188u8d0b8 u8d0bdu8d0b5-u8d0b3u8d183u8d0bbu8d18fu8d0bdu8d18cu8d18f u8d0bfu8d0beu8d0b4 u8d0bbu8d183u8d0bdu8d0beu8d0b9 u8d0b7u8d0b0 u8d181u8d0beu8d0bbu8d0bdu8d186u8d0b5 u8d0bdu8d0b5 u8d183 u8d0bdu8d0b0u8d181 u8d0bdu8d0b0u8d0b4 u8d0b3u8d0beu8d0bbu8d0beu8d0b2u8d0b0u8d0bcu8d0b8 u8d0b7u8d0b0 u8d182u8d0be u8d187u8d182u8d0be u8d0b2u8d18b u8d0b1u8d0beu8d0bbu8d18cu8d0bdu8d18b - u8d183u8d0b2u8d18b - u8d0bdu8d0b5 u8d0bcu8d0bdu8d0beu8d0b9 u8d0b7u8d0b0 u8d182u8d0be u8d187u8d182u8d0be u8d18f u8d0b1u8d0beu8d0bbu8d18cu8d0bdu8d0b0 - u8d183u8d0b2u8d18b - u8d0bdu8d0b5 u8d0b2u8d0b0u8d0bcu8d0b8. itu800 pleases meu800 that iu800 amu800 notu800 your hurt itu800 pleases meu800 that youu800 areu800 notu800 myu800 illness that asu800 weu800 step upon theu800 heavy earth itsu800 solidness shall never drift beneath usu800. itu800 pleases meu800 that iu800 donu8e28099t have tou800 flirt that iu800 canu800 beu800 relaxed andu800 wordplay-needless inu800 suffocating blush notu800 feeling stirred byu800 accidental touch oru800 random glimpses. itu800 pleases meu800 asu800 well that when weu800 meet youu800 would embrace another oneu800 serenely that youu800 donu8e28099t fantasize ofu800 hellish heat consuming meu800 foru800 kissing someone keenly that myu800 sweet name will notu800 emerge myu800 sweet from your fair lips - inu800 vain - come dayu800 oru800 evening andu800 inu800 au800 silent temple brightly litu800 weu8e28099ll never wedu800 tou800 hallelujah singing. andu800 with myu800 heart andu800 hand comes thanking word; foru800 youu800 - without theu800 knowledge tou800 admit this u8e28093 sou800 fancy meu800; foru800 sleep sou800 undisturbed foru800 rareness ofu800 ouru800 dates inu800 twilight dimness foru800 walking-notu800 across au800 moonlit court foru800 rising sunu800 weu800 dou800 notu800 ever witness foru800 fact that iu800 - alas - amu800 notu800 your hurt foru800 fact that youu800 - alas - areu800 notu800 myu800 illness. u8d0b5u8d0b2u8d0b3u8d0b5u8d0bdu8d0b8u8d18f u8d181u8d0b0u8d180u8d0bau8d0b8u8d181u8d18cu8d18fu8d0bdu8d186 category u8d0b8u8d180u8d0beu8d0bdu8d0b8u8d18f u8d181u8d183u8d0b4u8d18cu8d0b1u8d18b category u8d186u8d0b2u8d0b5u8d182u8d0b0u8d0b5u8d0b2u8d0b0 u8d0bcu8d0b0u8d180u8d0b8u8d0bdu8d0b0 u8d0b8u8d0b2u8d0b0u8d0bdu8d0beu8d0b2u8d0bdu8d0b0 category u8d0b5u8d0b2u8d0b3u8d0b5u8d0bdu8d0b8u8d18f u8d181u8d0b0u8d180u8d0bau8d0b8u8d181u8d18cu8d18fu8d0bdu8d186 category u8d0bfu8d183u8d0b3u8d0b0u8d187u8d191u8d0b2u8d0b0 u8d0b0u8d0bbu8d0bbu8d0b0 u8d0b1u8d0beu8d180u8d0b8u8d181u8d0beu8d0b2u8d0bdu8d0b0 category u8d0b8u8d180u8d0b8u8d0bdu8d0b0 u8d0bau8d180u8d183u8d182u8d0beu8d0b2u8d0b0 category u8d182u8d0b0u8d180u8d0b8u8d0b2u8d0b5u8d180u8d0b4u8d0b8u8d0b5u8d0b2 u8d0bcu8d0b8u8d0bau8d0b0u8d18du8d0bb u8d0bbu8d0b5u8d0beu8d0bdu8d0beu8d0b2u8d0b8u8d187 ')

Backtrace

#0 /../w/includes/libs/rdbms/database/Database.php(1683): Wikimedia\Rdbms\Database->getQueryException('Data too long f...', 1406, 'REPLACE INTO `s...', 'SearchMySQL::up...')
#1 /../w/includes/libs/rdbms/database/Database.php(1658): Wikimedia\Rdbms\Database->getQueryExceptionAndLog('Data too long f...', 1406, 'REPLACE INTO `s...', 'SearchMySQL::up...')
#2 /../w/includes/libs/rdbms/database/Database.php(1227): Wikimedia\Rdbms\Database->reportQueryError('Data too long f...', 1406, 'REPLACE INTO `s...', 'SearchMySQL::up...', false)
#3 /../w/includes/libs/rdbms/database/DatabaseMysqlBase.php(1367): Wikimedia\Rdbms\Database->query('REPLACE INTO `s...', 'SearchMySQL::up...', 128)
#4 /../w/includes/libs/rdbms/database/Database.php(3240): Wikimedia\Rdbms\DatabaseMysqlBase->doReplace('searchindex', Array, Array, 'SearchMySQL::up...')
#5 /../w/includes/libs/rdbms/database/DBConnRef.php(68): Wikimedia\Rdbms\Database->replace('searchindex', Array, Array, 'SearchMySQL::up...')
#6 /../w/includes/libs/rdbms/database/DBConnRef.php(496): Wikimedia\Rdbms\DBConnRef->__call('replace', Array)
#7 /../w/includes/search/SearchMySQL.php(354): Wikimedia\Rdbms\DBConnRef->replace('searchindex', 'si_page', Array, 'SearchMySQL::up...')
#8 /../w/includes/deferred/SearchUpdate.php(106): SearchMySQL->update('3380', 'u8d0bcu8d0b0u8d...', ' pu800 tlu800 r...')
#9 /../w/maintenance/rebuildtextindex.php(114): SearchUpdate->doUpdate()
#10 /../w/maintenance/rebuildtextindex.php(68): RebuildTextIndex->populateSearchIndex()
#11 /../w/maintenance/doMaintenance.php(107): RebuildTextIndex->execute()
#12 /../w/maintenance/rebuildtextindex.php(162): require_once('/var/www/html/w...')
#13 {main}

Not sure why this is happening. If the title is too long I guess it should not be stored anyways. Hmm ....

Related to T274707? Same wiki.

Related Objects

Mentioned In: T231827: Data too long for column 'si_title'
T274707: updateSearchIndex.php fails due to "Error 1100: Table 'temp_rev_comment' was not locked with LOCK TABLES"
Mentioned Here: T252853: updateSearchIndex.php sql error not all tables locked
T274707: updateSearchIndex.php fails due to "Error 1100: Table 'temp_rev_comment' was not locked with LOCK TABLES"
rMW3ff1919737e3: Upgrade wikimedia/less.php from 3.0.0 to 3.1.0

Event Timeline

Kghbln created this task.Feb 13 2021, 5:07 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 13 2021, 5:07 PM

Kghbln mentioned this in T274707: updateSearchIndex.php fails due to "Error 1100: Table 'temp_rev_comment' was not locked with LOCK TABLES".Feb 13 2021, 5:11 PM

Kghbln updated the task description. (Show Details)

Aklapper edited projects, added MediaWiki-Maintenance-system; removed Discovery-ARCHIVED.Feb 13 2021, 5:41 PM

Reedy renamed this task from "rebuildtextindex.php" fails due to "Error 1406: Data too long for column 'si_title' at row 1" to rebuildtextindex.php fails due to "Error 1406: Data too long for column 'si_title' at row 1".Feb 15 2021, 12:04 AM

Reedy added a project: MediaWiki-Search.

Restricted Application added a project: Discovery-Search. · View Herald TranscriptFeb 15 2021, 12:05 AM

• MPhamWMF triaged this task as Low priority.Feb 17 2021, 3:27 PM

• MPhamWMF moved this task from needs triage to ML & Data Pipeline on the Discovery-Search board.

• MPhamWMF moved this task from ML & Data Pipeline to making others happy on the Discovery-Search board.

Kghbln mentioned this in T231827: Data too long for column 'si_title'.Feb 23 2021, 5:14 PM

@Reedy Is this similar to T252853 which was recently fixed?

Nope

Huu, I was hoping ... Anyhow thanks for your info.

TheVoidwalker subscribed.Dec 24 2021, 9:06 PM

Universal_Omega subscribed.Dec 24 2021, 9:46 PM

Is there any way this error can be fixed manually on a case-by-case basis?

Is there any way to figure out which page is / which pages are causing the issue?

Query: REPLACE INTO `searchindex` (si_page,si_title,si_text) VALUES ('3380','u8d0bcu8d0b0u8d180u8d0b8u8d0bdu8d0b0

Second string in the VALUES brackets would be the title... With some normalisation or something looking at it?

But si_page is the page_id; so you can look that up in the page table...

Cool, thanks a bunch. I will give this a shot by going to the page table. Hopefully, it is just this one page.

I looked for the offending page in the database with SELECT * FROM page WHERE page_id = 3380;and got the following result:

+---------+----------------+----------------------------------------------------------------------------------------------------+-------------------+------------------+-------------+----------------+----------------+-------------+----------+--------------------+--------------------+-----------+
| page_id | page_namespace | page_title                                                                                         | page_restrictions | page_is_redirect | page_is_new | page_random    | page_touched   | page_latest | page_len | page_content_model | page_links_updated | page_lang |
+---------+----------------+----------------------------------------------------------------------------------------------------+-------------------+------------------+-------------+----------------+----------------+-------------+----------+--------------------+--------------------+-----------+
|    3380 |              0 | Марина_Цветаева_-_Мне_нравится,_что_вы_больны_не_мной...                                           |                   |                0 |           0 | 0.653659974253 | 20231107144904 |      411305 |     3015 | wikitext           | 20231107144904     | NULL      |
+---------+----------------+----------------------------------------------------------------------------------------------------+-------------------+------------------+-------------+----------------+----------------+-------------+----------+--------------------+--------------------+-----------+
1 row in set (0.000 sec)

I am unsure why a page like "Марина_Цветаева_-_Мне_нравится,_что_вы_больны_не_мной..." would create pain here.

@Kghbln, What are the results of show create table searchindex?

MariaDB [DB99093932820210211]> show create table searchindex;
+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table       | Create Table                                                                                                                                                                                                                                        |
+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| searchindex | CREATE TABLE `searchindex` (
  `si_page` int(10) unsigned NOT NULL,
  `si_title` varchar(255) NOT NULL DEFAULT '',
  `si_text` mediumtext NOT NULL,
  UNIQUE KEY `si_page` (`si_page`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_general_ci |
+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.000 sec)

MacFan4000 subscribed.Mar 13 2024, 2:16 PM

Paladox closed this task as a duplicate of T231827: Data too long for column 'si_title'.Thu, Oct 31, 12:21 PM

rebuildtextindex.php fails due to "Error 1406: Data too long for column 'si_title' at row 1"Closed, DuplicatePublicActions

Description

Related Objects

Event Timeline

rebuildtextindex.php fails due to "Error 1406: Data too long for column 'si_title' at row 1"
Closed, DuplicatePublic
Actions