[go: up one dir, main page]

CN113051186B - Method and device for processing page bump in memory recovery and electronic equipment - Google Patents

Method and device for processing page bump in memory recovery and electronic equipment Download PDF

Info

Publication number
CN113051186B
CN113051186B CN202110252780.9A CN202110252780A CN113051186B CN 113051186 B CN113051186 B CN 113051186B CN 202110252780 A CN202110252780 A CN 202110252780A CN 113051186 B CN113051186 B CN 113051186B
Authority
CN
China
Prior art keywords
page
bumpy
memory
application scene
processor
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110252780.9A
Other languages
Chinese (zh)
Other versions
CN113051186A (en
Inventor
黄朝阳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Ziguang Zhanrui Communication Technology Co Ltd
Original Assignee
Beijing Ziguang Zhanrui Communication Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Ziguang Zhanrui Communication Technology Co Ltd filed Critical Beijing Ziguang Zhanrui Communication Technology Co Ltd
Priority to CN202110252780.9A priority Critical patent/CN113051186B/en
Publication of CN113051186A publication Critical patent/CN113051186A/en
Priority to PCT/CN2022/079747 priority patent/WO2022188778A1/en
Application granted granted Critical
Publication of CN113051186B publication Critical patent/CN113051186B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02Addressing or allocation; Relocation
    • G06F12/0223User address space allocation, e.g. contiguous or non contiguous base addressing
    • G06F12/023Free address space management
    • G06F12/0253Garbage collection, i.e. reclamation of unreferenced memory
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/17Details of further file system functions
    • G06F16/172Caching, prefetching or hoarding of files

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Memory System Of A Hierarchy Structure (AREA)

Abstract

The invention provides a method and a device for processing page jolt in memory recovery and electronic equipment. The method comprises the following steps: data acquisition is carried out on a plurality of application scenes, and a bumpy page data set corresponding to each application scene is obtained; generating a page set bumpy in each application scene according to the bumpy page data set corresponding to each application scene; and storing files corresponding to the page set which is bumpy in each application scene into a shared memory file system, wherein the shared memory file system is positioned in a random access memory. The invention can reduce the influence of page jolt on the system performance.

Description

Method and device for processing page bump in memory recovery and electronic equipment
Technical Field
The invention relates to the technical field of computers, in particular to a method and a device for processing page jolt in memory recovery and an electronic device.
Background
At present, the Linux kernel adopts the use principle of 'available and usable, and recovery in busy hours' for the memory. "over" reclamation of memory may cause some frequently used document pages to be scanned twice in succession, so that activity quickly decreases to INACTIVE, and eventually is reclaimed to the Buddy system. Subsequent applications in turn have access to the corresponding locations of the file, so that the act of allocating physical pages or reading in the contents of the file occurs again, a process known as page thrashing. The thrashing of one or two pages is not problematic, but if a large number of document pages are recycled due to memory requirements and then read in, the situation causes the performance of the system to be reduced, which is manifested as the unsmooth and unsmooth system.
Disclosure of Invention
The page jolt processing method and device in memory recovery and the electronic equipment provided by the invention can reduce the influence of the page jolt on the system performance.
In a first aspect, the present invention provides a method for handling page thrashing in memory recycling, where the method includes:
acquiring data of a plurality of application scenes to acquire a bumpy page data set corresponding to each application scene;
generating a page set which is bumpy in each application scene according to the bumpy page data set corresponding to each application scene;
and storing files corresponding to the page set which is bumpy in each application scene into a shared memory file system, wherein the shared memory file system is positioned in a random access memory.
Optionally, the method further comprises:
after the system is started, whether a file is placed in the shared memory file system is confirmed through a cat/proc/meminfo command and a file system searching mode.
Optionally, the acquiring data of the plurality of application scenes, and the acquiring a bump page data set corresponding to each application scene includes:
extracting pagecache adding/deleting records in ftrace information for each application scene, and finding out the adding/deleting record which accords with an equality principle from the adding/deleting records as an element in the bumpy page data set;
the equality principle is as follows: add records and delete records of the same type and of the same page in the add/delete records.
Optionally, the same types include: pid, time, dev, ino, or ofs.
In a second aspect, the present invention provides a device for processing page thrashing in memory recycling, the device comprising:
the device comprises an acquisition unit, a processing unit and a control unit, wherein the acquisition unit is used for acquiring data of a plurality of application scenes and acquiring a bumpy page data set corresponding to each application scene;
the generating unit is used for generating a page set which is bumpy in each application scene according to the bumpy page data set corresponding to each application scene;
and the processing unit is used for storing the files corresponding to the page sets which are bumpy in each application scene into a shared memory file system, wherein the shared memory file system is positioned in a random access memory.
Optionally, the apparatus further comprises:
and the confirming unit is used for confirming whether a file is placed in the shared memory file system or not through a cat/proc/meminfo command and a file system searching mode after the system is started.
Optionally, the obtaining unit is configured to extract a pagecache add/delete record in ftrace information for each application scenario, and find an add/delete record that meets an equality principle from the add/delete record as an element in the bumpy page data set;
the equality principle is as follows: add records and delete records of the same type and of the same page in the add/delete records.
In a third aspect, the present invention provides an electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method for handling page thrashing in memory reclamation.
In a fourth aspect, the present invention provides a chip, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method for handling page thrashing in memory reclamation.
In a fifth aspect, the present invention provides a computer-readable storage medium, where the computer-readable storage medium stores computer instructions, and the computer instructions, when executed by a processor, implement the method for handling page thrashing in memory reclamation.
The method, the device and the electronic equipment for processing the page jolt in the memory recovery provided by the embodiment of the invention are used for carrying out quantitative analysis on the page jolt caused in the memory recovery process of a Linux system, accurately obtaining the file with the page jolt, then putting the file with the page jolt into a shared memory file system, and reducing the influence of the page jolt on the performance by utilizing the characteristic that the shared memory can quickly read in the memory from a block device.
Drawings
FIG. 1 is a flowchart illustrating a method for handling page thrashing in memory recycling according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a processing apparatus for page thrashing in memory recycling according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
An embodiment of the present invention provides a method for processing page thrashing in memory recovery, where the method is applied to an electronic device, and as shown in fig. 1, the method includes:
and S11, collecting data of a plurality of application scenes, and acquiring a bumpy page data set corresponding to each application scene.
And S12, generating a page set with jounce in each application scene according to the jounce page data set corresponding to each application scene.
And S13, storing the file corresponding to the page set which is bumpy in each application scene into a shared memory file system, wherein the shared memory file system is located in a random access memory.
The method for processing page jolts in memory recovery provided by the embodiment of the invention is used for carrying out quantitative analysis on the page jolts caused in the memory recovery process of the Linux system, accurately obtaining the files with the page jolts, then putting the files with the page jolts into the shared memory file system, and reducing the influence of the page jolts on the performance by utilizing the characteristic that the shared memory can quickly read the memory from the block device.
The following describes the method for handling page thrashing in memory recycling according to the present invention in detail with reference to specific embodiments.
The method for processing page thrashing in memory recovery provided by the embodiment comprises the following steps:
s21, collecting data of a plurality of application scenes, and acquiring a bump page data set page _ thrashing [ ] corresponding to each application scene.
Specifically, for each application scenario, extracting pagecache (page cache) add/delete records in Ftrace information, and finding out an add/delete record that conforms to the equality principle from the add/delete records as an element in the bumpy page data set.
The Ftrace is an embedded debugging means using partial functions in a GNU tool chain, and the Ftrace is mature in function framework, supports arbitrary extension and has small load interference on a system. Ftrace has preset trace _ events (points for outputting trace information) at the positions where the file is read into a memory (page tree of the inode) and is recycled by the memory, the trace _ events comprise the numbers of block devices, inodes (index nodes) and offsets of file pages, and the information forms a data object for analyzing memory recycling and page thrashing.
Wherein, the equality principle is as follows: add records and delete records of the same type and of the same page in the add/delete records.
The implementation method specifically comprises the following steps:
and running a plurality of application scenes, wherein each time one application scene is run, the system extracts the pagecache adding records and the pagecache deleting records in the ftrace information, so that a plurality of pagecache adding records and a plurality of pagecache deleting records are obtained, the same page of the same type is searched for from the pagecache adding records and the pagecache deleting records, and the same page is contained in the pagecache adding records and the pagecache deleting records.
The same types described above may include: pid, time, dev, ino, or ofs, but is not limited thereto.
S22, intersecting the elements in the bumpy page data set page _ thrashing [ ] to obtain a page set list _ thrashing _ common in which all the applications are bumpy; and (3) solving a set of each element in the bumpy page data set page _ thrashing [ ] to obtain all page sets list _ thrashing _ all which are bumpy.
Specifically, intersection is calculated for each element in the bumpy page data set, and pages which are bumpy in all application scenes are collected to list _ thrashing _ common. For the element in list _ thrashing _ common, namely, the page which is bumpy in all application scenes;
and (3) solving a collection of each element in the bumpy page data set, and collecting pages which are bumpy once or more in all application scenes into a list _ thrashing _ all. For the element in the list _ thrashing _ all, namely the page in which the thrashing occurs in at least one application scene.
And S23, placing files corresponding to the list _ thrashing _ common page set in the plurality of application scenes into a shared memory file system.
Depending on the characteristic that the file system performs exchange after compression on a swap area based on a DDR SDRAM (double data rate synchronous dynamic random access memory), even if the file system bumps, the file system can return to the memory from the swap area at a higher speed, and the effect of reducing the bumping influence is achieved.
The shmem file is a file form based on memory storage provided by a Linux operating system, and compared with other file forms stored in a nonvolatile memory, the shmem file can provide a faster file reading speed and a smaller page thrashing penalty. Compared with the conventional MLock file page mode, the file corresponding to the page set list _ thrashing _ common in the multiple application scenes is placed into the shmem file system, so that more available memory can be released for the system to use, and the penalty of page thrashing is reduced to the minimum.
Further, after the system is started, whether a file is placed in the shmem file system can be confirmed through a cat/proc/meminfo command and a file system searching mode, so that whether the file is in the page thrashing processing mode under the operated system is ensured.
By performing verification on a plurality of platforms, the method provided by the embodiment is adopted for cold start of an application program, including facebook, messenger, templerun, camera and the like, so that time optimization is about 10%.
The method for processing page jolts in memory recovery provided by the embodiment of the invention is used for carrying out quantitative analysis on the page jolts caused in the memory recovery process of a Linux system, accurately obtaining the files with the page jolts, then putting the files with the page jolts into a shared memory file system, and reducing the influence of the page jolts on the performance by utilizing the characteristic that a shared memory can quickly read in the memory from a block device, and avoiding the reduction of available memories in non-jolt scenes caused by adopting a mode of locking jolt pages to a certain extent.
An embodiment of the present invention further provides a device for processing page thrashing in memory recycling, where the device is located in an electronic device, as shown in fig. 2, and the device includes:
the acquiring unit 11 is configured to perform data acquisition on a plurality of application scenes, and acquire a bumpy page data set corresponding to each application scene;
a generating unit 12, configured to generate a page set in which a bump occurs in each application scene according to the bump page data set corresponding to each application scene;
and a processing unit 13, configured to store a file corresponding to the page set that is bumpy in each application scenario into a shared memory file system, where the shared memory file system is located in a random access memory.
The page jolt processing device in memory recovery provided by the embodiment of the invention carries out quantitative analysis on the page jolt caused in the memory recovery process of the Linux system, can accurately obtain the file with the page jolt, then puts the file with the page jolt into the shared memory file system, and reduces the influence of the page jolt on the performance by utilizing the characteristic that the shared memory can quickly read in the memory from the block device.
Optionally, the apparatus further comprises:
and the confirming unit is used for confirming whether a file is placed in the shared memory file system or not through a cat/proc/meminfo command and a file system searching mode after the system is started.
Optionally, the obtaining unit 11 is configured to extract a pagecache add/delete record in ftrace information for each application scenario, and find an add/delete record that meets an equality principle from the add/delete record as an element in the bumpy page data set;
the equality principle is as follows: add records and delete records of the same type and of the same page in the add/delete records.
The apparatus of this embodiment may be configured to implement the technical solutions of the above method embodiments, and the implementation principles and technical effects are similar, which are not described herein again.
An embodiment of the present invention further provides an electronic device, where the electronic device includes:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method for handling page thrashing in memory reclamation.
An embodiment of the present invention further provides a chip, where the chip includes:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method for handling page thrashing in memory reclamation.
The embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium stores computer instructions, and the computer instructions, when executed by a processor, implement the method for processing page thrashing in memory reclamation.
It will be understood by those skilled in the art that all or part of the processes of the embodiments of the methods described above may be implemented by a computer program, which may be stored in a computer-readable storage medium, and when executed, may include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
The above description is only for the specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims (10)

1. A method for handling page thrashing in memory recovery, comprising:
acquiring data of a plurality of application scenes to acquire a bumpy page data set corresponding to each application scene;
generating a page set which is bumpy in each application scene according to the bumpy page data set corresponding to each application scene;
and storing files corresponding to the page set which is bumpy in each application scene into a shared memory file system, wherein the shared memory file system is positioned in a random access memory.
2. The method of claim 1, further comprising:
after the system is started, whether a file is placed in the shared memory file system is confirmed through a cat/proc/meminfo command and a file system searching mode.
3. The method according to claim 1 or 2, wherein the acquiring data of a plurality of application scenes, and the acquiring a bumpy page data set corresponding to each application scene comprises:
extracting pagecache adding/deleting records in ftrace information for each application scene, and finding the adding/deleting record which accords with an equality principle from the adding/deleting records to serve as an element in the bumpy page data set;
the equality principle is as follows: add record and delete record of the same type and of the same page in add/delete record.
4. The method of claim 3, wherein the same type comprises: pid, time, dev, ino, or ofs.
5. A device for handling page thrashing in memory recycling, the device comprising:
the device comprises an acquisition unit, a processing unit and a control unit, wherein the acquisition unit is used for acquiring data of a plurality of application scenes and acquiring a bumpy page data set corresponding to each application scene;
the generating unit is used for generating a page set which is bumpy in each application scene according to the bumpy page data set corresponding to each application scene;
and the processing unit is used for storing the files corresponding to the page sets which are bumpy in each application scene into a shared memory file system, wherein the shared memory file system is positioned in a random access memory.
6. The apparatus of claim 5, further comprising:
and the confirming unit is used for confirming whether a file is placed in the shared memory file system or not through a cat/proc/meminfo command and a file system searching mode after the system is started.
7. The apparatus according to claim 5 or 6, wherein the obtaining unit is configured to, for each application scenario, extract pagecache add/delete records in ftrace information, and find out an add/delete record that meets an equality principle from the add/delete records as an element in the bumpy page data set;
the equality principle is as follows: add records and delete records of the same type and of the same page in the add/delete records.
8. An electronic device, characterized in that the electronic device comprises:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 4.
9. A chip, wherein the chip comprises:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 4.
10. A computer readable storage medium, wherein the computer readable storage medium stores computer instructions which, when executed by a processor, implement the method of any one of claims 1 to 4.
CN202110252780.9A 2021-03-08 2021-03-08 Method and device for processing page bump in memory recovery and electronic equipment Active CN113051186B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202110252780.9A CN113051186B (en) 2021-03-08 2021-03-08 Method and device for processing page bump in memory recovery and electronic equipment
PCT/CN2022/079747 WO2022188778A1 (en) 2021-03-08 2022-03-08 Method and apparatus for processing page thrashing in memory recovery, and electronic device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110252780.9A CN113051186B (en) 2021-03-08 2021-03-08 Method and device for processing page bump in memory recovery and electronic equipment

Publications (2)

Publication Number Publication Date
CN113051186A CN113051186A (en) 2021-06-29
CN113051186B true CN113051186B (en) 2022-06-24

Family

ID=76510740

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110252780.9A Active CN113051186B (en) 2021-03-08 2021-03-08 Method and device for processing page bump in memory recovery and electronic equipment

Country Status (2)

Country Link
CN (1) CN113051186B (en)
WO (1) WO2022188778A1 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113051186B (en) * 2021-03-08 2022-06-24 北京紫光展锐通信技术有限公司 Method and device for processing page bump in memory recovery and electronic equipment

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1149736A (en) * 1995-06-05 1997-05-14 北京航空航天大学 Distributing storage sharing management method
CN102004675A (en) * 2010-11-11 2011-04-06 福建星网锐捷网络有限公司 Cross-process data transmission method, device and network equipment
CN111666226A (en) * 2020-06-16 2020-09-15 北京紫光展锐通信技术有限公司 Page bump protection method for operating system memory recovery and user equipment
CN111666227A (en) * 2020-06-16 2020-09-15 北京紫光展锐通信技术有限公司 Page bump protection method and device for memory recovery of operating system

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5630097A (en) * 1991-06-17 1997-05-13 Digital Equipment Corporation Enhanced cache operation with remapping of pages for optimizing data relocation from addresses causing cache misses
CN102110073B (en) * 2011-02-01 2013-01-30 中国科学院计算技术研究所 A replacement device and method for on-chip shared cache and corresponding processor
CN106371765B (en) * 2016-08-29 2020-09-18 成都科鸿达科技有限公司 Method for removing memory jitter by LTL model detection of efficient large-scale system
CN113051186B (en) * 2021-03-08 2022-06-24 北京紫光展锐通信技术有限公司 Method and device for processing page bump in memory recovery and electronic equipment

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1149736A (en) * 1995-06-05 1997-05-14 北京航空航天大学 Distributing storage sharing management method
CN102004675A (en) * 2010-11-11 2011-04-06 福建星网锐捷网络有限公司 Cross-process data transmission method, device and network equipment
CN111666226A (en) * 2020-06-16 2020-09-15 北京紫光展锐通信技术有限公司 Page bump protection method for operating system memory recovery and user equipment
CN111666227A (en) * 2020-06-16 2020-09-15 北京紫光展锐通信技术有限公司 Page bump protection method and device for memory recovery of operating system

Also Published As

Publication number Publication date
CN113051186A (en) 2021-06-29
WO2022188778A1 (en) 2022-09-15

Similar Documents

Publication Publication Date Title
CN103870492A (en) Data storing method and device based on key sorting
CN103970788A (en) Webpage-crawling-based crawler technology
US20140207820A1 (en) Method for parallel mining of temporal relations in large event file
CN102541691B (en) Log check point recovery method applied to memory data base OLTP (online transaction processing)
CN103744875B (en) Data quick migration method and system based on file system
US11204935B2 (en) Similarity analyses in analytics workflows
CN111797104A (en) Method and device for acquiring data change condition and electronic equipment
CN110019063B (en) Method for computing node data disaster recovery playback, terminal device and storage medium
CN112965939A (en) File merging method, device and equipment
CN111221634A (en) Method, device and equipment for processing merging request and storage medium
CN105787012A (en) Method for improving small file processing capability of storage system and storage system
CN113051186B (en) Method and device for processing page bump in memory recovery and electronic equipment
CN113687783B (en) Object aggregation method, system, device and computer storage medium
CN119166048A (en) Cross-file data access method, system and embedded device based on data aggregation
CN112698866A (en) Code line life cycle tracing method based on Git and electronic device
CN108090128B (en) Recovery method and device for merged storage space and electronic equipment
CN110019295B (en) Database retrieval method, device, system and storage medium
CN115421856A (en) Data recovery method and device
CN109460500A (en) Focus incident finds method, apparatus, computer equipment and storage medium
CN106776654B (en) Data searching method and device
CN113536047A (en) Graph database data deleting method, system, electronic equipment and storage medium
CN111367992B (en) Data processing method and device, computer storage medium and electronic equipment
CN112486734A (en) NTFS deleted file recovery method and device and electronic equipment
CN108958968B (en) File processing method and device
CN113535714A (en) Data storage method, reading method and computer equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant