CN107766158A - A kind of EMS memory management process based on gperftools - Google Patents
A kind of EMS memory management process based on gperftools Download PDFInfo
- Publication number
- CN107766158A CN107766158A CN201711071210.XA CN201711071210A CN107766158A CN 107766158 A CN107766158 A CN 107766158A CN 201711071210 A CN201711071210 A CN 201711071210A CN 107766158 A CN107766158 A CN 107766158A
- Authority
- CN
- China
- Prior art keywords
- memory
- thread
- gperftools
- internal memory
- memory management
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
- G06F9/5016—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Memory System Of A Hierarchy Structure (AREA)
Abstract
本申请发明一种基于gperftools的内存管理方法，通过本发明所述的内存管理方法，可以改进并实现在内存申请时将线程缓存机制与内存的大页机制相结合，即减少了多线程环境中线程申请内存对锁的竞争，同时提升了访存时的TLB命中率，从而提升了访存性能。
The present application invents a memory management method based on gperftools. Through the memory management method described in the present invention, it is possible to improve and realize the combination of the thread cache mechanism and the large page mechanism of the memory when applying for memory, which reduces the number of problems in the multi-thread environment. Threads apply for memory-to-lock competition, and at the same time improve the TLB hit rate when accessing memory, thereby improving memory access performance.
Description
Technical field
The present invention relates to server memory management field, and in particular to a kind of memory management side based on gperftools
Method.
Background technology
Glibc is the libc storehouses of GNU issues, i.e. c Runtime Libraries.Glibc is the api of the bottom in Linux system, Ji Huqi
Its any Runtime Library can all depend on glibc.Glibc is in addition to the system service that encapsulation linux operating systems are provided, and it is originally
Body also provides the realization of some many other necessary function services.Because glibc enumerates what almost all of UNIX passed through
Standard, its content can be infered and embraced a wide spectrum of ideas.And just as other unix systems, its file group included is scattered in system
Directory tree structure in, prop up whole operation system as a support.In GNU/Linux systems, its C functions storehouse hair
Exhibition history has pointed out several important milestones of GNU/Linux evolution, is GNU/Linux by the use of glibc as the C functions storehouse of system
One important milestone of evolution.
In (SuSE) Linux OS, internal memory is applied in the memory management in the glibc storehouses used at present in multi-thread environment
The competition that Shi Huiyou largely locks, the application of internal memory is low with the efficiency comparison discharged, and gperftools provides one kind and is based on
The memory management storehouse of thread cache, memory access performance under multi-thread concurrent environment is lifted, after being improved based on gperftools
Memory management storehouse can further lift the memory access performance under multi-thread concurrent environment with reference to internal memory big page mechanism.gperftools
It is a very useful tool set of google exploitations, mainly includes mallocfree memory allocators tcmalloc；It is based on
Tcmalloc heap memory detection and memory overflow analysis tool heap-profiler, heap-checker；Based on tcmalloc
The program cpu performance monitoring instrument cpu-profiler of realization.Performance point of the above-mentioned three kinds of described instruments in server processes
Analysis monitoring, memory leak positioning, finds performance focus, improves in the various aspects of mallocfree Memory Allocation performances all very
Success.
In (SuSE) Linux OS, the EMS memory management process of gperftools offers is provided, subtracted using thread cache
The competition of lock during few internal memory application, by replacing malloc the and free functions used in current environment, using new
Malloc and free functions.Although can solve internal memory well using thread cache using gperftools under linux system
The race problem of lock during application, but the storehouse does not use big page mechanism when to system application page, compares in amount of ram
The performance of memory access can decline due to TLB a large amount of miss in big environment.
For such case, a kind of EMS memory management process based on gperftools of the present application, pass through institute of the present invention
The EMS memory management process stated, it can improve and realize and mutually tie thread cache mechanism and the big page mechanism of internal memory in internal memory application
Close.
The content of the invention
The main implementation of the present invention is that will be combined based on the memory management of thread cache with the big page mechanism of system,
TLB hit rate when reducing competition of the multi-thread environment thread application internal memory to lock, while improving memory access, so as to carry
Memory access performance is risen.
Specifically, a kind of EMS memory management process based on gperftools is claimed in the application, it is characterised in that the party
Method specifically comprises the following steps：
A number of big page is pre-set in systems；
It is in advance a part of internal memory of each thread cache in operation program；
If thread application internal memory, directly obtains internal memory from the caching of this thread；
When the low memory in thread cache to during system application internal memory using the internal memory big page mechanism pre-set, and
Internal memory is put into thread cache.
EMS memory management process based on gperftools as described above, is further characterized in that, pre-sets in systems
A number of big page is to set a number of big page in systems by/proc/sys/vm/nr_hugepages.
EMS memory management process based on gperftools as described above, is further characterized in that, the EMS memory management process is also
The multithreading running environment that can be expanded under smp environment.
Brief description of the drawings
Fig. 1, EMS memory management process of the present invention schematic flow sheet
Embodiment
The specific implementation of this method is by two multithreading memory allocation managing module, Installed System Memory application module module realities
It is existing.Wherein multithreading memory allocation managing module is used for the Memory Allocation management based on thread cache, Installed System Memory application module
For when the low memory in thread cache to system application internal memory.
The major function of multithreading memory allocation managing module is to be first a part of internal memory of each thread cache, work as thread
Apply directly obtaining internal memory from the caching of this thread during internal memory, avoid the competition of lock, accelerate Memory Allocation efficiency；Work as thread
To system application internal memory and internal memory is put into during low memory in caching in thread cache.
The major function of Installed System Memory application module is, first by/proc/sys/vm/nr_hugepages in system
It is middle that a number of big page is set, internal memory big page mechanism is then used when to system application internal memory, lifts TLB lives during memory access
Middle rate, so as to the memory access performance of lifting system.Specific implementation step of the present invention is carried out below by one embodiment
Explanation.
Idiographic flow is as shown in Figure 1.
A number of big page is pre-set in systems；
It is in advance a part of internal memory of each thread cache in operation program；
If thread application internal memory, directly obtains internal memory from the caching of this thread；
When the low memory in thread cache to during system application internal memory using the internal memory big page mechanism pre-set, and
Internal memory is put into thread cache.
It should be evident that illustrated above is only the specific embodiment of the present invention, for the common skill in this area
For art personnel, on the premise of not paying creative work, other technical schemes can also be obtained according to above-described embodiment,
And the equivalent variations made in the scope of protection of the invention all should be fallen within the scope of protection of the present invention, and belong to the present invention
The scope of protection.
In summary, the multithreading running environment that method of the present invention can be extended under smp environment, uses this
After method can also lifting system memory access performance.
Claims (3)
1. a kind of EMS memory management process based on gperftools, it is characterised in that this method specifically comprises the following steps：
A number of big page is pre-set in systems；
It is in advance a part of internal memory of each thread cache in operation program；
If thread application internal memory, directly obtains internal memory from the caching of this thread；
When the low memory in thread cache to during system application internal memory using the internal memory big page mechanism pre-set, and will in
Deposit in thread cache.
2. the EMS memory management process based on gperftools as claimed in claim 1, is further characterized in that, in systems in advance
It is to set a number of big page in systems by/proc/sys/vm/nr_hugepages to set a number of big page.
3. the EMS memory management process based on gperftools as claimed in claim 2, is further characterized in that, the memory management side
The multithreading running environment that method can be extended under smp environment.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711071210.XA CN107766158A (en) | 2017-11-03 | 2017-11-03 | A kind of EMS memory management process based on gperftools |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711071210.XA CN107766158A (en) | 2017-11-03 | 2017-11-03 | A kind of EMS memory management process based on gperftools |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107766158A true CN107766158A (en) | 2018-03-06 |
Family
ID=61272634
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711071210.XA Pending CN107766158A (en) | 2017-11-03 | 2017-11-03 | A kind of EMS memory management process based on gperftools |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107766158A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108664419A (en) * | 2018-04-03 | 2018-10-16 | 郑州云海信息技术有限公司 | A kind of method and its device of determining memory big page number |
| CN110727517A (en) * | 2019-10-12 | 2020-01-24 | 福建顶点软件股份有限公司 | Memory allocation method and device based on partition design |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105893269A (en) * | 2016-03-31 | 2016-08-24 | 武汉虹信技术服务有限责任公司 | Memory management method used in Linux system |
| US20170163538A1 (en) * | 2015-12-02 | 2017-06-08 | Macau University Of Science And Technology | PacketUsher: Accelerating Computer-Intensive Packet Processing |
| CN106970882A (en) * | 2017-03-10 | 2017-07-21 | 浙江大学 | A kind of easy extension page architecture based on Linux big page internal memories |
-
2017
- 2017-11-03 CN CN201711071210.XA patent/CN107766158A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170163538A1 (en) * | 2015-12-02 | 2017-06-08 | Macau University Of Science And Technology | PacketUsher: Accelerating Computer-Intensive Packet Processing |
| CN105893269A (en) * | 2016-03-31 | 2016-08-24 | 武汉虹信技术服务有限责任公司 | Memory management method used in Linux system |
| CN106970882A (en) * | 2017-03-10 | 2017-07-21 | 浙江大学 | A kind of easy extension page architecture based on Linux big page internal memories |
Non-Patent Citations (1)
| Title |
|---|
| DZQABC: ""大页内存（HugePages）"", 《HTTP://WWW.CNBLOGS.COM/DONGZHIQUAN/P/5043912.HTML》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108664419A (en) * | 2018-04-03 | 2018-10-16 | 郑州云海信息技术有限公司 | A kind of method and its device of determining memory big page number |
| CN110727517A (en) * | 2019-10-12 | 2020-01-24 | 福建顶点软件股份有限公司 | Memory allocation method and device based on partition design |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Xu et al. | vCAT: Dynamic cache management using CAT virtualization | |
| US8949295B2 (en) | Cooperative memory resource management via application-level balloon | |
| US7472249B2 (en) | Kernel memory free algorithm | |
| EP1966703B1 (en) | Method and apparatus for hardware-based dynamic escape detection in managed run-time environments | |
| US10248175B2 (en) | Off-line affinity-aware parallel zeroing of memory in non-uniform memory access (NUMA) servers | |
| US20050172099A1 (en) | Method and apparatus for memory management in a multi-processor computer system | |
| CN106354543B (en) | A kind of NUMA memory converted based on virtual machine and host memory address moves a page method | |
| US10949342B2 (en) | Persistent memory garbage collection | |
| US20040034742A1 (en) | Stack allocation system and method | |
| KR20140131089A (en) | Apparatus and method for allocating resource | |
| US20150178219A1 (en) | Application Startup Page Fault Management in a Hardware Multithreading Environment | |
| US7882505B2 (en) | Method and apparatus for switching between per-thread and per-processor resource pools in multi-threaded programs | |
| CN107766158A (en) | A kind of EMS memory management process based on gperftools | |
| CN112860381B (en) | Method and system for virtual machine memory expansion based on Shenwei processor | |
| US8103849B2 (en) | Reducing memory usage of kernel memory management structures | |
| Macêdo et al. | The mechanics of memory-related software aging | |
| Wang et al. | Memento: architectural support for ephemeral memory management in serverless environments | |
| US9274946B2 (en) | Pre-leak detection scan to identify non-pointer data to be excluded from a leak detection scan | |
| US8375194B2 (en) | System and method for freeing memory | |
| US7783849B2 (en) | Using trusted user space pages as kernel data pages | |
| US8725974B2 (en) | Page-protection based memory access barrier traps | |
| WO2007073926A2 (en) | Memory architecture and access method | |
| US11150929B2 (en) | Enhanced memory management for virtual machines | |
| CN108563507A (en) | A kind of EMS memory management process, device, equipment and readable storage medium storing program for executing | |
| Norlinder et al. | Compressed Forwarding Tables Reconsidered |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180306 |