CN110825708A - Multi-process file reading and writing method - Google Patents
Multi-process file reading and writing method Download PDFInfo
- Publication number
- CN110825708A CN110825708A CN201910897931.9A CN201910897931A CN110825708A CN 110825708 A CN110825708 A CN 110825708A CN 201910897931 A CN201910897931 A CN 201910897931A CN 110825708 A CN110825708 A CN 110825708A
- Authority
- CN
- China
- Prior art keywords
- file
- original file
- time
- original
- writing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/176—Support for shared access to files; File sharing support
- G06F16/1767—Concurrency control, e.g. optimistic or pessimistic approaches
- G06F16/1774—Locking methods, e.g. locking methods for file systems allowing shared and concurrent access to files
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method for reading and writing multi-process files, which comprises the following steps: copying an original file and establishing a target file; before reading and writing, obtaining the latest modification time of the original file; any process reads and writes the target file; after reading and writing are completed, the latest modification time of the original file is obtained; comparing the time obtained twice; and if the time is consistent, replacing the original file with the target file to finish the original file updating of the current process. The invention effectively avoids the condition that the server is down due to exclusive resource by operating the copied file to read and write the file; each process can carry out the same operation, adverse effects cannot be caused among the processes, the processes do not need to be queued one by one, and the file processing efficiency is greatly improved; by comparing the latest modification time of the original file obtained before and after reading and writing, whether the original file is being updated or not can be quickly known, and the update loss of other processes caused by direct update of the original file is avoided.
Description
Technical Field
The invention relates to the technical field of file reading and writing, in particular to a multi-process file reading and writing method.
Background
With the popularization and development of internet technology, a server often performs massive data request processing at the same time, for example: and one commodity is killed in seconds, the server can instantly receive a plurality of requests in the same time, and the database file for storing the stock can be simultaneously read and written by a plurality of processes. When the data request amount is small, the purpose of reading and writing files can be easily achieved by using a common file operation method. However, if the request amount is large at the same time, a plurality of processes can read and write the file at the same time. If the file is directly read and written and updated under the condition that exclusive protection is not carried out on the file, data modified by other processes are easily lost. When a process reads, writes and updates a file, it needs to protect the file by a method, and it also needs to ensure that other processes are not affected to read and write the file.
At present, a commonly adopted method is that when one process performs read-write operation on a certain file, a file lock is added to the file, the file is locked exclusively, and other processes participating in the read-write of the file are queued and blocked artificially. Only the current process has the highest authority for the file, the file can be read and written, other processes only have the read authority, and once the other processes try to write and update, the file is refused. And after the process for monopolizing the file finishes reading, writing and updating, unlocking immediately to release resources, so that the file can be restored to a changeable state, and other processes can monopolize the file one by one according to the queue to finish corresponding reading, writing and updating of the file.
In the prior art, a file lock is used to lock a file exclusively when a multi-process performs file reading and writing, but when a file lock is used in a PHP (first-in first-out queuing), the file lock is usually not unlocked in time to release the exclusive or the exclusive, which results in deadlock of resources, so that a server can maintain a very high cpu occupancy rate for a long time, and thus the server is usually crashed. Meanwhile, when a file lock is added to a file in the prior art, other processes generate queues, and the authority for writing the original file is not available, and the next process can recover the highest authority of the file only after the previous process finishes updating the file; the treatment efficiency is low.
Disclosure of Invention
Aiming at the defects in the problems, the invention provides a multi-process file reading and writing method, which is used for solving the problem of deadlock caused by single process monopolizing resources and solving the problems of time phase conflict and waiting requirement of multi-process file reading and writing.
The invention discloses a method for reading and writing multi-process files, which comprises the following steps:
copying an original file and establishing a target file;
before reading and writing, obtaining the latest modification time of the original file;
any process reads and writes the target file;
after reading and writing are completed, the latest modification time of the original file is obtained;
comparing the time obtained twice;
and if the time is consistent, replacing the original file with the target file to finish the original file updating of the current process.
As a further improvement of the present invention, the original file is copied to establish a target file; the method comprises the following steps:
creating a temporary folder and generating a random number with uniqueness;
and defining a target file path by using the temporary folder name, the random number and the original file name, and copying the original file into the target file path.
As a further improvement of the present invention, before defining the target file path, the file cache needs to be cleared.
As a further improvement of the invention, in the process of copying the original file, if the target file exists, the target file is directly overwritten.
As a further improvement of the present invention, before reading and writing, the latest modification time of the original file is obtained; the method comprises the following steps:
acquiring the latest modification time of an original file and storing the latest modification time in a variable;
defining an array, and sequentially storing the target file name, the original file name, the generated random number and the time variable in the array according to the sequence.
As a further improvement of the present invention, the arbitrary process reads and writes the target file; the method comprises the following steps:
and opening the target file, writing the content to be modified into the target file, and storing and updating.
As a further improvement of the present invention, after completing the reading and writing, the latest modification time of the original file is obtained; the method comprises the following steps:
clearing the file cache;
and re-acquiring the latest modification time of the original file according to the original file name stored in the array.
As a further improvement of the present invention, the comparison is performed for two times; the method comprises the following steps:
and comparing the newly acquired latest modification time with the time variable stored in the array.
As a further improvement of the present invention, if the time is consistent, the target file is replaced with the original file, and the original file update of the current process is completed; the method comprises the following steps:
if the time is consistent;
then, the target file name is renamed to the original file name and is moved to the original file directory to cover the original file, the temporary folder is deleted, and the original file updating of the current process is completed.
As a further improvement of the invention, the method also comprises the following steps:
if the time is inconsistent;
and deleting the target file, returning false, and re-reading and writing next time.
Compared with the prior art, the invention has the beneficial effects that:
under the condition of not using a file lock, the aim of reading and writing files is achieved by operating the copied files, and the original files are prevented from being directly operated, so that the condition that a server is down due to exclusive resource is effectively avoided;
each process of the invention can carry out the same operation, the processes can not generate adverse effect mutually, and other processes do not need to go through the process of queuing one by one, thus greatly improving the file processing efficiency;
the invention can quickly know whether the original file is updated or not by comparing the latest modification time of the original file obtained before and after reading and writing, thereby avoiding the direct update of the original file and the loss of the update of other processes.
Drawings
Fig. 1 is a flowchart of a method for reading and writing a multi-process file according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
The invention is described in further detail below with reference to the attached drawing figures:
as shown in fig. 1, the present invention discloses a method for reading and writing a multi-process file, which is used for solving the problem of deadlock caused by monopolizing resources by a single process, and solving the problems of time conflict and waiting when the multi-process reads and writes the file; the method is an operation of a file layer, and improves the file processing efficiency while ensuring the data security of the file;
the method comprises the following steps:
s1, copying the original file and establishing a target file;
the method specifically comprises the following steps:
s11, creating a temporary folder, and generating a random number:
creating a temporary folder for storing copied files;
generating a unique random number (random string) prevents other processes from using the same name for naming the copied file;
s12, copy the original file and name it using random number:
before generating a target file name, clearing the file cache by using a clearstatcache () function; the method prevents that file attributes in the cache are not updated in time when file read-write operation is carried out in multiple threads, so that the data accessed to the last update time is not real data easily;
splicing the temporary folder name, the random number and the original file name to define a target file path, namely generating a complete target file name;
and copying the original file into the target file by using a copy () function, and directly covering if the target file exists, so as to ensure that the obtained data is always the latest data.
S2, before reading and writing, obtaining the latest modification time of the original file;
the method specifically comprises the following steps:
acquiring the latest modification time of the original file by using a filetime () function, and storing the latest modification time in a variable;
and defining an array, and sequentially storing the target file name, the original file name, the generated random number and the time variable in the array according to the sequence, so that the target file name, the original file name, the generated random number and the time variable can be conveniently taken and used in the subsequent judgment.
S3, reading and writing the target file by any process;
the method specifically comprises the following steps:
when the single process needs to read and write the original file content, only the target file needs to be opened, and the content needing to be modified is written into the file and is stored and updated.
S4, obtaining the latest modification time of the original file after completing the reading and writing;
the method specifically comprises the following steps:
before time comparison, file cache still needs to be cleared, and the latest data is ensured to be operated;
and re-acquiring the latest modification time of the original file through the original file name stored in the array.
S5, comparing the time of the two acquisition;
the method specifically comprises the following steps:
and comparing the newly acquired latest modification time with the time variable stored in the array.
And S6, if the time comparison result is consistent, rename () is used to rename the target file name to the original file name, the target file name is moved to the original file directory to cover the original file, and then the temporary folder is deleted, so that the created temporary folder is prevented from occupying the memory all the time.
S7, if the time comparison result is inconsistent, indicating that other processes are modifying the original file; then, the target file is deleted and false is returned, and S1-S7 are repeated.
The invention has the advantages that:
under the condition of not using a file lock, the aim of reading and writing files is achieved by operating the copied files, and the original files are prevented from being directly operated, so that the condition that a server is down due to exclusive resource is effectively avoided; each process of the invention can carry out the same operation, the processes can not generate adverse effect mutually, and other processes do not need to go through the process of queuing one by one, thus greatly improving the file processing efficiency; the invention can quickly know whether the original file is updated or not by comparing the latest modification time of the original file obtained before and after reading and writing, thereby avoiding the direct update of the original file and the loss of the update of other processes.
The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes will occur to those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A method for reading and writing a multi-process file is characterized by comprising the following steps:
copying an original file and establishing a target file;
before reading and writing, obtaining the latest modification time of the original file;
any process reads and writes the target file;
after reading and writing are completed, the latest modification time of the original file is obtained;
comparing the time obtained twice;
and if the time is consistent, replacing the original file with the target file to finish the original file updating of the current process.
2. The method of claim 1, wherein the copying of the original file creates a target file; the method comprises the following steps:
creating a temporary folder and generating a random number with uniqueness;
and defining a target file path by using the temporary folder name, the random number and the original file name, and copying the original file into the target file path.
3. The method of claim 2, wherein the file cache is cleared prior to defining the target file path.
4. The method of claim 2, wherein the overwriting is performed directly if the target file exists during the copying of the original file.
5. The method of claim 1, wherein before reading and writing, the latest modification time of the original file is obtained; the method comprises the following steps:
acquiring the latest modification time of an original file and storing the latest modification time in a variable;
defining an array, and sequentially storing the target file name, the original file name, the generated random number and the time variable in the array according to the sequence.
6. The method of claim 1, wherein the target file is read from and written to by any process; the method comprises the following steps:
and opening the target file, writing the content to be modified into the target file, and storing and updating.
7. The method of claim 5, wherein after completing the reading and writing, obtaining the latest modification time of the original file; the method comprises the following steps:
clearing the file cache;
and re-acquiring the latest modification time of the original file according to the original file name stored in the array.
8. The method of claim 7, wherein the time between two acquisitions is aligned; the method comprises the following steps:
and comparing the newly acquired latest modification time with the time variable stored in the array.
9. The method of claim 1, wherein if the time is consistent, the target file is replaced by the original file to complete the update of the original file of the current process; the method comprises the following steps:
if the time is consistent;
then, the target file name is renamed to the original file name and is moved to the original file directory to cover the original file, the temporary folder is deleted, and the original file updating of the current process is completed.
10. The method of claim 1, further comprising:
if the time is inconsistent;
and deleting the target file, returning false, and re-reading and writing next time.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910897931.9A CN110825708A (en) | 2019-09-23 | 2019-09-23 | Multi-process file reading and writing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910897931.9A CN110825708A (en) | 2019-09-23 | 2019-09-23 | Multi-process file reading and writing method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110825708A true CN110825708A (en) | 2020-02-21 |
Family
ID=69548271
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910897931.9A Pending CN110825708A (en) | 2019-09-23 | 2019-09-23 | Multi-process file reading and writing method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110825708A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111638966A (en) * | 2020-05-28 | 2020-09-08 | 维沃移动通信有限公司 | Resource acquisition method and device and electronic equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6009440A (en) * | 1997-09-19 | 1999-12-28 | Mci Communications Corporation | Method and system for automatic object refresh |
| CN102195953A (en) * | 2010-03-19 | 2011-09-21 | 朱俭 | Method for protecting file downloading |
| CN103678295A (en) * | 2012-08-29 | 2014-03-26 | 北京百度网讯科技有限公司 | Method and device for providing files for user |
| CN104021355A (en) * | 2014-06-20 | 2014-09-03 | 浪潮电子信息产业股份有限公司 | Safety inspection method for simultaneously operating same file through multiple processes |
| CN105160030A (en) * | 2015-09-30 | 2015-12-16 | 北京金山安全软件有限公司 | Method and device for transmitting media cache files on mobile terminal |
-
2019
- 2019-09-23 CN CN201910897931.9A patent/CN110825708A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6009440A (en) * | 1997-09-19 | 1999-12-28 | Mci Communications Corporation | Method and system for automatic object refresh |
| CN102195953A (en) * | 2010-03-19 | 2011-09-21 | 朱俭 | Method for protecting file downloading |
| CN103678295A (en) * | 2012-08-29 | 2014-03-26 | 北京百度网讯科技有限公司 | Method and device for providing files for user |
| CN104021355A (en) * | 2014-06-20 | 2014-09-03 | 浪潮电子信息产业股份有限公司 | Safety inspection method for simultaneously operating same file through multiple processes |
| CN105160030A (en) * | 2015-09-30 | 2015-12-16 | 北京金山安全软件有限公司 | Method and device for transmitting media cache files on mobile terminal |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111638966A (en) * | 2020-05-28 | 2020-09-08 | 维沃移动通信有限公司 | Resource acquisition method and device and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11003689B2 (en) | Distributed database transaction protocol | |
| US6295610B1 (en) | Recovering resources in parallel | |
| US8266290B2 (en) | Scalable queues on a scalable structured storage system | |
| US6185577B1 (en) | Method and apparatus for incremental undo | |
| US6647510B1 (en) | Method and apparatus for making available data that was locked by a dead transaction before rolling back the entire dead transaction | |
| AU2004304873B2 (en) | Method and apparatus for data storage using striping | |
| CN109643309B (en) | System and method for performing range queries on skip list data structures | |
| US10430389B1 (en) | Deadlock-free locking for consistent and concurrent server-side file operations in file systems | |
| JP5658124B2 (en) | Method, apparatus and computer program for providing multiple contexts in a redirect-on-write file system | |
| EP2011045A1 (en) | Concurrency control within an enterprise resource planning system | |
| US10191936B2 (en) | Two-tier storage protocol for committing changes in a storage system | |
| JP4759570B2 (en) | Techniques for providing locks for file operations in database management systems | |
| US10678755B2 (en) | File metadata verification in a distributed file system | |
| US7818749B2 (en) | Data processing method, data processing apparatus, and data processing program | |
| CN110704199B (en) | Data compression method, device, computer equipment and storage medium | |
| US20080177959A1 (en) | System and method for executing transactions | |
| US11599504B2 (en) | Executing a conditional command on an object stored in a storage system | |
| CN113342507A (en) | Distributed lock service implementation method and device and computer equipment | |
| CN110825708A (en) | Multi-process file reading and writing method | |
| US10515066B2 (en) | Atomic updates of versioned data structures | |
| US8341368B2 (en) | Automatic reallocation of structured external storage structures | |
| US10108624B1 (en) | Concurrent directory move operations using ranking rules | |
| US10942912B1 (en) | Chain logging using key-value data storage | |
| CN110377614B (en) | Order processing lock system under distributed environment | |
| CN112328637A (en) | High-speed distributed data caching method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200221 |