CN119383577B - A data storage and uploading method and apparatus for a data acquisition terminal - Google Patents
A data storage and uploading method and apparatus for a data acquisition terminalInfo
- Publication number
- CN119383577B CN119383577B CN202411279594.4A CN202411279594A CN119383577B CN 119383577 B CN119383577 B CN 119383577B CN 202411279594 A CN202411279594 A CN 202411279594A CN 119383577 B CN119383577 B CN 119383577B
- Authority
- CN
- China
- Prior art keywords
- data
- storage
- address
- stored
- read
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/30—Services specially adapted for particular environments, situations or purposes
- H04W4/38—Services specially adapted for particular environments, situations or purposes for collecting sensor information
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/12—Applying verification of the received information
- H04L63/123—Applying verification of the received information received data contents, e.g. message integrity
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/40—Network security protocols
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Techniques For Improving Reliability Of Storages (AREA)
Abstract
The invention belongs to the technical field of data storage and communication, and particularly relates to a data storage and uploading method and device of a data acquisition terminal. The method comprises the following steps of firstly enabling a data acquisition terminal to conduct MCU main control on acquired data after the data acquisition is completed, then uploading the acquired data to a cloud platform through a wireless communication module, secondly adopting MD5 checking algorithm to check whether the data received by the platform are consistent with the transmitted data, if not, failing to transmit the data, then writing the data failing to transmit into an external Flash memory chip for storage, finally detecting the networking state at regular time, and when the networking state is good and the stored data exist, retransmitting the data stored in the Flash memory chip to the platform, and resetting a memory area. The problem of data loss caused by network problems is solved, a light-weight data storage scheme is provided for outdoor small-sized embedded equipment, the cost is reduced, and the service life of the Flash memory chip is prolonged.
Description
Technical Field
The invention belongs to the technical field of data storage and communication, and particularly relates to a data storage and uploading method and device of a data acquisition terminal.
Background
With the development of network technology, modern society becomes more informationized, and the development of information technology is promoted by the proposal of intelligent water conservancy, intelligent industry, intelligent agriculture and the like, and various industries are digitally transformed and upgraded. The main data acquisition mode at present is to acquire data through a sensor and then upload the data to a platform network through a data acquisition terminal, or the sensor directly acquires the data and uploads the data to the platform network, and the data acquisition terminal is often required to transfer the data due to the defects of high cost, few data acquisition types and the like.
The Chinese patent document CN114257978A discloses a transmission method of sensing data of outdoor power transmission equipment, which comprises the steps of acquiring the sensing data when the sensor sends the sensing data to a transit node, wherein the sensing data is the sensing data acquired in the process of monitoring the outdoor power transmission equipment by the sensor, the sensing data represents the equipment state of the outdoor power transmission equipment, judging whether the transit node receives the sensing data or not, and if the transit node is determined not to receive the sensing data, sending the sensing data to an intelligent gateway through the candidate node, so that retransmission can be avoided when transmission failure is caused due to too long transmission distance when the transmission distance of each hop is increased, and the transmission efficiency of the sensing data is effectively improved.
In urban areas with better network conditions, the data acquisition terminal usually cannot fail in data transmission due to network problems, but in the practical application process, the data acquisition terminal is often installed in a wide field or a mountain, and network signals are difficult to guarantee. Therefore, data loss and the like often occur due to poor network conditions. At present, more methods are used to store data and then upload the data, and in an embedded system, the capacity expansion is often performed by adopting a plug-in memory mode due to the limitation of storage resources. In small embedded devices, EEPROM or Flash is usually selected as an external memory, where EEPROM has the advantages of long service life and capability of being erased in bytes, but is relatively expensive and has relatively small memory capacity, while Flash has the advantages of relatively low price and large memory capacity, but needs to be erased in sectors, so that the service life is short. The existing data storage mode at present often has the defects of high cost, complex system design and the like, and how to design a lighter and lower-cost data storage and uploading method is important.
Therefore, the invention designs a data storage and uploading method of the data acquisition terminal.
Disclosure of Invention
The invention aims to overcome the defects of the prior art, and provides a data storage and uploading method of a data acquisition terminal, which effectively improves the data stability of a simple embedded acquisition terminal, improves the efficiency, prolongs the service life of a Flash memory and reduces the terminal cost.
The invention also discloses a device loaded with the data storage and uploading method of the data acquisition terminal.
The detailed technical scheme of the invention is as follows:
a data storage and uploading method of a data acquisition terminal, the method comprising the steps of:
s1, after data acquisition is completed by a data acquisition terminal, acquired data is firstly subjected to MCU main control and then uploaded to a cloud platform through a wireless communication module;
s2, checking whether the data received by the cloud platform is consistent with the transmitted data or not by adopting an MD5 checking algorithm, and if not, failing to transmit the data;
S3, writing the data which is failed to be sent into an external Flash memory chip for storage;
the Flash memory chip determines memory allocation in the following way, wherein a sector is divided into N parts, N represents the number of data stored in the sector, M sectors are selected to store data, if the size of the Flash sector is X, the storage interval size is X/N, and because Flash can only be erased by taking the sector as a unit, one sector is used as a transition zone, and M sectors are used for storing N (M-1) data;
And S4, detecting the networking state at fixed time by adopting a heartbeat mechanism, and when the networking state is in connection, the signal is good and the stored data exists, retransmitting the data stored in the Flash memory chip to the cloud platform and resetting the memory area.
According to the invention, the step S2 specifically means that after the data is acquired, MD5 check is performed through the MCU main control, the check code and the plaintext data are transmitted, the platform side calculates the comparison check code again through MD5 to determine whether the data is complete, if the data is complete, the cloud platform returns a response code, after the data acquisition terminal receives the response code, the data is represented as complete transmission, and if the data acquisition terminal cannot receive the response code or the MD5 check code is inconsistent, the data transmission fails.
According to the present invention, in the step S3, writing the data with failed transmission into the external Flash memory chip includes the following steps:
S31, judging whether the number of the stored data is larger than or equal to the set maximum value, namely N (M-1), when the number of the stored data is smaller than the maximum value, proving that the number of the stored data is smaller than the maximum value, the storage space has a margin, then automatically increasing the number variable of the stored data by one and entering the judgment of the storage head address;
S32, judging the storage head address and the storage tail address, if both the storage head address and the storage tail address are smaller than the storage head address of the (N-1) -th data of the Mth sector, changing the writing address into the storage tail address, automatically increasing the storage interval size of the storage tail address and the storage head address, and then entering the writing address judgment; if the address of the storage head is not less than the address of the storage head of the N-1 data of the M-th sector, the writing address is changed into the address of the storage tail, the address of the storage head is changed into the address of the first end, the address of the storage tail is automatically increased by the size of the storage interval, and then the writing address judgment is carried out;
S33, since the transition area is adopted to receive the data of one sector to reduce the erasing times of Flash so as to prolong the service life of the Flash, the sector where the writing address is located needs to be judged, when the number of the stored data reaches the preset maximum value, if the writing address is at the first address of one sector, the sector is the transition area, and the data can be directly erased and then stored;
Judging whether the value of the write address for taking the remainder of the sector capacity is 0, if so, indicating that the sector is a transition area, erasing, then writing the data to be written, and simultaneously writing the number of stored data, the storage head address and the storage tail address, if not, not erasing, directly writing the data to be written, and simultaneously writing the number of stored data, the storage head address and the storage tail address.
According to the present invention, in the step S4, the specific steps of retransmitting the data stored in the Flash memory chip to the cloud platform are as follows:
S41, reading the number of stored data, a storage head address and a storage tail address to a system memory;
S42, reading data according to the number of stored data, and transmitting the read data, wherein if the read data fails to be transmitted, judging whether the read data is not failed to be transmitted for the first time and is transmitted successfully before, if not, entering a judgment area for whether all the data is read and transmitted for one time, if so, writing the current data into a space where a storage tail address is located, the self-increasing storage interval size of the storage tail address and the storage tail address into an external storage, and then entering a judgment area for judging whether all the data is read and transmitted for one time;
If the read data is successfully transmitted for the first time, judging whether the read data is successfully transmitted for the first time, if so, subtracting one from the stored data number variable, changing the storage tail address into the current read address, writing the storage tail address and the stored data number into an external storage, and entering the judgment of the current read address;
S43, because the data is circularly stored in Flash, the current read address needs to be judged, if the current read address is smaller than the storage first address of the N-1 data of the Mth sector, the current read address automatically increases the storage interval size, and then whether all data are read and sent to a first pass judgment area is judged;
S44, because there may be multiple stored data, the stored data need to be read out and sent successively, if not all read out and sent once, reading and sending are continued, if all read out and sent once, judging whether all sending of the stored data is completed, if all sending is completed, writing the empty mark of the stored data to external storage, if all sending is completed, if not, the reading and sending of the stored data are completed, and the data which is not sent successfully continue and are sent preferentially next time.
In another aspect of the present invention, there is provided a data acquisition apparatus for implementing a data storage and upload method of a data acquisition terminal, the apparatus comprising:
the external Flash memory chip is used for storing data which is not sent successfully;
the MCU master control is used for being responsible for overall control and management of the whole system;
And the wireless communication module is used for uploading data to the cloud platform.
Compared with the prior art, the invention has the beneficial effects that:
the invention provides a data storage and uploading method of a data acquisition terminal, aiming at the problems of data packet loss, data transmission failure when network conditions are poor, short storage life of external Flash and the like of the data acquisition terminal, and the data storage and uploading method of the data acquisition terminal verifies whether data transmission is complete or not each time by adopting an MD5 verification algorithm, ensures effective data transmission, ensures effective data storage of transmission failure data by adopting Flash external storage and setting a transitional storage area, prolongs the service life of Flash, and ensures that stored data can be timely transmitted to a platform by detecting network conditions at regular time.
Drawings
FIG. 1 is a flow chart of the data acquisition terminal of the present invention;
FIG. 2 is a block diagram of the Flash memory chip structure of the present invention;
FIG. 3 is a flow chart of the data storage of the present invention;
FIG. 4 is a flow chart of the memory data reading and transmitting method of the present invention;
fig. 5 is a diagram showing an implementation effect of the data acquisition terminal in the embodiment of the present invention.
Detailed Description
The invention is further described below with reference to the drawings and examples.
Example 1,
Referring to fig. 1, a data storage and uploading method of a data acquisition terminal, the method includes the following steps:
s1, after data acquisition is completed by a data acquisition terminal, acquired data is firstly subjected to MCU main control and then uploaded to a cloud platform through a wireless communication module;
s2, checking whether the data received by the cloud platform is consistent with the transmitted data or not by adopting an MD5 checking algorithm, and if not, failing to transmit the data;
S3, writing the data which is failed to be sent into an external Flash memory chip for storage;
And S4, detecting the networking state at fixed time by adopting a heartbeat mechanism, and when the networking state is in connection, the signal is good and the stored data exists, retransmitting the data stored in the Flash memory chip to the cloud platform and resetting the memory area.
The step S2 specifically means that an MD5 verification algorithm is adopted to detect the integrity of the data, after the data is collected, the data is subjected to MD5 verification by the MCU master control, and the verification code and the plaintext data are transmitted. The platform side determines whether the data is complete by carrying out MD5 calculation and comparison check codes again, if the data is complete, the platform returns a response code, the data is completely transmitted after the data acquisition terminal receives the response code, and if the data acquisition terminal cannot receive the response code or the MD5 check codes are inconsistent, the data transmission is considered to be failed.
As shown in FIG. 2, the Flash memory chip determines memory allocation by dividing a sector into N parts, where N represents the number of data pieces stored in the sector, and selecting M sectors for data storage. If the Flash sector size is X, the storage interval size is X/N. Since Flash can only be erased in sector units, one sector is used as a transition area, and M sectors are used to store N (M-1) pieces of data.
As shown in fig. 3, in step S3, writing the data of the transmission failure to the external Flash memory chip includes the steps of:
S31, firstly judging whether the number of the stored data is greater than or equal to a set maximum value, namely N (M-1), when the number of the stored data is smaller than N (M-1), proving that the number of the stored data is not greater than the maximum value, the storage space has a margin, then automatically increasing the number of the stored data by one variable and entering the judgment of a storage head address;
S32, judging a storage head address and a storage tail address, if both the storage head address and the storage tail address are smaller than the storage head address of the (N-1) -th data of the Mth sector, changing a writing address into the storage tail address, automatically increasing the storage interval size of the storage tail address and the storage head address, and then entering the writing address judgment; if the address of the storage head is not less than the address of the storage head of the N-1 data of the M-th sector, the writing address is changed into the address of the storage tail, the address of the storage head is changed into the address of the first end, the address of the storage tail is automatically increased by the size of the storage interval, and then the writing address judgment is carried out;
And S33, the scheme adopts the transition area to receive the data of one sector to reduce the erasing times of Flash so as to prolong the service life of the Flash, so that the sector where the written address is located needs to be judged, and when the number of the stored data reaches the preset maximum value, if the written address is at the first address of a certain sector, the sector is the transition area, and the data can be directly erased and then stored.
Therefore, whether the value of the write address for taking the remainder of the sector capacity is 0 needs to be judged, if so, the sector is indicated to be a transition area and can be erased, then the data to be written is written, and the number of the stored data, the storage head address and the storage tail address are written at the same time; if not, the data to be written is directly written without erasing, and the number of the stored data, the address of the storage head and the address of the storage tail are written.
The stored data stripe number in fig. 3 indicates the number of data already stored in the current Flash;
the maximum value represents the number of data which can be stored in the set Flash, and is N (M-1);
The invention adopts a ring storage mode, and the storage head address represents the head address of the storage space where the earliest piece of data is positioned in all the currently stored data;
the storage tail address represents the head address of the storage space where the last piece of data in all the currently stored data is located;
The tail data address represents the head address of the storage space where the last piece of data of the unsent data is stored, which is distributed in Flash, and is the storage head address of the (N-1) th piece of data of the Mth sector;
The write address represents the first address of the memory space to which the data is to be written;
the head address represents the head address of all the spaces which are distributed in Flash and used for storing the failed data;
Sector capacity means the number of bytes that a sector of Flash can store, here X;
the storage space size represents the size of the space occupied by a piece of data in Flash, here X/N.
As shown in fig. 4, in step S4, the data stored in the Flash memory chip is retransmitted to the platform, and the stored data needs to be read and transmitted, which specifically includes the following steps:
S41, reading the number of stored data, a storage head address and a storage tail address to a system memory;
S42, reading data according to the number of stored data, and sending the read data;
S43, if the data transmission fails, judging whether the data is not failed in the first transmission and the previous transmission is successful, if not, entering a judging area of whether all the data is read and transmitted once, if so, writing the current data into a space where a storage tail address is located, the self-increasing storage interval size of the storage tail address and the storage tail address into an external storage, and then entering a judging area of whether all the data is read and transmitted once;
s44, if the read data is successfully transmitted, judging whether the read data is successfully transmitted for the first time, if so, subtracting one from the number variable of the stored data, changing the storage tail address into the current read address, writing the storage tail address and the number of the stored data into an external storage, and entering the judgment of the current read address;
S45, judging the current read address because the data is circularly stored in Flash, if the current read address is smaller than the storage first address of the N-1 data of the Mth sector, automatically increasing the storage interval size of the current read address, and then entering a judgment area for judging whether all data are read and sent once;
S46, because there may be a plurality of stored data, the stored data needs to be read out and sent successively, if not all read out and sent once, reading and sending are continued, if all read out and sent once, judging whether all sending of the stored data is completed, if all sending is completed, writing a storage data empty mark to external storage, and if not all sending is completed, the round of storage data reading and sending is ended. The data that was not successfully transmitted will continue and be preferentially transmitted next time.
EXAMPLE 2,
The embodiment provides a data acquisition device for realizing a data storage and uploading method of a data acquisition terminal, wherein the device comprises:
the external Flash memory chip is used for storing data which is not sent successfully;
the MCU master control is used for being responsible for overall control and management of the whole system;
And the wireless communication module is used for uploading data to the cloud platform.
The data acquisition terminal is used for data acquisition of agricultural sensors in mountainous areas, the agricultural sensors generally support RS485 protocol transmission, so that the data acquisition terminal adopts RS485 input, is packaged after being processed by MCU (micro control unit) main control, and is uploaded to a monitoring cloud platform through a 4G module. The data acquisition terminal provided by the invention can timely send out the data when the signal is good, timely store the data of the agricultural sensor when the signal is bad, and upload the data in that time period after the signal is recovered, so that the phenomenon of time discontinuity of the data of the agricultural sensor can not occur. The implementation effect is shown in fig. 5:
The device comprises a 1:4G antenna, a 2:vertical rod, a 3:waterproof box, a 4:data acquisition terminal, a 5:power line, a 6:agricultural sensor and a 7:RS485 data line.
Claims (4)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411279594.4A CN119383577B (en) | 2024-09-12 | 2024-09-12 | A data storage and uploading method and apparatus for a data acquisition terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411279594.4A CN119383577B (en) | 2024-09-12 | 2024-09-12 | A data storage and uploading method and apparatus for a data acquisition terminal |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN119383577A CN119383577A (en) | 2025-01-28 |
| CN119383577B true CN119383577B (en) | 2025-11-14 |
Family
ID=94334500
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202411279594.4A Active CN119383577B (en) | 2024-09-12 | 2024-09-12 | A data storage and uploading method and apparatus for a data acquisition terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN119383577B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104536699A (en) * | 2014-12-11 | 2015-04-22 | 中国科学院声学研究所 | Streamed data write-in method based on embedded file system |
| CN118626012A (en) * | 2024-06-12 | 2024-09-10 | 四川九洲电器集团有限责任公司 | A data management method and system for improving FLASH storage times |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2000050997A1 (en) * | 1999-02-22 | 2000-08-31 | Hitachi, Ltd. | Memory card, method for allotting logical address, and method for writing data |
| US9286198B2 (en) * | 2005-04-21 | 2016-03-15 | Violin Memory | Method and system for storage of data in non-volatile media |
| US7984084B2 (en) * | 2005-08-03 | 2011-07-19 | SanDisk Technologies, Inc. | Non-volatile memory with scheduled reclaim operations |
| WO2009104330A1 (en) * | 2008-02-20 | 2009-08-27 | 株式会社ソニー・コンピュータエンタテインメント | Memory control method and device, and memory access control method, computer program, and recording medium |
| JP2018148358A (en) * | 2017-03-03 | 2018-09-20 | 富士通株式会社 | Receiving device and transmission device |
-
2024
- 2024-09-12 CN CN202411279594.4A patent/CN119383577B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104536699A (en) * | 2014-12-11 | 2015-04-22 | 中国科学院声学研究所 | Streamed data write-in method based on embedded file system |
| CN118626012A (en) * | 2024-06-12 | 2024-09-10 | 四川九洲电器集团有限责任公司 | A data management method and system for improving FLASH storage times |
Also Published As
| Publication number | Publication date |
|---|---|
| CN119383577A (en) | 2025-01-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7003710B2 (en) | Communications method, communications apparatus and communications system using same communications apparatus | |
| CN112953839A (en) | Data framing transmission method, telemetering equipment and data center platform | |
| KR101428778B1 (en) | Apparatus and method for upgrading firmware of automatic meter reading terminal on the remote | |
| CN119383577B (en) | A data storage and uploading method and apparatus for a data acquisition terminal | |
| US9258476B2 (en) | Automatic image-capturing system | |
| CN120950009A (en) | Data processing methods, products, electronic devices, and computer-readable storage media | |
| US10009574B2 (en) | Transferring multiple records including video data to/from host | |
| CN112804122A (en) | Message link fault diagnosis method and device and computer readable storage medium | |
| CN113301284B (en) | Law enforcement recorder system and process | |
| US20040153766A1 (en) | Communication system,transmission device, reception device, and communication system having them | |
| CN115190444A (en) | Wireless Internet of things alarm method, system, equipment and medium | |
| CN107111927B (en) | Sink node, sensor network system, information acquisition method, and information acquisition program | |
| JP5736612B2 (en) | Quick MAP recovery method in case of error in MoCA | |
| EP4494317B1 (en) | DATA TRANSFER | |
| CN116150078B (en) | Inter-board data communication transmission method, electronic device and computer-readable storage medium | |
| JP2007249469A (en) | Nonvolatile semiconductor memory device and memory system | |
| CN112817889B (en) | Data acquisition method and system | |
| JP3006561B2 (en) | Data communication retransmission method | |
| CN114257978B (en) | Method, device and computer equipment for transmitting sensor data of outdoor power transmission equipment | |
| CN118118567A (en) | Method, device and system for transmitting electricity usage information | |
| JPH11251988A (en) | Satellite data transmitter and receiver | |
| JP3373428B2 (en) | Data transmission device, data transmission method, recording medium recording data transmission program, and data transmission program | |
| CN116955960A (en) | BIM modeling-based energy consumption analysis method, system and storage medium | |
| CN114443095A (en) | Air conditioner upgrading method and air conditioner | |
| CN121356738A (en) | Data transmission methods, systems, devices, and storage media based on the PSI5 protocol |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |