CN109614372B - Object storage and reading method and device and service server - Google Patents
Object storage and reading method and device and service server Download PDFInfo
- Publication number
- CN109614372B CN109614372B CN201811261886.XA CN201811261886A CN109614372B CN 109614372 B CN109614372 B CN 109614372B CN 201811261886 A CN201811261886 A CN 201811261886A CN 109614372 B CN109614372 B CN 109614372B
- Authority
- CN
- China
- Prior art keywords
- data
- database
- splitting
- data fragments
- storage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The object storage method comprises the following steps: carrying out serialization processing on the target object to obtain serialization data of the target object; splitting the serialized data; determining a plurality of data fragments of the serialized data based on the splitting processing result, storing the plurality of data fragments in a database, and storing index information for representing storage positions of the plurality of data fragments in the database; the object reading method comprises the following steps: determining a target storage position in a database according to pre-stored index information, wherein the index information is used for representing storage positions of a plurality of data fragments in the database; reading the data fragments from the database based on the target storage locations; merging the read data fragments; and performing deserialization on the data obtained by the merging processing to obtain a target object. By applying the object storage method and the object reading method, the access performance of the object can be improved.
Description
Technical Field
The embodiment of the specification relates to the technical field of data processing, in particular to an object storage method, an object reading device and a service server.
Background
In business systems, storage and reading of very large objects is often involved. In the prior art, a file type is generally used for storing oversized objects, and the storage capacity of the file type is infinite, however, if the oversized objects are stored in the form, in the process of searching oversized objects later, each file logical block is searched first, the logical blocks are scattered on physical blocks, and the physical blocks are scattered on different sectors, so that the whole objects can be searched only by searching layer by layer, and finally, the reading of the objects can be completed. It follows that the access performance of storing very large objects using the file form is very poor.
Disclosure of Invention
Aiming at the technical problems, the embodiment of the specification provides an object storage and reading method, an object storage and reading device and a service server, and the technical scheme is as follows:
according to a first aspect of embodiments of the present specification, there is provided an object storage method, the method comprising:
carrying out serialization processing on a target object to obtain serialization data of the target object;
splitting the serialized data;
based on the splitting processing result, a plurality of data fragments of the serialized data are determined, the plurality of data fragments are stored in a database, and index information for representing storage positions of the plurality of data fragments in the database is stored.
According to a second aspect of embodiments of the present specification, there is provided an object reading method, the method comprising:
determining a target storage position in a database according to pre-stored index information, wherein the index information is used for representing storage positions of a plurality of data fragments in the database;
reading a data fragment from the database based on the target storage location;
merging the read data fragments;
and performing deserialization on the data obtained by the merging processing to obtain a target object.
According to a third aspect of embodiments of the present specification, there is provided an object storage apparatus, the apparatus comprising:
the serialization module is used for carrying out serialization processing on the target object to obtain serialized data of the target object;
the splitting module is used for splitting the serialized data;
and the storage module is used for determining a plurality of data fragments of the serialized data based on the splitting processing result, storing the data fragments into a database and storing index information for representing the storage positions of the data fragments in the database.
According to a fourth aspect of embodiments of the present specification, there is provided an object reading apparatus, the apparatus comprising:
the determining module is used for determining a target storage position in a database according to pre-stored index information, wherein the index information is used for representing storage positions of a plurality of data fragments in the database;
the reading module is used for reading the data fragments from the database based on the target storage position;
the merging module is used for merging the read data fragments;
and the deserializing module is used for deserializing the data obtained by the joint processing to obtain the target object.
According to a fifth aspect of the embodiments of the present specification, there is provided a service server, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the object storage method provided by the embodiments of the present specification when executing the program.
According to a sixth aspect of the embodiments of the present specification, there is provided a service server, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the method for reading an object provided by the embodiments of the present specification when the processor executes the program.
According to the technical scheme provided by the embodiment of the specification, the target object is subjected to serialization processing to obtain the serialized data of the target object, the serialized data is split, a plurality of data fragments of the serialized data are determined based on a splitting processing result, the plurality of data fragments are stored in the database, and index information for representing the storage positions of the plurality of data fragments in the database is stored, so that the limitation of bottom storage can be broken through, and the storage performance of the oversized object is improved.
According to the technical scheme provided by the embodiment of the specification, the target storage position is determined in the database according to the pre-stored index information, the data fragments are read from the database based on the target storage position, and the read data fragments are combined; and performing deserialization on the data obtained by the merging and processing to obtain a target object, so that the reading performance of the oversized object can be improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the embodiments of the disclosure.
Further, not all of the effects described above need be achieved in any of the embodiments of the present specification.
Drawings
In order to more clearly illustrate the embodiments of the present description or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments described in the embodiments of the present description, and other drawings may be obtained according to these drawings for a person having ordinary skill in the art.
FIG. 1 is a flowchart of an embodiment of an object storage method according to an exemplary embodiment of the present disclosure;
FIG. 2 is a flowchart of an embodiment of an object reading method according to an exemplary embodiment of the present disclosure;
FIG. 3 is a block diagram of an embodiment of an object store according to an exemplary embodiment of the present disclosure;
FIG. 4 is a block diagram of an embodiment of an object reading apparatus according to an exemplary embodiment of the present disclosure;
fig. 5 is a schematic diagram of a more specific service server hardware structure according to an embodiment of the present disclosure.
Detailed Description
In order for those skilled in the art to better understand the technical solutions in the embodiments of the present specification, the technical solutions in the embodiments of the present specification will be described in detail below with reference to the drawings in the embodiments of the present specification, and it is apparent that the described embodiments are only some embodiments of the present specification, not all embodiments. All other embodiments obtained by a person skilled in the art based on the embodiments in the present specification shall fall within the scope of protection.
Since the access performance of storing an oversized object in a file format is very poor in the prior art, the embodiment of the present disclosure proposes an object storing method and an object reading method to improve the access performance of the object.
First, the object storing method and the object reading method are implemented based on an Hbase database, that is, the object is stored in the Hbase database, and then the object is read from the Hbase database. The following embodiments are shown to respectively explain the object storing method and the object reading method:
first, an object storage method will be described:
referring to fig. 1, a flowchart of an embodiment of an object storing method according to an exemplary embodiment of the present disclosure may include the following steps:
step 102: and carrying out serialization processing on the target object to obtain the serialization data of the target object.
First, in the present embodiment, for convenience of description, an object to be processed is referred to as a target object.
In order to enable the target object to be persisted, the target object may be serialized to obtain a set of bytes, where the set of bytes is serialized data of the target object.
In the serializing process of the target object, json, kryo, jdk Serialize, or the like may be used, which is not limited in the embodiment of the present disclosure.
In addition, after the serialized data of the target object is obtained, the serialized data can be further compressed, and then the compressed serialized data is stored. By the processing, the data quantity of the data to be stored can be further reduced, and the access performance of the data is improved.
Step 104: and splitting the serialized data.
In an embodiment, the physical splitting process may be directly performed on the serialized data obtained in step 102, where the "physical splitting process" refers to splitting the serialized data into a plurality of data slices in the system memory, for example, assuming that the serialized data includes 100 bytes, splitting the 100 bytes into 10 packets, each packet includes 10 bytes, and each packet is one data slice, so as to obtain 10 data slices.
In an embodiment, the serialized data obtained in step 102 may be subjected to a logic splitting process, where "logic splitting process" refers to obtaining, in a system memory, a plurality of splitting positions in the serialized data based on a preset splitting rule, for example, assuming that the serialized data includes 100 bytes, the preset splitting rule is to split the serialized data into 10 data slices, where each data slice includes 10 bytes, and then the obtained splitting positions include: 10. 20, 30, 40, 50, 60, 70, 80, 90, it will be appreciated by those skilled in the art that based on the 9 split positions, serialized data may be split into 10 data slices, where the data at the 1-10 byte positions is one data slice, the data at the 11-20 byte positions is one data slice, and so on, for a total of 10 data slices.
As can be seen from the above description, performing physical splitting processing on the serialized data in the memory generates a plurality of data fragments in the memory, thereby occupying the memory, and performing logical splitting processing on the serialized data in the memory only involves the occupation of the memory by a plurality of splitting positions, so that compared with performing physical splitting processing on the serialized data, performing logical splitting processing on the serialized data can reduce the occupation amount of the memory in the object storage process, and improve the system performance.
Step 106: based on the splitting processing result, a plurality of data fragments of the serialized data are determined, the plurality of data fragments are stored in a database, and index information for indicating storage positions of the plurality of data fragments in the database is stored.
In the embodiment of the present disclosure, if the "physical splitting manner" is adopted in step 104, after splitting, a plurality of data fragments may be directly obtained, and the plurality of data fragments are temporarily stored in the memory, and in the subsequent storage process, the plurality of data fragments in the memory may be stored in the database; if the "logical splitting processing manner" is adopted in step 104, after splitting processing, a plurality of data fragments may be determined in the original serialized data based on the acquired splitting position, and in the subsequent storage process, the determined data fragments are directly stored in the database.
In an embodiment, based on the Hase database, different data slices may be stored in different row records of the Hbase database, respectively, and one skilled in the art will appreciate that in the Hbase database, one row record has a unique rowkey value.
Based on this, in the embodiment of the present specification, index information for indicating the storage positions of the plurality of data slices in the Hbase database, that is, rowkey values of the row records in which the plurality of data slices are located in the Hbase database, may also be stored in the Hbase database, for example, in one row record of the Hbase database.
In an embodiment, the rowkey values recorded by the rows of the Hbase database where the plurality of data slices are located may be stored directly, for example, the stored rowkey values include: rowkey #01, rowkey #02, rowkey #03, … …, rowkey #10.
In another embodiment, the rowkey value recorded by the row in which the plurality of data slices are located in the Hbase database may also be stored "indirectly", where "indirectly" storing means that the number of data slices is stored, e.g. 10, a prefix of the rowkey value is stored, e.g. "#", and a starting sequence number of the rowkey value is stored, e.g. "01". It follows that by storing the rowkey values recorded in the row of the Hbase database for a plurality of data fragments "indirectly", the data storage amount can be further reduced.
It will be appreciated by those skilled in the art that by storing the above index information, the process of reading the object can be used for subsequent reading of the object, and the following description of the embodiment shown in fig. 2 will be omitted here.
According to the technical scheme provided by the embodiment of the specification, the target object is subjected to serialization processing to obtain the serialized data of the target object, the serialized data is split, a plurality of data fragments of the serialized data are determined based on a splitting processing result, the plurality of data fragments are stored in the database, and index information for representing the storage positions of the plurality of data fragments in the database is stored, so that the limitation of bottom storage can be broken through, and the storage performance of the oversized object is improved.
Next, an object reading method will be described:
referring to fig. 2, a flowchart of an embodiment of an object reading method according to an exemplary embodiment of the present disclosure may include the following steps based on the object storage method illustrated in fig. 1:
step 202: and determining a target storage position in the database according to pre-stored index information, wherein the index information is used for representing the storage positions of the plurality of data fragments in the database respectively.
As can be seen from the description of the embodiment shown in fig. 1, in the process of storing an object, index information of storage locations of multiple data slices of the object in the database may be stored, and then, in the process of reading the object, a target storage location, for example, a target rowkey value, may be determined in the database according to the pre-stored index information.
Step 204: based on the target storage location, the data shards are read from the database.
In this step, the data fragment may be read from the database based on the target storage location obtained in step 202.
As will be appreciated by those skilled in the art, the read data fragments are temporarily stored in memory.
Step 206: and merging the read data fragments.
In one embodiment, the data fragments read in step 204 may be physically merged, where "physically merging" refers to stitching multiple data fragments in a system memory to obtain complete serialized data, and those skilled in the art will understand that the complete serialized data is also temporarily stored in the memory.
In one embodiment, the data slices read in step 204 may be logically combined, where "logically combined" may refer to the encapsulation of the read data slices into an input stream. It will be appreciated by those skilled in the art that an input stream refers to an object capable of reading a sequence of bytes, and that when data is to be read, a program can read the source of the data from the input stream, whereby it can be seen that by this process, merging of the data fragments can be logically achieved.
As can be seen from the above description, performing physical merging processing on multiple data slices in the memory generates a complete serialized data in the memory, so as to occupy the memory, and performing logical merging processing on the serialized data in the memory, which does not involve memory occupation, so that compared with performing physical merging processing on multiple data slices, performing logical merging processing on multiple data slices can reduce the memory occupation in the process of reading objects, and improve the system performance.
Step 208: and performing deserialization on the data obtained by the merging processing to obtain a target object.
In the object storage process, the serialized data of the object is stored, so that in the step, the deserialization processing can be performed on the serialized data obtained by the combination processing to obtain the target object.
In addition, if the serialized data is compressed during the object storage process, in this step, the decompressed data may be decompressed first, and then the decompressed data may be deserialized, so as to obtain the target object.
In addition, when decompressing the data obtained by the combination processing, the decompression processing may be preferably performed by a stream nesting method.
According to the technical scheme provided by the embodiment of the specification, the target storage position is determined in the database according to the pre-stored index information, the data fragments are read from the database based on the target storage position, and the read data fragments are combined; and performing deserialization on the data obtained by the merging and processing to obtain a target object, so that the reading performance of the oversized object can be improved.
Corresponding to the above embodiment of the method illustrated in fig. 1, the embodiment of the present disclosure further provides an object storage device, referring to fig. 3, which is a block diagram of an embodiment of the object storage device provided in an exemplary embodiment of the present disclosure, where the device may include: a serialization module 31, a splitting module 32, and a storage module 33.
The serialization module 31 is configured to perform serialization processing on a target object to obtain serialized data of the target object;
a splitting module 32, configured to split the serialized data;
a storage module 33, configured to determine a plurality of data fragments of the serialized data based on a splitting result, store the plurality of data fragments in a database, and store index information indicating storage positions of the plurality of data fragments in the database.
In one embodiment, the splitting module 32 may be specifically configured to:
based on a preset splitting rule, a plurality of splitting positions in the serialized data are obtained.
In an embodiment, different data fragments are respectively stored in different row records of the database;
the index information includes: the rowkey value recorded in the row where the data slice is located.
It should be understood that the serialization module 31, the splitting module 32, and the storage module 33 may be configured in the apparatus as three modules with independent functions, or may be configured in the apparatus separately as shown in fig. 3, and thus the configuration shown in fig. 3 should not be construed as limiting the embodiment of the present disclosure.
Corresponding to the above-mentioned method embodiment illustrated in fig. 2, the present embodiment further provides an object reading apparatus, referring to fig. 4, which is a block diagram of an embodiment of the present disclosure, where the apparatus may include: a determination module 41, a reading module 42, a merging module 43, and a de-serialization module 44.
Wherein, the determining module 41 is configured to determine a target storage location in a database according to pre-saved index information, where the index information is used to represent storage locations of a plurality of data fragments in the database;
a reading module 42, configured to read a data fragment from the database based on the target storage location;
a merging module 43, configured to merge the read data slices;
the deserializing module 44 is configured to deserialize the data obtained by the merging process to obtain the target object.
In one embodiment, the merging module 43 is specifically configured to:
the read data fragments are packaged into an input stream.
It will be appreciated that the determining module 41, the reading module 42, the combining module 43, and the reverse serializing module 44 may be configured in the apparatus simultaneously as shown in fig. 4, or may be configured in the apparatus separately, and thus the configuration shown in fig. 4 should not be construed as limiting the embodiment of the present disclosure.
In addition, the implementation process of the functions and roles of each module in the above device is specifically shown in the implementation process of the corresponding steps in the above method, and will not be described herein again.
The embodiment of the present disclosure also provides a service server, which at least includes a memory, a processor, and a computer program stored on the memory and capable of running on the processor, where the processor implements the foregoing object storage method when executing the program. The method at least comprises the following steps: carrying out serialization processing on a target object to obtain serialization data of the target object; splitting the serialized data; based on the splitting processing result, a plurality of data fragments of the serialized data are determined, the plurality of data fragments are stored in a database, and index information for representing storage positions of the plurality of data fragments in the database is stored.
The embodiment of the present disclosure also provides another service server, which at least includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, where the processor implements the foregoing method for reading an object when executing the program. The method at least comprises the following steps: determining a target storage position in a database according to pre-stored index information, wherein the index information is used for representing storage positions of a plurality of data fragments in the database; reading a data fragment from the database based on the target storage location; merging the read data fragments; and performing deserialization on the data obtained by the merging processing to obtain a target object.
Fig. 5 shows a more specific hardware architecture of a service server according to an embodiment of the present disclosure, where the apparatus may include: processor 510, memory 520, input/output interface 530, communication interface 540, and bus 550. Wherein processor 55, memory 520, input/output interface 530, and communication interface 540 enable communication connections within the device between each other via bus 550.
The processor 510 may be implemented by a general-purpose CPU (Central Processing Unit ), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, etc. for executing relevant programs to implement the technical solutions provided in the embodiments of the present disclosure.
The Memory 520 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory ), static storage device, dynamic storage device, or the like. Memory 520 may store an operating system and other application programs, and when the embodiments of the present disclosure are implemented in software or firmware, the associated program code is stored in memory 520 and executed by processor 55.
The input/output interface 530 is used for connecting with an input/output module to realize information input and output. The input/output module may be configured as a component in a device (not shown) or may be external to the device to provide corresponding functionality. Wherein the input devices may include a keyboard, mouse, touch screen, microphone, various types of sensors, etc., and the output devices may include a display, speaker, vibrator, indicator lights, etc.
The communication interface 540 is used to connect with a communication module (not shown in the figure) to enable communication interaction between the present device and other devices. The communication module may implement communication through a wired manner (such as USB, network cable, etc.), or may implement communication through a wireless manner (such as mobile network, WIFI, bluetooth, etc.).
It should be noted that although the above device only shows the processor 510, the memory 520, the input/output interface 530, the communication interface 540, and the bus 550, in the implementation, the device may further include other components necessary for achieving normal operation. Furthermore, it will be understood by those skilled in the art that the above-described apparatus may include only the components necessary to implement the embodiments of the present description, and not all the components shown in the drawings.
The present specification embodiment also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the aforementioned object storage method. The method at least comprises the following steps: carrying out serialization processing on a target object to obtain serialization data of the target object; splitting the serialized data; based on the splitting processing result, a plurality of data fragments of the serialized data are determined, the plurality of data fragments are stored in a database, and index information for representing storage positions of the plurality of data fragments in the database is stored.
The present specification embodiment also provides another computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the aforementioned object reading method. The method at least comprises the following steps: determining a target storage position in a database according to pre-stored index information, wherein the index information is used for representing storage positions of a plurality of data fragments in the database; reading a data fragment from the database based on the target storage location; merging the read data fragments; and performing deserialization on the data obtained by the merging processing to obtain a target object.
Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
From the foregoing description of embodiments, it will be apparent to those skilled in the art that the present embodiments may be implemented in software plus a necessary general purpose hardware platform. Based on such understanding, the technical solutions of the embodiments of the present specification may be embodied in essence or what contributes to the prior art in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the embodiments or some parts of the embodiments of the present specification.
The system, apparatus, module or unit set forth in the above embodiments may be implemented in particular by a computer chip or entity, or by a product having a certain function. A typical implementation device is a computer, which may be in the form of a personal computer, laptop computer, cellular telephone, camera phone, smart phone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or a combination of any of these devices.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for the device embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference is made to the description of the method embodiments for relevant points. The apparatus embodiments described above are merely illustrative, in which the modules illustrated as separate components may or may not be physically separate, and the functions of the modules may be implemented in the same piece or pieces of software and/or hardware when implementing the embodiments of the present disclosure. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
The foregoing is merely a specific implementation of the embodiments of this disclosure, and it should be noted that, for a person skilled in the art, several improvements and modifications may be made without departing from the principles of the embodiments of this disclosure, and these improvements and modifications should also be considered as protective scope of the embodiments of this disclosure.
Claims (8)
1. An object storage method, the method comprising:
carrying out serialization processing on a target object to obtain serialization data of the target object;
splitting the serialized data;
determining a plurality of data fragments of the serialized data based on a splitting processing result, storing the plurality of data fragments in a database, and storing index information for representing storage positions of the plurality of data fragments in the database;
the splitting processing of the serialized data comprises the following steps:
based on a preset splitting rule, a plurality of splitting positions in the serialized data are obtained;
and splitting the serialized data based on the plurality of splitting positions.
2. The method of claim 1, wherein different data slices are stored in different row records of the database, respectively;
the index information includes: the rowkey value recorded in the row where the data slice is located.
3. An object reading method based on the object storing method according to any one of claims 1 to 2, the method comprising:
determining a target storage position in a database according to pre-stored index information, wherein the index information is used for representing storage positions of a plurality of data fragments in the database;
reading a data fragment from the database based on the target storage location;
merging the read data fragments;
performing deserialization on the data obtained by the merging processing to obtain a target object;
the merging processing of the read data fragments comprises the following steps:
the read data fragments are packaged into an input stream.
4. An object storage device, the device comprising:
the serialization module is used for carrying out serialization processing on the target object to obtain serialized data of the target object;
the splitting module is used for splitting the serialized data;
the storage module is used for determining a plurality of data fragments of the serialized data based on a splitting processing result, storing the data fragments into a database and storing index information used for representing storage positions of the data fragments in the database;
the splitting module is specifically used for:
based on a preset splitting rule, a plurality of splitting positions in the serialized data are obtained;
and splitting the serialized data based on the plurality of splitting positions.
5. The apparatus of claim 4, different data fragments being stored in different row records of the database, respectively;
the index information includes: the rowkey value recorded in the row where the data slice is located.
6. An object reading device based on the object storage device of any of claims 4 to 5, the device comprising:
the determining module is used for determining a target storage position in a database according to pre-stored index information, wherein the index information is used for representing storage positions of a plurality of data fragments in the database;
the reading module is used for reading the data fragments from the database based on the target storage position;
the merging module is used for merging the read data fragments;
the anti-serialization module is used for carrying out anti-serialization processing on the data obtained by the merging processing to obtain a target object;
the merging module is specifically configured to:
the read data fragments are packaged into an input stream.
7. A service server comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any of claims 1-2 when the program is executed by the processor.
8. A service server comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of claim 3 when executing the program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811261886.XA CN109614372B (en) | 2018-10-26 | 2018-10-26 | Object storage and reading method and device and service server |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811261886.XA CN109614372B (en) | 2018-10-26 | 2018-10-26 | Object storage and reading method and device and service server |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109614372A CN109614372A (en) | 2019-04-12 |
| CN109614372B true CN109614372B (en) | 2023-06-02 |
Family
ID=66001648
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811261886.XA Active CN109614372B (en) | 2018-10-26 | 2018-10-26 | Object storage and reading method and device and service server |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109614372B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110287199B (en) * | 2019-07-01 | 2021-11-16 | 联想(北京)有限公司 | Database processing method and electronic equipment |
| CN110633283A (en) * | 2019-09-18 | 2019-12-31 | 达疆网络科技(上海)有限公司 | Method for changing data from horizontal expansion to vertical expansion |
| CN111049729A (en) * | 2019-11-29 | 2020-04-21 | 苏州浪潮智能科技有限公司 | Persistent message transmission method and device |
| CN111857890B (en) * | 2020-07-24 | 2021-08-31 | 苏州浪潮智能科技有限公司 | Service processing method, system, device and medium |
| CN114063888B (en) * | 2020-07-31 | 2024-11-26 | 中移(苏州)软件技术有限公司 | Data storage system, data processing method, terminal and storage medium |
| CN114328501A (en) * | 2020-09-29 | 2022-04-12 | 华为技术有限公司 | Data processing method, device and equipment |
| CN113342824A (en) * | 2021-06-30 | 2021-09-03 | 平安资产管理有限责任公司 | Data storage method, device, equipment and medium based on target storage equipment |
| CN115221140A (en) * | 2022-08-12 | 2022-10-21 | 芜湖云一新材料科技有限公司 | Method, system and computer readable medium for storing and reading production data |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101122885A (en) * | 2007-09-11 | 2008-02-13 | 腾讯科技（深圳）有限公司 | Data cache processing method, system and data cache device |
| US8553041B1 (en) * | 2008-09-10 | 2013-10-08 | Nvidia Corporation | System and method for structuring an A-buffer to support multi-sample anti-aliasing |
| CN106933868A (en) * | 2015-12-30 | 2017-07-07 | 阿里巴巴集团控股有限公司 | A kind of method and data server for adjusting data fragmentation distribution |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TW201122527A (en) * | 2009-12-21 | 2011-07-01 | Ind Tech Res Inst | Positioning method and communication system using thereof |
| CN101894238B (en) * | 2010-08-09 | 2012-07-04 | 中国人民解放军海军工程大学 | Double authentication-based word document electronic seal system and method |
| CN104794231B (en) * | 2015-05-04 | 2018-05-25 | 罗敬宁 | A kind for the treatment of method and apparatus of the unstructured data based on Map/Reduce |
| CN105007308B (en) * | 2015-06-24 | 2018-06-19 | 国家电网公司 | A kind of document transmission method under database isolating device environment |
| CN105138649B (en) * | 2015-08-26 | 2018-11-30 | 小米科技有限责任公司 | Searching method, device and the terminal of data |
| CN106326377A (en) * | 2016-08-15 | 2017-01-11 | 东软集团股份有限公司 | Serialization method and device, deserialization method and device, and system |
-
2018
- 2018-10-26 CN CN201811261886.XA patent/CN109614372B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101122885A (en) * | 2007-09-11 | 2008-02-13 | 腾讯科技（深圳）有限公司 | Data cache processing method, system and data cache device |
| US8553041B1 (en) * | 2008-09-10 | 2013-10-08 | Nvidia Corporation | System and method for structuring an A-buffer to support multi-sample anti-aliasing |
| CN106933868A (en) * | 2015-12-30 | 2017-07-07 | 阿里巴巴集团控股有限公司 | A kind of method and data server for adjusting data fragmentation distribution |
Non-Patent Citations (3)
| Title |
|---|
| Matin Hashemi 等.Throughput-memory footprint trade-off in synthesis of streaming software on embedded multiprocessors.《ACM Transactions on Embedded Computing Systems》.2013,第13卷(第3期),1-26. * |
| Xuewen Cui 等.Directive-Based Partitioning and Pipelining for Graphics Processing Units.《2017 IEEE International Parallel and Distributed Processing Symposium (IPDPS)》.2017,575-584. * |
| 黄贵 等.OceanBase分布式存储引擎.《华东师范大学学报(自然科学版)》.2014,(第2014(05)期),164-172. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109614372A (en) | 2019-04-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109614372B (en) | Object storage and reading method and device and service server | |
| TWI743458B (en) | Method, device and system for parallel execution of blockchain transactions | |
| CN109359118B (en) | Data writing method and device | |
| CN111241040B (en) | Information acquisition method and device, electronic equipment and computer storage medium | |
| CN108154211A (en) | Quick Response Code generation, method for processing business, device and equipment and Quick Response Code | |
| CN113885935A (en) | Resource packaging method and device, electronic equipment and computer readable storage medium | |
| CN113873013B (en) | Offline package reorganization method and system | |
| CN108804625B (en) | LSM tree optimization method and device and computer equipment | |
| CN111506670B (en) | Data processing method, device and equipment | |
| CN109542531B (en) | File layout adjustment method and device | |
| CN112379915B (en) | Source code processing method and device | |
| CN115757397A (en) | Data reforming method and device for database table, medium and computer equipment | |
| CN109656805B (en) | Method and device for generating code link for business analysis and business server | |
| CN114925127A (en) | Two-dimensional graph generation method and device for cascade structure data and storage medium | |
| CN115080305A (en) | Data recovery method and system based on repeated reading and electronic equipment | |
| CN113641643A (en) | File writing method and device | |
| WO2020258287A1 (en) | Image correction method, image correction apparatus, and electronic device | |
| CN112653595B (en) | Chip network function testing method and device, storage medium and equipment | |
| CN112035078A (en) | Display method and system of screen content of intelligent terminal and vehicle-mounted terminal | |
| CN111209062A (en) | Information acquisition method and device, electronic equipment and computer storage medium | |
| CN109582449B (en) | Wind control task grouping method and device in wind control service system and computer equipment | |
| CN112463798B (en) | Cross-database data extraction method and device, electronic equipment and storage medium | |
| CN115510783A (en) | Method, apparatus and storage medium for implementing sequential logic user-defined primitive | |
| US20150178377A1 (en) | System for self-distinguishable identifier and generation | |
| CN107391029B (en) | File storage and reading method and corresponding system thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20201012 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant after: Advanced innovation technology Co.,Ltd. Address before: A four-storey 847 mailbox in Grand Cayman Capital Building, British Cayman Islands Applicant before: Alibaba Group Holding Ltd. Effective date of registration: 20201012 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant after: Innovative advanced technology Co.,Ltd. Address before: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Applicant before: Advanced innovation technology Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |