CN114020689B - Data processing method, data processing device, electronic device and storage medium - Google Patents
Data processing method, data processing device, electronic device and storage medium Download PDFInfo
- Publication number
- CN114020689B CN114020689B CN202111329627.8A CN202111329627A CN114020689B CN 114020689 B CN114020689 B CN 114020689B CN 202111329627 A CN202111329627 A CN 202111329627A CN 114020689 B CN114020689 B CN 114020689B
- Authority
- CN
- China
- Prior art keywords
- data
- archive
- file
- archived
- field
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/11—File system administration, e.g. details of archiving or snapshots
- G06F16/113—Details of archiving
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
本公开提供了一种数据处理方法、数据处理装置、电子设备及存储介质，涉及计算机技术领域。该方法应用于文件服务系统，文件服务系统具有多位用户，该方法包括：获取用户上传的待归档数据；将待归档数据写入预设归档模板中，生成归档文件；通过调用业务接口，将归档文件存入第一存储组件中，得到返回的归档文件的标签和存储地址；以及将归档文件的标签和存储地址存入与用户对应的第二存储组件中。
The present disclosure provides a data processing method, a data processing device, an electronic device and a storage medium, which relate to the field of computer technology. The method is applied to a file service system, which has multiple users, and comprises: obtaining data to be archived uploaded by the user; writing the data to be archived into a preset archive template to generate an archive file; storing the archive file in a first storage component by calling a business interface, obtaining a label and a storage address of the returned archive file; and storing the label and the storage address of the archive file in a second storage component corresponding to the user.
Description
Technical Field
The present disclosure relates to the field of computer technology, and more particularly, to a data processing method, a data processing apparatus, an electronic device, a readable storage medium, and a computer program product.
Background
Data archiving is a process of moving data to a single storage device for long-term storage, and can avoid the occurrence of the situation of losing stored data caused by the halt of a host.
In the process of realizing the disclosed conception, the inventor finds that at least the following problems exist in the related technology, namely, because the interfaces of application systems of different users and file service systems are not uniform, the workload of interface development is increased, and the risk of nonstandard interface development exists.
Disclosure of Invention
In view of this, the present disclosure provides a data processing method, a data processing apparatus, an electronic device, a readable storage medium and a computer program product.
One aspect of the disclosure provides a data processing method applied to a file service system, wherein the file service system is provided with a plurality of users, the method comprises the steps of obtaining data to be archived uploaded by the users, writing the data to be archived into a preset archiving template to generate an archived file, storing the archived file into a first storage component through a call service interface to obtain a returned label and a storage address of the archived file, and storing the label and the storage address of the archived file into a second storage component corresponding to the users.
According to the embodiment of the disclosure, the data to be archived comprises a plurality of data fields, wherein the writing of the data to be archived into a preset archiving template generates an archive file comprises the steps of obtaining an archive data table, processing the archive data table based on the plurality of data fields of the data to be archived to obtain imported data, and writing the imported data into the preset archiving template to generate the archive file.
According to the embodiment of the disclosure, the data field of the data to be archived has a first field identification, the empty field in the archive data table has a second field identification, and the processing of the archive data table based on the plurality of data fields of the data to be archived comprises the steps of sequentially matching the first field identification of the data field with the second field identification of the plurality of empty fields in the archive data table for each data field of the data to be archived, and replacing the current empty field in the archive data table with the data field when the first field identification of the data field is matched with the second field identification of the current empty field.
According to the embodiment of the disclosure, the data processing method further comprises the steps of responding to the file downloading request of the user, obtaining a label of a target archive file carried in the file downloading request, determining a storage address corresponding to the label of the target archive file from a second storage component corresponding to the user through calling the service interface, wherein the storage address comprises a pre-signed address, and accessing a first storage component based on the storage address of the target archive file so as to download the target archive file from the first storage component.
According to the embodiment of the disclosure, the method for determining the storage address corresponding to the label of the target archive file from the second storage component corresponding to the user by calling the service interface comprises the steps of acquiring the identity information of the user, calling the service interface, determining the target second storage component corresponding to the user based on the identity information of the user, and determining the storage address corresponding to the label of the target archive file from the target second storage component.
According to the embodiment of the disclosure, the method for determining the storage address corresponding to the label of the target archive file from the second storage component corresponding to the user by calling the service interface further comprises the steps of authenticating the user after calling the service interface, determining the target second storage component corresponding to the user based on the identity information of the user when authentication is successful, and sending feedback information of failure in downloading to the user when authentication is failed.
The data processing device comprises an acquisition module, a writing module, a processing module and a storage module, wherein the acquisition module is used for acquiring data to be archived uploaded by a user, the writing module is used for writing the data to be archived into a preset archiving template to generate an archived file, the processing module is used for storing the archived file into a first storage component through calling a service interface to obtain a returned label and a returned storage address of the archived file, and the storage module is used for storing the label and the storage address of the archived file into a second storage component corresponding to the user.
Another aspect of the disclosure provides an electronic device comprising one or more processors and a memory for storing one or more instructions, wherein the one or more instructions, when executed by the one or more processors, cause the one or more processors to implement the method as described above.
Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions that, when executed, are configured to implement a method as described above.
Another aspect of the present disclosure provides a computer program product comprising computer executable instructions which, when executed, are for implementing a method as described above.
According to the embodiment of the disclosure, by setting the archiving template, data to be archived, which are uploaded by different users through different application systems, can be written into the archiving template to generate an archiving file in a unified format, and the archiving file can be transmitted to the first storage component through the service interface and metadata of the archiving file is stored through the second storage component. By the technical means, the data in different application systems are archived by using the same service interface, the technical problem of increased repeated development workload of the interfaces caused by non-uniform interfaces of the application systems of different users is at least partially solved, further the implementation efficiency of the butt joint of the different users and the file service system is effectively improved, and the development cost and the interface development risk are reduced.
Drawings
The above and other objects, features and advantages of the present disclosure will become more apparent from the following description of embodiments thereof with reference to the accompanying drawings in which:
FIG. 1 schematically illustrates a system architecture of a related art application data processing method;
FIG. 2 schematically illustrates an exemplary system architecture for applying a data processing method according to an embodiment of the present disclosure;
FIG. 3 schematically illustrates a flow chart of a data processing method according to an embodiment of the disclosure;
FIG. 4 schematically illustrates a flow chart of a data processing method according to another embodiment of the present disclosure;
fig. 5 schematically illustrates an application scenario diagram of a data processing method according to an embodiment of the present disclosure;
FIG. 6 schematically illustrates a block diagram of a data processing apparatus according to an embodiment of the present disclosure, and
Fig. 7 schematically illustrates a block diagram of an electronic device implementing a data processing method according to an embodiment of the disclosure.
Detailed Description
Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is only exemplary and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. In addition, in the following description, descriptions of well-known structures and techniques are omitted so as not to unnecessarily obscure the concepts of the present disclosure.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and/or the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.
All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the terms used herein should be construed to have meanings consistent with the context of the present specification and should not be construed in an idealized or overly formal manner.
Where a convention analogous to "at least one of A, B and C, etc." is used, in general such a convention should be interpreted in accordance with the meaning of one of skill in the art having generally understood the convention (e.g., "a system having at least one of A, B and C" would include, but not be limited to, systems having a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).
Fig. 1 schematically shows a schematic diagram of a system architecture to which a data processing method is applied in the related art.
As shown in fig. 1, different users have respective application systems, for example, user a, user B, user C, and user D correspond to application system a, application system B, application system C, and application system D, respectively. Different users can upload structured data and/or unstructured data to a big data cloud platform for archiving through interfaces of an application system and a file service system so as to facilitate the invoking of subsequent data. For example, the user can merge the id of the accessory to be uploaded into a dat file through the application system, call a service interface with the file service system, and push the dat file to the big data cloud platform.
However, the archive data structures of the application systems of the users are not uniform, and have respective service fields and data structures, and part of the application systems archive the unstructured files and simultaneously archive the structured data. In the related art, the interfaces of each application system are usually developed respectively, so that a large amount of repeated development work is generated, and the risks of irregular development and the like are generated while time and labor are consumed.
In view of this, the embodiment of the disclosure provides a standardized, single-interface data docking scheme, which realizes the unification of interfaces of application systems and ensures the safe transmission of data archiving.
In particular, embodiments of the present disclosure provide a data processing method, a data processing apparatus, an electronic device, a readable storage medium, and a computer program product. The method comprises the steps of obtaining data to be archived uploaded by a user, writing the data to be archived into a preset archiving template to generate an archive file, storing the archive file into a first storage component through a call service interface to obtain a label and a storage address of the returned archive file, and storing the label and the storage address of the archive file into a second storage component corresponding to the user.
In the technical scheme of the disclosure, the acquisition, storage, application and the like of the related personal information of the user all conform to the regulations of related laws and regulations, necessary security measures are taken, and the public order harmony is not violated.
Fig. 2 schematically illustrates an exemplary system architecture 200 in which a data processing method may be applied according to an embodiment of the present disclosure. It should be noted that fig. 2 is only an example of a system architecture to which embodiments of the present disclosure may be applied to assist those skilled in the art in understanding the technical content of the present disclosure, but does not mean that embodiments of the present disclosure may not be used in other devices, systems, environments, or scenarios.
As shown in fig. 2, the system architecture 200 according to this embodiment may include terminal devices 201, 202, 203, a network 204, and a server 205. The network 204 is the medium used to provide communication links between the terminal devices 201, 202, 203 and the server 205. The network 204 may include various connection types, such as wired and/or wireless communication links, and the like.
The user may interact with the server 205 using the terminal devices 201, 202, 203 via the network 204 to upload or download archive files, etc. Various client applications may be installed on the terminal devices 201, 202, 203, such as uploading file class applications, downloading class applications, instant messaging tools, mailbox clients and/or social platform software, etc. (by way of example only).
The terminal devices 201, 202, 203 may be a variety of electronic devices having a display screen and supporting the uploading or downloading of archive files, including but not limited to smartphones, tablets, laptop portable computers, desktop computers, and the like.
The server 205 may be a server providing various services, such as a background management server (by way of example only) providing support for archive files uploaded or downloaded by a user using the terminal devices 201, 202, 203. The background management server may analyze and process the received data such as the user file download request, and feed back the processing result (for example, whether the user requests to download the target archive file successfully or not) to the terminal device.
It should be noted that the data processing method provided in the embodiments of the present disclosure may be generally performed by the server 205. Accordingly, the data processing apparatus provided by the embodiments of the present disclosure may be generally disposed in the server 205. The data processing method provided by the embodiments of the present disclosure may also be performed by a server or a server cluster that is different from the server 205 and is capable of communicating with the terminal devices 201, 202, 203 and/or the server 205. Accordingly, the data processing apparatus provided by the embodiments of the present disclosure may also be provided in a server or a server cluster different from the server 205 and capable of communicating with the terminal devices 201, 202, 203 and/or the server 205. Or the data processing method provided by the embodiment of the present disclosure may be performed by the terminal device 201, 202, or 203, or may be performed by another terminal device different from the terminal device 201, 202, or 203. Accordingly, the data processing apparatus provided by the embodiments of the present disclosure may also be provided in the terminal device 201, 202, or 203, or in another terminal device different from the terminal device 201, 202, or 203.
It should be understood that the number of terminal devices, networks and servers in fig. 2 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Fig. 3 schematically illustrates a flow chart of a data processing method according to an embodiment of the present disclosure.
As shown in FIG. 3, the data processing method is applied to a file service system having a plurality of users, and the method includes operations S310 to S340.
It should be noted that, unless there is an execution sequence between different operations or an execution sequence between different operations in technical implementation, the execution sequence between multiple operations may be different, and multiple operations may also be executed simultaneously in the embodiment of the disclosure.
In operation S310, data to be archived uploaded by a user is acquired.
In operation S320, the data to be archived is written into a preset archive template, and an archive file is generated.
In operation S330, the archive file is stored in the first storage component by calling the service interface, and the tag and the storage address of the returned archive file are obtained.
In operation S340, the tag and the storage address of the archive file are stored in a second storage component corresponding to the user.
According to embodiments of the present disclosure, the data to be archived may include structured data and/or unstructured data. Wherein the structured data may characterize data having a fixed format or a limited length, such as database tables, metadata, etc. Unstructured data may characterize data of varying length or in non-fixed format, such as business data for mail, word, excel, and pdf documents.
According to embodiments of the present disclosure, the preset archive template may be set based on the interface configuration of all application systems interfacing with the file service system.
According to an embodiment of the present disclosure, the preset archive template may be a Dat file, and the archive file generated accordingly may also be a Dat file.
According to embodiments of the present disclosure, the business interface may include, but is not limited to, a data acquisition component of a big data cloud platform, such as an application programming interface, e.g., an API. The first storage component may include a cloud archive server side of a big data cloud platform, such as an object storage service (Cloud Object Service, COS) or a data storage system (Hadoop Distributed FILE SYSTEM, HDFS). The second storage component may include a data management component of a big data cloud platform.
According to the embodiment of the disclosure, by setting the archiving template, data to be archived, which are uploaded by different users through different application systems, can be written into the archiving template to generate an archiving file in a unified format, and the archiving file can be transmitted to the first storage component through the service interface and metadata of the archiving file is stored through the second storage component. By the technical means, the data in different application systems are archived by using the same service interface, the technical problem of increased repeated development workload of the interfaces caused by non-uniform interfaces of the application systems of different users is at least partially solved, further the implementation efficiency of the butt joint of the different users and the file service system is effectively improved, and the development cost and the interface development risk are reduced.
According to an embodiment of the present disclosure, the data to be archived includes a plurality of data fields.
According to an embodiment of the present disclosure, writing data to be archived in a preset archive template to generate an archive file may include the operations of obtaining an archive data table, wherein the archive data table includes a plurality of empty fields arranged in a preset order. The archive data table is processed based on a plurality of data fields of the data to be archived, resulting in imported data. The imported data is written into a preset archive template to generate an archive file.
According to embodiments of the present disclosure, the archive data table may be configured with field attributes of the type, length, whether empty, order, etc. of each field.
According to embodiments of the present disclosure, archive data tables may be processed using a variety of policies. For example, the data field of the data to be archived may have a first field identification, the empty field in the archive data table may have a second field identification, the first field identification and the second field identification may be names, primary keys, unique keys, etc. of the fields, without limitation, the first field identification of the data field may be matched with the second field identifications of the plurality of empty fields in the archive data table for each data field of the data to be archived in turn, and the current empty field in the archive data table may be replaced with the data field if the first field identification of the data field matches with the second field identification of the current empty field.
According to an embodiment of the disclosure, after writing the imported data into the preset archive template, unstructured data contained in the archive data may be packaged into a tar package, and a path of the tar package is added in a last field of the preset archive template, so as to generate an archive file, where metadata of the archive file may include information such as an ip port of an incoming server, a path of the tar package, and the like.
Fig. 4 schematically shows a flow chart of a data processing method according to another embodiment of the present disclosure.
As shown in FIG. 4, the data processing method may further include operations S410-S430.
In operation S410, in response to the file download request of the user, a tag of the target archive file carried in the file download request is acquired.
In operation S420, a storage address corresponding to the tag of the target archive file is determined from the second storage component corresponding to the user by invoking the service interface, wherein the storage address includes a pre-signed address.
In operation S430, the first storage component is accessed based on the storage address of the target archive file to download the target archive file from the first storage component.
According to embodiments of the present disclosure, an archive file has metadata from which a user to whom the archive file belongs can be determined.
According to an embodiment of the present disclosure, determining a storage address corresponding to a tag of a target archive file from a second storage component corresponding to a user by invoking a service interface may include the operations of obtaining identity information of the user. And calling a service interface, and determining a target second storage component corresponding to the user based on the identity information of the user. A storage address corresponding to a tag of the target archive file is determined from the target second storage component.
According to an embodiment of the present disclosure, determining a storage address corresponding to a tag of the target archive file from a second storage component corresponding to the user by invoking the service interface may further include authenticating the user after invoking the service interface. And under the condition that the authentication is successful, determining a target second storage component corresponding to the user based on the identity information of the user. And under the condition that authentication fails, sending feedback information of failure in downloading to the user.
In accordance with an embodiment of the present disclosure, in the case where a user downloads a target archive file, it is necessary to determine whether the user has rights to access the second storage component. And under the condition that authentication is successful, determining a target second storage component corresponding to the user according to the identity information of the user, thereby acquiring the target archive file from the target second storage component. And under the condition of authentication failure, sending feedback information to the user so as to remind the user that the file downloading request cannot be completed. Wherein a successful authentication characterizes the user as having the right to access the second storage component and a failed authentication characterizes the user as not having the right to access the second storage component.
Fig. 5 schematically illustrates an application scenario of a data processing method according to an embodiment of the present disclosure.
As shown in fig. 5, the data to be archived 501 uploaded by the user may generate an archive file 502 according to the method of operation S302. In the process of storing the archive file 502, the archive file is firstly stored in the first storage component 504 appointed by the user by calling the service interface 503, after the first storage component 504 completes file storage, the storage address and the file label of the archive file 502 are returned, and then the storage address, the file label and other metadata of the archive file 502 can be stored in the second storage component 505.
On the other hand, after the storage of the archive file 502 is completed, the service interface 503 may transmit service related data to the external platform 506, so that a user of the external system may download the archive file 502 through the external platform.
Upon receiving a request from any user to download the target archive file 507, the storage address of the target archive file 507 may be determined from the second storage component 505 by the tag of the target archive file 507 carried in the request. Upon accessing the second storage component 505 through the service interface 503, the second storage component 505 authenticates the user to determine whether the user has the right to obtain the target archive 507, and after successful authentication, the second storage component 505 returns the pre-signed storage address of the second storage component 505 to the service interface 503. Based on the storage address, a target archive file 507 may be obtained from the corresponding first storage component 504.
Fig. 6 schematically shows a block diagram of a data processing apparatus according to an embodiment of the disclosure.
As shown in fig. 6, the data processing apparatus 600 may include an acquisition module 610, a writing module 620, a processing module 630, and a storage module 640.
And the acquiring module 610 is configured to acquire data to be archived uploaded by a user.
The writing module 620 is configured to write the data to be archived in a preset archive template, and generate an archive file.
And the processing module 630 is configured to store the archive file in the first storage component by calling the service interface, and obtain a tag and a storage address of the returned archive file.
And the storage module 640 is used for storing the label and the storage address of the archived file into a second storage component corresponding to the user.
According to the embodiment of the disclosure, by setting the archiving template, data to be archived, which are uploaded by different users through different application systems, can be written into the archiving template to generate an archiving file in a unified format, and the archiving file can be transmitted to the first storage component through the service interface and metadata of the archiving file is stored through the second storage component. By the technical means, the data in different application systems are archived by using the same service interface, the technical problem of increased repeated development workload of the interfaces caused by non-uniform interfaces of the application systems of different users is at least partially solved, further the implementation efficiency of the butt joint of the different users and the file service system is effectively improved, and the development cost and the interface development risk are reduced.
According to an embodiment of the present disclosure, the data to be archived includes a plurality of data fields.
According to an embodiment of the present disclosure, the writing module 620 may include a first acquisition unit, a processing unit, and a writing unit.
And the first acquisition unit is used for acquiring an archive data table, wherein the archive data table comprises a plurality of empty fields arranged in a preset sequence.
And the processing unit is used for processing the archive data table based on a plurality of data fields of the data to be archived to obtain imported data.
And the writing unit is used for writing the imported data into a preset archive template to generate an archive file.
According to an embodiment of the present disclosure, a data field of data to be archived has a first field identification and a null field in an archive data table has a second field identification.
According to an embodiment of the present disclosure, the processing unit may include a first processing subunit and a second processing subunit.
And the first processing subunit is used for matching the first field identification of the data field with the second field identifications of the plurality of empty fields in the archive data table for each data field of the data to be archived in sequence.
And the second processing subunit is used for replacing the current null field in the archive data table into the data field under the condition that the first field identification of the data field is matched with the second field identification of the current null field. According to an embodiment of the present disclosure, the data processing apparatus 600 may further include a second acquisition module, a determination module, and a download module.
And the second acquisition module is used for responding to the file downloading request of the user and acquiring the tag of the target archive file carried in the file downloading request.
And the determining module is used for determining a storage address corresponding to the label of the target archive file from the second storage component corresponding to the user by calling the service interface, wherein the storage address comprises a pre-signature address.
And the downloading module is used for accessing the first storage component based on the storage address of the target archive file so as to download the target archive file from the first storage component.
According to an embodiment of the present disclosure, the determining module may include a second acquiring unit, a first determining unit, and a second determining unit.
And the second acquisition unit is used for acquiring the identity information of the user.
And the first determining unit is used for calling the service interface and determining a target second storage component corresponding to the user based on the identity information of the user.
And the second determining unit is used for determining a storage address corresponding to the label of the target archive file from the target second storage component.
According to an embodiment of the present disclosure, the determining module may further include an authentication unit, a third determining unit, and a transmitting unit.
And the authentication unit is used for authenticating the user after the service interface is called.
And the third determining unit is used for determining a target second storage component corresponding to the user based on the identity information of the user under the condition that the authentication is successful.
And the sending unit is used for sending feedback information of the downloading failure to the user under the condition of authentication failure.
Any number of the modules, units, sub-units, or at least some of the functionality of any number of the modules, units, sub-units, or sub-units according to embodiments of the present disclosure may be implemented in one module. Any one or more of the modules, units, sub-units according to embodiments of the present disclosure may be implemented as split into multiple modules. Any one or more of the modules, units, sub-units according to embodiments of the present disclosure may be implemented at least in part as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or in hardware or firmware in any other reasonable manner of integrating or packaging the circuits, or in any one of or in any suitable combination of three of software, hardware, and firmware. Or one or more of the modules, units, sub-units according to embodiments of the disclosure may be at least partially implemented as computer program modules which, when executed, may perform the corresponding functions.
For example, any of the acquisition module 610, the writing module 620, the processing module 630, and the storage module 640 may be combined in one module/unit/sub-unit or any of the modules/units/sub-units may be split into multiple modules/units/sub-units. Or at least some of the functionality of one or more of these modules/units/sub-units may be combined with at least some of the functionality of other modules/units/sub-units and implemented in one module/unit/sub-unit. According to embodiments of the present disclosure, at least one of the acquisition module 610, the write module 620, the processing module 630, and the storage module 640 may be implemented at least in part as hardware circuitry, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented in hardware or firmware in any other reasonable way of integrating or packaging circuitry, or in any one of or a suitable combination of three of software, hardware, and firmware. Or at least one of the acquisition module 610, the writing module 620, the processing module 630 and the storage module 640 may be at least partially implemented as a computer program module, which when executed may perform the corresponding functions.
It should be noted that, in the embodiments of the present disclosure, the data processing apparatus portion corresponds to the data processing method portion in the embodiments of the present disclosure, and the description of the data processing apparatus portion specifically refers to the data processing method portion and is not described herein.
Fig. 7 schematically illustrates a block diagram of an electronic device adapted to implement a data processing method according to an embodiment of the disclosure. The electronic device shown in fig. 7 is merely an example and should not be construed to limit the functionality and scope of use of the disclosed embodiments.
As shown in fig. 7, a computer electronic device 700 according to an embodiment of the present disclosure includes a processor 701 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 702 or a program loaded from a storage section 708 into a Random Access Memory (RAM) 703. The processor 701 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or an associated chipset and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), or the like. The processor 701 may also include on-board memory for caching purposes. The processor 701 may comprise a single processing unit or a plurality of processing units for performing different actions of the method flows according to embodiments of the disclosure.
In the RAM 703, various programs and data necessary for the operation of the electronic apparatus 700 are stored. The processor 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. The processor 701 performs various operations of the method flow according to the embodiments of the present disclosure by executing programs in the ROM 702 and/or the RAM 703. Note that the program may be stored in one or more memories other than the ROM 702 and the RAM 703. The processor 701 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in the one or more memories.
According to an embodiment of the present disclosure, the electronic device 700 may further include an input/output (I/O) interface 705, the input/output (I/O) interface 705 also being connected to the bus 704. The electronic device 700 may also include one or more of an input portion 706 including a keyboard, mouse, etc., an output portion 707 including a Cathode Ray Tube (CRT), liquid Crystal Display (LCD), etc., and speaker, etc., a storage portion 708 including a hard disk, etc., and a communication portion 709 including a network interface card such as a LAN card, modem, etc., connected to the I/O interface 705. The communication section 709 performs communication processing via a network such as the internet. The drive 710 is also connected to the I/O interface 705 as needed. A removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 710 as necessary, so that a computer program read therefrom is mounted into the storage section 708 as necessary.
According to embodiments of the present disclosure, the method flow according to embodiments of the present disclosure may be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable storage medium, the computer program comprising program code for performing the method shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via the communication portion 709, and/or installed from the removable medium 711. The above-described functions defined in the system of the embodiments of the present disclosure are performed when the computer program is executed by the processor 701. The systems, devices, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the disclosure.
The present disclosure also provides a computer-readable storage medium that may be included in the apparatus/device/system described in the above embodiments, or may exist alone without being assembled into the apparatus/device/system. The computer-readable storage medium carries one or more programs which, when executed, implement methods in accordance with embodiments of the present disclosure.
According to embodiments of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. Such as, but not limited to, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
For example, according to embodiments of the present disclosure, the computer-readable storage medium may include ROM 702 and/or RAM 703 and/or one or more memories other than ROM 702 and RAM 703 described above.
Embodiments of the present disclosure also include a computer program product comprising a computer program comprising program code for performing the methods provided by the embodiments of the present disclosure, the program code for causing an electronic device to implement the data processing methods provided by the embodiments of the present disclosure when the computer program product is run on the electronic device.
The above-described functions defined in the system/apparatus of the embodiments of the present disclosure are performed when the computer program is executed by the processor 701. The systems, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the disclosure.
In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, or the like. In another embodiment, the computer program may also be transmitted, distributed over a network medium in the form of signals, downloaded and installed via the communication section 709, and/or installed from the removable medium 711. The computer program may comprise program code that is transmitted using any appropriate network medium, including but not limited to wireless, wireline, etc., or any suitable combination of the preceding.
According to embodiments of the present disclosure, program code for performing computer programs provided by embodiments of the present disclosure may be written in any combination of one or more programming languages, and in particular, such computer programs may be implemented in high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. Programming languages include, but are not limited to, such as Java, c++, python, "C" or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user's computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected through the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions. Those skilled in the art will appreciate that the features recited in the various embodiments of the disclosure and/or in the claims may be combined in various combinations and/or combinations, even if such combinations or combinations are not explicitly recited in the disclosure. In particular, the features recited in the various embodiments of the present disclosure and/or the claims may be variously combined and/or combined without departing from the spirit and teachings of the present disclosure. All such combinations and/or combinations fall within the scope of the present disclosure.
The embodiments of the present disclosure are described above. These examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the embodiments cannot be used advantageously in combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be made by those skilled in the art without departing from the scope of the disclosure, and such alternatives and modifications are intended to fall within the scope of the disclosure.
Claims (8)
1. A data processing method applied to a file service system having a plurality of users, the method comprising:
acquiring data to be archived uploaded by the user;
Writing the data to be archived into a preset archiving template to generate an archiving file;
storing the archive file in a first storage component by calling a service interface to obtain a returned tag and storage address of the archive file, and
Storing the label and the storage address of the archive file into a second storage component corresponding to the user;
Wherein the data to be archived includes a plurality of data fields;
The writing the data to be archived in a preset archiving template to generate an archiving file includes:
acquiring an archive data table, wherein the archive data table comprises a plurality of empty fields arranged according to a preset sequence;
processing the archive data table based on the plurality of data fields of the data to be archived to obtain imported data, and
Writing the imported data into the preset archive template to generate the archive file;
The data field of the data to be archived has a first field identification, and the empty field in the archived data table has a second field identification;
Wherein the processing the archive data table based on the plurality of data fields of the data to be archived comprises:
for each data field of the data to be archived, matching a first field identification of the data field with a second field identification of a plurality of empty fields in the archived data table, and
And replacing the current null field in the archive data table with the data field under the condition that the first field identification of the data field is matched with the second field identification of the current null field.
2. The method of claim 1, further comprising:
responding to a file downloading request of the user, and acquiring a tag of a target archive file carried in the file downloading request;
Determining a storage address corresponding to a tag of the target archive file from a second storage component corresponding to the user by invoking the service interface, wherein the storage address comprises a pre-signed address, and
Accessing a first storage component based on a storage address of the target archive file to download the target archive file from the first storage component.
3. The method of claim 2, wherein the determining, by invoking the business interface, a storage address corresponding to a tag of the target archive file from a second storage component corresponding to the user comprises:
acquiring identity information of the user;
Invoking the service interface, determining a target second storage component corresponding to the user based on the identity information of the user, and
And determining a storage address corresponding to the label of the target archive file from the target second storage component.
4. A method according to claim 3, further comprising:
Authenticating the user after the service interface is called;
determining the target second storage component corresponding to the user based on the identity information of the user in case of successful authentication, and
And under the condition of authentication failure, sending feedback information of failure in downloading to the user.
5. A data processing apparatus comprising:
the acquisition module is used for acquiring data to be archived uploaded by a user;
the writing module is used for writing the data to be archived into a preset archiving template to generate an archiving file;
A processing module for storing the archive file into a first storage component by calling a service interface to obtain a returned label and storage address of the archive file, and
The storage module is used for storing the label and the storage address of the archive file into a second storage component corresponding to the user;
Wherein the data to be archived includes a plurality of data fields;
wherein, write module, include:
The first acquisition unit is used for acquiring an archive data table, wherein the archive data table comprises a plurality of empty fields arranged according to a preset sequence;
a processing unit for processing the archive data table based on the multiple data fields of the data to be archived to obtain imported data, and
The writing unit is used for writing the imported data into the preset archiving template to generate the archiving file;
The data field of the data to be archived has a first field identification, and the empty field in the archived data table has a second field identification;
wherein the processing unit comprises:
A first processing subunit, configured to match, for each data field of the data to be archived in turn, a first field identifier of the data field with a second field identifier of a plurality of empty fields in the archive data table, and
And the second processing subunit is used for replacing the current null field in the archive data table with the data field under the condition that the first field identification of the data field is matched with the second field identification of the current null field.
6. An electronic device, comprising:
One or more processors;
a memory for storing one or more instructions,
Wherein the one or more instructions, when executed by the one or more processors, cause the one or more processors to implement the method of any of claims 1 to 4.
7. A computer readable storage medium having stored thereon executable instructions which when executed by a processor cause the processor to implement the method of any of claims 1 to 4.
8. A computer program product comprising computer-executable instructions, the computer executable instructions, when executed, are for implementing the method of any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111329627.8A CN114020689B (en) | 2021-11-10 | 2021-11-10 | Data processing method, data processing device, electronic device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111329627.8A CN114020689B (en) | 2021-11-10 | 2021-11-10 | Data processing method, data processing device, electronic device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114020689A CN114020689A (en) | 2022-02-08 |
| CN114020689B true CN114020689B (en) | 2024-12-20 |
Family
ID=80063253
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111329627.8A Active CN114020689B (en) | 2021-11-10 | 2021-11-10 | Data processing method, data processing device, electronic device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114020689B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114661722A (en) * | 2022-03-23 | 2022-06-24 | 天津同阳科技发展有限公司 | Data storage method, data indexing method and device |
| CN114979056B (en) * | 2022-05-20 | 2023-12-29 | 甬矽电子(宁波)股份有限公司 | E-mail processing method and device, storage medium and electronic equipment |
| CN116737663A (en) * | 2023-06-29 | 2023-09-12 | 中国银行股份有限公司 | A file archiving method, device, equipment and readable storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111581171A (en) * | 2020-04-30 | 2020-08-25 | 中国工商银行股份有限公司 | Log processing method and device, computing equipment and medium |
| CN112732641A (en) * | 2020-12-30 | 2021-04-30 | 山东浪潮通软信息科技有限公司 | Electronic file filing method, device and medium |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7769709B2 (en) * | 2004-09-09 | 2010-08-03 | Microsoft Corporation | Method, system, and apparatus for creating an archive routine for protecting data in a data protection system |
| CN104090987B (en) * | 2014-07-28 | 2018-03-13 | 华中科技大学 | A kind of history data store and indexing means |
| US11281621B2 (en) * | 2018-01-08 | 2022-03-22 | International Business Machines Corporation | Clientless active remote archive |
| CN110716895B (en) * | 2019-09-17 | 2023-05-26 | 平安科技（深圳）有限公司 | Target data archiving method, device, computer equipment and medium |
| CN113610396B (en) * | 2021-08-06 | 2022-02-11 | 三峡高科信息技术有限责任公司 | Method and system for structuring matrix designer based on construction quality acceptance table |
-
2021
- 2021-11-10 CN CN202111329627.8A patent/CN114020689B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111581171A (en) * | 2020-04-30 | 2020-08-25 | 中国工商银行股份有限公司 | Log processing method and device, computing equipment and medium |
| CN112732641A (en) * | 2020-12-30 | 2021-04-30 | 山东浪潮通软信息科技有限公司 | Electronic file filing method, device and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114020689A (en) | 2022-02-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11368447B2 (en) | Oauth2 SAML token service | |
| US20130067100A1 (en) | Multi-desktop interaction using nested remote desktop sessions | |
| CN111400061A (en) | A data processing method and system | |
| CN114020689B (en) | Data processing method, data processing device, electronic device and storage medium | |
| US20110167114A1 (en) | Automatically synchronizing new contacts across multiple social networking sites | |
| CN111598575B (en) | Business process control method, business process control device, electronic equipment and readable storage medium | |
| US11025623B2 (en) | Updating database drivers for client applications through a database server push | |
| US10586187B2 (en) | Managing assets | |
| US11470068B2 (en) | System and methods for securely storing data for efficient access by cloud-based computing instances | |
| US20170324686A1 (en) | System and method for secure and efficient communication within an organization | |
| US9948694B2 (en) | Addressing application program interface format modifications to ensure client compatibility | |
| CN112559024A (en) | Method and device for generating transaction code change list | |
| CN113132400B (en) | Business processing method, device, computer system and storage medium | |
| CN108022079A (en) | Sharing method, device and the computer-readable recording medium of Service Source | |
| CN107657155B (en) | Method and device for authenticating user operation authority | |
| US10997004B2 (en) | Detecting co-resident services in a container cloud | |
| CN107077490B (en) | Data query job submission management | |
| CN115098840B (en) | An identity authentication method, device, equipment, medium and product | |
| CN116737662A (en) | Business data processing methods, devices, electronic equipment and storage media | |
| CN116975200A (en) | Methods, devices, equipment and media for controlling the working status of servers | |
| US11429362B2 (en) | Device experience application launcher | |
| CN114189365A (en) | Universal multi-tenant service authorization method and device based on field mapping | |
| US20200364093A1 (en) | System and methods for generating secure ephemeral cloud-based computing resources for data operations | |
| CN112835609A (en) | A method and device for modifying the download address of a dependency package | |
| CN113778969B (en) | A log processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |