CN116894023A - Data migration method and device, storage medium and electronic device - Google Patents

Abstract

Embodiments of the present application provide a data migration method and device, a storage medium and an electronic device, wherein the method includes: receiving a first instruction triggered by touching a first button on a target display interface; Triggered by the first instruction, the target interface for managing the source data set and the target data set is called; receiving the second instruction triggered by touching the second button on the target display interface; triggered by the second instruction , migrate the target data in the source data set to the target data set. Through this application, the problems of low data migration efficiency and poor versatility existing in related technologies are solved.

Description

Data migration method and device, storage medium and electronic device

Technical field

Embodiments of the present application relate to the field of computers, specifically, to a data migration method and device, a storage medium, and an electronic device.

Background technique

With the development of data technology, for example, the development of big data technology, in the context of the rapid explosion of data information in the Internet era, the era of big data has arrived, and big data has also become the trend of future technology development. The research organization Gartner gave "big data" The following definition is made: Big data is a massive, high-growth and diverse information asset that requires new processing models to have stronger decision-making power, insight discovery and process optimization capabilities.

In the big data environment, due to the development of user business, business relationships and other reasons, big data migration will be involved, that is, the data in the big data cluster needs to be migrated to other big data clusters. However, some manufacturers currently use open source Migration tools such as distcp are used to migrate underlying commands. The steps are cumbersome and the work efficiency is low. Moreover, the migration operation needs to be performed by professionals. It has limited applicability and poor versatility.

To address the problems of low data migration efficiency and poor versatility in related technologies, no effective solution has yet been proposed.

Contents of the invention

Embodiments of the present application provide a data migration method and device, a storage medium, and an electronic device to at least solve the problems of low data migration efficiency and poor versatility existing in related technologies.

According to an embodiment of the present application, a data migration method is provided, including: receiving a first instruction triggered by touching a first button on a target display interface; and calling a user when triggered by the first instruction. A target interface for managing the source data set and the target data set; receiving a second instruction triggered by touching a second button on the target display interface; triggered by the second instruction, converting the source data set into The target data is migrated into the target data set.

In an optional embodiment, after migrating the data in the source data set to the target data set, the method further includes: receiving a third button triggered by touching a third button on the target display interface. Three instructions: when triggered by the third instruction, display information on the data migrated to the target data set on the target display interface.

In an optional embodiment, when HADOOP data is stored in the source data set: calling the target interface for managing the source data set and the target data set triggered by the first instruction includes: in the Triggered by the first instruction, the HADOOP interface for managing the source data set and the target data set is called; triggered by the second instruction, the data in the source data set is migrated to the target data Concentrating includes: triggering the second instruction, creating a first migration task, wherein the first migration task is used to select the target data in the source data set and transfer the target data to the target data set.

In an optional embodiment, when HIVE data is stored in the source data set: calling the target interface for managing the source data set and the target data set triggered by the first instruction includes: in the Triggered by the first instruction, the HDFS and HIVE interfaces used to manage the source data set and the target data set are called; triggered by the second instruction, the data in the source data set is migrated to the The target data set includes: under the trigger of the second instruction, exporting the target data in the source data set to the target data set through the HDFS file system.

In an optional embodiment, when triggered by the second instruction, exporting the target data in the source data set to the target data set through the HDFS file system includes: in the second instruction When triggered, perform the following operations: create an export task, wherein the export task is used to export the target data in the source data set to the source HDFS file system; create a second migration task, wherein the second The migration task is used to migrate the target data on the source HDFS file system to the target HDFS file system; create an import task, wherein the import task is used to migrate the target data on the target HDFS file system. Import into the target dataset.

In an optional embodiment, when HBASE data is stored in the source data set: calling the target interface for managing the source data set and the target data set triggered by the first instruction includes: in the Triggered by the first instruction, the HDFS and HBASE interfaces used to manage the source data set and the target data set are called; triggered by the second instruction, the data in the source data set is migrated to the The target data set includes: under the trigger of the second instruction, exporting the target data in the source data set to the target data set through the HDFS file system.

In an optional embodiment, when triggered by the second instruction, exporting the target data in the source data set to the target data set through the HDFS file system includes: in the second instruction When triggered, perform the following operations: create an HBASE snapshot task, where the HBASE snapshot task is used to create an HBASE snapshot for the data in the source data set; create an HBASE snapshot export and migration task, where the HBASE snapshot export and migration The task is used to export the HBASE snapshot to the source HDFS file system, and migrate the HBASE snapshot on the source HDFS file system to the target HDFS file system; create a table task, wherein the table task is used to Create an HBASE table on the target data set according to the table attributes in the source data set; create a restore snapshot task, wherein the restore snapshot task is used to restore the HBASE snapshot on the target HDFS file system to the target HDFS file system. described in the HBASE table created on the target data set.

According to another embodiment of the present application, a data migration device is provided, including: a first receiving module, configured to receive a first instruction triggered by touching a first button on a target display interface; a calling module, configured to call a target interface for managing the source data set and the target data set when triggered by the first instruction; a second receiving module configured to receive a third command triggered by touching a second button on the target display interface; Two instructions; a migration module, configured to migrate the target data in the source data set to the target data set when triggered by the second instruction.

In an optional embodiment, the device further includes a third receiving module, configured to receive a third message on the target display interface through touch after migrating the data in the source data set to the target data set. A third instruction triggered by the three buttons; when triggered by the third instruction, information about data migrated to the target data set is displayed on the target display interface.

In an optional embodiment, when HADOOP data is stored in the source data set: the calling module includes: a first calling unit, configured to call a method for managing all data triggered by the first instruction. The HADOOP interface of the source data set and the target data set; the migration module includes: a creation unit, configured to create a first migration task triggered by the second instruction, wherein the first migration task uses Select the target data in the source data set and transfer the target data to the target data set.

In an optional embodiment, when HIVE data is stored in the source data set: the calling module includes: a second calling unit, configured to call, under the trigger of the first instruction, a method for managing all the HIVE data. The HDFS and HIVE interfaces of the source data set and the target data set; the migration module includes: a first export unit, configured to transfer the source data set through the HDFS file system under the trigger of the second instruction. The target data is exported into the target data set.

In an optional embodiment, the first export unit includes: a first execution sub-unit, configured to perform the following operations when triggered by the second instruction: create an export task, wherein the export task is Exporting the target data in the source data set to the source HDFS file system; creating a second migration task, wherein the second migration task is used to migrate the target data on the source HDFS file system to the target HDFS file system; create an import task, where the import task is used to import the target data on the target HDFS file system into the target data set.

In an optional embodiment, when HBASE data is stored in the source data set: the calling module includes: a third calling unit, configured to call, under the trigger of the first instruction, a method for managing all The HDFS and HBASE interfaces of the source data set and the target data set; the migration module includes: a second export unit, configured to export the source data set through the HDFS file system under the trigger of the second instruction. The target data is exported into the target data set.

In an optional embodiment, the second export unit includes: a second execution subunit, configured to perform the following operations when triggered by the second instruction: create an HBASE snapshot task, wherein the HBASE snapshot The task is used to create an HBASE snapshot for the data in the source data set; create an HBASE snapshot export and migration task, wherein the HBASE snapshot export and migration task is used to export the HBASE snapshot to the source HDFS file system, and Migrate the HBASE snapshot on the source HDFS file system to the target HDFS file system; create a table task, wherein the table task is used to create HBASE on the target data set according to table attributes in the source data set Table; create a restore snapshot task, where the restore snapshot task is used to restore the HBASE snapshot on the target HDFS file system to the HBASE table created on the target data set.

According to yet another embodiment of the present application, a computer-readable storage medium is also provided. A computer program is stored in the computer-readable storage medium, wherein the computer program is configured to execute any of the above methods when running. Steps in Examples.

According to yet another embodiment of the present application, an electronic device is also provided, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to perform any of the above. Steps in method embodiments.

Through this application, what is used is to receive the first instruction triggered by touching the first button on the target display interface; and call the target interface for managing the source data set and the target data set when triggered by the first instruction. ; Receive a second instruction triggered by touching the second button on the target display interface; and, triggered by the second instruction, migrate the target data in the source data set to the target data set, That is, buttons for performing relevant data migration operations will be displayed on the target display interface, and corresponding functions will be linked to each relevant button in advance. Therefore, when data migration is required, data migration can be achieved by simply touching the relevant buttons. The purpose of migration is to make the operation simpler and more convenient for users. Data migration can be achieved without writing professional underlying commands, thereby expanding the use of data migration, improving the versatility of data migration, and improving data migration to a certain extent. It improves the work efficiency of data migration and effectively solves the problems of low data migration efficiency and poor versatility in related technologies.

Description of the drawings

Figure 1 is a hardware structure block diagram of a mobile terminal of a data migration method according to an embodiment of the present application;

Figure 2 is a flow chart of a data migration method according to an embodiment of the present application;

Figure 3 is a schematic diagram of the migration process of HBASE data according to an embodiment of the present application;

Figure 4 is a structural block diagram of a data migration device according to an embodiment of the present application.

Detailed ways

The embodiments of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

It should be noted that the terms "first", "second", etc. in the description and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

First, the terminology involved in the embodiment of the present invention is explained:

HDFS: The full name is Hadoop Distributed File System, that is, distributed file system. This system refers to a distributed file system (DistributedFile System) designed to run on common hardware (commodity hardware). It has a lot in common with existing distributed file systems. But at the same time, the difference between it and other distributed file systems is also obvious. HDFS is a highly fault-tolerant system suitable for deployment on cheap machines. HDFS can provide high-throughput data access and is very suitable for applications on large-scale data sets. HDFS relaxes some POSIX constraints to achieve the purpose of streaming file system data. HDFS was originally developed as the infrastructure for the Apache Nutch search engine project. HDFS is part of the Apache Hadoop Core project. In addition, HDFS has high fault-tolerant characteristics and is designed to be deployed on low-cost hardware. And it provides high throughput to access application data, suitable for applications with large data sets. HDFS relaxes POSIX requirements so that streaming access to data in the file system can be achieved.

HIVE: HIVE (also called hive) is a data warehouse tool based on Hadoop, used for data extraction, transformation, and loading. This is a mechanism that can store, query, and analyze large-scale data stored in Hadoop. The HIVE data warehouse tool can map structured data files into a database table, and provides SQL query functions, which can convert SQL statements into MapReduce tasks for execution. The advantage of Hive is that it has low learning cost and can implement fast MapReduce statistics through SQL-like statements, making MapReduce simpler without having to develop a specialized MapReduce application. hive is very suitable for statistical analysis of data warehouses.

HBASE: HBASE (also known as HBase) is a distributed, column-oriented open source database. This technology comes from the paper "Bigtable: A distributed storage system for structured data" written by Fay Chang. Just like Bigtable utilizes the distributed data storage provided by the File System, HBase provides Bigtable-like capabilities on top of Hadoop. HBase is a subproject of Apache's Hadoop project. HBase is different from general relational databases in that it is a database suitable for unstructured data storage. Another difference is that HBase is column-based rather than row-based.

Distcp: (Distributed Copy, also known as DistCp) is a tool for large inter-cluster/intra-cluster replication. The effect of executing cp and scp on Linux is the same. The difference is that cp copies the files and directories of the local machine to other places on the machine, while scp can copy the files or directories of machine A to machine B. What Distcp can achieve is copying the data of cluster A (HDFS) to cluster B (HDFS). When the data is copied in a distributed manner, the DN node of level A group can send data to the DN node of cluster B at the same time, which is a breakthrough. The network card rate limit for stand-alone copying makes copying more efficient. At the same time, Distcp uses Map/Reduce tasks to implement file distribution, error handling and recovery, and report generation. It uses the list of files and directories as the input of the map task, and each task will complete the copy of some files in the source list (actually Distcp only uses map, not reduce, and reads data from the old cluster in the map task. Then write it to the new cluster to complete the data migration).

Snapshot: Snapshot technology is the core backup and recovery tool for the HBase database. It is widely used in backup and has been applied to arrays and hosts very early. It mainly uses the Copy on Write algorithm, which is usually volume-based and processed at the block level.

How the present invention solves the problems existing in related technologies will be described below in conjunction with the embodiments:

The method embodiments provided in the embodiments of this application can be executed in a mobile terminal, a computer terminal, or a similar computing device. Taking running on a mobile terminal as an example, FIG. 1 is a hardware structure block diagram of a mobile terminal according to the data migration method according to the embodiment of the present application. As shown in Figure 1, the mobile terminal may include one or more (only one is shown in Figure 1) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the above-mentioned mobile terminal may also include a transmission device 106 and an input and output device 108 for communication functions. Persons of ordinary skill in the art can understand that the structure shown in Figure 1 is only illustrative, and it does not limit the structure of the above-mentioned mobile terminal. For example, the mobile terminal may also include more or fewer components than shown in FIG. 1 , or have a different configuration than shown in FIG. 1 .

The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer programs corresponding to the data migration method in the embodiment of the present application. The processor 102 executes the computer program by running the computer program stored in the memory 104. Various functional applications and data processing implement the above methods. Memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory located remotely relative to the processor 102, and these remote memories may be connected to the mobile terminal through a network. Examples of the above-mentioned networks include but are not limited to the Internet, intranets, local area networks, mobile communication networks and combinations thereof.

Transmission device 106 is used to receive or send data via a network. Specific examples of the above-mentioned network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC for short), which can be connected to other network devices through a base station to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (Radio Frequency, RF for short) module, which is used to communicate with the Internet wirelessly.

This embodiment provides a data migration method running on the above-mentioned terminal. Figure 2 is a flow chart of the data migration method according to the embodiment of the present application. As shown in Figure 2, the process includes the following steps:

Step S202: Receive the first instruction triggered by touching the first button on the target display interface;

Step S204: Call the target interface for managing the source data set and the target data set when triggered by the above-mentioned first instruction;

Step S206: Receive a second instruction triggered by touching the second button on the target display interface;

Step S208, triggered by the second instruction, migrate the target data in the source data set to the target data set.

The above-mentioned target display interface can be a display interface on a specific terminal, or a display interface of an application installed on a specific terminal. Of course, it can also be other types of display interfaces, as long as it is a visual interface. The execution subject of the above steps may be a terminal displaying the above target display interface, or a terminal or control device or server that controls other devices to display the above target display interface, or other devices with similar processing capabilities. or system.

In the above embodiment, the target display interface is configured with multiple buttons (or keys, for example, touch buttons, which of course can also be physical buttons, and the specific type of buttons shall be subject to the actual situation), and each The functions configured for each button can be different. A certain button can trigger the corresponding function by touching it (for physical buttons, it is a pressing operation). Among them, the triggered function is a function related to data migration. A specific button corresponds to Which function can be configured based on the actual situation. In addition, the position of the touch button on the display interface can also be adjusted, and users can flexibly adjust it based on their own usage habits.

In the above embodiment, different types of data can be migrated through the target display interface. For example, HDFS data, HIVE data, and HBASE data can be migrated across big data clusters. Of course, other types of data can also be migrated. It should be noted that in the above embodiments, when performing data migration, all data in the source data set can be migrated to the target data set, or part of the data in the source data set can be migrated to the target data set. Specifically, which data should be migrated? Data can be selected by the user through the target display interface. In addition, the number of target data sets can be one or multiple. The specific number is determined by the actual amount of data to be migrated and the free storage space of the target data set. A switch button can also be configured in the target display interface, and the migration mode corresponding to the specific data type to be migrated can be selected by touching the switch button. The number of switch buttons can be one or more. When the number of is one, the mode switched to is different depending on the number of times of pressing, or the mode switched to is different depending on the pressing force, or the mode switched to is different depending on the pressing duration. , In addition, the specific switching method can also be flexibly set and adjusted based on the user's usage habits; when there are multiple switching buttons, the current mode can be switched to other modes by touching different switching buttons.

In the above embodiment, the method is to receive the first instruction triggered by touching the first button on the target display interface; and to call the target for managing the source data set and the target data set when triggered by the above first instruction. Interface; receiving a second instruction triggered by touching a second button on the target display interface; triggering the above second instruction, the operation of migrating the target data in the above source data set to the above target data set, that is, Buttons for performing relevant data migration operations will be displayed on the target display interface, and corresponding functions will be linked to each relevant button in advance. Therefore, when data migration is required, data migration can be achieved by simply touching the relevant buttons. The purpose is to make the operation simpler and more convenient for users. Data migration can be achieved without writing professional underlying commands, thereby expanding the use of data migration, improving the versatility of data migration, and improving data efficiency to a certain extent. The work efficiency of migration effectively solves the problems of low data migration efficiency and poor versatility in related technologies.

In an optional embodiment, after migrating the data in the source data set to the target data set, the method further includes: receiving a third instruction triggered by touching a third button on the target display interface; Triggered by the above third instruction, information about the data migrated to the above target data set is displayed on the above target display interface. In this embodiment, the data migration can also be verified through the target display interface, that is, to verify whether the data migration has been completed. After touching the third button, the data migration can be verified in a specific area of the target display interface. Display information about the data that has been migrated. For example, display the list of data that has been migrated, or the folder where the data is located, etc. In addition, it should be noted that the display operation in this embodiment can also be performed in real time, that is, it does not need to be executed under the trigger of the third instruction, but can be performed on the target after detecting a data migration operation. A specific area of the display interface displays the corresponding migration progress or displays the corresponding information on the data that has been migrated. Through this embodiment, data migration integrity can be verified, allowing users to intuitively see the data migration process and whether the migration is completed, thereby reducing unnecessary waiting time.

As can be seen from the description of the foregoing embodiments, embodiments of the present invention can realize the migration of multiple types of data. The following describes the migration process of multiple types of data according to situations:

Case 1: HADOOP data is stored in the above source data set

Calling the target interface for managing the source data set and the target data set when triggered by the above-mentioned first instruction includes: calling the target interface for managing the above-mentioned source data set when triggered by the above-mentioned first instruction (in this case, the source data set includes the source data set). HDFS file system) and the HADOOP interface of the above-mentioned target data set (in this case, the source data set includes the target HDFS file system, or the destination HDFS file system); triggered by the above-mentioned second instruction, the above-mentioned source data set Migrating data to the above-mentioned target data set includes: triggering the above-mentioned second instruction, creating a first migration task, wherein the above-mentioned first migration task is used to select the above-mentioned target data in the above-mentioned source data set and transfer the above-mentioned target data to The above target data set. In addition, after completing the above migration operation, the integrity of the data can also be verified through the visual UI (ie, the above-mentioned target display interface) (the integrity can be verified by viewing the total number of file data).

In the above scenario 1, this can be accomplished through the open source migration tool Distcp. Distcp is a tool suitable for data copying within and between large-scale clusters. It uses Map/Reduce to implement file distribution, error handling and recovery, and report generation. It takes a list of files and directories as input to a map task, and each task completes a copy of some files in the source list. Due to the use of the MapReduce method, this tool has special features in semantics and execution.

Case 2: HIVE data is stored in the above source data set

Calling the target interface for managing the source data set and the target data set when triggered by the above-mentioned first instruction includes: calling the target interface for managing the above-mentioned source data set when triggered by the above-mentioned first instruction (in this case, the source data set includes the source data set). HIVE data warehouse) and the HDFS and HIVE interfaces of the above-mentioned target data set (in this case, the source data set includes the target HIVE data warehouse, or the target HIVE data warehouse); triggered by the above-mentioned second instruction, the above-mentioned source data Migrating the centralized data to the above-mentioned target data set includes: under the trigger of the above-mentioned second instruction, exporting the above-mentioned target data in the above-mentioned source data set to the above-mentioned target data set through the HDFS file system.

In an optional embodiment, in the above situation 2, under the trigger of the above second instruction, exporting the above target data in the above source data set to the above target data set through the HDFS file system includes: in the above second instruction Triggered by Migrating the target data on the source HDFS file system to the target HDFS file system; creating an import task, wherein the import task is used to import the target data on the target HDFS file system into the target data set.

In case 2 above, HIVE data migration can use HIVE's own import and export functions to import and export the table structure and data in the HIVE table. In addition, after completing the above migration operation, the integrity of the data can also be verified through the visual UI (ie, the above-mentioned target display interface) (the integrity can be verified by viewing the total number of table data).

Scenario 3: The above source data set stores HBASE (or Hbase) data

Calling the target interface for managing the source data set and the target data set when triggered by the above-mentioned first instruction includes: calling the HDFS and HBASE interfaces for managing the above-mentioned source data set and the above-mentioned target data set when triggered by the above-mentioned first instruction. ; Triggered by the above-mentioned second instruction, migrating the data in the above-mentioned source data set to the above-mentioned target data set includes: Triggered by the above-mentioned second instruction, exporting the above-mentioned target data in the above-mentioned source data set to the above-mentioned target data set through the HDFS file system target data set.

In an optional embodiment, in the above situation 3, under the trigger of the above second instruction, exporting the above target data in the above source data set to the above target data set through the HDFS file system includes: in the above second instruction When triggered by Export the above-mentioned HBASE snapshot to the source HDFS file system, and migrate the above-mentioned HBASE snapshot on the above-mentioned source HDFS file system to the target HDFS file system; create a table task, wherein the above-mentioned table task is used to create a table task based on the above-mentioned source data set The table attribute creates an HBASE table on the above-mentioned target data set; creates a restore snapshot task, wherein the above-mentioned restore snapshot task is used to restore the above-mentioned HBASE snapshot on the above-mentioned target HDFS file system to the above-mentioned HBASE table created on the above-mentioned target data set. .

In case 3 above, HBASE data migration can be achieved using HBASE's snapshot and exprotsnapshot and restore_snapshot core functions. For example, when exporting the above HBASE snapshot to the source HDFS file system, you can export the above HBASE snapshot to the source HDFS file system through exportsnapshot technology, and then restore the above HBASE snapshot on the above target HDFS file system to the above mentioned target. When the HBASE table is created on the data set, the HBASE snapshot can be restored to the HBASE table through the restore_snapshot technology. In addition, after completing the above migration operation, the integrity of the data can also be verified through the visual UI (ie, the above-mentioned target display interface) (the integrity can be verified by viewing the total number of table data). The migration process of HBASE data can be seen in Figure 3.

Through several situations in the above embodiments, visual migration of various types of data can be effectively realized to meet customer usage needs, improve migration efficiency, and simplify migration steps.

In addition, it should be noted that the aforementioned data migration situations can actually be performed in secure mode (kerberos) and non-secure mode (non-kerberos). Among them, in full mode, the migration operation needs to be performed Security verification, for example, calls to the management interface must be security verified, and subsequent data migration operations will only be performed after the verification is passed.

Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus the necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is Better implementation. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product in essence or that contributes to the existing technology. The computer software product is stored in a storage medium (such as ROM/RAM, disk, CD), including several instructions to cause a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described above in various embodiments of the present application.

This embodiment also provides a data migration device, which is used to implement the above embodiments and preferred implementations. What has already been described will not be described again. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.

Figure 4 is a structural block diagram of a data migration device according to an embodiment of the present application. As shown in Figure 4, the device includes:

The first receiving module 42 is used to receive the first instruction triggered by touching the first button on the target display interface; the calling module 44 is used to call the source data set management module 44 when triggered by the first instruction. The target interface of the target data set; the second receiving module 46 is used to receive the second instruction triggered by touching the second button on the target display interface; the migration module 48 is used to trigger the second instruction. Migrate the target data in the above source data set to the above target data set.

In an optional embodiment, the above device further includes a third receiving module, configured to receive the data received by touching the third button on the target display interface after migrating the data in the source data set to the target data set. The third instruction triggered; when triggered by the above third instruction, the information of the data migrated to the above target data set is displayed on the above target display interface.

In an optional embodiment, when HADOOP data is stored in the above source data set: the above calling module 44 includes: a first calling unit, configured to call for managing the above source data when triggered by the above first instruction. set and the HADOOP interface of the target data set; the migration module 48 includes: a creation unit, configured to create a first migration task triggered by the second instruction, wherein the first migration task is used to select the source data set of the above target data, and transfer the above target data to the above target data set.

In an optional embodiment, in the case where HIVE data is stored in the above source data set: the above calling module 44 includes: a second calling unit, configured to call for managing the above source data when triggered by the above first instruction. set and the HDFS and HIVE interfaces of the above-mentioned target data set; the above-mentioned migration module 48 includes: a first export unit, configured to export the above-mentioned target data in the above-mentioned source data set to the above-mentioned source data set through the HDFS file system when triggered by the above-mentioned second instruction. target data set.

In an optional embodiment, the above-mentioned first export unit includes: a first execution sub-unit, configured to perform the following operations when triggered by the above-mentioned second instruction: create an export task, wherein the above-mentioned export task is used to convert the above-mentioned Export the above target data in the source data set to the source HDFS file system; create a second migration task, wherein the above second migration task is used to migrate the above target data on the above source HDFS file system to the target HDFS file system; create An import task, wherein the above-mentioned import task is used to import the above-mentioned target data on the above-mentioned target HDFS file system into the above-mentioned target data set.

In an optional embodiment, when HBASE data is stored in the source data set: the calling module 44 includes: a third calling unit, configured to call for managing the source data when triggered by the first instruction. set and the HDFS and HBASE interfaces of the above-mentioned target data set; the above-mentioned migration module 48 includes: a second export unit, configured to export the above-mentioned target data in the above-mentioned source data set to the above-mentioned source data set through the HDFS file system when triggered by the above-mentioned second instruction. target data set.

In an optional embodiment, the above-mentioned second export unit includes: a second execution sub-unit, configured to perform the following operations when triggered by the above-mentioned second instruction: create an HBASE snapshot task, wherein the above-mentioned HBASE snapshot task is used to Create an HBASE snapshot for the data in the above source data set; create an HBASE snapshot export and migration task, in which the above HBASE snapshot export and migration task is used to export the above HBASE snapshot to the source HDFS file system, and transfer it to the above source HDFS file system. Migrate the above HBASE snapshot to the target HDFS file system; create a table task, wherein the above table task is used to create an HBASE table on the above target data set according to the table attributes in the above source data set; create a restore snapshot task, wherein the above restore The snapshot task is used to restore the above HBASE snapshot on the above target HDFS file system to the above HBASE table created on the above target data set.

It should be noted that each of the above modules can be implemented through software or hardware. For the latter, it can be implemented in the following ways, but is not limited to this: the above modules are all located in the same processor; or the above modules can be implemented in any combination. The forms are located in different processors.

Embodiments of the present application also provide a computer-readable storage medium that stores a computer program, wherein the computer program is configured to execute the steps in any of the above method embodiments when running.

In this embodiment, the above-mentioned computer-readable storage medium may be configured to store a computer program for performing the following steps:

S1, triggering the above first instruction, calls the target interface for managing the source data set and the target data set;

S2, receiving the second instruction triggered by touching the second button on the target display interface;

S3, triggered by the above second instruction, migrate the target data in the above source data set to the above target data set.

In an exemplary embodiment, the computer-readable storage medium may include but is not limited to: USB flash drive, read-only memory (ROM), random access memory (Random Access Memory, RAM) , mobile hard disk, magnetic disk or optical disk and other media that can store computer programs.

An embodiment of the present application also provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

In an exemplary embodiment, the above-mentioned processor may be configured to perform the following steps through a computer program:

S1, triggering the above first instruction, calls the target interface for managing the source data set and the target data set;

S2, receiving the second instruction triggered by touching the second button on the target display interface;

S3, triggered by the above second instruction, migrate the target data in the above source data set to the above target data set.

In an exemplary embodiment, the above-mentioned electronic device may further include a transmission device and an input-output device, wherein the transmission device is connected to the above-mentioned processor, and the input-output device is connected to the above-mentioned processor.

For specific examples in this embodiment, reference may be made to the examples described in the above-mentioned embodiments and exemplary implementations, and details will not be described again in this embodiment.

Obviously, those skilled in the art should understand that the above-mentioned modules or steps of the present application can be implemented using general-purpose computing devices, and they can be concentrated on a single computing device, or distributed across a network composed of multiple computing devices. They may be implemented in program code executable by a computing device, such that they may be stored in a storage device for execution by the computing device, and in some cases may be executed in a sequence different from that shown herein. or the described steps, or they are respectively made into individual integrated circuit modules, or multiple modules or steps among them are made into a single integrated circuit module. As such, the application is not limited to any specific combination of hardware and software.

The above descriptions are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and changes. Any modifications, equivalent replacements, improvements, etc. made within the principles of this application shall be included in the protection scope of this application.

Claims (10)

1. A method of migrating data, comprising:

receiving a first instruction triggered by a first button on a touch target display interface;

invoking a target interface for managing a source data set and a target data set under the triggering of the first instruction;

receiving a second instruction triggered by a second button on the touch target display interface;

and under the triggering of the second instruction, migrating the target data in the source data set to the target data set.

2. The method of claim 1, wherein after migrating the data in the source dataset to the target dataset, the method further comprises:

receiving a third instruction triggered by a third button on the touch target display interface;

and under the triggering of the third instruction, displaying information of the data migrated to the target data set on the target display interface.

3. The method according to claim 1, wherein in case HADOOP data is stored in the source dataset:

invoking a target interface for managing a source data set and a target data set under the triggering of the first instruction comprises: invoking a HADOOP interface for managing the source data set and the target data set under the triggering of the first instruction;

under the triggering of the second instruction, migrating the data in the source data set to the target data set includes: and under the triggering of the second instruction, creating a first migration task, wherein the first migration task is used for selecting the target data in the source data set and transferring the target data into the target data set.

4. The method according to claim 1, wherein in case HIVE data is stored in the source dataset:

invoking a target interface for managing a source data set and a target data set under the triggering of the first instruction comprises: invoking an HDFS and HIVE interface for managing the source data set and the target data set under the triggering of the first instruction;

under the triggering of the second instruction, migrating the data in the source data set to the target data set includes: and under the triggering of the second instruction, exporting the target data in the source data set to the target data set through an HDFS file system.

5. The method of claim 4, wherein exporting the target data in the source data set to the target data set through an HDFS file system under the trigger of the second instruction comprises:

under the triggering of the second instruction, the following operations are executed:

creating an export task, wherein the export task is used for exporting the target data in the source data set onto a source HDFS file system;

creating a second migration task, wherein the second migration task is used for migrating the target data on the source HDFS file system to a target HDFS file system;

and creating an import task, wherein the import task is used for importing the target data on the target HDFS file system into the target data set.

6. The method according to claim 1, wherein in case HBASE data is stored in the source dataset:

invoking a target interface for managing a source data set and a target data set under the triggering of the first instruction comprises: invoking an HDFS and HBASE interface for managing the source data set and the target data set under the triggering of the first instruction;

Under the triggering of the second instruction, migrating the data in the source data set to the target data set includes: and under the triggering of the second instruction, exporting the target data in the source data set to the target data set through an HDFS file system.

7. The method of claim 6, wherein exporting the target data in the source data set to the target data set through an HDFS file system under the trigger of the second instruction comprises:

under the triggering of the second instruction, the following operations are executed:

creating an HBASE snapshot task, wherein the HBASE snapshot task is used for creating an HBASE snapshot for data in the source data set;

creating an HBASE snapshot export and migration task, wherein the HBASE snapshot export and migration task is used for exporting the HBASE snapshot to a source HDFS file system and migrating the HBASE snapshot on the source HDFS file system to a target HDFS file system;

creating a table task, wherein the table task is used for creating an HBASE table on the target data set according to the table attribute in the source data set;

and creating a recovery snapshot task, wherein the recovery snapshot task is used for recovering the HBASE snapshot on the target HDFS file system into the HBASE table created on the target data set.

8. A data migration apparatus, comprising:

the first receiving module is used for receiving a first instruction triggered by a first button on the touch target display interface;

the calling module is used for calling a target interface for managing the source data set and the target data set under the triggering of the first instruction;

the second receiving module is used for receiving a second instruction triggered by a second button on the touch target display interface;

and the migration module is used for migrating the target data in the source data set to the target data set under the triggering of the second instruction.

9. A computer readable storage medium, characterized in that a computer program is stored in the computer readable storage medium, wherein the computer program, when being executed by a processor, implements the steps of the method according to any of the claims 1 to 7.

10. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method of any one of claims 1 to 7 when the computer program is executed.