CN101741904B - Method for building distributed space computation service node and gateway device - Google Patents

Abstract

The invention discloses a construction method of a distributed space computing service node and a gateway device to provide space computing services by integrating multiple platforms. The method includes: asynchronously invoking spatial computing resources based on the Web service structure; responding to events in a multi-threaded manner in a first-in-first-out order to implement functions corresponding to events; configuring actual connection computing resources through configuration files Connectors to connect to actual computing resources. The present invention enables the owner of spatial computing resources to simply provide computing resources on various platforms and systems owned by them, and the node construction tool serves them to form spatial computing with a unified self-describing interface. Serve.

Description

A method for constructing a distributed spatial computing service node and a gateway device

technical field

The invention relates to a Web geographic information system (GIS) and a Web remote sensing technology, in particular to a method for constructing a distributed space computing service node and a gateway device.

Background technique

Although the existing development methods and systems based on geographic information network service framework have their own advantages and disadvantages, most of them depend on specific GIS software, and some even rely on specific operating systems, which cannot support cluster and parallel computing environments.

The existing development technology based on the geographic information network service framework includes the database used to develop the geographic information system network service framework, and the database contains spatial data. In addition, the GIS web service framework provides a GIS web service to users. GIS-based functions are web services combined with geographic information systems. The function of users accessing geographic information system is to use the network service of this geographic information system.

At present, the existing geographic information system interface service technology is based on the data foundation of GIS interface service. The GIS interface matches the external application system, so that the external application system can use the GIS interface service function, so that it is related to the external application system. The information is positioned and distributed on GIS.

At present, there is a grid service group establishment technology and grid service discovery technology. Through interaction between several GIS instances, a service group member list is generated and a cascade network connection is established, and then the service group is established. In the grid service discovery technology, after the GIS instance receives the service query request, when there is a service group of the service type, it judges that the GIS instance is a member of the service group and forwards the request to other members of the service group. Otherwise, the request is forwarded to any member of the service group, who forwards it to other members. This grid service discovery technology improves the efficiency of service query and avoids the defects of long response time and large network overhead caused by the flooding method in the prior art.

Existing methods each have their own focus, but often rely on specific GIS software, rely on specific operating systems, and cannot support clusters and parallel computing environments. These methods focus on database operations for geographic information, so they are not flexible enough, because the actual The geographic information file formats and spatial computing needs are varied.

Contents of the invention

The technical problem to be solved by the present invention is to provide a method for constructing a distributed spatial computing service node and a gateway device, so as to integrate multiple platforms to provide spatial computing services.

In order to solve the above technical problems, the present invention provides a distributed spatial computing service node construction method, including:

Based on the Web service structure, make asynchronous calls to spatial computing resources;

According to the order of first-in-first-out, respond to the event in a multi-threaded manner, and realize the corresponding function of the event;

Configure the connector that actually connects to the computing resource through the configuration file to realize the connection with the actual computing resource.

Preferably, the step of asynchronously invoking the spatial computing resources includes:

Encapsulate the user's computing needs into computing tasks described in Extensible Markup Language;

Submit the computing task to the spatial computing service node through a predefined web service interface, and generate a task instance with a unique identifier;

The user initiates and controls the operation of the task instance until the task instance is executed and the execution result is recovered.

Preferably, monitoring and management are implemented on the node itself and each task deployed on the node.

Preferably, the step of responding to the event in a multi-threaded manner in a first-in-first-out order includes:

Add user requests and system state changes as events to the event queue, and respond to events in a multi-threaded manner in the order of first-in-first-out.

In order to solve the above technical problems, the present invention also provides a gateway device for a distributed spatial computing service node, including a unified interface for spatial computing services, a node gateway engine, and a resource interface, wherein:

The uniform interface of the spatial computing service is based on the Web service structure, and is used for asynchronously invoking spatial computing resources;

The node gateway engine is connected to the uniform interface of the spatial computing service, and is used to respond to events in a multi-threaded manner in a first-in-first-out order, and realize functions corresponding to events;

The resource interface is connected to the node gateway engine, adopts a plug-in-configuration structure, and configures a connector for actually connecting computing resources through a configuration file, so as to realize connection with actual computing resources.

Preferably, according to the self-description of Extensible Markup Language, the unified interface of the spatial computing service has the ability of self-description.

Preferably, the node gateway engine is configured to search for a computing resource interface support class according to the configuration information provided by the task description and the configuration file used when creating the task, so as to realize the connection with this type of computing resource.

Preferably, the node gateway engine supports desktop systems, condor or cluster computing resource platforms, and supports Linux or Windows operating systems.

Preferably, the device further comprises:

The node monitoring and management service module is configured to monitor and manage the service node and each task deployed on the service node.

The technical solution of the present invention is mainly designed and implemented systematically in the distributed computing of spatial computing services and web-based spatial computing, and provides a powerful and easy-to-use technical means, so that the owner of spatial computing resources can simply use the Computing resources on different platforms and systems are provided, and the node construction tools serve them to form spatial computing services with a unified self-describing interface, which can be relatively simple by installing middleware and simply Configuration, communicates with other spatial computing service nodes installed with the same middleware and similar configurations to form a distributed spatial computing environment with multiple nodes and multiple support environments, which overcomes the shortcomings of previous technologies and products in this regard. Users of spatial Web computing services can easily use these spatial computing services through any browser without installing other plug-ins.

Additional features and advantages of the invention will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention may be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

Description of drawings

The accompanying drawings are used to provide a further understanding of the present invention, and constitute a part of the description, and are used together with the embodiments of the present invention to explain the present invention, and do not constitute a limitation to the present invention. In the attached picture:

Fig. 1 is a schematic structural diagram of an embodiment of a spatial computing service node gateway device proposed by the present invention;

FIG. 2 is a schematic diagram of logical execution of the unified interface of the spatial computing service shown in FIG. 1;

FIG. 3 is a schematic flowchart of an embodiment of a method for constructing a distributed spatial computing service node in the present invention.

Detailed ways

The implementation of the present invention will be described in detail below in conjunction with the accompanying drawings and examples, so as to fully understand and implement the process of how to apply technical means to solve technical problems and achieve technical effects in the present invention.

It should be noted that, if there is no conflict, the embodiments of the present invention and various features in the embodiments can be combined with each other, and all are within the protection scope of the present invention. In addition, the steps shown in the flow diagrams of the figures may be performed in a computer system, such as a set of computer-executable instructions, and, although a logical order is shown in the flow diagrams, in some cases, the sequence may be different. The steps shown or described are performed in the order herein.

The calling model adopted by the traditional Web spatial computing service is basically a synchronous call, that is, the caller will be blocked when calling the spatial computing service, and the execution cannot continue until the computing service completes the work and returns control to the caller. This synchronous call will bring a series of problems, the most serious of which is the timeout problem. In order to solve the above-mentioned problems, aiming at the defects of large amount of calculation and long calculation time of the existing spatial computing service, the present invention adopts an asynchronous invocation model of the spatial computing service when constructing the Web spatial computing service node.

FIG. 1 is a schematic composition diagram of an embodiment of a spatial computing service node gateway device proposed by the present invention. As shown in Figure 1, the device embodiment mainly includes a spatial computing service unified interface 110, a node gateway engine 120, a resource interface 130, and a node monitoring and management service module 140, wherein:

Spatial computing service unified interface 110, based on the Web service (Service) structure, is used for asynchronously invoking spatial computing resources;

Because spatial computing usually has a large amount of calculation and takes a long time, the synchronous call mode of traditional Web Service cannot be used as a unified interface for spatial computing services due to timeout restrictions; the above-mentioned unified interface for spatial computing services 110 is based on the structure of traditional Web Services. It is used to make asynchronous calls to spatial computing resources, and endow the above-mentioned unified interface 110 of spatial computing services with self-describing capabilities, so as to facilitate the use of spatial computing services.

Self-describing refers to a feature of Extensible Markup Language (XML) documents, that is, XML documents usually contain a document type declaration, so XML documents are self-describing. Not only humans can understand XML documents, but computers can also process them. The way of representing data in XML is really independent of the application system, and the data can be reused. XML documents are regarded as databaseization of documents and documentation of data. In the present invention, this technology is utilized to realize the self-description of the spatial computing service interface.

Considering the efficiency and convenience of use, the basic implementation process of the asynchronous mode of the spatial computing service unified interface 110 is: encapsulating the user's computing needs into a computing task described in Extensible Markup Language (XML). Submit to the spatial computing service node through a pre-defined Web Service interface, generate a task instance with a unique ID (ID), and the user starts and controls the operation of the task instance, and recycles the task until the execution of the task instance is completed the result of the execution.

Specifically, the spatial computing service unified interface 110 provides service users with the following functions to meet the needs of creating, destroying and controlling tasks:

Create: Create a new spatial computing task instance according to the task description provided by the user, and return the unique ID used to identify the task instance.

Start: start a task instance, and the node gateway engine 120 will decide when and which resources to use to run the task instance according to the status of the platform and system providing the service.

Pause/resume (Pause/resume): Pause/resume the running of a task instance, which can only be used when the task implementation supports pause/resume, that is, some tasks cannot be paused/resume.

Terminate: Forcefully end the running of a task instance.

Destroy: Destroys an instance of a task.

In addition to the interfaces required by the above control tasks, the spatial computing service unified interface 110 also defines the following functions to achieve more control and monitoring of tasks by users:

Parameter setting (setParameter): Set the parameters required for the task instance to run. The parameters will be automatically encoded according to their types. The data types supported by the current parameter/result transmission (that is, the data types that can be automatically encoded and decoded) are: Java simple types, characters Strings, iava beans, and arrays of the above types.

Parameter read (getParameter): read the parameters of the task instance running.

Result acquisition (getResult): Get the result of the task instance running. It should be noted that the task instance does not necessarily have to reach the completed (finished) state before it can output the result. The getResult interface can also read the running result of the task instance at any time. If the result has not been output, it returns a null value.

Status acquisition (getStatus): Get the running status of the task instance, and the client can poll to obtain the status of the task instance.

Message acquisition (getMessage): read the messages output during the running of the task instance, and these messages are set by the implementation code of the task.

The above functions are all invoked through Web Service, have a clear Web Service Description Language (WSDL) description, and provide some client support libraries to facilitate their use.

The node gateway engine 120 is connected to the space computing service interface 110, and is used to respond to events in a multi-threaded manner in a first-in-first-out order, and realize functions corresponding to events;

The event-based node gateway engine 120, with the event queue as the center, adds various user requests and various system state changes to the event queue as events, and responds to the events in a multi-threaded manner in the order of first-in-first-out. Realize the corresponding function.

The events defined by the node gateway engine 120 include a task creation request event for requesting to create a task, a task establishment completion event for completing task creation, a task start request event for requesting to start a task, a task start event for starting a task, a task completion event for completing a task, and a Task failure events for task failures, etc. The functions realized by the computing resources defined by the node gateway engine 120 include the following:

Task start (startJob): Start the operation of a task instance.

Task termination (terminateJob): Terminate the running of a task instance.

Task initialization (onCreate): Called when the task instance is created to provide a mechanism for initializing the corresponding computing resources before the task instance actually runs.

Task cleanup (onDestroy): Called before the task instance is destroyed to provide a mechanism for cleaning up the corresponding computing resources before the task instance is destroyed.

Task suspension (stopJob): suspend the running of a task instance.

Task resume (resumeJob): Resume the operation of a suspended task instance.

Among them, startJob must have a functional implementation to ensure that the task can be run; while the remaining functions can be implemented empty if the computing resources do not need or support it.

At the same time, the node gateway engine 120 also provides the definition of task instance context necessary for task execution. The task instance context includes information such as the ID, description, parameters, status, message, and result of the task instance. At the same time, the task instance context also indicates which computing resource interface support the task instance depends on.

The resource interface 130 is connected to the node gateway engine 120, and is used to adopt a plug-in-configuration structure. The node gateway engine 120 defines the functions that must be realized by the computing resource interface in the form of a Java interface, etc., and configures the actual connection to the computing resource through a configuration file. Connectors to connect to actual computing resources.

The implemented computing resource interface can be configured through an XML file, and the node gateway engine 120 will search for a suitable computing resource interface support class according to the task description used by the user when creating the task and the configuration information provided by the configuration file, so as to realize the The connection of various types of computing resources can support three different computing resource platforms of desktop system, condor and cluster, as well as two different operating systems of Linux and Windows.

The node monitoring and management service module 140 is connected to the node gateway engine 120, and is used to monitor and manage the node itself and the tasks deployed on the node. The main tasks include the following:

Get node-related information, including static information (such as node architecture, operating system, platform version, etc.) Conditions (such as task name, task characteristics, etc.) are used to search for tasks, and provide convenient means of task deployment; obtain task interfaces for users to call tasks. At the same time, tools are provided to automatically generate clients for invoking computing service tasks based on known task interfaces.

FIG. 2 is a schematic diagram of logical execution of the unified interface of the spatial computing service shown in FIG. 1 . In combination with the embodiment of the gateway device shown in FIG. 1, the logic execution schematic diagram shown in FIG. 2 mainly includes the following steps:

Step S210, create (create) a new spatial computing task instance according to the task description provided by the user, and complete the initialization (initialized);

Step S220, start (Start) the operation of the task instance, and then be in the running (running) state;

Step S230, during the running process, a Pause operation can be performed on some tasks; after the Pause operation is performed, the task instance is in a Pause state;

Step S240, perform a recovery operation on the task instance in the pause (pause) state, and return to the running (running) state;

It should be noted that only some tasks support the pause/resume function;

Step S250, in the running (running) state and after (accomplished), enter the completed (finished) state, and step S280 can be executed to destroy (Destroy) the task instance, so that the task instance is in the non-existing (Not Exist) state.

As shown in step S260 in Figure 2, during the initialization (initialized) process, in the running (running) state or in the pause (pause) state, the operation of the task instance can be forcibly terminated (Terminate), and the task instance is in failure (failed) at this time. state.

As shown in step S270, in the running state, the task instance may be in a failed state due to an exception state.

In the failed (failed) state or the completed (finished) state, step S280 may be executed to destroy (Destroy) the task instance, so that the task instance is in a non-existing (Not Exist) state.

When the task instance is in the state of not existing (Not Exist), it means that this is a new task instance, and returns to step S210.

FIG. 3 is a schematic flowchart of an embodiment of a method for constructing a distributed spatial computing service node in the present invention. Combining the embodiment of the gateway device shown in Figure 1 and the logic execution shown in Figure 2, the method of the present invention shown in Figure 3 mainly includes the following steps:

Step S310, based on the Web service structure, make an asynchronous call to the spatial computing resource;

Step S320: Respond to the event in a multi-threaded manner according to the order of first-in-first-out, so as to realize the functions corresponding to the event;

In step S330, the connector for actually connecting to the computing resource is configured through the configuration file, so as to realize the connection with the actual computing resource.

The above steps of asynchronously invoking the space computing resources mainly include:

Encapsulate the user's computing needs into computing tasks described in Extensible Markup Language;

Submit the computing task to the spatial computing service node through a predefined web service interface, and generate a task instance with a unique identifier;

The user initiates and controls the operation of the task instance until the task instance is executed and the execution result is recovered.

In the method embodiment shown in FIG. 3 , monitoring and management can be implemented on the node itself and each task deployed on the node.

The above-mentioned steps of responding to events in a multi-threaded manner according to the order of first-in-first-out may include:

Add user requests and system state changes as events to the event queue, and respond to events in a multi-threaded manner in the order of first-in-first-out.

The present invention is different from the prior art in achieving distributed computing of spatial computing services and web-based spatial computing by constructing spatial computing service nodes. Therefore, the core invention of the present invention is the process of constructing spatial computing service nodes to realize the distributed computing of spatial computing services, the asynchronous calling model of spatial computing services, and the support methods of computing resource interfaces of various platforms.

The method of the present invention adopts the basic theoretical model of Service-Oriented Architecture (SOA) and the technical framework of Web service, utilizes technologies such as Java technology and service encapsulation, can be applicable to multiple geographic information systems (GIS), and overcomes the present grid computing technology well Insufficient flexibility and poor scalability, and effectively support the migration of the operating environment and applications in the grid environment, and also support the original grid computing model. The method of the invention can make the application system make good use of GIS without modifying the application system, and greatly reduces the realization cost.

Obviously, those skilled in the art should understand that each module or each step of the above-mentioned present invention can be realized by a general-purpose computing device, and they can be concentrated on a single computing device, or distributed in a network formed by multiple computing devices Optionally, they can be implemented with program codes executable by computing devices, thus, they can be stored in storage devices and executed by computing devices, or they can be made into individual program modules, or their Multiple modules or steps are implemented as a single program module.

Although the embodiments disclosed in the present invention are as above, the described content is only an embodiment adopted for the convenience of understanding the present invention, and is not intended to limit the present invention. Anyone skilled in the technical field to which the present invention belongs can make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed by the present invention, but the patent protection scope of the present invention, The scope defined by the appended claims must still prevail.

Claims (8)

1. a distributed space computation service node construction method is characterized in that, comprising:

The Web-based services structure is carried out asynchronous call to space computing resources; Specifically comprise: user's computation requirement is packaged into the calculation task of describing with extend markup language; By predefined Web service interface described calculation task is submitted to space computation service node, generation has the uniquely identified task instances; By starting and control the operation of described task instances, until carrying out, described task instances finishes the rear execution result that reclaims by the user;

According to the order of first-in first-out, event is responded the corresponding function of realization event in the mode of multithreading;

Dispose the connector of actual connection space computational resource by configuration file, realize and being connected of space computing resources.

2. the method for claim 1 is characterized in that:

Each task-cycle to space computation service node itself and space computation service node deploy monitors and management.

3. the method for claim 1 is characterized in that, according to the order of first-in first-out,, comprising the step that event responds in the mode of multithreading:

User request and system state change are joined event queue as event, according to the order of first-in first-out, in the mode of multithreading event is responded.

4. the gateway apparatus of a distributed space computation service node is characterized in that, comprises space calculation services unified interface, node gateway engine and resource interface, wherein:

Described space calculation services unified interface, the Web-based services structure is used for space computing resources is carried out asynchronous call; Specifically comprise: user's computation requirement is packaged into the calculation task of describing with extend markup language; By predefined Web service interface described calculation task is submitted to space computation service node, generation has the uniquely identified task instances; By starting and control the operation of described task instances, until carrying out, described task instances finishes the rear execution result that reclaims by the user;

Described node gateway engine links to each other with described space calculation services unified interface, is used for the order according to first-in first-out, event is responded the corresponding function of realization event in the mode of multithreading;

Described resource interface links to each other with described node gateway engine, adopts plug-in unit-configuration structure, disposes the connector of actual connection space computational resource by configuration file, realizes and being connected of space computing resources.

5. device as claimed in claim 4 is characterized in that:

According to the self descriptiveness of extend markup language, described space calculation services unified interface possesses the ability of self-described.

6. device as claimed in claim 4 is characterized in that:

Described node gateway engine is used for employed task description and configuration file provide when setting up task configuration information and seeks the computational resource interface and support class, to realize and being connected of this kind computational resource.

7. device as claimed in claim 6 is characterized in that:

Described node gateway engine support table plane system, condor or cluster computational resource platform are supported Linux or Windows operating system.

8. device as claimed in claim 4 is characterized in that, this device further comprises:

Node monitors and management services module, is used for providing each task-cycle to described space computation service node and described space computation service node deploy to monitor and management.

Images