JP7159027B2 - User interface providing device, user interface providing method, program and embedded device - Google Patents

Description

The present invention relates to a user interface providing device, a user interface providing method, a program, and an embedded device.

2. Description of the Related Art Conventionally, as a GUI (Graphical User Interface) for embedded devices, a technique using a browser program for displaying structured documents such as HTML files is known. For such embedded devices, prepare a program written in native code (hereinafter referred to as "native program") that can acquire and set internal information, and link the native program and the browser program. , the state display and various settings of the embedded device are realized using a browser program (see Patent Document 1, for example).

In order to implement a GUI in a browser program, content that can be interpreted and drawn by the browser program (hereinafter abbreviated as "browser displayable content") is required to construct a screen. Therefore, based on the screen design, it is necessary to perform coding using HTML, CSS, Javascript (registered trademark), etc. (hereinafter referred to as "HTML coding").

Japanese Unexamined Patent Application Publication No. 2011-8490

Compared to coding using a native language such as the C language, HTML coding has the advantage that tools are relatively inexpensive and the skills required of coders are relatively low.

However, you still need to do some coding. There is also a need to test whether browser-viewable content created by HTML coding is appropriate. In particular, to make the presentation of the GUI dynamic, scripts such as Javascript must be incorporated into the browser-viewable content. In this case, the browser program interprets and executes the script, modifies part of the HTML file, interprets and renders the modified HTML file, and updates part of the screen. For this reason, it is necessary to actually interpret the browser-viewable content in the browser program, execute the script, and try to update the screen appropriately. Therefore, even with HTML coding, as with coding using a native language, there is a risk that the development period will be long and the cost will increase.

Also, in order to cause the embedded device to execute the operation desired by the user, for example, a script is embedded in the browser-displayable content. Then, the browser program interprets and executes the script, links with the native program, and causes the native program to control the hardware of the embedded device. It is necessary to test whether such control is properly performed by actually interpreting browser-displayable content with a browser program and executing a script.

In this way, whether or not the GUI operates normally and whether or not the embedded device can appropriately execute the user's operation largely depends on the performance and specifications of the browser program. As a result, the number of test items is large, the development period is lengthened, and the cost may increase.

The present invention has been made in view of this point, and provides a user interface providing device, a user interface providing method, a program, and an embedded device that can develop a GUI rich in dynamic expressions for an embedded device in a short period of time and at low cost. One of the purposes is to provide

According to one aspect of the user interface providing apparatus of the present invention, static content , at least a part of which is structured document file, is generated according to an event based on a template and screen design information for the user interface. Acquiring user operation information related to a user's operation on the screen from a dynamically generated static content generating means and a browser program that interprets the static content and displays the screen on a display unit. and control request means for outputting control information relating to control of the embedded device corresponding to the user operation information to a native program that controls the embedded device.

One aspect of the user interface providing method according to the present invention is executed by an embedded device, and according to an event, static content , at least a part of which is structured document file, is transferred to a template and screen for user interface. and a browser program that interprets the static content and displays the screen on a display unit, the user operation information about the user's operation performed on the screen. obtaining and outputting control information related to control of the embedded device corresponding to the user operation information to a native program that controls the embedded device.

One aspect of the program according to the present invention dynamically generates static content , at least a part of which is structured document file, based on template and screen design information for user interface in response to an event. A native device that acquires user operation information related to a user's operation performed on the screen from a browser program that interprets the procedure and the static content and displays the screen on a display unit, and controls the embedded device. The program is characterized by executing a procedure of outputting control information relating to control of the embedded device corresponding to the user operation information.

According to the present invention, a GUI rich in dynamic expressions can be developed in a short period of time and at a low cost for an embedded device.

2 is a block diagram showing the configuration of a GUI providing unit included in the user interface providing device according to this embodiment; FIG. 2 is a block diagram showing the configuration of a browser program according to the embodiment; FIG. 3 is a block diagram showing the configuration of a native program 300 according to this embodiment; FIG. 4 is a flow chart showing an example of a user interface providing method according to the present embodiment; FIG. 4 is a schematic diagram showing an example of a dynamic GUI provided by the user interface providing method according to the embodiment; FIG. 4 is an explanatory diagram showing an example of a structured document that constitutes dynamic content; FIG. 4 is an explanatory diagram showing an example of a template used by the user interface providing device according to the present embodiment; FIG. 4 is an explanatory diagram showing an example of static content generated by the user interface providing device according to the embodiment; FIG. 4 is an explanatory diagram showing an example of static content generated by the user interface providing device according to the embodiment; 1 is a diagram showing an example of a functional configuration of an embedded device to which a user interface providing device according to this embodiment is applied; FIG. 1 is a diagram showing an example of a hardware configuration of an embedded device to which a user interface providing device according to this embodiment is applied; FIG.

[Summary of Invention]
The size and resolution of displays mounted in built-in devices such as home electric appliances are increasing. GUIs provided by such displays are required to have a higher user experience (hereinafter referred to as UX), and tend to incorporate more dynamic expressions.

In order to implement dynamic expressions in browser-displayable content, it is common to use dynamic content that causes a script such as Javascript to be executed on a browser program.

Such dynamic content is likely to contain many bugs, so in order to check whether the browser program can properly display it and achieve the desired dynamic presentation, test items are more important than static content. will increase.

Also, in order for the hardware to perform the control desired by the user from the GUI implemented with the browser-displayable content, the browser program can execute scripts, interact with the native program, and let the native program control the hardware. Common.

Control of the hardware of an embedded device using such a script is performed by outputting control information (commands, etc.) to native programs (drivers, etc.) that control the hardware. Then, the script dynamically changes the GUI when the control information is output, and does not reflect feedback from the native program (including changes in hardware status, etc.). For this reason, it is difficult to avoid increasing the time and effort of testing to confirm whether or not the GUI is changed at appropriate timing in accordance with the change in the operating state of the hardware.

In a GUI platform that uses structured documents instead of native languages, the inventors have achieved further reductions in development costs and development periods while realizing hardware control via the GUI without deteriorating UX. With the aim of doing so, we conducted extensive research.

The inventor of the present invention has focused on the fact that a GUI with excellent UX can be constructed without using dynamic contents by realizing dynamic expressions by dynamically switching static contents according to events. However, in order to realize this, static content must be prepared for the number of updates of the constituent elements in the screen. As a result, the number of test items can be reduced, but the number of man-hours required to create browser-displayable content becomes enormous.

Accordingly, the inventors of the present invention have found that the number of man-hours for creating static content can be reduced by dynamically generating static content based on template and screen design information according to an event.

On the other hand, with the adoption of static content, it was decided to let the native program control the hardware of the embedded device instead of executing the script in the browser program to control the hardware. More specifically, first, the browser program acquires user operation information related to the user's operation on the screen. The browser program then outputs hardware control information corresponding to the user operation information to the native program. The native program controls the hardware of the embedded device according to the hardware control information.

The present inventor has completed the present invention based on the above new knowledge. That is, the user interface providing apparatus of the present invention dynamically generates static content, at least a portion of which is composed of a structured document file, based on a template and screen design information for a user interface according to an event. Acquisition of user operation information related to a user's operation performed on the screen from a static content generating means and a browser program that interprets the static content and displays the screen on a display unit, and controls the embedded device. and control request means for outputting control information relating to control of the embedded device corresponding to the user operation information to a native program that performs the processing.

In the present invention, an event includes, for example, an event that occurs in a browser program or a native program, and that causes an action to update at least part of screen components.

More specifically, the event includes, for example, an operation by the user performed on the screen for the user interface detected by the browser program.

Events also include those that cause actions to update at least some of the components of the screen from the native program.

In the present invention, a template is a data group that constitutes content that can be displayed in a browser for constructing a screen. files, etc.). The external resource may be included in the template in advance, may be separately stored in the storage unit, or may be acquired from the network via the communication unit.

A template can be said to be content that cannot be properly displayed by a browser program as it is, or that can be displayed by a browser program as it is, but a part of which is provisional display content.

For example, as an example of a template, a structured document describes a tag "img" that embeds an image file for a button, an attribute "width" and an attribute "height", but an attribute "src" is described. There are things that have not been done. In this case, since the attribute "src" is not described, the button will not be displayed when the browser program interprets and renders the structured document that constitutes the template. Such a state is referred to as "proper display is not possible".

Another example of a template is a structured document in which a provisional character string representing hardware control corresponding to a button is described for the button. For example, there may be content in which a provisional character string "ON" is described for a button, which means activation of a function provided by hardware. This temporary character string "ON" can be replaced with another character string such as "Turn on the light" if the hardware is a lighting device. In this way, the temporary character string can be described in the content so as to be replaceable with another character string.

Also, in the present invention, a screen for user interface means a single screen in GUI screen design. GUI screen design refers to designing the layout of screens operated by users, display items, and the like in GUI development for embedded devices and the like. GUI screen design includes screen elements that make up individual screens and a single screen design that determines the layout of the screen elements, screen transition design that determines the relationship between multiple screens, and user operations on screen elements. and an action design that defines the relationship between the events of the event and the actions that occur due to the event. Therefore, it can be said that the single screen in screen design is the individual screens targeted in the above-described single screen design.

In the present invention, screen design information refers to information relating to screen design. The design information includes, for example, screen layout information, screen transition information, and action information.

The screen layout information includes screen elements that make up the screen and their arrangement, attributes of the screen elements, and events that occur due to user operations on screen elements and their timing. good.

Screen elements include text boxes, list boxes, radio buttons, buttons (action occurs when clicked or touched), labels (character strings), images (including still images and moving images), etc., and these is not limited to

The screen transition information is information representing the flow of multiple screens. The screen transition information includes, for example, a plurality of screens to be displayed, the order relationship of those screens, and an event that triggers the transition from screen to screen, but may include information other than these. .

Action information is information that indicates the relationship between an event and an action that occurs due to the event.

In the context of the present invention, static content refers to browser-displayable content that is interpreted and rendered by a browser program and does not include scripts that modify the content.

Further, in the present invention, user operation information is information indicating that the user has performed an operation on a screen element that configures the screen. The user operation information can also be said to be information related to an event that has occurred in the browser program and that is based on an operation performed by the user on the screen.

The user interface providing device of the present invention is suitable for constructing a GUI for embedded equipment. In the present invention, embedded equipment is a general term for equipment controlled using software in a broad sense, but in a narrow sense it does not include general-purpose computers such as personal computers. It is a general term for devices with embedded software intended to perform limited functions.

Also, an embedded device can be said to be a device controlled by firmware. Firmware is software for controlling hardware, which is pre-written in an integrated circuit such as a ROM and incorporated in a device. Thus, in one aspect of the invention, native programs and browser programs can be such firmware.

Furthermore, the user interface providing apparatus of the present invention is suitable for constructing a GUI of a device that has a unique IP address and can be connected to the Internet, that is, an IoT (Internet of Things) device among built-in devices. An IoT device can also be said to be an embedded device having a communication unit for connecting to the Internet.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings. It should be noted that the present invention is not limited to the embodiments described below, and various modifications can be made within the spirit and scope of the invention.

[GUI provider]
FIG. 1 is a block diagram showing the configuration of a GUI providing unit included in a user interface providing device according to this embodiment.

In the user interface providing device 100, the GUI providing unit 101 is implemented by, for example, a native program executed by the control unit 11, which will be described later. The GUI providing unit 101 has a static content generating unit 102 and a screen control unit 103 .

Based on the template 104, the static content generation unit 102 refers to the screen design information 105, which is an example of the screen design information of the present invention, and dynamically generates static content. The details of generating static content will be described later.

The static content generator 102 is configured to be able to link with the browser program 200 . The static content generator 102 can receive information about event occurrences from the browser program 200 and generate static content. The static content generation unit 102 can also output the generated static content and cause the browser program 200 to display it on the display unit 111 .

Also, the static content generator 102 is configured to be able to link with at least one native program 300 . The static content generator 102 can receive information about event occurrences from the native program 300 and generate static content.

On the other hand, the screen control unit 103, which is an example of the control request means of the present invention, outputs control information regarding control of the embedded device corresponding to the event to the native program 300 in response to the event generated in the browser program 200. FIG. Here, the event is detection of an operation by the user performed via the input unit 112 on the screen displayed on the display unit 111 by the browser program.

The screen control unit 103 is configured to be linked with the browser program 200 . The screen control unit 103 can receive information about the occurrence of an event from the browser program 200 and acquire control information corresponding to the event based on the screen design information 105 .

Furthermore, the screen control unit 103 is configured to be able to link with the native program 300 . The screen control unit 103 can output the acquired control information and cause the native program 300 to control the hardware 121 corresponding to the control information.

The screen control unit 103 also receives information about the operating state of the hardware 121 (hereinafter referred to as “state information”) from the native program 300 .

Based on the screen transition information included in the screen design information 105, the screen control unit 103 dynamically generates static content according to the operating state of the hardware 121, and displays the static content on the browser program 200. to display.

The linkage between the static content generation unit 102 and the screen control unit 103 of the GUI providing unit 101 as described above, and the browser program 200 and the native program 300 is not particularly limited. This can be realized by dynamically or statically linking the browser program 200 and the native program 300 with the browser program 200 and the native program 300, hereinafter referred to as "GUI providing program". Further, it can be realized by providing a software interface between the GUI providing program and the browser program 200 and the native program 300 so that data can be exchanged.

[browser program]
Next, the browser program 200 will be described with reference to FIG. FIG. 2 is a block diagram showing the configuration of the browser program according to this embodiment. The browser program 200 acquires a structured document file written in a structured document such as HTML or XML, analyzes the structured document file, and based on the analysis result, embeds in the structured document file or , which has the ability to draw the referenced element to the screen.

The browser program 200 includes a user interface (UI) 201 , browser engine 202 , rendering engine 203 , networking module (communication section) 204 , script engine 205 and data storage (storage section) 206 .

The user interface 201 displays UI elements (back button, forward button, address bar, bookmark, update button, stop button, home button, etc.) for general web browsing on the display unit 111 (see FIG. 1), It is configured to receive input from the user to the relevant screen element via the input unit 112 (see FIG. 1). However, when browser program 200 is used to build a GUI as in this embodiment, user interface 201 does not necessarily need to provide UI elements for general web browsing.

The browser engine 202 is configured to be able to transmit input from the user interface 201 to the rendering engine 203 and operate the rendering engine 203 according to input from the user.

The rendering engine 203 is configured to interpret, render, and display browser-viewable content on the display.

The networking module 204 is an example of a communication unit, and is configured to transmit HTTP requests to servers, for example.

Script engine 205 is configured, for example, to interpret and execute script files contained in browser-viewable content. In this embodiment, the script is Javascript, but is not particularly limited. However, in the present embodiment, the script engine 205 is not necessarily required if the browser program 200 only interprets and draws static content, which will be described later, and displays a UI screen on the display unit.

Data storage 206 is configured to store data such as cookies.

Furthermore, the browser program 200 is configured to be able to cooperate with the GUI providing section 101 as described above.

Processing in the rendering engine 203 will be described below. Rendering engine 203 parses the HTML file and generates a DOM tree. The rendering engine 203 converts tags described in the HTML file into DOM nodes on the tree to generate a DOM tree. A DOM tree is a tree-like collection of all elements contained in an HTML file. A DOM node is each element contained in the DOM tree.

If the HTML file contains CSS (Cascading Style Sheet) or is linked to an external CSS file, the rendering engine 203 analyzes the style data consisting of style elements and renders the CSS. Convert to tree.

Processing for analyzing such HTML files and CSS files is called analysis (parsing) processing.

The rendering engine 203 also builds a rendering tree by combining the DOM tree and the rendering tree.

The rendering engine 203 also determines the locations of DOM nodes stored in the rendering tree. This processing is called layout processing. More specifically, the rendering engine 203 adds "position" and "size" information to renderers added to the rendering tree.

Also, the rendering engine 203 displays the DOM nodes stored in the rendering tree on the display unit 111 according to "position" and "size". This process is called painting process.

Such rendering tree construction, layout processing, and painting processing are collectively referred to as drawing processing.

[Native program]
Next, the native program 300 will be described with reference to FIG. FIG. 3 is a block diagram showing the configuration of the native program 300 according to this embodiment. As shown in FIG. 3A, native program 300 may control hardware 121 via middleware 301 and operating system (OS) 302, and as shown in FIG. The hardware 121 may be directly controlled without a system (OS). For convenience of explanation, only one native program 300 is shown here, but a plurality of native programs 300 may exist at the same time.

[User interface provision method]
A method for providing a user interface according to this embodiment will be described below. FIG. 4 is a flow chart showing an example of a user interface providing method according to this embodiment.

First, static content generation section 102 of GUI providing section 101 (see FIG. 1) dynamically generates static content based on template 104 and screen design information 105 according to an event (ST101).

Next, browser program 200 interprets the static content and causes display unit 111 to display a screen (ST102).

Next, screen control section 103 receives information about event occurrence from browser program 200 (ST103).

After that, the screen control unit 103 outputs control information regarding control of the hardware 121 of the embedded device corresponding to the event to the native program 300 (ST104).

FIG. 5 is a schematic diagram showing an example of a dynamic GUI provided by the user interface providing method according to this embodiment.

One example of a GUI described herein is a dynamic GUI for a lighting control device. As shown in FIG. 5, first, a menu screen 410 is displayed for selecting an object to be controlled from Light A or Light B, which are two lighting devices.

When the first menu button 411 of the two menu buttons 411 and 412 displayed on the menu screen 410 is selected, a control screen 420 for controlling Light A is displayed. The control screen 420 includes an ON button 421 and an OFF button 422 . When the control screen 420 is first displayed, it can be seen at a glance that the ON button 421 is enabled (operable state) and the OFF button 422 is disabled (unoperable state). The OFF button 422 is grayed out.

When the user touches the ON button 421 on the control screen 420 to turn on Light A, the ON button 421 of the control screen 420 is disabled, the OFF button 422 is enabled, and the ON button 421 is disabled. As can be seen at a glance, the ON button 421 is grayed out. On the other hand, the OFF button 422 changes to its standard display state so that it can be seen at a glance that the OFF button 422 is enabled. In this manner, the display states of the ON button 421 and the OFF button 422 on the control screen are updated in response to the occurrence of an event in which the ON button 421 is touched by the user's operation on the control screen 420 .

A case in which such a dynamic GUI is realized by the user interface providing apparatus 100 according to this embodiment will be described below.

First, a case where a dynamic GUI is realized by dynamic contents in a conventional user interface providing device will be described. FIG. 6 is an explanatory diagram showing an example of a structured document forming dynamic content. A structured document 600 is composed of a style sheet part 601 , a content part 602 and a script part 603 .

The stylesheet part 601 includes a stylesheet that controls the display format (appearance, appearance, etc.) of the structured document 600 . A stylesheet is described using a stylesheet language such as CSS, for example.

The content part 602 is the main body of the structured document 600, and the content described here is displayed on the screen. In this embodiment, content part 602 includes, for example, at least one block. Here, a block refers to each of a plurality of parts when the content described in the content part 602 is divided into a plurality of parts using div elements, for example.

In the content part 602, for example, the attribute "class" can be used together with the div element and the tag "button" to individually specify the display format for the display content included in the block.

For example, in this embodiment, first, a class name is given to a block. Then, in the style sheet part 601, a class name can be specified for the style with the attribute "class", and the style can be applied only to the blocks to which the class name is assigned.

A script part 603 contains a script that dynamically processes the structured document. A script is written in a scripting language such as Javascript, for example.

Note that the structured document is composed of one file in the description here, but may be composed of a plurality of files. For example, the stylesheet part 601 may be a separate file (external stylesheet), and the script part 603 may be a separate file (external script).

The browser program interprets the structured document 600 for dynamic content as described above, applies the style sheet described in the style sheet part 601 to the content described in the content part 602, and displays the GUI on the screen. Along with displaying, the script described in the script part 603 can be executed to add dynamic processing to the GUI.

On the other hand, in the user interface providing apparatus 100 according to the present embodiment, the static content generation unit 102 is configured such that at least a portion of the static content generation unit 102 is composed of a structured document file based on the template and screen design information for the user interface. Generate static content dynamically.

FIG. 7 is an explanatory diagram showing an example of a template used by the user interface providing apparatus 100 according to this embodiment. A template 700 shown in FIG. 7 is composed of a structured document.

A template 700 is composed of a style sheet part 701 and a content part 702 . Stylesheet part 701 and content part 702 are similar to stylesheet part 601 and content part 602 of structured document 600 for dynamic content described with reference to FIG. In this example, template 700 does not contain a script part.

In the selector "container" of the style sheet part 701 of the template 700, the size of the control screen 420 (see FIG. 5), the layout of the elements included in the content part 702, and the background color are defined in the same manner as the dynamic content. More specifically, in the template 700, for example, the argument (value) of the property "background-color" that specifies the background color is set to the variable name "bgc1", thereby changing the background color of the control screen 420. display format.

Also, the size, font size and color of the ON button 421 and the OFF button 422, and the background color are defined in the selector "btn". More specifically, in the template 700, for example, the argument of the property "color" that specifies the font color is the variable name "fontc1", and the argument of the property "background-color" that specifies the background color is the variable name " bgc2", by which the font color and background color of the ON button 421 and the OFF button 422 can be changed.

Also, in the selector "btn-on-active", a background color is defined when the ON button 421 and the OFF button 422 are active. In the template 700, the variable name "bgc3" is set as the argument of the property "background-color" that specifies the background color, and the ON button 421 and the OFF button 422 are enabled and operable. The display format is such that the background color can be changed for ease of viewing.

In the selector "btn-off-active", a background color is defined when the ON button 421 and the OFF button 422 are not active. In the template 700, the argument of the property "background-color" that specifies the background color is set to the variable name "bgc4". It has a color changeable display format.

In the block with the class name "container" of the content part 702, two elements, that is, the ON button 421 and the OFF button 422 (see FIG. 5) are defined using the tag "button". First, in the tag "button", character strings "ON" and "OFF" to be displayed on the ON button 421 and the OFF button 422 are defined in the portions enclosed by <button> and </button>.

Also, in the tag "button", the class name specified by the selector in the style sheet part 701 is given using the attribute "class". " are set to variable names "class1" and "class2", respectively, so that the display format to which the ON button 421 and the OFF button 422 are applied can be changed.

Note that this is just an example of properties and attributes whose arguments are defined with variable names. It may be set so that the display format can be changed.

Also, in the tag "button", the part enclosed by <button> and </button> is set with a variable name so that the character strings displayed on the ON button 421 and the OFF button 422 can be changed. good.

In user interface providing apparatus 100 according to the present embodiment, static content generating section 102 (see FIG. 1) of GUI providing section 101 refers to screen design information 105 based on template 700 as described above, and generates static content. Generate targeted content.

8 and 9 are explanatory diagrams showing an example of static content generated by the user interface providing device according to this embodiment. 8 shows static content A when the ON button 421 is enabled on the control screen 420 shown in FIG. 5, and FIG. 9 shows static content A when the OFF button 422 is enabled on the control screen 420. B.

First, as a point common to the static contents A and B, in the style sheet parts 801 and 901, the variable name "bgc1 ", "bgc2", "bgc3", "bgc4", and "fontc1" are set with color codes.

Also, as shown in FIG. 8, in the content part 802 of the static content A, the variable name "class1" of the attribute "class" of the first tag "button" (corresponding to the ON button 421) has the class name "btn". ", "btn-on-active" is set.

Also, in the content part 802 of the static content A, the class name "btn" is set to the variable name "class2" of the attribute "class" of the second tag "button" (corresponding to the OFF button 422), The attribute "disable" is set and the OFF button 422 is disabled.

On the other hand, as shown in FIG. 9, in the content part 902 of the static content B, the variable name "class1" of the attribute "class" of the first tag "button" (corresponding to the ON button 421) has the class name "btn". " is set, the attribute "disable" is set, and the ON button 421 is disabled.

In the content part 902 of the static content B, the class names "btn", "btn-on- active" is set.

In this embodiment, the screen control unit 103 of the GUI providing unit 101 (see FIG. 1) controls the static content A, B based on the screen transition information included in the screen design information 105 according to the operation status of the lighting equipment. is processed by the browser program 200 and displayed on the display unit 111 .

For example, when the lighting device is not turned on, the screen control unit 103 causes the browser program 200 to process the static content A based on the screen transition information, and controls the display unit 111 so that the ON button 421 is enabled. Display screen 420 .

When the ON button 421 of the control screen 420 is operated by the user via the input unit 112 , the screen control unit 103 receives information regarding the occurrence of an operation event for the ON button 421 from the browser program 200 . The screen control unit 103 acquires control information corresponding to the operation event based on the screen design information 105, outputs the control information to the native program 300, and executes control to turn on the lighting device.

When the control is successfully executed, the native program 300 sends status information indicating that the lighting device is turned on. In response, the screen control unit 103 causes the browser program 200 to process the static content B based on the screen transition information, and causes the display unit 111 to display the control screen 420 with the ON button 421 disabled.

As described above, the control screen 420 is switched as shown in FIG.

As described above, according to the user interface providing apparatus 100 according to the present embodiment, a dynamic GUI can be realized using static content. , test items can be reduced.

In addition, since the screen control unit 103 of the GUI providing unit 101 switches screens by reflecting feedback from the native program 300, the GUI can be changed at an appropriate timing according to changes in the operating state of the embedded device. You can save the trouble of testing to see if it is

According to the present embodiment described above, a GUI rich in dynamic expressions can be developed in a short period of time and at low cost for an embedded device.

[Configuration of embedded device]
FIG. 10 is a diagram showing an example of the functional configuration of an embedded device to which the user interface providing device according to this embodiment is applied. As shown in FIG. 10 , the embedded device 10 has a control section 11 , a storage section 12 , a communication section 13 , an input section 14 and an output section 15 . It should be noted that FIG. 10 mainly shows the functional blocks characteristic of the present embodiment, and the embedded device 10 may also have other functional blocks necessary for other processing. Moreover, it is good also as a structure which does not include some functional blocks.

The control unit 11 controls the embedded device 10 . The control unit 11 can be configured by a controller, a control circuit, or a control device, which will be explained based on common recognition in the technical field related to the present invention.

For example, the control unit 11 may perform controls necessary for a specific application of the embedded device 10 . The control unit 11 may receive data or transmit data through communication of the communication unit 13 .

Further, for example, if the embedded device 10 is a controller for lighting equipment, the control unit 11 may include functions for controlling lighting, turning off, brightness, color tone, and the like of the lighting equipment.

Further, for example, when the embedded device 10 is a controller of an automatic washing machine, the control unit 11 may include a function of controlling washing time, rinsing time, spin-drying time, etc. of the automatic washing machine.

The storage unit 12 stores (holds) information used in the embedded device 10 . For example, the storage unit 12 may store the template 104 and screen design information 105 (see FIG. 1). The storage unit 12 can be configured by, for example, a memory, a storage, a storage device, and the like, which are explained based on common recognition in the technical field of the present invention.

The communication unit 13 may communicate with other communication devices (embedded devices, servers, etc.) via the Internet. Further, the communication unit 13 may output various received information to the control unit 11 .

The communication unit 13 can be configured by a transmitter/receiver, a transmitting/receiving circuit, or a transmitting/receiving device described based on common recognition in the technical field related to the present invention. Note that the communication unit 13 may be composed of a transmission unit and a reception unit.

The input unit 14 receives an input by a user's operation. Also, the input unit 14 may be connected to a predetermined device, storage medium, or the like, and receive input of data. The input unit 14 may output the input result to the control unit 11, for example.

The input unit 14 can be composed of input devices such as a keyboard, mouse, and buttons, input/output terminals, input/output circuits, and the like, which will be explained based on common recognition in the technical field of the present invention. Also, the input unit 14 may be integrated with the display unit (for example, a touch panel display).

The output unit 15 draws and outputs content. For example, the output unit 15 may include a display unit that displays text, images, and the like, an audio output unit that outputs audio, and the like.

The display unit 111 (see FIG. 1) can be configured by, for example, a display device such as a display or a monitor, which are explained based on common recognition in the technical field of the present invention. Also, the audio output unit can be configured by an output device such as a speaker, which is explained based on the common recognition in the technical field related to the present invention.

The output unit 15 includes, for example, an arithmetic unit, an arithmetic circuit, an arithmetic device, a player, an image/video/audio processing circuit, an image/video/audio processing device, It can be configured to include an amplifier or the like.

[Hardware configuration]
It should be noted that the block diagrams used in the description of the above embodiments show blocks in units of functions. These functional blocks (components) are implemented by any combination of hardware and/or software. Further, means for realizing each functional block is not particularly limited. In other words, each functional block may be realized by one physically connected device, or may be realized by two or more physically separated devices connected by wire or wirelessly. good.

FIG. 11 is a diagram showing an example of the hardware configuration of an embedded device to which the user interface providing device according to this embodiment is applied. The embedded device 10 described above may be physically configured as a computer device including a processor 1001, a memory 1002, a storage 1003, a communication device 1004, an input device 1005, an output device 1006, a bus 1008, and the like.

Note that in the following description, the term "apparatus" can be read as a circuit, device, unit, or the like. The hardware configuration of the embedded device 10 may be configured to include one or more of each device shown in the figure, or may be configured without some of the devices.

For example, although only one processor 1001 is shown, there may be multiple processors. Also, processing may be performed by one processor, or processing may be performed by one or more processors concurrently, serially, or otherwise.

Each function in the embedded device 10 is performed by the processor 1001 performing calculations by loading predetermined software (programs) on hardware such as the processor 1001 and the memory 1002, communication by the communication device 1004, and communication in the memory 1002 and the storage 1003. It is realized by controlling reading and/or writing of data.

The processor 1001, for example, operates an operating system to control the entire computer. The processor 1001 may be configured by a central processing unit (CPU) including an interface with peripheral devices, a control device, an arithmetic device, registers, and the like. Note that each unit such as the control unit 11 described above may be implemented by the processor 1001 . Processor 1001 may be implemented by one or more chips.

The processor 1001 also reads programs (program codes), software modules, data, etc. from the storage 1003 and/or the communication device 1004 to the memory 1002, and executes various processes according to them. As the program, a program that causes a computer to execute at least part of the operations described in the above embodiments is used. For example, the control unit 11 may be implemented by a control program stored in the memory 1002 and running on the processor 1001, and other functional blocks may be similarly implemented.

The memory 1002 is a computer-readable recording medium, for example, ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically EPROM), RAM (Random Access Memory), and at least other appropriate storage media. may be configured by one. The memory 1002 may also be called a register, cache, main memory (main storage device), or the like. The memory 1002 can store executable programs (program codes), software modules, etc. for implementing the advertisement distribution method according to the present embodiment.

The storage 1003 is a computer-readable recording medium, for example, a flexible disk, a floppy (registered trademark) disk, a magneto-optical disk (for example, a compact disk (CD-ROM (Compact Disc ROM), etc.), a digital versatile disk, Blu-ray disc), removable disc, hard disk drive, smart card, flash memory device (e.g., card, stick, key drive), magnetic stripe, database, server, or other suitable storage medium may be configured by Storage 1003 may also be called an auxiliary storage device. Note that the storage unit 12 described above may be implemented by the memory 1002 and/or the storage 1003 .

The communication device 1004 is hardware (transmitting/receiving device) for communicating between computers via a wired and/or wireless network, and is also called a network device, network controller, network card, communication module, or the like. Note that the communication unit 13 described above may be implemented by the communication device 1004 .

The input device 1005 is an input device (for example, keyboard, mouse, etc.) that receives input from the outside. The output device 1006 is an output device (eg, display, speaker, etc.) that outputs to the outside. Note that the input device 1005 and the output device 1006 may be integrated (for example, a touch panel). The input unit 14 and the output unit 15 described above may be realized by the input device 1005 and the output device 1006, respectively.

Devices such as the processor 1001 and the memory 1002 are connected by a bus 1008 for communicating information. The bus 1008 may be configured with a single bus, or may be configured with different buses between devices.

The embedded device 10 includes hardware such as a microprocessor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array). may be configured, and a part or all of each functional block may be realized by the hardware. For example, processor 1001 may be implemented with at least one of these hardware.

[Program structure]
A program that implements the user interface providing method according to the present embodiment includes, for example, a GUI providing program, a browser program 200, and a native program 300, as described with reference to FIG. It is not limited to this. For example, the GUI providing program, browser program 200 and native program 300 may be one program. Also, the GUI providing program and the browser program 200 may be one program.

In addition, the browser program 200 may be configured to interpret and draw Internet content in the same manner as a general Internet browser program. It may be a configuration that can interpret and draw only.

Also, the GUI providing program may be in the form of a library. A library is a collection of highly versatile programs in a reusable form, and cannot be operated as a single program, that is, does not have to be in the form of an executable file. In other words, a library may be a collection of code that provides some functionality to other programs.

[Modification]
In addition, although an example was shown in the above-described embodiment, the present invention is not limited to this. In the above-described embodiments, the case where the user interface providing device is the embedded device itself has been described as an example. I don't mind.

Also, the terms described in this specification and/or terms necessary for understanding this specification may be replaced with terms having the same or similar meanings.

The information, parameters, etc. described herein may be represented by absolute values, by relative values from a predetermined value, or by corresponding other information. Also, the names used for parameters and the like in this specification are not restrictive in any respect.

Information, signals, etc. described herein may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may refer to voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, light fields or photons, or any of these. may be represented by a combination of

Information, signals, etc. may be input and output through multiple network nodes. Input/output information, signals, and the like may be stored in a specific location (for example, memory), or may be managed using a table. Input and output information, signals, etc. may be overwritten, updated or appended. Output information, signals, etc. may be deleted. Input information, signals, etc. may be transmitted to other devices.

In addition, the notification of the predetermined information (for example, the notification of "being X") is not limited to being performed explicitly, but may be performed implicitly (for example, by not notifying the predetermined information). may

Software, whether referred to as software, firmware, middleware, microcode, hardware description language or otherwise, includes instructions, instruction sets, code, code segments, program code, programs, subprograms, and software modules. , applications, software applications, software packages, routines, subroutines, objects, executables, threads of execution, procedures, functions, and the like.

Software, instructions, information, etc. may also be sent and received over a transmission medium. For example, the software uses wired technology (coaxial cable, fiber optic cable, twisted pair and digital subscriber line (DSL), etc.) and/or wireless technology (infrared, microwave, etc.) to access websites, servers, or other When transmitted from a remote source, these wired and/or wireless technologies are included within the definition of transmission media.

As used herein, the terms "system" and "network" are used interchangeably.

Each aspect/embodiment described in this specification may be used alone, may be used in combination, or may be used by switching according to execution. Also, the processing procedures, sequences, flowcharts, etc. of each aspect/embodiment described in this specification may be rearranged in order as long as there is no contradiction. For example, the methods described herein present elements of the various steps in a sample order, and are not limited to the specific order presented.

As used herein, the phrase "based on" does not mean "based only on," unless expressly specified otherwise. In other words, the phrase "based on" means both "based only on" and "based at least on."

Any reference to elements using the "first," "second," etc. designations used herein does not generally limit the quantity or order of those elements. These designations may be used herein as a convenient method of distinguishing between two or more elements. Thus, references to first and second elements do not imply that only two elements may be employed or that the first element must precede the second element in any way.

Although the invention according to the present invention has been described in detail above, it will be apparent to those skilled in the art that the invention according to the present invention is not limited to the embodiments described herein. The invention according to the present invention can be implemented as modifications and changes without departing from the spirit and scope of the invention determined based on the description of the claims. Therefore, the descriptions in this specification are for the purpose of illustration, and do not impose any limitation on the invention according to the present invention.

As described above, there is an effect that a GUI rich in dynamic expressions can be developed in a short period of time and at a low cost for embedded devices, and it is particularly useful for IoT devices.

10 embedded device 11 control unit 12 storage unit 13 communication unit 14 input unit 15 output unit 100 user interface providing device 101 GUI providing unit 102 static content generation unit (static content generation means)
103 Screen control unit (control request means)
104 template 105 screen design information (screen design information)
111 display unit 112 input unit 121 hardware 200 browser program 201 user interface 202 browser engine 203 rendering engine 204 networking module 205 script engine 206 data storage 300 native program 301 middleware 410 menu screen 411 menu button 412 menu button 420 control screen 421 ON button 422 OFF Button 600 Structured Document 601, 701, 801, 901 Style Sheet Part 602, 702, 802, 902 Content Part 603 Script Part 700 Template

Claims (5)

static content generation means for dynamically generating static content , at least a portion of which is composed of a structured document file, according to an event based on template and screen design information for the user interface ;
To the native program that interprets the static content, acquires user operation information regarding the user's operation on the screen from the browser program that displays the screen on the display unit, and controls the embedded device and control request means for outputting control information relating to control of the embedded device corresponding to the user operation information. 2. The user interface according to claim 1, wherein the event is a user's operation performed on the screen detected by the browser program, and updates at least a part of the components of the screen. Device. An embedded device comprising the user interface providing device according to claim 1 or 2. executed by an embedded device,
dynamically generating, in response to an event, static content , at least a portion of which is a structured document file, based on template and screen design information for the user interface ;
To the native program that interprets the static content, acquires user operation information about the user's operation on the screen from the browser program that displays the screen on the display unit, and controls the embedded device a step of outputting control information related to control of the embedded device corresponding to the user operation information;
A method for providing a user interface, comprising: embedded devices,
a procedure for dynamically generating static content , at least a portion of which is composed of a structured document file, according to an event, based on a template and screen design information for a user interface ;
To the native program that interprets the static content, acquires user operation information about the user's operation on the screen from the browser program that displays the screen on the display unit, and controls the embedded device a step of outputting control information related to control of the embedded device corresponding to the user operation information;
A program characterized by causing the execution of

Images