DE10150141A1 - Device for acquisition and representation of movements, e.g. for use in capture of handwriting or sketches drawn on a plasma type display, and also for use as a conventional mouse type device with single hand operation - Google Patents

Abstract

Device for acquisition and presentation of movements comprises a housing (110), a position acquisition unit attached to the housing for acquisition of current position data or a reference or cursor point, data processing means for processing the position data and a display unit (112) for presentation of an image (122) of the acquired processed position data. The invention also relates to a corresponding method for control of an inventive device.

Description

The invention relates to a method for controlling a Order, an order to carry out the method and a program product.

A pin-shaped mobile phone is known from EP 907 278 A3. For this pen-shaped mobile phone, there is in addition to the Entering numbers, letters and other characters Need to use the rest of the mobile phone this type of input so that a user can connect not always between a pen position in which he Cellphone like writing with a pen on the surface, and have to change a keyboard position. So it is for Example annoying, after entering an SMS the cell phone from the Pen posture to take the keyboard posture to initially complete the entry, send the SMS with selection of address data from a telephone book, an acknowledgment of the associated phone number and confirming the Send process to the point that the SMS can really be shipped.

In the cell phones on the market today it is purely keyboard-operated and additionally pen-operated devices, the so-called smartphones. The purely keyboard-operated Devices do not allow the user direct manipulation the user interface. In contrast, with PDAs (Personal Digital Assistants) or smartphones a direct one Manipulation of a graphic displayed on the phone User interface possible. However, one is Manipulation always on a pointing device (a pen or a Mouse) bound to one of them more or less non-mobile pointing surface carries out the manipulations. is the manipulation area is too small or the pointing point is located outside the visible area, so must be from the direct Manipulation switched to scrolling or navigation become. This means that in the first phase the User interface in a non-mobile way by the user is operated with the help of a pen, while in the second Phase the user interface is moved so that the desired target object can be reached again in the sense of the first Phase is. Thus, the cognitive and handling effort just by switching between the two phases considerably.

Proceeding from this, the invention is based on the object Arrangement control method and arrangement to provide for the implementation of the procedure at the one user the change between different Control and operating concepts are spared. Farther should be avoided that the user unnecessarily by it is charged with manual scrolling. After all, complex operations are intended for the user become clearer. When arranging different, previously necessary input components become dispensable, which makes the arrangement more cost-effective will be manufactured and based on the aforementioned Criteria the handling of the arrangement is facilitated.

This task is accomplished through a method of controlling a Arrangement, an arrangement that is established and means has to carry out the method and a program product solved with the features of the independent claims.

In the method of controlling an arrangement are through the arrangement of objects can be processed. The objects are in Form of actions and content. The actions are for yourself or executable on the content. The objects are for Operation spatially arranged in an operating room. Separate the objects spatially arranged in the operating room can now can be strung together by the user of the arrangement by the respective position of the objects in the operating room becomes. Each object whose position in the Operating room is driven, in a row added. The sequence can in particular be one Be stack. The alignment of the objects is done by a final operation completed.

After that, the stringing of objects can either be direct be processed by the arrangement or it is used as a macro saved. Then the sequence can be modified, by later canceling the final operation.

Alternatively, the sequence can already be during the Worked in a row.

The final operation can be done in a variety of ways become. So it is conceivable to stop in the operating room takes longer than a given period of time to withdraw the mobile phone in pen form from the ground, a run over a final action as an object in the control room or, for example the push of a button of the arrangement.

In methods for controlling arrangements according to the prior art In technology, the objects are strung together in time. It is usually only possible to have two objects to marry once. One can also look at known ones Move user interfaces around the room by looking at individual Objects in menus and submenus or folders and subfolders place. Here, however, there is no string of Objects instead of just individual objects consulted. More than two objects on different Positions are in the process of controlling the arrangement, can only be connected to each other by that first married the first two objects and then after performing the related actions by arranging the result of the actions with a third object is married and immediately.

In contrast to the one-dimensional time path for procedures according to the state of the art can be claimed Moving multidimensionally in space to objects to line up. Although the spatial representation such a string of objects to represent Structure and flow diagrams for creating software known to be implemented in a control method however, there is no arrangement.

The control room is virtual. To give the user alternatives in to operate the arrangement, he must be multidimensional. A two-dimensional control room leaves with those available at the time of registration Display means particularly well and is therefore particularly preferred. A three-dimensional control room can on the other hand be useful if there are more operating alternatives should result or to the way in the operating room at the Make the line-up of objects as short as possible. Navigation in an operating room with more than three Experience has shown that dimensions prepare the user Trouble.

In general, when performing the method of multi-dimensional control room on a one-dimensional Stringing reduced by the serial arrangement is processed. For the control of parallel processing Arranging is also creating multiple, together linked, at least partially parallel, as a rule but one-dimensional alignment is possible again. The same applies if there are several similar objects on the same place in the sequence the same action is to be carried out. Then you can look at the sequence at this point either to the several similar objects split up and then think together again or one considers the several similar objects as one Total object.

The control room is preferably at least partially shown. This is almost essential, especially when the user first starts using the procedure for Control familiarizing the arrangement or with such Wants to control a new arrangement unknown to him. In the further course, however, this need does not apply if the user uses the routes he uses in the control room has internalized.

As a rule, the entire operating room is not shown can be, the section of the Operating room that is currently being visited.

The operating room is for a particularly easy orientation shown as a map. The possible routes can also be used be displayed in the operating room. Usually are by no means all lines up. Objects make sense. By showing the possible routes, the possible ones Sequences visualized. The representation of the possible routes are made by lines between the Objects.

Analogous to and regardless of the representation of the possible It also makes sense to only use the possible routes in the Specify control room and / or the user only the possible Let paths be taken in the operating room. This can an operator error already when the objects are lined up avoided, which would otherwise only occur when a incorrect sequence of objects processed would.

In order to be able to find the objects in the operating room, the Objects do not have to be punctiform, but an extension of have reasonable size. To do this, the objects are as cells educated. In particular, the cells are bubbles and / or Cells are represented as bubbles.

Specifying or enabling possible routes in the operating room is a dynamic process. Depending on which objects previously the possible paths change. Accordingly, certain ways in. Control room only released if the associated requirements are met were. You can do this with the analogy of a Illustrate traffic lights. As long as the user doesn't the necessary objects in the form of content or actions strung together, the traffic light stands for the of red path. The user can on the other hand, recognize that the search by the red Traffic lights on blocked roads under different conditions is possible in principle. Therefore, the user first takes another way in the control room to get the necessary Requirements. Then he returns to the Traffic light back, which now shows green and the from User releases the desired route as the currently possible route or pretends.

It is crucial for the efficiency of the process to make the control room as sensible as possible. One criterion is that the length of the paths in the Operating room from one current object to the next Objects are dependent on the respective probability, with the respective next object and the current object is strung together. As short as possible in the course of the request Paths in the control room will be the next object with the largest Probability associated with the shortest path. So you can optimize the process by the user in the Routes to be traveled are minimized overall.

The control room is optimized at the Creation, usually by the manufacturer of the product controlling arrangement is made or initiated. Alternatively or in addition, the operating room can also be operated by the User, especially interactive, created, changed and / or be optimized.

The operating room can also be operated by the user other aspect can be changed. The user can namely change the control room so that certain characteristic movements very specific Sequences of objects can be assigned. Thereby a certain movement becomes a macro for a certain control of the system by stringing together Actions and content. So the straight line is ideal for the most commonly used string of objects. A straight Control movement by the user can therefore for that Macro stand: Send SMS to Jim. One movement, one Angle describes, for example, the following Trigger control of the arrangement: Send SMS-to-Tom. Other Movements that lend themselves to macros are those that simple geometric objects, such as a circle or Triangle, or lines, for example wave or zigzag, describe.

Preferably one can from a current one. Object to the next Object most likely to match the current object strung together, get through a jump operation (Tele Porter). This saves manual driving the path between the two objects. Such Jump operation can be done, for example, by pressing a button on the Order or triggered by other gestures. After this You can either trigger the jump operation immediately most likely object, what the fastest course of action is, or you can in the representation of the control room automatically follow the way, what in particular is clearer in an unknown operating room. Depending on Awareness of the operating room for the user is a Switching between the two alternatives makes sense.

Typically, small graphic display means with the Problematic burden that the user on a complex and extensive user interface more or less with the "Lost-in-Space" problem is faced: Navigates to Example in a directed count, it can be common occur that the display means empty and without any Information for him is.

To solve the "lost in space" problem are the following Process refinements proposed. First of all while already lining up, the one already set up Sequence of objects shown. The representation can be done, for example, in a window next to a marker that indicates the current position of the Indicates user in the operating room. This will give the user the covered distance represented what its orientation and its cognitive ability of the context is very beneficial.

Furthermore, the distance to the various next linguistically meaningful, that is possible objects in the control room. The distance between two objects Probability of lining up the two objects given in the given context. If the objects in Control room can already be arranged as described above, that the path difference of the probability of Stringing corresponds, the distance is simply that Path difference. The direction of the object with the minimum Distance is shown in the representation of the control room, for example an arrow. Simultaneously or alternatively the next object to the current object, which on most likely strung together with the current object is displayed in a preview. The preview is done as the review already mentioned, preferably in one Window. The window will look in the direction of the last one Movement in the control room is shown next to the mark that indicates the current position of the user in the operating room.

Alternatively, you can preview the most likely object also appear at the top of the arrow indicating the Direction to the most likely object. This is particularly useful if not only that most likely object, that is the object with the smallest distance, in a preview and or with a Directional arrow is displayed, but also that second most likely, that is, the object with the second smallest distance and possibly also that third most likely, that is, the object with the third smallest distance.

You get to the object shown in the preview either by following the direction of the arrow, or by the jump operation described above.

An object can be made up of a series of objects be taken out by again, especially in reverse direction. This is especially true for the last object in the sequence.

Selecting an object from a list of objects that is arranged at a position in the control room preferably by a movement in the operating room, which in Essentially perpendicular to the movement between the position of the List and the previous object is. The list of In turn, objects is an object.

The sequence preferably contains at least three, especially at least five objects.

With the claimed method, there is also a completely novel and easy-to-use way of controlling and operation of the arrangement in a natural language and map their grammar. Under a natural language is human language understood in the linguistic sense, for example in the form of English, Chinese or German.

Today's user interfaces use human language in a more or less limited way to Make operating processes understandable. This limitation is expressed, for example, in the fact that menus in Applications mostly with a subject, for example "file" or "View", then start with an associated one Verb, for example "save" or "open" continue. Sentences occur similarly, which are also only from a verb and a subject, but for which the Order is reversed, leaving the verb in front of the subject stands. Such primitive sentences are then in one confusing variety offered and the user stands facing the problem of a complex task by a particular one Coping with these primitive sentences. This will make the user aware of the possibility of it potential language skills cheated and what special is annoyed, forced into a kind of language that him sometimes completely foreign, restrictive and sometimes degrading must occur. For example, if the user State-of-the-art cell phones pursue the goal write an SMS and then send it, so the Users move through menus and submenus for example the words "messages", "SMS", "new text create "etc.

The procedure gives him the opportunity to Simply put it in the form: "I want one Write a message to Mr. X and send it as SMS " the procedure becomes a grammatical for the user supported formulation of the problem. The User combines verbs, nouns and prepositions with Help the procedure so that on the one hand the classic Preserve functionality of the menu navigation of a mobile phone remains, but also a meaningful sentence den Describes overall control flow.

This goal is achieved by arranging the Objects in the operating room and / or the possible routes in the Control room follows the rules of a natural language or consequences. The grammar is especially under the rules understood the natural language. To achieve this goal reach, it may be possible that the control room during the lining up of objects interactively by the process is changed, for example by the position of objects is changed, old objects deleted or new objects to be added. Sometimes this is the only way take into account the complexity of a natural language. With the interactive change of the operating room by the Procedure, but also when creating an operating room, that takes into account the rules of a natural language, linguistic rules can be advantageously used, especially those that are already for mechanical Translations are formulated.

An interactive change of the operating room through the Procedures can also be useful in other cases.

When performing the method, it is advantageous if a haptic and / or acoustic feedback is made. Such feedback can, for example, in the event of an error, especially following an impossible path, respectively. However, if a macro is recognized, the means a recognized movement or gesture that leads to a preferred sequence of objects, are reproduced. This will be advantageous different macros different feedback assigned. Another option is an acoustic one Feedback. A buzz is particularly useful here, when the jump operation is performed.

Furthermore, a text-to-speech function can be provided with which objects are read aloud. This is especially for those shown in the preview and / or the review Objects make sense. An arrangement that is set up one to carry out the described processes can be carried out at Example by programming and setting up accordingly a data processing system.

The arrangement preferably has an input means that contains a motion sensor.

So the movement of the user or the hand of the Users in real space in one movement in the operating room transform.

The motion sensor can be a mechanical, an optical or be an acoustic motion detector. In particular, he can a mouse ball, a trackball, a gyro sensor, an image sensor and / or an interferrometric sensor.

Preferably, the arrangement including the Input means integrated in a housing.

The arrangement is preferably designed as a mobile terminal, for example as a mobile phone, as a smartphone, as a PDA, as a handheld or as a notebook.

The arrangement expediently also has Display means for displaying the control room.

The arrangement is particularly a pen-shaped mobile phone designed as described in PCT / DE01 / 01979. On in this regard, this document is fully related taken. The arrangement can also be a Interface to another arrangement. This Interface can serve two independent purposes serve. For one thing, the further arrangement can be a larger one Display means have as the arrangement, the Operating room can be shown on the large display. Thereby the arrangement can be small for various uses and be designed to be handy while certain Application purposes the large display of the further arrangement is available. Second, the arrangement for Serve the further arrangement. There is a To construct the control room for the further arrangement and the To provide arrangement. In both cases it is Arrangement expediently a handy mobile terminal, while the further arrangement will be a notebook or a PC can. The interface is especially wireless according to the IRDA or Bluetooth standard implemented.

Further essential advantages and features of the invention result from the description of an exemplary embodiment based on the drawing. It shows

Fig. 1 shows an arrangement which is adapted to perform a method for controlling the assembly;

Fig. 2 is an illustration of an operating space;

Fig. 3 is a representation of the operating area of Figure 2 after a change.

Fig. 4 shows a control room with a lock and

Fig. 5 shows a control room with preview and review.

In FIG. 1 one recognizes an arrangement 1, which is adapted to perform a method for controlling the assembly 1. The arrangement 1 is designed in the form of a mobile telephone which also offers the functionalities which are usually implemented in a PDA or a handheld. These include, for example, appointment management, word processing, audio and video recording and playback as well as spreadsheets. The arrangement 1 has a microphone 3 and a loudspeaker 2 for recording and reproducing sound signals. Furthermore, the arrangement 1 is equipped with a keyboard 4 and a display means 5 . The display means 5 is designed to be touch-sensitive, so that inputs into the arrangement 1 can be made via the display means 5 . A user who wants to operate the arrangement 1 in a conventional manner can therefore do so via the keyboard 4 and / or the display means 5 .

The arrangement 1 also has a movement sensor 6 , which measures the movement of the arrangement 1 with respect to a surface 7 . The motion sensor 6 also contains a distance meter with which the distance of the arrangement 1 from the surface 7 is measured. The movements measured by the movement sensor 6 are only included in the method for controlling the arrangement 1 if the distance falls below a predetermined minimum. Additional control functions can also be triggered via the distance meter, for example an end operation described below, which is carried out as soon as the distance meter reports that a maximum distance between the arrangement 1 and the surface 7 has been exceeded.

Finally, the arrangement 1 also has a wireless interface 8 , which is designed in particular as an infrared interface according to the IRDA standard or as a radio interface according to the Bluetooth standard. The arrangement 1 can communicate with a further arrangement 9 via the interface 8 . The further arrangement 9 is realized in the exemplary embodiment shown as a PC. The further arrangement 9 in turn has an interface 10 for communication with the arrangement 1 . The further arrangement 9 also has a display means 11 . In contrast to the arrangement 1 , which is designed as a mobile arrangement, the further arrangement 9 is designed as a stationary arrangement. The display means 11 of the further arrangement 9 can thus be made much larger than the display means 5 of the arrangement 1 .

The arrangement 1 is a cell phone that can be operated on the surface 7 like a pen (pen cell phone). So you can make 7 text entries by writing movement of the arrangement 1 on the surface. To do this, either use a special alphabet that can be recognized particularly well and easily by arrangement 1 , or arrangement 1 has means for recognizing the writing of normal cursive.

In addition, however, it is also possible to operate or control the arrangement 1 by moving the arrangement 1 on the surface 7 .

If the arrangement 1 is moved on the surface 7 , a virtual movement of the user interface shown on the display means 5 of the arrangement 1 is initially coupled with it. If the user moves arrangement 1 to the left, the user interface on the display means of arrangement 1 scrolls to the right at this moment. As a result, the user has the illusion that only arrangement 1 is moving, but not the user interface, and that he is always getting new sections on the user interface of any size.

In FIG. 2, one can see the representation of an operating room. Objects 12 , 13 , 14 , 15 , 16 are arranged in the operating room. The objects 12 , 13 , 14 , 15 , 16 are designed as cells which have the shape of bubbles. The objects (files) 12 , 13 , 14 , 15 , 16 are actions (programs) and contents (documents, attributes). Actions can be carried out on their own or on content.

A section 17 of the operating room is shown as part of the user interface on the display means 5 of the arrangement 1 . The position of the user in the control room is usually shown in the center of the display of the control room. This can correspond to the center of display 5 , but it does not have to be. The possible paths between objects 12 , 13 , 14 , 15 , 16 in the operating room are shown by lines.

If the user moves through the control room by moving the arrangement 1 on the surface 7 , the section 17 of the control room shown on the display means 5 in the virtual control room is shifted analogously to the real movement of the user.

The user can thus navigate through the movement of the arrangement 1 in the operating room.

The arrangement 1 communicates with the further arrangement 9 via the interfaces 8 , 10 . On the larger display means of the further arrangement 9 , depending on the size of the display means 11 and the control room, the control room can be shown completely or in a larger section than the cutout 17 on the display means 5 of the arrangement 1 . Thus, the user of the arrangement 1 can get an overview of the control room and the context of his current position in the control room by looking at the display means 11 of the further arrangement 9 . For this purpose, the section 17 , which is displayed on the display means 5 of the arrangement 1 , is preferably symbolized in the display means 11 of the further arrangement 9 by a frame. It is thus easy for the user to see which section of the control room he is currently seeing on the display means 5 of the arrangement 1 .

The operating room can either be an operating room for the control of the arrangement 1 or an operating room for the control of the further, remote arrangement 9 . In the first case, the further arrangement 9 serves to present a larger overview of the operating space of the arrangement 1 . In the second case, arrangement 1 serves as an input device for further arrangement 9 . The arrangement 1 then functions for the further arrangement 9, in particular as a mouse substitute, whereby it simultaneously replaces the keyboard and other operating, control and input means by the method described below. In both cases, the operation of the arrangement 1 , 9 to be operated is facilitated by the simultaneous use of the respective other arrangement 9 , 1 .

For the method for controlling the arrangement 1 , the objects 12 , 13 , 14 , 15 , 16 are spatially arranged in the operating room. The objects 12 , 13 , 14 , 15 , 16 are included in a sequence in the method for controlling the arrangement 1 . For this purpose, their respective position in the operating room is sought. For this purpose, the user moves the arrangement 1 in the real room, as a result of which he moves through the operating room. He searches for an object 12 , 13 , 14 , 15 , 16 in the operating room by moving the arrangement 1 such that the object 12 , 13 , 14 , 15 , 16 is in the center of the display means 5 of the arrangement 1 . Instead of the center of the display means 5 of the arrangement 1 , another area can also be selected, so that an object 12 , 13 , 14 , 15 , 16 is considered to be sought when it comes into contact with this area or is moved into the area. The center or the other area can be indicated by a marking, for example in the form of a cross or a frame.

. In the operating room shown in FIG 2 have the objects 12, 13, 14, 15, 16 the following meaning:
Object 12 "Send"
Object 13 "text"
Object 14 "as email"
Object 15 "as SMS"
Object 16 "to <list of persons>"

First, object 12 is visited in the operating room. Object 12 is an action, namely "send". By searching, the object 12 is included as the first object in the sequence of objects.

Then object 13 is sought. Object 13 is a content, namely "text". Basically, there could be several objects to choose from in the form of a list. List objects are described in detail at object 16 . The present object 13 is, however, a certain content, that is to say text that was previously created by the user by keyboard input or by voice input with subsequent voice recognition by the arrangement 1 or with the participation of the arrangement 1 . Object 13 is included as a second object in the sequence.

Now the user can decide, because there are two possible ways. The first of the two possible paths leads to object 14 . The object 14 has the attribute "as email". This attribute specifies the "send" action of object 12 . Alternatively, the user can visit object 15 in the second possible way. Object 15 has the attribute "as SMS" as content. This attribute also specifies the "send" action of object 12 . In contrast to object 14, the specification is made in such a way that object 13 is not sent as an email, but as an SMS. Furthermore, it is assumed that the user chooses the route via the object 15 by visiting it. This object 15 is then included as a third object in the sequence.

Finally, starting from object 15 , object 16 is sought. In contrast to the previously treated objects, object 16 is not a single object, but a list object. A list object is a content that consists of a list of objects. List object 16 consists of an address book with names and addresses. In the exemplary embodiment shown, the address book contains two entries, a first name 18 "Jim" and a second name 19 "Tom". When the user has reached the list object 16 , he moves the arrangement 1 with a movement component which runs in or against the direction of the list object 16 listing. Accordingly, the list of list object 16 is scrolled up or down. This allows the name of the desired addressee to be marked. In the illustrated embodiment, the name 18 "Jim" is marked. After marking, the name is selected by moving out of the object 16 . The name 18 "Jim" is thus extracted from the object 16 and the object 16 is included as "Jim" in the sequence.

After that, a final operation is performed to complete the sequence. The end operation can consist in pressing a key on the keyboard 4 of the arrangement 1 or in lifting the arrangement 1 from the surface 7 , which is registered by the distance meter of the motion detector 6 .

The objects 12 , 13 , 15 , 16 contained in the sequence are then processed. In the exemplary embodiment explained, this means that the arrangement 1 sends the text to Jim as an SMS.

As an alternative or in addition to processing the sequence, the sequence can also be stored in the arrangement after another end operation, for example by pressing another key on the keyboard 4 .

Instead of including the objects in the sequence by searching only, it may be advantageous to only include the objects in the sequence when the object to be recorded or its cell has been (completely) passed through in the operating room or when the object or object to be recorded or his cell in the control room, another action is taken by the user, which may consist, for example, of pressing a key on the keyboard 4 .

Driving through an object occurs when the object or his cell in the operating room again after the visit is left.

The illustrated method for controlling the arrangement 1 is a generic method for operating arrangements whose physical input movement is directly linked to the display of their user interface. The method is therefore the first choice for all mobile telephones, PDAs, handhelds or other manageable, compact arrangements, in which the display on the display means is also moved when the arrangement is moved.

Over time, the user of the arrangement 1 will identify certain physical movements of the arrangement 1 with certain rows of objects. For the sequence of objects he uses most frequently, the user of the arrangement 1 will therefore no longer be dependent on the display of the control room. He simply executes the familiar movement with the arrangement 1 , which he connects with the sequence of objects that he just wants.

If the user gets used to the efficient operation, it is of great advantage if he is doing more complex Stringing together objects as a whole or wrong can recognize correctly. Here he is informed that his string of objects just made is nonsensical or impossible or the sequence becomes confirmed as correct. To do this, the user receives an acoustic or haptic feedback that conveyed him over Accuracy or incorrectness of the entry in knowledge puts.

In addition, it is also possible to make certain Stringing together determined acoustic or haptic Assign feedback. For example, if Stringing "Send text as SMS to Jim" a major Triad assigned and the sequence "Send text as an email to Tom "a minor triad, the user knows when sounding the appropriate sound not only that he a syntactically correct sequence of objects but also whether it is correct in terms of content Stringing was or not.

A further possibility of acoustic feedback is reading the list element 18 , 19 directly after the selection of the list element 18 , 19 from a list or reading out the sequence before, during or after its processing with the aid of a speech synthesis method.

The operating space can advantageously be changed by the user. For this purpose, for example, a tool can be made available to him with which he can move objects in the operating room. The possible routes are preferably updated automatically. In Fig. 3 is 2 detects the operating room of Fig., After which it was modified by moving the first object 12 and second object 13. Furthermore, in list object 16, list element 18 "Jim" was moved to the center.

While in the arrangement of the objects in the control room according to FIG. 2 created by the manufacturer of the arrangement 1 , the paths for sending text as e-mail or as SMS are the same length, in the arrangement of the objects in the control room according to FIG . clearly the sending of text as an SMS to Jim preferential treatment. 3 To assemble the corresponding series, it only has to pull through arrangement 1 relatively straight from left to right. This is roughly the simplest movement that can be carried out with the arrangement 1 . The associated sequence should accordingly be the most frequently used operating sequence for arrangement 1 . The straight pull is then a gestural macro for this sequence of operations.

Basically, movements in the control room are conceivable in all directions. If, in the exemplary embodiments shown, the main direction of movement in the operating room leads from left to right, it is because this is the easiest procedure for a user who usually writes from left to right. In culture rooms in which writing is usually done from right to left or from top to bottom, the operating room should be modified accordingly in order to make the operation of the arrangement 1 as simple as possible for users from these cultural circles.

In FIG. 4 to 21 22 23 recognizes another operating room, the five objects 20, 24 and includes a lock 25. In this operating room, the arrangement of the objects is first put together as in the previously described operating room. The object 20 and the object 21 are traversed one after the other and thereby included in the sequence. The user then arrives at the lock 25 in the operating room on the following path to the object 24 . The lock 25 is displayed in the form of a traffic light, which shows red if the way to object 24 is blocked for the user and green if the way is currently possible for the user. If the user comes directly from the object 21 to the lock 25 , this is initially active, for which the traffic light shows red. As a result, the user is required to add another object between the object 21 and the object 24 , for example for syntax reasons. This further object can be either object 22 or object 23 . The user therefore moves from the lock 15 either to the object 22 or to the object 23 and thereby takes the selected object in a row. The user then moves back to the lock 25 in the operating room, which is now no longer active, which is why the traffic light shows green, so that the user moves to the object 24 in the then currently possible way and completes the sequence by recording this object 24 can.

An advanced navigation method is explained with reference to FIG. 5, which makes it easier for the user to navigate in the operating room. The operating room of FIG. 5 contains six objects 26, 27, 28, 29, 30, 31. In the display means 5 of the arrangement 1 , due to the design of the control room, only a section 32 of the control room can be shown, in which only the object 28 can be recognized as the only object. Without navigation aids, this would lead to a "lost-in-space" problem for the user, since he cannot see where he is coming from or where he has to go.

As a guide, a review is initially displayed in a review window 33 . The sequence of objects that has been carried out to date is shown in the rear view window 33 by the objects themselves or their names being listed on the user interface in the sequence in which they are arranged. In the example shown, object 26 and object 27 are shown in the rear window 33 . The back window 33 is displayed against the last valid direction of movement next to the current position in the operating room.

In the center of the display means 5 of the arrangement 1, in which in each case the current position is displayed in the operating room is shown in the object 28 shown in Fig. 3 moment. Various paths lead from object 28 to other objects in the operating room. For the sake of clarity, the various paths are not all shown. Rather, three arrows on the three next and / or most likely objects 29 , 30 , 31 are displayed on the object 28 . The length of the arrows represents the distance to the respective object. Thus, the shortest arrow points to the closest object 29 . The second shortest arrow points to the next closest sixteenth object 31 . The longest arrow points to the fifteenth object 30, which is closest to the third. A preview window 34 , 35 , 36 is shown at the top of each arrow, in which a preview of the object 29 , 30 , 31 to which the arrow leads can be seen.

Using a corresponding input operation, the user can select a desired object as the object following the object 28 in the sequence and is automatically guided to this object via a jump operation without having to move the arrangement 1 . He either jumps immediately to the object, which speeds up the operation of the arrangement 1 considerably, or the path through the control room is displayed to him at an appropriate speed on the display means 5 of the arrangement 1 , which makes it easier for him to keep an overview in the control room.

Instead of displaying the next objects 29 , 30 , 31 in preview windows 34 , 35 , 36 , the name of the next object (s) 29 , 30 , 31 can also be displayed at the point on the edge of the representation of the operating room at which the path between the current object 28 and the next object 29 , 30 or 31 intersects the edge of the representation.

During the movement through the control room is continuous both the distance in the perpendicular to the corresponding neighboring Edges as well as the actual distance to neighboring ones Objects measured themselves. With simple distance measurement always the next edge or the next object taken. Thus, using this procedure with small geometric movements a variety of different Operating sequences are run over. This is especially true on end of the sequences: Here are especially in German to find the determining verbs. So with a few Movements of the input device an abundance of different operations are recorded in language, Comfortable and - if necessary - with the Jump operation can be started.

The linguistic standard completion for a partially complete sequence can also be displayed. Every shortest meaningful geometric distance determination implies a presumed standard completion of the previously incomplete sequence. These standards have either been explicitly set beforehand or they are reconfigured by evaluating a number of operating processes. The suspected missing, still to be recorded objects of the sequence are displayed in a separate area, preferably the lower area, of the display means 5 of the arrangement 1 by displaying their name as text. The parameters to be filled are either displayed with the object with which they are filled by default, or openly, for example with ellipsis, in the form of a text. If the display means 5 of the arrangement 1 is large enough, several alternatives can also be visualized. It is also sensible to offer a jump operation for this display of the linguistic standard completion of the sequence.

The described navigation aids enable the user both a micro and a macro orientation at all times. This is particularly important for orders whose Display means are limited in size. Here you can reorient the user at "white" places at any time and Shorten paths with the jump operation. Will this Procedures coupled with a speech synthesis process, so the user can sometimes completely without display means and visual representation of the operating room in the acoustic Gesture the display of the operating room and shorten the routes.

• - Objekte bzw. Subjekte (zum Beispiel Personen, Meldungen, Nachrichten, Anrufen, Notizen, Text etc.), die erzeugt, gefunden, benutzt, verschickt, verändert etc. werden können,
• - Verben (Senden, Schreiben, Erfassen, Erzeugen, Suchen etc.), die das Gerät in Bezug auf die zuvor ermittelten Objekte als Aktionen anwenden kann, und
• - Präpositionen (zum Beispiel als, an, in, den, über), die Erweiterungen der Objekt-Subjekt-Verb-Beziehungen darstellen.

In a special embodiment of the invention, the control room can be designed in such a way that the method can also be used to create a series of objects which correspond to sentences with more than three or five parts of the sentence and which nevertheless follow the, in particular grammatical, rules of a natural human language. This supports the user in formulating his string of objects by continuing to use his linguistic and, in particular, grammatical skills. He combines verbs, nouns and prepositions with the help of the method, so that on the one hand the classic functionality of the menu navigation of arrangements is preserved, but at any time a meaningful sentence or part of the sentence also describes the overall course of the action. When the control room is created, the addressable words are first made available. For this purpose, a kind of ontological analysis of all functions is carried out that the arrangement to be controlled can carry out. This results in the addressable words as:

• - Objects or subjects (for example people, messages, messages, calls, notes, text etc.) that can be created, found, used, sent, changed etc.,
• - Verbs (sending, writing, capturing, generating, searching, etc.) that the device can use as actions in relation to the previously determined objects, and
• - prepositions (for example, as, at, in, the, about) that represent extensions of the object-subject-verb relationship.

The next step is the addressable sentences provided. The operating room is ideally so created that the user of his linguistic imagination the formulation of his desired control process Arrangement. In the next better case, it will be new Can formulate sentences in order to achieve his goal. In any case, he will be on the way to his goal, the Arrangement to carry out the function desired by him cause to always see meaningful sentences as ariadne thread and with additional use of a speech synthesis process also hear.

The next step is to provide the different variants. A distinction is made between single word variants and sentence variants. Single-word variants are always advantageous if the user can choose from a number of grammatically equivalent elements, but the grammatical structure and meaning of the sentence is retained in principle. A note to Mr. X, Y or Z can be represented by the three single-word variant sentences "I want to write an SMS and send it to Mr. X (Y, Z)". In the operating room, this is achieved by moving list objects vertically, as explained for FIGS . 2 and 3 on list object 16 . The connection that a text should be sent to a person is not lost.

Multi-word variants leave the user in the same state of affairs or functionality in different ways express using different sentences. For example the goal of texting and sending it to Tom the multi-word variant sentences "I want to write a note and send an SMS to Tom "or" I want Tom to send a new SMS write and send "equally accessible.

The objects are represented with or by their names, so that the user can clearly identify them.

In another example, the user wants one Word processor to say the sentence: "Because of the X may write a letter of complaint to Mr. Y ", so he doesn't just say the sentence of the order if he doesn't knows exactly that the machine chose exactly this one chosen by him Understands variant of the sequence. Instead, can but the user in the control room according to his linguistic and grammatical skills correct and therefore for the Arrange workable and possible routes until he has achieved its goal.

Are the user a few sentences or rows of recurring meaning, so he can this as Sentence macros, i.e. as whole sentences, about their endpoints start an engine. So on this last level of observation complex operating sequences in the form of a sequence of Objects to the user as a whole, very easy to distinguish and visualized sentences to be grasped in their meaning and / or auditioned using a speech synthesis process.

Claims (30)

1. A method for controlling an arrangement ( 1 ), whereby objects ( 12-16 , 20-24 , 26-31 ) can be processed in the form of actions and contents by the arrangement, in that the actions can be carried out and the contents can be processed by the actions .
in which the objects ( 12-16 , 20-24 , 26-31 ) are spatially arranged in an operating room and
in which the objects ( 12-16 , 20-24 , 26-31 ) are recorded in a row, for which purpose their respective position in the operating room is sought. 2. The method according to at least claim 1, in which the objects are lined up by a Final operation is completed. 3. The method according to at least one of the preceding claims, wherein the stringing of the objects after completion by the end operation of the arrangement ( 1 ) is processed. 4. The method according to at least one of the preceding Expectations, where the control room is two-dimensional. 5. The method according to at least one of the preceding Expectations, in which the control room, at least in part, is displayed in particular on a user interface. 6. The method according to at least claim 5, in which the section ( 17 , 32 ) is shown around the position of the control room that is being visited. 7. The method according to at least claim 5, where the control room is shown as a map. 8. The method according to at least one of the preceding Expectations, where possible routes are specified in the operating room and / or being represented. 9. The method according to at least one of the preceding claims, wherein the objects ( 12-16 , 20-24 , 26-31 ) are designed as cells. 10. The method according to at least claim 9, in which the cells are bubbles and / or as bubbles being represented. 11. The method according to at least one of the preceding claims, in which a lock ( 25 ) is arranged in the operating room, which is only released depending on certain conditions. 12. The method according to at least one of the preceding claims, wherein the length of the paths in the operating room from a current object ( 12-16 , 20-24 , 26-31 ) to the neighboring objects depends on the respective probability with which the respective neighboring one Object and that current object are strung together, with the neighboring object being assigned the shortest path with the greatest probability. 13. The method according to at least one of the preceding Expectations, in which the control room can be changed by a user. 14. The method according to at least one of the preceding claims, in which one jumps from a current object ( 12-16 , 20-24 , 26-31 ) to the neighboring object, which is most likely to be strung together with the current object. 15. The method according to at least one of the preceding claims, in which the object ( 12-16 , 20-24 , 26-31 ) that is most likely to be included next in the sequence is displayed in a preview. 16. The method according to at least one of the preceding claims, in which an object ( 12-16 , 20-24 , 25-31 ) is removed from the sequence by being sought again. 17. The method according to at least one of the preceding Expectations, where the selection of the element from a list object by a movement in the control room, its direction at least partially in or against the direction of the listing the elements in the list object point and / or their direction partially perpendicular to the movement of the in line previous object to the list object. 18. The method according to at least one of the preceding Expectations, in which the lining up of at least three objects contains. 19. The method according to at least one of the preceding Expectations, in which the arrangement of the objects in the operating room and / or possible routes in the operating room following the rules of a natural Language follows or will follow. 20. The method according to at least one of the preceding Expectations, where a haptic and / or acoustic feedback he follows. 21. An arrangement which is set up and has means Method according to at least one of the preceding claims perform. 22. The arrangement according to at least claim 21, characterized in that the arrangement ( 1 ) has an input means which contains a motion sensor ( 6 ). 23. The arrangement according to at least claim 22, characterized in that the motion sensor ( 6 ) is a mouse ball, a trackball, a gyro sensor, an image sensor and / or an interferrometric sensor. 24. The arrangement according to at least claim 22, characterized in that the arrangement ( 1 ) including the input means is integrated in a housing. 25. The arrangement according to at least claim 21, characterized in that the arrangement ( 1 ) is a mobile terminal. 26. The arrangement according to at least one of claims 21 to 25, characterized in that the arrangement ( 1 ) has display means ( 5 ) for displaying the control room. 27. The arrangement according to at least one of claims 21 to 26, characterized in that the arrangement ( 1 ) has an interface ( 8 ) to a further arrangement ( 9 ). 28. The arrangement according to at least claim 21, which is set up to carry out a method according to at least claim 4, characterized in that the representation of the control room is coupled to the movement which is measured by the movement sensor ( 6 ). 29. Arrangement according to at least one of claims 21 to 26, characterized in that the arrangement ( 1 ) is a mobile terminal, in particular a mobile phone. 30. Program product for a data processing system, the Contains software code sections with which a method according to at least one of claims 1 to 20 on the Data processing system can be run.