CN100531248C - Method for replacing key operation by detecting camera motion of camera phone - Google Patents

Abstract

The invention relates to a method for replacing key operation by detecting the motion of a camera mobile phone, which corresponds different motion parameters of the camera with corresponding key operation of the mobile phone by detecting parameters of different motion distances, motion speeds, motion directions and the like of the camera on the camera mobile phone so as to control a mobile phone program, thereby realizing the purpose that different motions of the camera represent different key operations and replacing manual key operation.

Description

A method of replacing button operation by detecting camera movement of a camera-capable mobile phone

technical field

The invention belongs to the fields of mobile phone application and mobile phone management, mobile phone games and mobile phone entertainment, and in particular relates to a method for replacing key operation by detecting the movement of a camera of a mobile phone capable of taking pictures.

Background technique

With the development of mobile phone technology, the functions of mobile phones are becoming more and more abundant. In addition to making calls, there are more and more applications for mobile games and mobile entertainment, such as surfing the Internet, listening to songs, and playing games. Among them, key operation is an important means for people to use mobile phones to run programs at present, and mobile phone keys mainly complete functions such as opening programs, closing programs, selecting programs, executing programs, and completing data or character input.

In addition, the Chinese patent application with the application number 200410155088 and the title of the invention "Mobile Phone with Camera and Its Method for Inputting Video Data to Computer" discloses a method of how to capture data through the camera on the mobile phone and input it to the computer, which can replace Computer camera. The Chinese patent application with the application number 200310108642 and the title of invention "Mobile phone monitoring multifunctional camera" discloses a method for installing a camera in an electrical appliance and remotely monitoring the electrical appliance through a mobile phone or a computer. The Chinese patent application with the application number 03128141 and the invention name "MMS image alarm device" discloses a method of using MMS mobile phones to realize remote monitoring of distant moving targets and sites, that is, when the user wants to watch the monitoring site, send Send a command text message to the alarm device, and the alarm device will immediately collect the current image and send it back to the user, or automatically send the MMS image to the user's mobile phone when the monitoring site changes, and realize the alarm. All of the above belong to the related patent applications in the field of mobile phone camera or monitoring. However, at present, there is no such application that can make the operation and use of mobile phones faster, more convenient and more humanized, and more entertaining and more convenient for the elderly, children and the disabled. technology that people can use without barriers.

Contents of the invention

In view of the above, the object of the present invention is to provide a method for replacing the button operation by detecting the movement of the camera of the mobile phone that can take pictures. Different motion parameters correspond to the corresponding key operations of the mobile phone, so as to control the mobile phone program, so that different actions of the camera represent different key operations, instead of manual key operations.

In order to achieve the above object, a kind of method of the present invention replaces button operation by detecting camera movement of a mobile phone that can take pictures, comprising the following steps:

Step 1: Obtain video through the camera on the mobile phone, that is, when the selected program starts to run, start the camera operation in the background at the same time, and complete the shooting of the scene environment;

Step 2: Preprocessing the video sequence obtained in the above step 1, performing preliminary analysis and classifying the video frames in the video sequence;

Step 3: Using motion estimation and image tracking technology to extract motion parameters from the video sequence, wherein the motion parameters include at least one or more parameters of motion direction, motion speed, and motion distance;

Step 4: Perform action matching, that is, match the above-mentioned extracted motion parameters with the key operations in the key classification sequence; for example, the motion direction represents the direction selection key of the mobile phone, the motion speed represents the number input key, and the motion distance represents the switching between programs Operations, such as switching from numbers to letters, etc., can be freely defined here to facilitate the realization of different functions and specific operations; the key classification sequence here includes the following implementation steps:

Step 401: Extracting the buttons on mobile phones of different styles and models;

Step 402: Analyze the button functions of different models of mobile phones, and analyze the main functions completed by the buttons;

Step 403: Through the analysis of the keys in the above step 402, the functions they represent are clear, and they are classified into one or more categories of numbers, letters, selections, and confirmations;

Step 404: queuing and warehousing the contents of the button classification in the above step 403, loading each type of button into the database according to the serial number classification, forming a button classification sequence;

Step 5: By moving the camera-capable mobile phone, the mobile phone automatically detects the motion parameters of the camera, and performs corresponding operations according to the different meanings it represents.

Wherein, the preprocessing of the video sequence in step 2 above includes at least one or more operations of filtering, denoising and edge enhancement.

The video sequence in the above step 3 includes the first frame video and the second frame video, and the above step 3 includes the following steps:

Step 301: Select a reference object in the first frame of video, where an 8×8 area at the center of the video can usually be selected, but any other appropriate area in the first frame of video can also be selected;

Step 302: Find the area corresponding to the reference object in the second frame of video, and after finding it, calculate the displacement and motion direction between it and the first frame of video, and then calculate the motion parameters of the reference object; If the reference object in the first frame is not found in the frame, all motion parameters can be assigned zero or other parameters, that is, there is no motion.

The above step 3 key includes the following two aspects:

1) Selection of reference objects: In order to ensure the accuracy of parameters, three reference objects are usually selected, all of which are 8×8 areas and distributed in a triangle; distributed in a manner;

2) Matching of reference objects: In order to improve the matching speed and ensure the accuracy, at least one or more of the motion estimation technology and the three-step fast algorithm can be used to realize the matching. Among them, the above-mentioned motion estimation technology and three-step fast algorithm are introduced in common video technology processing textbooks and books.

Certainly, under the situation that the method described in the present invention (promptly replacing the key operation of the mobile phone by detecting the movement of the camera) is in working condition, the user sometimes still wishes to return to the mode of pressing the key to carry out the relevant operation, which can be done through the mobile phone The method of pre-setting the "hot key" or the specific mobile phone motion mode (such as moving up and down multiple times in rapid succession) is realized after clicking the "hot key" or performing the specific motion mode operation of the mobile phone.

By adopting the method of the present invention, it is not only faster, more convenient and more humanized in operation and use, but also more entertaining and more convenient for the elderly, children and disabled people to use without barriers. And for the implementation of the method of the present invention, the mobile phone manufacturer can design and implement it by itself after obtaining the method of the present invention, and there is no problem of pre-designing a set of software for the mobile phone manufacturer to purchase or install. In addition, mobile phone manufacturers can apply the method described in the present invention to the management of any application in the mobile phone.

The specific implementation of the method of the present invention will be further described in detail below in conjunction with the accompanying drawings. The above and other objects, features and advantages of the present invention will be apparent to those skilled in the art from the detailed description of the method of the present invention.

Description of drawings

Figure 1 is a schematic diagram of the method of the present invention.

Detailed ways

The specific implementation of the method of the present invention will be further described in detail below in conjunction with the accompanying drawings.

The method of the present invention is to correspond the different motion parameters of the camera with the corresponding key operations of the mobile phone by detecting the different motion distances, motion speeds, and motion directions of the camera on the mobile phone that can take pictures, so as to control the mobile phone program, thereby realizing the function of the camera. Different actions of represent the purpose of different button operations. Due to the diversity of the movement distance, movement speed and movement direction of the camera on the camera phone, it can represent the meaning that people want to express to a certain extent, and can replace manual keys. In the application process of the camera phone or in the mobile communication application (the third generation mobile communication system, namely 3G), if people can manipulate the program only by the movement of the camera phone, it can make people feel more convenient and comfortable to use. fast.

Figure 1 is a schematic diagram of the method of the present invention. As shown in Figure 1, the method of the present invention comprises the steps:

Step 1: Obtain video through the camera on the mobile phone, that is, when the selected program starts to run, start the camera operation in the background at the same time, and complete the shooting of the scene environment;

Step 2: Preprocess the video sequence obtained in the above step 1, conduct preliminary analysis and classify the video frames in the video sequence, which at least includes one or more of filtering, denoising and edge enhancement operations operation;

Step 3: Use motion estimation and image tracking technology to extract the motion parameters of the video sequence; the motion parameters include at least one or more parameters in the direction of motion, speed of motion, and distance of motion; wherein first in the first frame Select a reference object from the video, where the 8×8 area at the center of the video can usually be selected, but any other appropriate area in the first frame of video can also be selected; then search for the corresponding reference object in the second frame of video After finding the area, calculate the displacement and motion direction between it and the first frame of video, and then calculate the motion parameters of the reference object; if the reference object in the first frame is not found in the second frame, all The motion parameter assignment is zero or other parameters, that is, there is no motion;

Step 4: Perform action matching, that is, match the above-mentioned extracted motion parameters with all key operations in the key classification sequence; for example, the motion direction represents the direction selection key of the mobile phone, the motion speed represents the number input key, and the motion distance represents the distance between programs. Switching operations, such as switching from numbers to letters, etc., can be freely defined here to facilitate the realization of different functions and specific operations;

Step 5: By moving the camera-capable mobile phone, the mobile phone automatically detects the motion parameters of the camera, and performs corresponding operations according to the different meanings it represents.

Wherein, the key classification sequence in the above-mentioned step 4 includes the following implementation steps:

Step 401: Extracting the buttons on mobile phones of different styles and models;

Step 402: Analyze the button functions of different models of mobile phones, and analyze the main functions completed by the buttons;

Step 403: Through the analysis of the buttons in the above step 402, the functions they represent are clear, and they are classified. Usually, they can be classified into at least one or more categories of numbers, letters, selections, and confirmations. ;

Step 404: Queue and store the content of the key classification in the above step 403, and load each type of key into the database according to the sequence number to form a key classification sequence.

For the implementation of the method of the present invention, the mobile phone manufacturer can design and implement it by itself after obtaining the method of the present invention, and there is no problem of pre-designing a set of software for the mobile phone manufacturer to purchase or install. In addition, mobile phone manufacturers can apply the method of the present invention to the management of any application of mobile phones, such as being applied to the playback of mobile phone MP3 songs, such as assuming mobile phone motion parameters (motion parameters can include direction, speed, distance, etc.) and The definition relationship of the buttons is: the mobile phone moves forward, which represents the up button; the mobile phone moves backward, which represents the down button; the mobile phone moves to the right, which represents the volume up button; The upward movement represents the selection button (selection is to start playing), and the downward movement of the mobile phone represents the cancel button (cancellation is to stop playing), which mainly includes the following operations:

1) select the mobile phone MP3 playing menu, open the available button or the method described in the present invention (the method for detecting camera movement); At this time, the mobile phone camera has been working in the background, and it is assumed that there are many first MP3 songs in the mobile phone;

2) Move the mobile phone backwards, the mobile phone confirms that the mobile phone moves backwards by detecting the movement of the camera, then the cursor in the mobile phone display screen moves to the position of the next song in the MP3 menu corresponding to the down button;

3) Move the mobile phone forward, and the mobile phone confirms that the mobile phone moves forward by detecting the movement of the camera, then corresponding to the up button, the cursor in the mobile phone display screen moves to the position of the previous song in the MP3 menu;

4) Move the mobile phone to the right, and the mobile phone confirms that the mobile phone moves to the right by detecting the movement of the camera, which corresponds to the volume up button, and the mobile phone automatically increases the playback volume;

5) Move the mobile phone to the left, and the mobile phone confirms that the mobile phone is moved to the left by detecting the movement of the camera, which corresponds to the volume down button, and the mobile phone automatically reduces the playback volume.

6) Move the mobile phone up, and the mobile phone confirms that the mobile phone moves up by detecting the movement of the camera, then it corresponds to the selection button, and the mobile phone automatically plays the selected MP3 song;

7) Move down the mobile phone, the mobile phone confirms that the mobile phone moves down by detecting the motion of the camera, then corresponds to the cancel button, and the mobile phone stops playing MP3 songs.

Another example is the operation of a mobile phone game. For example, this mobile game is "driving a car displayed on the screen of the mobile phone". In addition to the number keys and letter keys on the keyboard, there are four direction keys of up, down, left and right). Before this method was adopted, the key operations of this game were: click the up button to operate the car to run forward; click the down button to operate the car to run backward; click the left button to operate the car to turn left; click the right button key to operate the car to turn right. After adopting this method, for example, it is assumed that the definition relationship between the movement parameters of the mobile phone and the buttons is as follows: moving the mobile phone forward means clicking the up button; moving the mobile phone backward means clicking the down button; moving the mobile phone to the right means clicking the right button; The mobile phone moves to the left, which means clicking the left scroll button. It mainly includes the following operations:

1) Select the game of driving a car in the mobile phone, and the opening mode can be opened by a button or the method described in the present invention (that is, the method for detecting camera movement); at this time, the mobile phone camera has been working in the background;

2) Move the mobile phone forward, and the mobile phone confirms that the mobile phone moves forward by detecting the movement of the camera, which corresponds to the up button, and the car moves forward;

3) Move the mobile phone backwards, and the mobile phone confirms that the mobile phone is moved backward by detecting the movement of the camera, which corresponds to the down button, and the car moves backwards;

4) Move the mobile phone to the right, and the mobile phone confirms that the mobile phone moves to the right by detecting the movement of the camera, which corresponds to the right turn button, and the car turns right;

5) Move the mobile phone to the left, and the mobile phone confirms that the mobile phone moves to the left by detecting the movement of the camera, and then corresponds to the left turning button, and the car turns left.

The working principle of the present invention has been described in detail above, but this is only a vivid example for easy understanding, and should not be regarded as limiting the scope of the present invention. Similarly, any ordinary professional in the technical field can make various possible equivalent changes or replacements according to the technical solution of the present invention and the description of its preferred embodiments, but all these changes or replacements should belong to the scope of the present invention. The scope of the claims.

Claims (5)

1. A method for replacing button operations by detecting camera movement of a mobile phone that can take pictures, characterized in that it comprises the following steps: Step 1: Obtain video through the camera on the mobile phone, that is, when the selected program starts to run, start the camera operation in the background at the same time, and complete the shooting of the scene environment; Step 2: Preprocessing the video sequence obtained in the above step 1, performing preliminary analysis and classifying the video frames in the video sequence; Step 3: using motion estimation and image tracking technology to extract motion parameters from the video sequence, wherein the motion parameters include at least one or more parameters of motion direction, motion speed and motion distance; Step 4: Perform action matching, that is, match the above-mentioned extracted motion parameters with the button operations in the button classification sequence. The button classification sequence here includes the following implementation steps: Step 401: Extracting the buttons on mobile phones of different styles and models; Step 402: Analyze the button functions of different models of mobile phones, and analyze the main functions completed by the buttons; Step 403: through the analysis of the keys in the above step 402, they are classified into one or more categories of numbers, letters, selections, and confirmations; Step 404: queuing and warehousing the contents of the button classification in the above step 403, loading each type of button into the database according to the serial number classification, forming a button classification sequence; Step 5: By moving the camera-capable mobile phone, the mobile phone automatically detects the motion parameters of the camera, and performs corresponding operations according to the different meanings it represents. 2. The method according to claim 1, characterized in that the preprocessing of the video sequence in step 2 includes one or more operations of filtering, denoising and edge enhancement. 3. The method according to claim 1, wherein the video sequence in step 3 includes a first frame of video and a second frame of video, and said step 3 includes the following steps: Step 301: Select a reference object in the first frame of video; Step 302: Find the area corresponding to the reference object in the second frame of video, and calculate the displacement and motion direction between it and the first frame of video after finding it, and then calculate the motion parameters of the reference object. 4. The method according to claim 1, characterized in that said step 3 at least includes: 1) Selection of reference objects; 2) Matching of reference objects: At least one or more methods of motion estimation technology and three-step fast algorithm are used to achieve matching. 5. The method according to claim 1, characterized in that by pre-setting the "hot key" or the pre-set motion mode of the mobile phone, when the above method is in the working state, the user can Need, after clicking the "hot key" or performing the operation of the mobile phone's pre-set exercise mode, the method can be restored to perform related operations by pressing the button.

Images