KR100962511B1 - Electronic pen mouse and operating method thereof - Google Patents

Abstract

An electronic pen mouse and a method of operating the same are disclosed. A light source for projecting projection light having a predetermined wavelength to a target object located in a space, a lens unit to which light propagating from the target object is incident, and light incident to the lens unit at a first point of time is sensed to the target object And an image generator configured to generate a reference image related to the reference image, and generate a comparison image of the target object by sensing light incident at a second time point, wherein the reference image is compared with the comparison image to communicate with the electronic pen mouse. The electronic pen mouse, characterized in that the movement information of the pointer output to the electronic device is generated, writes letters directly to the electronic device like a pen, and displays coordinates for continuously positioning the pointer in a space separated from the surface. There is an effect to track from.

Description

Electronic pen mouse and operating method

The present invention relates to an electronic device, and more particularly, to an electronic pen mouse and a method of operating the same.

Various input devices such as a keyboard, a mouse, a stylus, an electronic pen, and the like are used to input command signals to a computer. A mouse is generally used as a positioning device used to move a pointer displayed on a dual computer screen. The mechanical ball mouse determines a position value of the mouse in response to the amount of rotation of the ball detected when moving on a uniform plane such as a mouse pad. An optical mouse recognizes reflected light emitted to the bottom. The method of calculating the amount of movement of the mouse is used.

However, conventional input devices have limitations that are insufficient to meet various needs of users. That is, the keyboard can only be used to enter a symbol such as a letter or move a pointer in a horizontal direction by using a predetermined button, and a mouse, a stylus, and an electronic pen are designated when they are separated from the surface of an object designated for operation. There is a limit to the performance of the operation. In addition, in the case of the electronic pen, a mechanical switch provided at the tip of the pen must be pressed to start the operation, which causes fatigue to the user.

The background art described above is technical information possessed by the inventors for the derivation of the present invention or acquired during the derivation process of the present invention, and is not necessarily a publicly known technique disclosed to the general public before the application of the present invention.

Disclosure of Invention The present invention provides an electronic pen mouse and a method of operating the same, such that a pen can be directly input to an electronic device by writing a character such as a pen to the electronic device, and the function as a positioning device can be maintained even in a space separated from the surface.

In addition, the present invention may be able to input a variety of information or perform a function with a single tool compared to a mouse or a keyboard, an electronic pen that can improve the user convenience by displaying the letters written on the bottom surface of the monitor compared to the stylus It is to provide a mouse and its operation method.

In addition, the present invention compared to the conventional electronic pen, the electronic pen mouse and its operation can be maintained in the character input function in both the contact or spaced state through the function as a positioning device that is held in a space away from the surface It is to provide a method.

In addition, the present invention allows the user to control the sensitivity such as moving the pointer a lot, even if a small displacement in the front and rear, left and right as the target object becomes smaller when moving closer to the surface, the more delicate the moving object is convenient for the user It is to provide an electronic pen mouse and a method of operation thereof that can be provided.

In addition, the present invention is an electronic pen mouse that can combine a mouse mode that does not recognize the coordinates when in the air and recognizes the coordinates only in contact with the surface and a pen mode that recognizes the coordinates in both the contact with the surface or in the air And to provide a method of operation thereof.

Technical problems other than the present invention will be easily understood through the following description.

According to an aspect of the present invention, in an electronic pen mouse, a light source for projecting a projection light having a predetermined wavelength to a target object located in space, a lens unit to which the light traveling from the target object is incident and the lens unit And an image generator configured to generate a reference image of the target object by sensing light incident at a first time point and to generate a comparison image of the target object by detecting light incident at a second time point. Is compared with the comparison image is provided an electronic pen mouse, characterized in that the movement information of the pointer output to the electronic device communicating with the electronic pen mouse is generated.

Herein, the light incident on the lens unit may be light reflected by the projection light to the target object. The embodiment may include a calculation unit configured to generate movement information of the pointer by comparing the reference image with the comparison image. The target object may further include a predetermined pattern.

The calculating unit may further include an attribute value extracting unit extracting at least one attribute value of brightness and color values of each pixel included in the image, and an attribute value comparing the attribute values of adjacent pixels among the pixels included in the image with each other. A comparison unit, a position extraction unit for extracting a position of a pixel when the difference of the compared attribute values is equal to or greater than a predetermined threshold value, and a first reference position and the comparison image with respect to the reference image extracted from the position extraction unit; And a horizontal movement amount calculation unit configured to calculate a horizontal movement amount of the electronic pen mouse by comparing the second reference positions with respect to each other.

Here, the threshold may be an average value of attribute values of all pixels included in the image.

The horizontal shift amount may be calculated using a difference value between the average value of the second reference position and the average value of the first reference position.

The calculation unit may be further configured to extract a first crossing position where the first reference lines formed by the first reference position cross each other from the reference image, and form a second reference line formed by the second reference position from the comparison image. After extracting the second intersecting positions that cross each other, a vertical movement amount calculator may be further configured to calculate the vertical movement amount of the electronic pen mouse by comparing the first intersection position and the second intersection position with each other.

Here, the vertical movement amount calculating unit may calculate the vertical movement amount of the electronic pen mouse by comparing the width of the figure formed by the plurality of first crossing positions with the width of the figure formed by the plurality of second crossing positions. .

The calculator may further include a tilt calculator configured to calculate an inclination of the electronic pen mouse by comparing an upper width and a lower width of the figure formed by the plurality of first crossing positions.

Here, the calculating unit may linearly analyze the plurality of first crossing positions to calculate a first rotation amount of the first reference line, and linearly analyze the plurality of second crossing positions to perform second analysis of the second reference line. After calculating the rotation amount, the rotation amount calculation unit for comparing the first rotation amount and the second rotation amount with each other to calculate the rotation amount of the electronic pen mouse may be further included.

In addition, the size of the screen rotated or output to the electronic device may be adjusted in proportion to the calculated rotation amount and rotation direction.

In addition, the operation unit may be configured to perform at least one of horizontal movement, enlargement, reduction, tilting, and rotation of the reference image while calculating attribute values for each pixel and attribute values for each pixel of the comparison image. Corresponding to the performing operation in which the sum of the absolute values of the differences is the minimum, one or more of the horizontal movement amount, the vertical movement amount, the tilt and the rotation amount of the electronic pen mouse may be calculated.

Here, the target object may be a mouse pad in which a predetermined pattern is formed.

In addition, the embodiment may further include a touch sensing unit for generating a touch sensing signal when contacting the target object.

The embodiment may further include a control unit for receiving a touch detection signal input from the touch sensing unit to turn on the light source, and further includes a light sensing sensor for sensing external illumination, the control is the external illumination The light source may be turned off when the reference illuminance is preset.

The controller may end one or more operations of the light source unit and the image generator when the extracted movement information of the pointer is maintained in the no movement state for a predetermined time.

In addition, the light source may emit light outside the visible region, the distance between one end of the object and the electronic pen mouse is 0 to 30cm, the distance between the image generating unit and the lens unit is greater than the focal length of the lens unit It may be less than twice the focal length of the lens unit.

According to another aspect of the present invention, in the electronic pen mouse, a camera unit for generating a reference image by capturing a target object located in the space at a first viewpoint, and generating a comparison image by capturing at a second viewpoint; And a mode switching unit for selecting any one of a pointer control mode for extracting movement information of a pointer output to an electronic device using a comparison result of the comparison image, and a handwriting mode for inputting text to the electronic device. An electronic pen mouse is provided.

Here, the mode switching unit may be implemented in a specific shape for distinguishing the pointer control mode and the writing mode, for example, an image, text, and the like, on a screen of an electronic device communicating with an electronic pen mouse. When the touch sensing unit is moved to the position of the contact with the target object may be switched to the mode where the cursor is located. To this end, the control unit may refer to the position information of the cursor when receiving the touch detection signal input from the touch sensing unit, and when the cursor is located at the position of the mode switching unit on the screen, the controller may switch to the mode where the cursor is located.

The mode switching unit may be a mechanical structure button or a touch switch button provided in the electronic pen mouse. The mode switching unit is selected as the writing mode when the mode switching unit is clicked, and the pointer control when the mode switching unit is released. The mode can be selected.

In addition, when the mode switch is clicked, the writing mode may be selected, and when the mode switch is clicked again, the pointer control mode may be selected.

According to another aspect of the present invention, a method of operating an electronic pen mouse, comprising the steps of: projecting a projection light having a predetermined wavelength to a target object located in the space, the step of light propagating from the target object, Generating a reference image of the target object by sensing the light incident at a first point of view, and generating a comparison image of the target object by detecting the light incident at a second point of time; The image is compared with the comparison image is provided an electronic pen mouse operating method characterized in that the movement information of the pointer output to the electronic device communicating with the electronic pen mouse is generated.

The incident light may be light in which the projection light is reflected by the target object, and the present embodiment may further include generating movement information of the pointer by comparing the reference image with the comparison image. Can be.

The extracting of the pointer movement information may include extracting at least one attribute value of brightness and color values of each pixel included in the image, comparing the attribute values of adjacent pixels among the pixels included in the image with each other; Extracting the position of the pixel when the difference of the compared attribute values is equal to or greater than a predetermined threshold value; and a first reference position with respect to the reference image extracted from the position extraction unit and a second reference with respect to the comparison image. Comparing the position with each other may include calculating a horizontal movement amount of the electronic pen.

Here, the threshold may be an average value of attribute values of all pixels included in the image, and the horizontal shift amount may be calculated using a difference value between the average value of the second reference position and the average value of the first reference position. have. In this case, when the attribute value is a color value and the mouse pad has two colors, the threshold value can determine the difference between two color vectors included in the image on the r, g, and b planes, and set the median of the difference vectors as the threshold value. have.

In addition, the present embodiment extracts a first crossing position where the first reference line formed by the first reference position intersects each other from the reference image, and the second reference position is formed from the comparison image. The method may further include calculating a vertical movement amount of the electronic pen by extracting a second crossing position where the reference lines cross each other, and comparing the first crossing position and the second crossing position.

The calculating of the vertical movement amount may include calculating a vertical movement amount of the electronic pen by comparing an area of a figure formed by a plurality of first crossing positions with an area of a figure formed by the plurality of second crossing positions. It may further include.

Also, the present embodiment may further include calculating an inclination of the electronic pen by comparing an upper width and a lower width of the figure formed by the plurality of first crossing positions.

Also, the present embodiment may linearly analyze a plurality of the first crossing positions to calculate a first rotation amount of the first reference line, and linearly analyze the plurality of second crossing positions to perform a second analysis of the second reference line. After calculating the rotation amount, the method may further include calculating the rotation amount of the electronic pen by comparing the first rotation amount and the second rotation amount with each other.

In addition, the embodiment may further include the step of adjusting the size of the screen is rotated or the screen output to the electronic device in proportion to the calculated rotation amount and the rotation direction.

In the extracting of the pointer movement information, the attribute value of each pixel and the comparison image calculated for the same coordinate value while performing any one or more of horizontal movement, enlargement, reduction, tilt, and rotation with respect to the reference image. One or more of the horizontal movement amount, the vertical movement amount, the tilt and the rotation amount of the electronic pen mouse may be calculated according to the performing operation in which the absolute value of the absolute value of the difference of the attribute values of each pixel is minimized.

Also, the present embodiment may be any one of a pointer control mode for extracting movement information of a pointer output to an electronic device using a comparison result of the reference image and the comparison image, and a handwriting mode for inputting text into the electronic device. The method may further include selecting a.

In addition, the size of the screen rotated by the pointer or output to the electronic device may be adjusted in proportion to the calculated rotation amount, and the length of the first reference line and the second reference line may be compared to determine the size of the electronic pen. The slope can be calculated.

According to still another aspect of the present invention, there is provided a record in which a program of instructions that can be executed by a digital processing apparatus is tangibly implemented to perform the above-described method of operating an electronic pen mouse, and a program that can be read by the digital processing apparatus. Medium is provided.

Other aspects, features, and advantages other than those described above will become apparent from the following drawings, claims, and detailed description of the invention.

The electronic pen mouse according to the present invention and a method of operating the same are inputted by writing characters directly into an electronic device, such as a pen, and tracking coordinates for positioning the pointer so as to continuously recognize the position of the pointer even in a space spaced from the surface. It is simple to input information with a tool compared to a mouse or keyboard, and compared to a stylus, it is convenient for users who are used to writing on the floor because the monitor is standing up and the writing is on the floor. Since it can be used in a desired position, there is an effect that it is easy to use on an electronic blackboard.

In addition, compared to the conventional electronic pen, the electronic pen mouse and its operation method according to the present invention can easily input characters because the position of the pointer can be determined to continuously recognize the position of the pointer even in a space away from the surface. If it is close to, the more delicately moving and the farther away, the smaller the object becomes, the smaller the object moves. In addition, it is possible to use a mouse mode that does not recognize coordinates in the air and recognizes coordinates only when it is on the surface, and a pen mode that continuously recognizes coordinates even when in the air.

As the invention allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all changes, equivalents, and substitutes included in the spirit and scope of the present invention.

Terms including ordinal numbers such as first and second may be used to describe various components, but the components are not limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component. And / or < / RTI > includes any combination of a plurality of related listed items or any of a plurality of related listed items.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. As used herein, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, action, component, part, or combination thereof described on the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

In the following description of the present invention with reference to the accompanying drawings, the same components are denoted by the same reference numerals regardless of the reference numerals, and redundant explanations thereof will be omitted. In the following description of the present invention, if it is determined that the detailed description of the related known technology may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted. In addition, before describing the preferred embodiments of the present invention in detail, the characteristics of the lens unit used in the electronic pen mouse will be described first.

1 is a view showing the characteristics of the lens unit used in the electronic pen mouse according to an embodiment of the present invention.

Light emitted from the light source is reflected on the object. The light reflected from one point of the object travels through the space, and once it passes through the lens, the light is collected at one point. If the light sensor is provided at the location where the light is collected, a clear image of the reflected object may be obtained. . If the location of the object is called ObjLoc (Object Location) and the location where a clear image is obtained is called ClImgLoc (Clean Image Location), the image of the object located in ObjLoc is most clear when it is created in ClImgloc, and it is far from ClImgloc. The more clarity, the less it blurs. The image generated in ClImgLoc can be processed by being located at CmosLoc (Cmos Location), which is the location of a photo sensor, and displayed on a computer screen.

First, a case in which an image is represented by a lens will be described. Referring to FIG. 1, fr is a focal length located on the right side of the lens, fl is a focal length located on the left side of the lens, and ctr is the center of the lens. If the position of the object (ObjLoc) adjacent to ctr is moved to the left, the size of the image of the object is increased and ClImgLoc is moved to the left of ctr. As the object moves further left to approach fl, the size of the image is infinite, and the position of ClImgLoc moves infinity to the left to form a virtual image. Moving the object further to the left in fl reduces the size of the image, which has grown to infinity, and ClImgLoc moves to 2fr from the right infinity. If the object moves further to the left and is located at 2fl, the size of the image is equal to the size of the object and ClImgLoc is located at 2fr. If the object continues to move to the left, the image will be reduced in size, and ClImgLoc will move toward fr.

Existing optical mouse creates and saves an image that is enlarged to the detail of the surface by ObjLoc, which is the position of the object, located near Cl of the lens and CmosLoc is located on the right side of 2fr. Comparing to calculate the moving distance of the optical mouse, and change the position of the pointer by the calculated distance.

In comparison, conventional webcams (Web Cams) used in conjunction with computers use short fr lenses, while ClImgLoc is located close to fr. The reason for using a short fr lens is that it is easy to manufacture and the ObjLoc can capture a long distance object relatively far away from fl, making it relatively clear to the camera. The reason for positioning the image plane close to the fr is because it can hold a large external image in a small camera. The reason for using a small camera is that the size of the CMOS camera used as the light sensor is small, so the equipment is cheap. to be.

The electronic pen mouse according to the present embodiment may be implemented such that the ObjLoc generates a clear image with respect to an area of, for example, 0 cm to 30 cm from the end of the pen. In this case ClImgLoc occurs between fr and 2fr, so CmosLoc is located between fr and 2fr. In addition, if the focal length fr or fl is too short, the size of the image changes too rapidly due to the change of ObjLoc. Therefore, an appropriate focal length lens is used so that the size of the image does not change too rapidly.

2A and 2B are block diagrams of an electronic pen mouse according to an embodiment of the present invention, and FIG. 2C is a diagram illustrating a character input method using an electronic pen mouse according to an embodiment of the present invention.

The electronic pen mouse 100 according to the present embodiment obtains an image of a target object contacted or spaced apart in space, and compares the successive images in time, so that the electronic pen mouse moves horizontally, vertically, tilts, and rotates. By detecting the motion, information for changing the coordinate value of the pointer displayed on the display device of the electronic device is provided. The pointer of the present embodiment may be display information (eg, an image moving on a display, etc.) for indicating an item to be selected in an electronic device (eg, computer, mobile phone, PMP, etc.).

The electronic pen mouse 100 may be embodied in a pen shape or the like itself, but may be embedded in any electronic device (for example, a mobile phone having a camera unit) to move the electronic device. Accordingly, the pointer displayed on the display of the electronic device or another electronic device connected through a wired / wireless communication network may be implemented to move. Hereinafter, the case where the electronic pen mouse 100 is independently implemented to have a tangible shape will be described.

2A and 2B, the electronic pen mouse 100 includes a touch detector 102, a light source 104, a camera 106, a storage 108, an operation unit 110, a communication unit 112, and the like. It may include a control unit 114. The calculating unit 110 includes a brightness value extracting unit 121, a brightness value comparing unit 122, a position extracting unit 123, a horizontal moving amount calculating unit 124, a vertical moving amount calculating unit 125, and a gradient calculating unit 126. And a rotation amount calculator 127. One or more components included in the calculator 110 may be implemented as a software program or hardware of a type.

Although not shown, the operation unit 110 may be implemented as a software program that is included as one component of an electronic device that communicates with the electronic pen mouse 100 or is installed and driven in the electronic device to display an image provided from the electronic pen mouse. Naturally, the recognition process may be used.

The touch sensing unit 102 is provided at the distal end of the electronic pen mouse 100 so as to sense that the electronic pen mouse 100 is in contact with the surface of the object. When it is detected by the touch detector 102 that the surface of the object is touched, the electronic pen mouse 100 may be preset to enter an arbitrary operation mode.

The electronic pen mouse 100 may operate while contacting the surface of the object or moving in space, and the operation mode may be, for example, a pointer control mode for moving a pointer, a writing mode for inputting characters, and the like.

In this case, the pointer control mode detects relative movement of the electronic pen mouse 100 by referring to an image photographed on the surface of the target object (for example, a pad) so that the position of the pointer displayed through the display device of the electronic device is changed. It is an operation mode. The pointer control mode may be limited to function only in a state of being in contact with the surface of the object, but may be adjusted so that the sensitivity of the pointer position adjustment varies depending on how far the electronic pen mouse 100 is vertically spaced on the pad surface. have.

In addition, the writing mode is a mode for performing a text input function by a movement of the electronic pen mouse 100 in contact with the pad or located in the space. As a character input method, there may be a method of scanning and inputting a printed character and a method of sequentially recognizing a movement of the electronic pen mouse 100 and inputting a corresponding character. Here, a mode switching unit (not shown) may be further provided around the finger (for example, the index finger) holding the electronic pen mouse 100 to recognize that the writing mode is operating. When an operation on the mode switching unit is detected, the recognition mode may be recognized as a writing mode, and a character included in the captured image may be recognized and input, or a character may be input by sequentially recognizing a movement trajectory of the electronic pen mouse 100. . The mode switching unit may be implemented by, for example, a button of a mechanical structure or a touch switch button. The mode switching unit is recognized as the writing mode only while being pressed, or the mode switching unit is selected as the writing mode when the mode switching unit is clicked, the pointer switching mode is selected when the mode switching unit is clicked again, or the writing mode initiation is started by the writing mode initiation / end selection. It may be selected so that it is recognized in the handwriting mode only until the end is selected.

Referring to FIG. 2C, which illustrates a character input method according to a movement trajectory recognition of the electronic pen mouse 100, the electronic pen mouse 100 may allow a letter “za” to be input by moving from ① to ⑧. . In this case, in the state where the electronic pen mouse 100 is in contact with the pad or in the space, the sections of ① to ②, ③ to ④, ⑤ to ⑥, and ⑦ to ⑧ operate in the handwriting mode by operating the mode switching unit. The sections of A, B, and C may function in the pointer control mode by operating the mode switching unit so that a desired letter may be correctly input by moving only the position of the pointer.

Of course, after recognizing the entire movement trajectory of the electronic pen mouse 100, after recognizing the length and the bending angle of each section, the most suitable letter is selected and input in contrast to the letters recognized in the storage unit 108 in advance. You could do it.

Referring again to FIG. 2A, the touch sensing unit 102 may be implemented as a button of a mechanical structure, a touch switch button, or a button for switching to distance measurement using an ultrasonic sensor. The touch sensing unit 102 may include, for example, a high frequency oscillation type proximity sensor, and may use an eddy current generated in a relationship with an iron component injected into a pad to which the electronic pen mouse 100 contacts. Here, the method of injecting the iron component into the pad may be used, for example, a variety of methods, such as how to put the powder on the pad, the method of arranging the wires and the method of attaching the iron plate.

In addition, as a method of allowing the touch sensing unit 102 to detect contact with the pad, for example, whether or not contact with the target object is performed by whether current flows from the distal end of the electronic pen mouse 100 to the contacted conductor. It can also be detected. In this case, the pad may include plastic or metal of a conductor component through which current can flow.

In addition, the conductive film is coated on the pad and two electrodes are provided at the end of the electronic pen mouse 100 so that when the two electrodes come into contact with the conductive film, current may flow to serve as a switch. In addition, by measuring the distance between the end of the electronic pen mouse 100 and the pad on the surface by using an ultrasonic sensor, if the distance is within a certain range it may be to perform a switching function of entering or canceling the operation mode.

In addition, when the electronic pen mouse 100 approaches or contacts the surface of the object, it is determined whether the size of the grid included in the image captured by the image generator 117 to be described later is greater than or equal to a predetermined size, and the specified size. In this case, the electronic pen mouse 100 may be entered into the operation mode described above. In this case, the touch detector 102 may be omitted. In this case, when the contact detection unit 102 starts or closes the contact state with the target object, that is, the electronic pen mouse 100 detects that a predetermined electric circuit is disconnected or connected through contact with the target object. May also cause a switching function (eg, button function of a normal mouse) to be performed.

The light source unit 104 emits light having a predetermined wavelength. The controller 114 may control the turn on / off of the light source unit 104 with reference to the touch detection signal input from the touch detector 102. Here, the emitted light may be light including wavelengths for at least one of an infrared region, a visible region, and an ultraviolet region.

The electronic pen mouse 100 may emit light having a predetermined wavelength so that the electronic pen mouse 100 may be used irrespective of shadows generated by night light and / or other light sources with or without ambient lighting.

For example, light may be emitted that includes wavelengths in the visible region to provide the amount of light needed for image imaging. In this case, the electronic pen mouse 100 may further include a light sensor (not shown), and when the external illuminance sensed by the light sensor is equal to or greater than a reference illuminance specified for capturing the captured image, the controller 114. May be controlled such that the light source unit 104 is not turned on.

In addition, light of a specific wavelength may be emitted so that only the reflected light reflected by the object is emitted from the light source unit 104. In this case, an optical filter that selectively passes only the light of the corresponding wavelength may be a camera. It may be further provided in the unit 106. In addition, it can be obvious that the light of various wavelength bands can be implemented.

Of course, as will be described below, the electronic pen mouse 100 according to the present embodiment may calculate and calculate boundary lines and / or intersection positions thereof in the image photographed using the light of the light source unit 104. To extract, attribute values such as brightness values and / or color values according to the detected light may be used regardless of the wavelength.

The camera unit 106 generates an image by capturing an external subject, and may include a lens unit and an image generator.

The camera unit 106 is set to generate an image by photographing an external subject as in a conventional camera apparatus, or to generate an image using the reflected light reflected by the target object from the light emitted from the light source unit 104. It may be set.

Hereinafter, the latter case will be described, but a person skilled in the art can easily understand the former case with reference to the following description. In addition, in order to explain a series of processes for sensing the movement of the electronic pen mouse 100 by comparing the sequentially generated images, for convenience, the image obtained at the first viewpoint, which is a pre-view point, may be referred to as a reference image, An image acquired at a second time point, which is a later point in time, is referred to as a comparison image, and these are collectively referred to as an image. In addition, the projection light projected to obtain the reference image is referred to as the first projection light, and the reflected light propagated by reflecting the first projection light to the object is referred to as the first reflection light. In addition, the projection light projected to obtain the comparison image is referred to as a second projection light, and the reflected light propagated by reflecting the second projection light to the object is referred to as a second reflection light.

The lens unit is a component that receives the reflected light reflected by the projection light emitted from the light source, and can adjust the size of the image of the target object to the processing size of the image generator. The lens unit may include one lens, but may include a combination of a plurality of lenses for injecting the above-described reflected light into the image generating unit.

Here, the focal length of the lens unit is a distance between the end of the electronic pen mouse 100, for example, a clear image is generated mainly between fr and 2fr for a target object having a distance of 0 cm to 30 cm, for example, so that CmosLoc is located between fr and 2fr. do. In addition, if the focal length fr or fl is too short, the size of the image changes too rapidly due to the change of ObjLoc. Therefore, an appropriate focal length lens is used so that the size of the image does not change too rapidly.

The image generator detects the reflected light incident through the lens unit and generates a reference image and a comparison image of the object. The image generator may include an optical sensor for detecting light, and may generate an image of an object in electronic form using the detected light. In addition, the image generator may further perform a function of converting the image data into predetermined image data so that the image analysis by the calculator 110 is possible. For example, the image generator 117 may be a CCD camera or a CMOS camera including an image sensor, an image signal processor (ISP), or the like.

The storage unit 108 stores images generated by the camera unit 106 (ie, one or more reference images and a comparison image). Images stored in the storage unit 108 may be in the form of image data converted to be processed by the operation unit 110. In addition, the storage unit 108 may further store a program for driving the electronic pen mouse 100.

The calculator 110 may calculate movement information of the electronic pen mouse 100 by comparing the reference image stored in the storage 108 with a comparison image. Here, the movement information is information about the movement direction and the movement amount of the electronic pen mouse 100, for example, used for the movement of a pointer displayed through the display device, or in an electronic device communicating with the electronic pen mouse 100. One or more of a predetermined function (eg, performing a writing function according to the pointer movement direction, or expanding information displayed on the display device when the electronic pen mouse 100 is rotated clockwise) is performed. Can be.

The calculating unit 110 compares attribute values (eg, brightness value, color value, etc.) with respect to each pixel of the reference image and the comparison image, and thus, the reference for the boundary line of the pattern included in the object or the intersection thereof. By detecting the position and recognizing how much the reference position varies between the reference image and the comparison image, movement information regarding the movement of the electronic pen mouse 100 may be generated. The generated movement information may be used as information for moving the pointer and / or information for performing a designated function in the electronic device as described above. However, since the information about the reference position of the reference image will be calculated in the previous process and stored in the storage unit 108, only the process for detecting the reference position of the comparison image may be performed.

As described above, the operation unit 110 may be provided in the electronic pen mouse 100, but may be provided in an electronic device coupled to the electronic pen mouse 100 through a wired / wireless communication network. In the latter case, since the electronic pen mouse 100 only needs to transmit an image generated by capturing an external subject or image data that can be interpreted by the operation unit 110 to the electronic device, the structure may be simpler. have.

As shown in FIG. 2B, the operation unit 110 may include a brightness value extractor 121 that extracts brightness values of each pixel included in an image, and a vertically and / or horizontally adjacent pixel among pixels included in the image. It may include a brightness value comparison unit 122 for comparing the brightness value, and a position extraction unit 123 for extracting the position of the pixel when the difference of the compared brightness value is more than a predetermined threshold value as a reference position. When the attribute value is used instead of the brightness value, the brightness value extractor 121 and the brightness value comparer 122 may be referred to as an attribute value extractor and an attribute value comparer. The following description will focus on the case of calculating the brightness value among the attribute values, and the following description is also applicable to the attribute values such as the color values.

Here, the threshold of the brightness value may be specified as, for example, any predetermined reference value or an average value of brightness values of all pixels included in the photographed image. For example, if each pixel in the image has a brightness value greater than the threshold value of 1, and the other pixels are separated by 0, the pixel position at the point of time from 0 to 1 or from 1 to 0 is described above. Can be extracted to a reference position. If the amount of light at a specific position is significantly different from the amount of light at another position, only the pixels corresponding to the specific position may be partially extracted and the average value of the brightness values may be used as a threshold.

The position extractor 123 may further extract an intersection position where the reference line in which the reference positions are continuous and the two reference lines intersect using the arrangement of the reference positions extracted by the above-described method. When the grid pattern exists in the target object as shown in FIG. 3, horizontal and vertical lines constituting the grid may be extracted as the reference line, and a position where the horizontal line and the vertical line cross each other may be extracted as the intersection position.

In addition, the operation unit 110 further includes a horizontal movement amount calculator 124, a vertical movement amount calculator 125, a slope calculator 126, and a rotation amount calculator 127, each of which is an electronic pen mouse 100. Is used to calculate the displacement relative to Each calculator performs an operation independently or calculates the displacement amount in combination with each other, so that the movement information of the electronic pen mouse 100 is finally calculated. As described above, the movement information may be used as change amount information for moving the position of the pointer or change amount information for performing a designated function (eg, writing function, enlargement, reduction, or the like).

The horizontal movement amount calculator 124 calculates the horizontal movement amount of the electronic pen mouse 100 by comparing the first reference position with respect to the reference image extracted by the position extraction unit 123 with the second reference position with respect to the comparison image. can do. The horizontal movement direction of the electronic pen mouse 100 may be interpreted as the opposite direction of the reference position between the images.

For example, the horizontal shift calculator 124 receives and stores information about the reference position of each image from the position extractor 123 and stores the information, or the reference position stored by the position extractor 123 in the storage 108. You will be able to read and use this information.

In addition, the vertical movement amount calculator 125 may calculate the vertical movement amount of the electronic pen mouse 100 by comparing the crossing positions in the reference image and the comparison image extracted by the position extraction unit 123, respectively. In this case, the vertical movement amount calculation unit 125 may include relationship information between intersection positions existing in the reference image (for example, one or more of the length between the intersection positions and / or the width of an area or a figure formed by the intersection positions). The amount of vertical movement of the electronic pen mouse 100 may be calculated by comparing the relationship information of the intersection positions existing in the comparison image with (eg, calculating the amount of reduction in length or area).

In addition, the tilt calculator 126 may compare the length of the upper side and the lower side of the figure formed by the plurality of intersection positions, or the tilt of the electronic pen mouse 100 using the area ratio of the vertically continuous figures. Can be calculated.

The rotation amount calculation unit 127 linearly analyzes intersection positions existing in the reference image and the comparison image, respectively, and calculates rotation amounts of the lines (that is, the reference lines) to which the intersection positions are connected, and then calculates the first rotation amount (that is, The rotation amount of the electronic pen mouse 100 may be calculated by comparing the rotation amount of the reference line of the reference image) and the second rotation amount (that is, the rotation amount of the reference line of the comparison image) with each other. Of course, the amount of rotation of the reference line with respect to the reference image may be calculated in a previous calculation process and may be included in the amount of rotation calculator 127 or may be stored and managed in the storage 108.

Each calculation unit will be described in detail below with reference to specific embodiments.

The communication unit 112 transmits mobile data to an electronic device connected through a communication network. The communication unit 112 may further perform a function of converting the mobile data to be transmitted into a data format that the electronic device can receive and process.

The controller 114 controls the above-described components to perform a specified operation. For example, the controller 114 may control turn on / off of the light source unit 104 or the like. Here, the control unit 114 controls the light source unit 104 to turn on when the contact detection unit 102 detects contact with the target object, and when the release of contact with the target object is detected, the light source unit 104 turns off. Can be controlled. Alternatively, the control unit 114 controls the light source unit 104 to be turned on when the contact detection unit 102 detects contact with the target object, and the operation unit (for a predetermined time interval regardless of whether the contact with the target object is released). When the movement information calculated by 110 is maintained in the no movement state of the electronic pen mouse 100, the operation of the camera unit 106 and / or the light source unit 104 may be terminated.

In addition, the controller 114 may generate the movement information calculated by the operation unit 110 as movement data, which is a data type that can be used by an electronic device connected through a communication network. Of course, if the electronic device can recognize and process the movement information that is the result of the operation of the operation unit 110, the controller 114 may process the movement information to be transmitted to the electronic device through the communication unit 112.

Although not illustrated in the drawing, an auto focusing unit may perform an auto-focusing process in software to generate a sharper subject image.

3 is a diagram illustrating a mouse pad which is a target object of an electronic pen mouse according to an embodiment of the present invention. Referring to FIG. 3, an operation process of the electronic pen mouse 100 using a mouse pad having a predetermined pattern or pattern will be described. For example, as shown in FIG. 3A, the mouse pad can make two regions having different brightnesses alternately in a lattice pattern.

The image generating unit of the camera unit 106 generates a reference image B for the photographing area of a predetermined size with respect to the mouse pad illustrated in FIG. 3. Here, Xp and Yp represent X and Y axes based on the mouse pad, and Xc and Yc represent X and Y axes based on the captured image.

The photographing area of the reference image may be designated as an area larger than the size of one unit grid in the state in which the electronic pen mouse 100 is in contact with the mouse pad. When the size of the shooting area is smaller than the size of one unit grid, the reference image and the comparison image are the same even when the movement of the electronic pen mouse 100 (horizontal movement, vertical movement, rotation, and tilting) is present. This is because motion cannot be detected. In addition, when the photographing area of the reference image is much larger than the size of the unit grid, the calculated movement direction may be unclear. As illustrated, the photographing area of the reference image may be set to include 1 to 2 unit grids. . Of course, when the electronic pen mouse 100 is vertically moved, a larger quantity of unit grids may be captured to be included in the reference image and the comparative image than when the electronic pen mouse 100 is in contact with the mouse pad.

In the reference image and the comparison image photographed in this manner, the reference position and / or the boundary position are detected by the operation unit 110, and the electronic pen mouse 100 contacts the mouse pad or in space using the corresponding position information. Movement information is generated by calculating the distance moved.

The operation of the electronic pen mouse 100 may be classified into four types such as horizontal / vertical movement, change of inclination, and rotational movement of rotating the electronic pen mouse 100 based on an extension object. Hereinafter, each operation will be described in turn.

4 is a diagram illustrating a method of calculating a horizontal movement amount of an electronic pen mouse according to an embodiment of the present invention. (A) is a reference image and (B) is a comparison image.

The horizontal movement calculator 124 may calculate the horizontal movement of the electronic pen mouse 100 by comparing the reference position of the reference image extracted from the position extractor 123 with the reference position of the comparison image.

For example, the position extractor 123 scans the brightness value of the reference image generated by the camera unit 106 at the first time point in the column direction for each pixel row and in the row direction for each pixel column. The brightness values of adjacent pixels are compared with each other while scanning, and thus, the positions (reference positions) are stored because portions where the brightness values suddenly change are the boundary portions of the grid formed on the mouse pad. The same procedure is repeated on the newly recognized comparison image at the second time point. Of course, the reference position with respect to the reference image recognized at the first time point may be already extracted and stored in the previous movement information generation process.

Thereafter, the horizontal shift calculator 124 subtracts the position (reference position) of the boundary obtained from the reference image from the position (reference position) of the grid boundary obtained from the comparison image. That is, it can be calculated as the distance that the movement amount of the boundary line having the same length moved. The subtracted value for the two boundary parts may be set to the distance that the electronic pen mouse 100 moves on the mouse pad. Such movement information is provided to the electronic device and used for movement of the pointer or operation of the writing mode.

Here, the time interval between the first time point and the second time point is set smaller than the time when the electronic pen mouse 100 moves to another unit grid. If the time interval between the first time point and the second time point is greater than the time that the electronic pen mouse 100 moves to another unit grid, the unit grid included in the comparison image is different from the unit grid included in the reference image. This is because the horizontal movement amount is difficult to measure.

For example, the time interval between the first time point and the second time point may be determined corresponding to the set number of frames per second (FPS). The transmission speed of the image frame transmitted from the electronic pen mouse 100 to the electronic device (ie, software of the electronic device) may be, for example, a speed of 33 to 66 FPS. The higher the number of frames per second, the more accurate the travel distance measurement can be, so the transmission rate may be specified as a larger value (eg 132 FPS).

According to another embodiment, the horizontal shift amount may be calculated based on the center point of the unit grid. For example, since the intersection point of the grid boundary corresponding to the first reference position described above corresponds to four corners of the unit grid, their average value becomes the center point of the unit grid, and resets it to the first reference position. In the same manner, the horizontal shift amount can be calculated by resetting the average value of the intersection points of the grid boundaries corresponding to the second reference position to the second reference position and then subtracting the first reference position from the second reference position.

5 is a diagram illustrating a method of calculating a vertical movement amount of an electronic pen mouse according to an embodiment of the present invention. (A) is a reference image and (B) is a comparison image. The differences from the above will be explained mainly.

The vertical movement amount calculator 125 may calculate the vertical movement amount of the electronic pen mouse 100 by comparing the crossing positions in the reference image and the comparison image extracted by the position extraction unit 123, respectively. In this case, an amount of change in the relationship information between the intersection positions existing in the reference image and the comparison image (for example, a reduction in the length of the intersection positions and / or the width of the area or the shape formed by the intersection positions) may be used. .

For example, when the electronic pen mouse 100 moves vertically with respect to a target object, the size of the square forming the grid is reduced, and the distance that the electronic pen mouse 100 moves vertically is calculated using the decrease amount. Can be.

First, the position extractor 123 scans the brightness value of the reference image generated by the camera unit 106 at the first viewpoint in the column direction for each pixel row and in the row direction for each pixel column. The scanning unit compares the brightness values of adjacent pixels with each other and extracts and stores a line (first reference line) connecting the positions since the portion where the brightness value suddenly changes is the boundary portion of the grid formed on the mouse pad. Then find the point where the row and column intersect (the first intersection) at the boundary position of the stored grid. Intersecting points are the upper left, lower left, upper right and lower right points of the mouse pad grid. The same procedure is repeated for the comparison image created at the second time point.

The vertical movement amount calculator 125 determines that the electronic pen mouse 100 is moved from the mouse pad based on the amount of change in the relationship information between the grid position (second crossing position) obtained from the comparison image and the grid position (first crossing position) obtained from the reference image. Calculate the distance climbed. After that, the distance is reflected in the pointer. The vertical movement amount may be used for adjusting the sensitivity of the electronic pen mouse 100, for example.

As described above, even when the electronic pen mouse 100 is vertically moved, the continuous operation in the writing mode is possible, and the function as the positioning device for changing the position of the mouse pointer can be continued.

6 is a diagram illustrating a method of calculating a tilt of an electronic pen mouse according to an embodiment of the present invention. (A) is a reference image and (B) is a comparison image. The differences from the above will be explained mainly.

The inclination calculator 126 may calculate the inclination of the electronic pen mouse 100 by comparing the length of the upper side and the lower side of the figure formed by the plurality of intersection positions or by using the area ratio of the vertically continuous figures. Can be.

For example, when the position extractor 123 detects four intersection positions such as a left upper point, a lower left point, an upper right point, and a lower right point with respect to the comparison image, the slope calculator 126 may place the upper left point and the upper right point. After calculating the distance (first distance) and the distance from the lower left point to the lower right point (second distance), and then calculate the subtraction value or ratio value between the first distance and the second distance of the electronic pen mouse 100 The slope can be calculated.

In addition, when the slope is calculated using the width of one grid included in the reference image, the intersection position may be a vertex of each grid, and an average value of two arbitrary vertices is calculated to calculate a mean of the left middle point and the right middle point. Set the point, upper middle point, and lower middle point. By calculating the width of the figure using the calculated four crossing positions and the average value, the upper width and the lower width of each grid can be calculated. When the electronic pen mouse 100 is tilted, the upper width and the lower width are different from each other, and thus the slope may be calculated in proportion to the difference value.

The inclination value may be used to correct the movement of the pointer and to accelerate the movement in the inclined direction. That is, when the tip of the electronic pen mouse 100 is in contact with the mouse pad, there is no movement of the pointer even if the inclination of the pen changes, but the tip of the electronic pen mouse 100 is located in a space spaced apart from the mouse pad. In this case, the distance that the pointer moves may be increased according to the inclination angle of the electronic pen mouse 100.

7 is a diagram illustrating a method of calculating an amount of rotation of an electronic pen mouse according to an embodiment of the present invention. (A) is a reference image and (B) is a comparison image. The differences from the above will be explained mainly.

The rotation amount calculation unit 127 linearly analyzes intersection positions existing in the reference image and the comparison image, respectively, and calculates rotation amounts of the lines (that is, the reference lines) to which the intersection positions are connected, and then calculates the first rotation amount (that is, The rotation amount of the electronic pen mouse 100 may be calculated by comparing the rotation amount of the reference line of the reference image) and the second rotation amount (that is, the rotation amount of the reference line of the comparison image) with each other. Of course, the amount of rotation of the reference line with respect to the reference image may be calculated in a previous calculation process and may be included in the amount of rotation calculator 127 or may be stored and managed in the storage 108.

For example, when the electronic pen mouse 100 is rotated clockwise or counterclockwise about its extended direction, the rotation amount calculating unit 127 may be configured such that the reference line of the reference image has an angle with respect to the coordinate axis. The degree of rotation (rotation value or angle change amount) of the electronic pen mouse 100 is calculated as the difference between the angles of the reference line of the comparative image with respect to the coordinate axis.

The rotation value may be information that allows the electronic device to rotate and display the pointer through the display device or may be used as information for performing a predetermined function. For example, when the pointer is displayed crosswise, the pointer may be rotated as much as the electronic pen mouse 100 rotates. In addition, when the electronic pen mouse 100 is rotated in a clockwise direction, an image of a screen displayed on the display is enlarged, and when the electronic pen mouse 100 is rotated in a counterclockwise direction, a function, a figure, etc., rotates in the rotation direction of the electronic pen mouse 100. A function such as rotating in the same direction may be implemented.

8 is a view illustrating a pattern of a target object of the electronic pen mouse according to another embodiment of the present invention. The differences from the above will be explained mainly.

As described above, the present embodiment has a feature that can be used in a desk having a pattern (for example, a square pattern, a wood grain pattern, etc.) rather than a continuous pattern. (A) illustrates an object such as a desk on which a predetermined pattern is formed, and (B) shows a reference image for the corresponding pattern.

As shown in FIG. 9, the horizontal movement amount of the electronic pen mouse 100 may be calculated by the following procedure. It is possible to calculate the horizontal movement amount of the electronic pen mouse 100 by moving any one of the reference image and the comparison image until the photographed pattern has the same position. Hereinafter, it is assumed that the comparison image is moved.

First, the operation unit 110 stores brightness values of respective pixels of the reference image of the desk surface photographed at the first time point, and performs the same procedure on the comparison image captured at the second time point to calculate brightness values of each pixel. Save it.

The comparison image is moved in the x-axis and / or y-axis directions with respect to the stored reference image to calculate a difference in brightness value for each pixel that is moved and co-located. At this time, when the difference in brightness value of each pixel in the process of moving the comparison image is 0 or the maximum number of pixels close to 0 (that is, the point of overlapping the position having the same pattern in the reference image and the comparison image as much as possible in the entire moving process) The distance and direction in which the comparison image is moved may be calculated as movement information of the electronic pen mouse 100.

In addition, as a method of calculating the movement information of the electronic pen mouse 100, a method of summing the absolute values of the difference values for each pixel may be applied. That is, the comparison image is moved in the x-axis and / or y-axis direction with respect to the stored reference image to calculate a difference in brightness values for each pixel that is moved and co-located, and adds the absolute values of the difference values for each pixel. . In this case, the position where the added value is close to zero or minimum may be referred to as a position where the reference image has the same pattern as the comparison image. Therefore, the distance and direction in which the comparison image is moved until the position having the same pattern may be calculated as the movement information of the electronic pen mouse 100.

In addition, there are various methods of finding a surface having the same pattern as the reference image by moving the comparison image left and right. For example, there may be a method of moving the reference image to the right first and then moving to the left if the difference does not decrease, and moving in a direction in which the difference decreases further if the difference is reduced from the left. Such a method is generally referred to as a minimum search method, and various minimum search methods other than the present invention may be applied.

According to another exemplary embodiment, the horizontal shift amount may be calculated by calculating a difference in brightness values only for pixels constituting the outer outline of the reference image and the comparison image. For example, if the above-described minimum search method is applied to only the pixels of the outline of the reference image and the pixels of the outline of the comparison image with respect to the lattice of FIG. There is an advantage to shrink.

The above-described procedure is applied not only to horizontal movement but also to a vertical movement with respect to a target object of the electronic pen mouse 100, a change in inclination, and a rotational movement occurring when the electronic pen mouse 100 is rotated about an extended axis. can do. Hereinafter, when a square pattern is formed on the desk, a processing method for each operation will be described in turn.

10 is a diagram illustrating a method of calculating a vertical movement amount of an electronic pen mouse according to another embodiment of the present invention. (A) is a reference image generated when the electronic pen mouse 100 is on the surface of the target object, and (B) is a comparative image generated when the electronic pen mouse 100 is spaced apart from the surface of the target object. The differences from the above will be explained mainly.

As described above, the operation unit 110 calculates and stores brightness values of respective pixels with respect to the reference image and the comparison image generated at the first and second views, respectively. The brightness value calculation and storage timing for each image may be different.

While reducing or enlarging the size of the stored reference image, the absolute value of the difference between the brightness value of each pixel of the reference image and the brightness value of each pixel of the comparison image is added to the same coordinate. As described above, when the added value is close to zero or becomes the minimum value, it may be recognized that the comparison image and the reference image are the same, and the reduction or enlargement ratio and the vertical shift amount of the reference image may be calculated using the comparison image.

Here, as described above, the vertical movement amount may be reflected in sensitivity adjustment for changing the actual moving distance of the pointer even with the same tilt change amount or horizontal movement amount.

FIG. 11 illustrates a method of calculating a tilt of an electronic pen mouse according to another embodiment of the present invention. (A) is a reference image generated with the electronic pen mouse 100 vertically facing the surface of the target object, and (B) is generated with the electronic pen mouse 100 tilted with respect to the surface of the target object. It is a comparison image.

When the electronic pen mouse 100 is inclined with respect to the target object, the front part of the target object close to the electronic pen mouse 100 looks large and the rear part looks small by perspective.

In order to calculate the slope, the calculator 110 may use brightness values of respective pixels stored for the reference image and the comparison image generated at the first and second views, respectively. That is, the reference image is variously modified by changing the slope as shown in (B), and then both absolute values of the difference between the brightness values of each pixel of the reference image and the brightness values of each pixel of the comparison image are added to the same coordinate. As described above, when the added value is close to zero or becomes the minimum value, it may be determined that the comparison image is the same as the reference image, and the slope of the comparison image may be calculated using the comparison image.

12 is a diagram illustrating a method of calculating an amount of rotation of an electronic pen mouse according to an embodiment of the present invention. (A) is a reference image generated before the electronic pen mouse 100 is rotated, (B) is a comparison image generated after the electronic pen mouse 100 is rotated.

In order to calculate the amount of rotation of the electronic pen mouse 100, the operation unit 110 may use brightness values of respective pixels stored with respect to the reference image and the comparison image generated at the first and second views, respectively. That is, after the reference image is rotated clockwise or counterclockwise, the absolute value of the difference between the brightness value of each pixel of the reference image and the brightness value of each pixel of the comparison image is added to the same coordinate. As described above, when the added value is close to zero or becomes the minimum value, it may be recognized that the comparison image and the reference image are the same, and the rotation amount of the comparison image may be calculated using the comparison image.

In addition, when the electronic pen mouse 100 according to the present embodiment performs all four above-described operations simultaneously on the surface of an object of irregular pattern, for example, when it is vertically raised, tilted, rotated, and moved horizontally By changing the horizontal movement, the vertical movement on the surface, the change of the tilt, and the degree of rotation, the state in which the brightness values of the above-described image are the same can be applied to the pointer coordinate value. In this case, the multi-dimension minimum search method is applicable to the present invention, which is obvious to those skilled in the art, and thus a detailed description thereof will be omitted.

13 is a flowchart illustrating a method of operating an electronic pen mouse according to an embodiment of the present invention.

As described above, the camera unit 106 of the electronic pen mouse 100 is set to generate an image by photographing an external subject as in a conventional camera apparatus, or the light emitted from the light source unit 104 is a target. It may be set to generate an image using the reflected light reflected by the object. In the former case, the light source unit 104 may be regarded as a means for providing an amount of light necessary for photographing, as in a conventional camera apparatus. Hereinafter, the latter case will be described, but a person skilled in the art can easily understand the former case with reference to the following description.

In step S10, the light source unit 104 projects the projection light having a predetermined wavelength toward the target object at the first time point.

In step S20, the reflected light propagating in different directions after the projection light emitted from the light source unit 104 is reflected on the object is incident on the lens unit of the camera unit 106, and in step S30, the projection unit of the camera unit 106 The image generator detects the reflected light incident on the lens unit and generates a reference image of the object.

Thereafter, steps S10 to S30 are sequentially performed at the second time point different from the first time point to generate the comparison image, and then store the same in the storage unit 108. In order to store each image in the storage unit 108, the camera unit 106 may further perform a process of converting the generated image into a form that may be processed by the operation unit 110.

In operation S40, the calculator 110 compares the generated reference image with the comparison image to generate movement information of the electronic pen mouse 100. The movement information may be used, for example, for the purpose of changing the position of the pointer output through the display device of the electronic device. In this case, the operation unit 110 may extract movement information of the electronic pen mouse 100 based on a difference between brightness values of pixels of the reference image and the comparison image generated as described above.

As described above, a method of extracting movement information of the electronic pen mouse 100 may be largely divided into steps S42 and S43 and steps S44 to S48.

For example, in step S42, the brightness value extractor 121 extracts brightness values of each pixel included in each image, and in step S43, the difference between brightness values between pixels for the same coordinate of the reference image and the comparison image is determined. By calculating and obtaining the minimum value of the absolute value of the difference, the movement information of the electronic pen mouse 100 can be extracted.

In step S44, the brightness value comparator 122 compares brightness values of adjacent pixels among the pixels included in the image with each other. In step S46, the location extractor 123 compares the brightness values with a predetermined threshold value. In this case, the pixel position can be extracted as the reference position.

In step S48, the horizontal movement amount, vertical movement amount, tilt and rotation amount of the electronic pen mouse 100 are calculated as described above with reference to this reference position, and the calculated information may be used for pointer movement and command control.

In addition, a specific form of the electronic pen mouse 100 according to an embodiment of the present invention, a common platform technology such as embedded system, O / S, communication protocol, interface standardization technology such as I / O interface, actuator, battery, camera, A detailed description of a part standardization technology, such as a sensor, will be omitted since it is obvious to those skilled in the art.

The electronic pen mouse operating method according to the present invention can be implemented in the form of program instructions that can be executed by various computer means and recorded in a computer readable medium. In other words, the recording medium may be a computer-readable recording medium having recorded thereon a program for causing the computer to execute the above-described steps.

The computer readable medium may include a program command, a data file, a data structure, etc. alone or in combination. Program instructions recorded on the media may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like.

Those skilled in the art will appreciate that various modifications and changes can be made in the present invention without departing from the spirit and scope of the invention as set forth in the claims below.

1 is a view showing the characteristics of the lens unit used in the electronic pen mouse according to an embodiment of the present invention.

Figure 2a is a block diagram of an electronic pen mouse in accordance with an embodiment of the present invention.

Figure 2b is a block diagram of the operation unit of the electronic pen mouse in accordance with an embodiment of the present invention.

Figure 2c is a diagram illustrating a character input method using an electronic pen mouse in accordance with an embodiment of the present invention.

3 is a view illustrating a mouse pad that is a target object of an electronic pen mouse according to an embodiment of the present invention.

4 is a diagram illustrating a method of calculating a horizontal movement amount of an electronic pen mouse according to an embodiment of the present invention.

5 is a diagram illustrating a method of calculating a vertical movement amount of an electronic pen mouse according to an embodiment of the present invention.

6 is a diagram illustrating a method for calculating a tilt of an electronic pen mouse according to an embodiment of the present invention.

7 is a diagram illustrating a method of calculating an amount of rotation of an electronic pen mouse according to an embodiment of the present invention.

8 is a view showing a pattern of the target object of the electronic pen mouse according to another embodiment of the present invention.

9 is a diagram illustrating a method of calculating a horizontal movement amount of an electronic pen mouse according to another embodiment of the present invention.

10 is a diagram illustrating a method of calculating a vertical movement amount of an electronic pen mouse according to another embodiment of the present invention.

FIG. 11 illustrates a method for calculating a tilt of an electronic pen mouse according to another embodiment of the present invention. FIG.

12 is a diagram illustrating a method of calculating an amount of rotation of an electronic pen mouse according to an embodiment of the present invention.

13 is a flowchart illustrating a method of operating an electronic pen mouse according to an embodiment of the present invention.

<Description of the symbols for the main parts of the drawings>

100: electronic pen mouse 102: contact detection unit

104: light source unit 106: camera unit

110: calculation unit 108: storage unit

112: communication unit 114: control unit

121: brightness value extraction unit 122: brightness value comparison unit

123: position extraction unit 124: horizontal movement amount calculation unit

125: vertical movement amount calculation unit 126: slope calculation unit

127: rotation amount calculation unit

Claims (42)

In the electronic pen mouse, A light source for projecting projection light having a predetermined wavelength to a target object located in space; A lens unit to which light traveling from the target object is incident; An image generation unit configured to generate a reference image of the object by sensing light incident at a first point of view in the lens unit, and generate a comparison image of the object by sensing light incident at a second point of view; And Comprising a comparison unit for generating a movement information of the pointer by comparing the reference image and the comparison image, The reference image is compared with the comparison image to generate the movement information of the pointer output to the electronic device communicating with the electronic pen mouse, The calculation unit, An attribute value extraction unit for extracting one or more attribute values of brightness and color values of each pixel included in the image; An attribute value comparison unit for comparing attribute values of adjacent pixels among pixels included in the image with each other; A position extraction unit for extracting a position of a pixel when the difference between the compared attribute values is equal to or greater than a predetermined threshold value; And And a horizontal movement amount calculator configured to calculate a horizontal movement amount of the electronic pen mouse by comparing the first reference position with respect to the reference image extracted from the position extraction unit with the second reference position with respect to the comparison image. Electronic pen mouse. The method of claim 1, The light incident on the lens unit is the electronic pen mouse, characterized in that the projection light is reflected to the target object. delete The method of claim 1, The target object is an electronic pen mouse, characterized in that a predetermined pattern is formed. delete The method of claim 1, The threshold value is an electronic pen mouse, characterized in that the average value of the attribute value of all pixels included in the image. The method of claim 1, The electronic pen mouse for calculating the horizontal movement amount by using a difference value between the average value of the second reference position and the average value of the first reference position. The method of claim 1, The calculation unit, A first intersection position at which the first reference line formed by the first reference position crosses each other is extracted from the reference image, and a second intersection line at which the second reference line formed by the second reference position crosses each other from the comparison image After extracting the intersection position, the electronic pen mouse further comprises a vertical movement amount calculating unit for calculating the vertical movement amount of the electronic pen mouse by comparing the first intersection position and the second intersection position with each other. The method of claim 8, The vertical movement amount calculation unit, The electronic pen mouse, wherein the amount of vertical movement of the electronic pen mouse is calculated by comparing the width of the figure formed by the plurality of first crossing positions with the width of the figure formed by the plurality of second crossing positions. The method of claim 1, The calculation unit, A first intersection position at which the first reference line formed by the first reference position crosses each other is extracted from the reference image, and a second intersection line at which the second reference line formed by the second reference position crosses each other from the comparison image After extracting the intersection position, And an inclination calculator configured to calculate an inclination of the electronic pen mouse by comparing an upper width and a lower width of a figure formed by a plurality of first crossing positions. The method of claim 1, The calculation unit, A first intersection position at which the first reference line formed by the first reference position crosses each other is extracted from the reference image, and a second intersection line at which the second reference line formed by the second reference position crosses each other from the comparison image After extracting the intersection position, And an inclination calculator configured to calculate an inclination of the electronic pen by comparing the distance between the first crossing positions and the distance between the second crossing positions. The method of claim 1, The calculation unit, A first intersection position at which the first reference line formed by the first reference position crosses each other is extracted from the reference image, and a second intersection line at which the second reference line formed by the second reference position crosses each other from the comparison image After extracting the intersection position, Linearly analyzing the plurality of first crossing positions to calculate a first rotation amount of the first reference line, and linearly analyzing the plurality of second crossing positions to calculate a second rotation amount of the second reference line And a rotation amount calculator configured to calculate the rotation amount of the electronic pen mouse by comparing the first rotation amount and the second rotation amount with each other. The method of claim 12, The electronic pen mouse, characterized in that the size of the screen is rotated or the screen output to the electronic device is adjusted in proportion to the calculated rotation amount and the rotation direction. The method of claim 1, The calculation unit, Absolute value for the difference between the attribute value for each pixel calculated for the same coordinate value and the attribute value for each pixel of the comparison image while performing any one or more of horizontal movement, enlargement, reduction, tilt, and rotation with respect to the reference image. And at least one of a horizontal movement amount, a vertical movement amount, an inclination, and a rotation amount of the electronic pen mouse, in correspondence with the performing operation of which the sum value is the minimum. The method of claim 1, The electronic pen mouse, characterized in that the target object is a mouse pad formed with a predetermined pattern. The method of claim 1, The electronic pen mouse further comprising a touch sensing unit for generating a touch sensing signal when contacting the target object. The method of claim 16, The electronic pen mouse further comprising a control unit for turning on the light source unit by receiving a touch detection signal input from the touch detection unit. The method of claim 17, Further comprising a light sensor for detecting external illumination, And the controller turns off the light source unit when the external illuminance is equal to or greater than a preset reference illuminance. The method of claim 17, And the control unit terminates at least one of the light source unit and the image generating unit when the extracted movement information of the pointer is maintained in the no movement state for a predetermined time. The method of claim 1, The light source is an electronic pen mouse, characterized in that for emitting light outside the visible region. The method of claim 1, Electronic pen mouse, characterized in that the distance between the target object and one end of the electronic pen mouse 0 to 30cm. The method of claim 21, The distance between the image generating unit and the lens unit is greater than the focal length of the lens unit and less than twice the focal length of the lens unit. delete delete delete delete In the way the electronic pen mouse works, Projecting light having a predetermined wavelength onto a target object located in space; Incident light propagating from the target object; Generating a reference image of the object by sensing the light incident at a first time point; Sensing the light incident at a second viewpoint to generate a comparison image of the object; And Comprising the step of comparing the reference image and the comparison image to generate the movement information of the pointer, The reference image is compared with the comparison image to generate the movement information of the pointer output to the electronic device communicating with the electronic pen mouse, Generating the movement information of the pointer, Extracting at least one attribute value among brightness values and color values of each pixel included in the image; Comparing attribute values of adjacent pixels among the pixels included in the image with each other; Extracting the position of the pixel when the difference of the compared attribute values is greater than or equal to a predetermined threshold value as a reference position; And And comparing the first reference position with respect to the reference image extracted from the position extraction unit with the second reference position with respect to the comparison image to calculate a horizontal movement amount of the electronic pen. How it works. The method of claim 27, And the incident light is light reflected by the projection light to the object. delete delete The method of claim 27, And the threshold value is an average value of attribute values of all pixels included in the image. The method of claim 27, And calculating the horizontal movement amount by using a difference value between the average value of the second reference position and the average value of the first reference position. The method of claim 27, A first intersection position at which the first reference line formed by the first reference position crosses each other is extracted from the reference image, and a second intersection line at which the second reference line formed by the second reference position crosses each other from the comparison image And extracting the crossing positions, and comparing the first crossing positions with the second crossing positions to calculate vertical movement amounts of the electronic pens. 34. The method of claim 33, The vertical movement amount calculating step, Calculating an amount of vertical movement of the electronic pen by comparing a width of a figure formed by a plurality of first crossing positions with a width of a figure formed by the plurality of second crossing positions. How the Pen Mouse Works. The method of claim 27, A first intersection position at which the first reference line formed by the first reference position crosses each other is extracted from the reference image, and a second intersection line at which the second reference line formed by the second reference position crosses each other from the comparison image After extracting the intersection position, And calculating an inclination of the electronic pen by comparing an upper width and a lower width of a figure formed by a plurality of first crossing positions. The method of claim 27, A first intersection position at which the first reference line formed by the first reference position crosses each other is extracted from the reference image, and a second intersection line at which the second reference line formed by the second reference position crosses each other from the comparison image After extracting the intersection position, And calculating a slope of the electronic pen by comparing the distance between the first crossing positions and the distance between the second crossing positions. The method of claim 27, A first intersection position at which the first reference line formed by the first reference position crosses each other is extracted from the reference image, and a second intersection line at which the second reference line formed by the second reference position crosses each other from the comparison image After extracting the intersection position, Linearly analyzing the plurality of first crossing positions to calculate a first rotation amount of the first reference line, and linearly analyzing the plurality of second crossing positions to calculate a second rotation amount of the second reference line And comparing the first rotation amount and the second rotation amount with each other to calculate a rotation amount of the electronic pen. The method of claim 37, And adjusting the size of a screen outputted to the electronic device or rotating the pointer in proportion to the calculated rotation amount and the rotation direction. The method of claim 27, Generating the movement information of the pointer, Absolute value for the difference between the attribute value for each pixel calculated for the same coordinate value and the attribute value for each pixel of the comparison image while performing any one or more of horizontal movement, enlargement, reduction, tilt, and rotation with respect to the reference image. And at least one of a horizontal movement amount, a vertical movement amount, an inclination, and a rotation amount of the electronic pen mouse, in correspondence with the performing operation of which the sum value is minimum. The method of claim 27, Selecting one of a pointer control mode for extracting movement information of a pointer output to an electronic device and a writing mode for inputting a character to the electronic device by using a comparison result of the reference image and the comparison image; Electronic pen mouse operation method comprising. The method of claim 28, And the projection light is light outside the visible light region. 42. A program of instructions that can be executed by a digital processing apparatus is tangibly implemented to perform the electronic pen mouse operating method according to any one of claims 27 to 28 and 31 to 41. Recording medium recording a program that can be read by.

Images