CN103036968B - A kind ofly shown by on-vehicle host and control the method and system of smart mobile phone - Google Patents

Abstract

The invention discloses a method and a system for displaying and controlling a smart phone through a vehicle host. The method includes: transmitting the screen interface of the smart phone to the vehicle-mounted host through the audio-video conversion device in the form of video stream; and transmitting the audio data of the smart phone to the vehicle-mounted host through the audio-video conversion device in the form of audio stream; The mouse hook software converts the position coordinates corresponding to the mouse event into mobile phone screen coordinate data according to the pixel size data of the smart phone screen; the smart phone simulates the mouse event and the mobile phone screen coordinate data and forms corresponding operation instructions , and control the smart phone to perform corresponding operations according to the operation instructions. By adopting the invention, the user can control the vehicle-mounted host computer in the car, and then conveniently perform operations such as dialing/answering mobile phone calls, and run various application programs of the smart phone.

Description

A method and system for displaying and controlling a smart phone through a vehicle host

technical field

The present invention relates to the field of automotive electronics for vehicle-mounted wireless networks and applications, in particular to a method and system for displaying and controlling a smart phone through a vehicle-mounted host.

Background technique

In recent years, the development of smart phones has been ahead of the car head unit in many aspects. Therefore, the industry has started to connect the smart phone with the car head unit, so that the user can display the screen of the mobile phone and play the sound output of the smart phone through the car head unit. Through the operation of the on-board host screen, the smart phone can answer/make calls and run the application program on the mobile phone, so that the user can conveniently operate the mobile phone during driving and reduce the risk factors caused by the user operating the mobile phone during driving.

In the existing technology, the technology that can connect the vehicle-mounted host computer and the smart phone mainly includes the MirrorLink protocol, but the defect of the MirrorLink protocol is that because the MirrorLink protocol has not been fully popularized in the mobile phone industry, many mobile phone manufacturers have not yet joined this association, and for Apple, a high-end Mobile phone manufacturers do not support the MirrorLink protocol, but have their own set of protocols and software similar to the MirrorLink function.

However, the defect of the MirrorLink protocol is that because the MirrorLink protocol has not yet been fully popularized in the mobile phone industry, many mobile phone manufacturers have not yet joined this association, and Apple, a high-end mobile phone manufacturer, does not support the MirrorLink protocol, and has its own set of protocols and software similar to MirrorLink functions. Therefore, many mobile phones do not support the MirrorLink protocol at present. This makes the wide application of the MirrorLink protocol bring great limitations.

Therefore, the prior art still needs to be improved and developed.

Contents of the invention

The technical problem to be solved by the present invention is to provide a method and system for displaying and controlling a smart phone through a vehicle-mounted host computer in view of the above-mentioned defects of the prior art, so that the user can operate on the touch screen of the vehicle-mounted host computer, and the mouse hook software monitoring of the vehicle-mounted host computer And receive the mouse event of the touch screen and the position coordinates corresponding to the mouse event, and convert the position coordinates corresponding to the mouse event according to the smart phone screen pixel size data, and convert the converted mobile phone screen coordinate data and the mouse The event is sent to the smart phone, and the smart phone simulates the mouse event on the screen coordinates of the mobile phone, that is, simulates operations such as clicking or sliding at this position. Through the present invention, the user can conveniently control the application program of the smart phone through the vehicle-mounted host computer during the process of driving the vehicle, thereby improving driving safety while providing convenience to the user.

The technical solution adopted by the present invention to solve technical problems is as follows:

A method for displaying and controlling a smart phone through a vehicle-mounted host, including steps;

A. The screen interface of the smart phone is transmitted to the vehicle-mounted host through the audio-video conversion device in the form of a video stream; and the audio data of the smart phone is transmitted to the vehicle-mounted host through the audio-video conversion device in the form of an audio stream; The touch screen of the vehicle-mounted host displays the screen interface of the smart phone, and the audio equipment connected to the vehicle-mounted host plays the audio data;

B. Establishing a wireless connection between the vehicle host and the smart phone;

C. When the mouse hook software on the vehicle-mounted host monitors that the vehicle-mounted host receives a mouse event input by the user through the touch screen, the mouse hook software acquires the mouse event and the position coordinates corresponding to the mouse event ;

D. The mouse hook software pre-receives the smart phone screen pixel size data sent by the smart phone in a wireless manner, and converts the position coordinates corresponding to the mouse event into mobile phone screen coordinates according to the smart phone screen pixel size data data, and the mobile phone screen coordinate data and the mouse event are sent to the smart phone wirelessly;

E. The smart phone receives the mouse event and the mobile phone screen coordinate data, simulates the mouse event and the mobile phone screen coordinate data to form corresponding operation instructions, and controls the smart phone to perform corresponding operations according to the operation instructions operate.

The step A further includes; the audio-video conversion device is respectively connected to the vehicle-mounted host and the smart phone through data lines.

The step C further includes, when the mouse hook software does not detect that the vehicle host receives a mouse event input by the user through the touch screen, the mouse hook software continues to monitor.

The step B also includes;

Establish a wireless connection with the second Bluetooth module installed on the smart phone through the first Bluetooth module installed on the vehicle host;

Alternatively, a wireless connection is established between the first WIFI module installed on the vehicle host and the second WIFI module installed on the smart phone.

The first Bluetooth module communicates with the second Bluetooth module through the SPP protocol, and the first WIFI module communicates with the second WIFI module through the TCP/IP protocol.

The step D also includes; the smart phone obtains the screen pixel size data of the smart phone through an application programming interface.

A system based on the above method, comprising: a vehicle-mounted host for installation in a car, and a smart phone connected to the vehicle-mounted host in communication, and it also includes a data line connected to the vehicle-mounted host and the smart phone respectively An audio-video conversion device, wherein the vehicle-mounted host is provided with;

The touch sensing and display module is used to receive the mouse event input by the user and display the screen interface of the smart phone;

The audio and video stream analysis module is used to receive and analyze the audio stream and video stream sent by the smart phone and converted by the audio and video conversion device, and send the analyzed video stream to the touch sensing and display module for display ; The audio stream will be analyzed and sent to the audio equipment connected to the vehicle host for playing;

The mouse hook module is used to receive the pixel size data of the smart phone screen sent by the smart phone wirelessly, monitor and obtain the mouse event input by the user through the touch sensing and display module and the position coordinates corresponding to the mouse event, And convert the position coordinates corresponding to the mouse event into mobile phone screen coordinate data according to the pixel size data of the smart phone screen; and send the mouse event and the mobile phone screen coordinate data to the smart phone wirelessly;

The smart phone is provided with;

Obtaining module, used to obtain smart phone screen pixel size data through application programming interface, and send described smart phone screen pixel size data to the vehicle-mounted host;

An analysis module, configured to receive the mouse event and the mobile phone screen coordinate data sent by the vehicle-mounted host, and simulate the mouse event and the mobile phone screen coordinate data to form corresponding operating instructions, and control the smart phone according to The operation instruction performs the corresponding operation;

Sending module, for sending the audio stream and the video stream of the smart phone to the vehicle host through the audio and video conversion device;

The audio-video conversion device is provided with;

The conversion module is used to convert the audio stream and video stream of the smart phone into an audio stream and a video stream consistent with the audio and video input format of the vehicle host according to the audio and video input formats of the vehicle host, and convert the The final audio stream and video stream are input to the vehicle host.

The vehicle host is also provided with a first bluetooth module and/or a first WIFI module, and the smart phone is provided with a second bluetooth module communicatively connected with the first bluetooth module and/or communicates with the first WIFI module The second WIFI module connected; where,

The first Bluetooth module communicates with the second Bluetooth module through the SPP protocol, or the first WIFI module communicates with the second WIFI module through the TCP/IP protocol.

The vehicle host is also provided with a CVBS video input interface or a VGA video input interface or an RGB video input interface, and an L/R audio input interface; the smart phone is provided with an MHL interface or an HDMI interface;

The conversion module of the audio-video conversion device is respectively connected with the CVBS video input interface or the VGA video input interface or the RGB video input interface of the vehicle-mounted main unit through a data line, and is connected with the L/R audio input interface;

It is also connected with the MHL interface or HDMI interface of the smart phone through a data cable.

A method and system for displaying and controlling a smart phone through a vehicle-mounted host computer provided by the present invention, through the mouse hook software set on the vehicle-mounted host computer, the mouse event input by the user on the touch screen of the vehicle-mounted host computer and the mouse event corresponding to the mouse event The position coordinates are converted into the corresponding coordinate data on the screen of the mobile phone, so that the smart phone can perform synchronous actions such as mouse click and movement according to the coordinate data and mouse events, and achieve the purpose of controlling the smart phone through the vehicle-mounted host. The displayed interface is synchronized with the display interface on the smartphone screen, and the audio stream from the phone is played through the car's high-quality audio equipment.

Description of drawings

FIG. 1 is a flow chart of steps in a preferred embodiment of a method for displaying and controlling a smart phone through a vehicle-mounted host computer provided by the present invention.

Fig. 2 is a schematic diagram of a preferred embodiment of the working principle of the audio-video conversion device provided by the present invention.

Fig. 3 is a schematic diagram of a preferred embodiment of the system of the present invention based on the method of displaying and controlling a smart phone through a vehicle-mounted host.

Detailed ways

In order to make the object, technical solution and advantages of the present invention more clear and definite, the present invention will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

The method provided by the present invention will be described in detail below in conjunction with FIG. 1 , wherein FIG. 1 is a flow chart of a preferred embodiment of a method for displaying and controlling a smart phone through a vehicle-mounted host computer provided by the present invention.

Step 101, transmit the screen interface of the smartphone to the vehicle-mounted host through the audio-video conversion device in the form of a video stream; and transmit the audio data of the smartphone to the vehicle-mounted host through the audio-video conversion device in the form of an audio stream; The touch screen of the vehicle-mounted host displays the screen interface of the smart phone, and the audio equipment connected to the vehicle-mounted host plays the audio data;

Wherein in conjunction with Fig. 2, the operating principle of the audio-video conversion device is described in detail, that is, the audio-video conversion device 310 is respectively connected with the CVBS video input interface or the VGA video input interface or RGB video input interface 320 and The L/R audio input interface 321 of the vehicle host is connected; it is also connected with the MHL interface 301 or the HDMI interface 302 of the smart phone through a data line.

That is, when the audio-video conversion device 310 is connected to the MHL interface 301 of the smart phone, the audio-video conversion device 320 first converts the audio-video stream in the MHL format to the HDMI format, and when the audio-video conversion device 310 is connected to the The HDMI interface 302 of the smart phone is connected to receive audio and video streams in HDMI format. Convert the audio and video stream in the HDMI format to a video stream in CVBS or VGA or RGB format and an L/R audio stream.

The video stream in CVBS or VGA or RGB format is connected to the CVBS or VGA or RGB video input interface 320 of the vehicle host, and the audio stream is connected to the L/R audio input interface 321 of the vehicle host.

Step 102, establishing a wireless connection between the vehicle host and the smart phone;

Wherein, the specific way of establishing the wireless connection is: through the first bluetooth module installed on the vehicle-mounted host and the second bluetooth module installed on the smart phone to establish a wireless connection; The first WIFI module establishes a wireless connection with the second WIFI module installed on the smart phone.

The first Bluetooth module communicates with the second Bluetooth module through the SPP protocol, and the first WIFI module communicates with the second WIFI module through the TCP/IP protocol.

Step 103, the mouse hook software on the vehicle-mounted host monitors whether the vehicle-mounted host receives a mouse event input by the user through the touch screen;

When the mouse hook software does not monitor, then enter step 104, the mouse hook software continues to monitor;

When the mouse hook software monitors it, then enter step 105, the mouse hook software acquires the mouse event and the position coordinates corresponding to the mouse event;

Step 106, the mouse hook software pre-receives the smart phone screen pixel size data sent by the smart phone wirelessly, and converts the position coordinates corresponding to the mouse event into a mobile phone screen according to the smart phone screen pixel size data Coordinate data, and the mobile phone screen coordinate data and the mouse event are sent to the smart phone wirelessly;

Wherein, the smart phone obtains the screen size data of the smart phone through an application programming interface, and the application programming interface is called API for short.

Step 107: The smart phone receives the mouse event and the mobile phone screen coordinate data, simulates the mouse event and the mobile phone screen coordinate data to form a corresponding operation instruction, and controls the smart phone to perform corresponding operations according to the operation instruction. operation.

Through this preferred embodiment, in the specific use process, the user first connects the smart phone to the vehicle-mounted host through an audio-video conversion device, and the touch screen of the vehicle-mounted host synchronously displays the screen interface of the smart phone, and is connected to the vehicle-mounted host The audio equipment of the smart phone plays the audio stream output by the smart phone. At this time, the user can perform a touch operation on the touch screen of the vehicle-mounted host, and the mouse hook software of the vehicle-mounted host monitors and obtains the mouse event and the position coordinates corresponding to the mouse event, and according to the smart The pixel size of the mobile phone screen converts the position coordinates corresponding to the mouse event into the coordinate data of the smart phone screen, and sends the mouse event and coordinate data to the smart phone, so that the user can control the smart phone on the touch screen of the vehicle host. Apps, such as answering/making calls, surfing the Internet, watching videos, etc., solve the hidden dangers of insecurity caused by direct operation of smartphones when users drive vehicles.

The present invention also provides a system, and the structure of the system will be described in detail below in conjunction with FIG. 3 , wherein FIG. 3 is a schematic structural diagram of a preferred embodiment of the system based on the method for controlling a smart phone through a vehicle-mounted host in the present invention.

As can be seen from FIG. 3 , the system includes a vehicle-mounted host 210 for being installed on a car, a smart phone 220 communicatively connected with the vehicle-mounted host 210, and a smart phone 220 connected to the vehicle-mounted host and the smart phone through data lines respectively. Audio and video conversion device 230;

The vehicle host 210 includes;

The touch sensing and display module 211 is used to receive the mouse event input by the user's touch, and it is also used to display the screen interface of the smart phone;

The audio and video stream analysis module 215 is used to receive and analyze the audio stream and video stream sent by the smart phone and converted by the audio and video conversion device, and send the analyzed video stream to the touch sensing and display module for further processing. display; the audio stream will be analyzed and sent to the audio equipment connected to the vehicle host for playing;

The mouse hook module 212 is used to receive the smart phone screen pixel size data sent by the smart phone 220, monitor and obtain the mouse event input by the user through the touch sensing module 211 and the position coordinates corresponding to the mouse event, and according to The pixel size data of the smart phone screen converts the position coordinates corresponding to the mouse event into mobile phone screen coordinate data; and sends the mouse event and the mobile phone screen coordinate data to the smart phone wirelessly;

The smart phone 220 is provided with;

Obtaining module 223, is used for obtaining smart phone screen pixel size data through application programming interface, and described smart phone screen pixel size data is sent to described on-board host computer;

The analysis module 224 is used to receive the mouse event and the mobile phone screen coordinate data sent by the vehicle host, and simulate the mouse event and the mobile phone screen coordinate data to form corresponding operation instructions, and control the smart phone Carry out corresponding operations according to the operation instructions;

The sending module 225 is used to send the audio and video stream of the smart phone to the vehicle host through the audio and video conversion device;

The audio-video converting device 230 is provided with;

The conversion module 231 is used for converting the audio stream and video stream of the smart phone into an audio stream and a video stream consistent with the audio and video input format of the vehicle host according to the audio and video input formats of the vehicle host, and converting the The converted audio stream and video stream are input to the vehicle host.

Preferably, the vehicle host 210 is also provided with a CVBS video input interface or a VGA video input interface or an RGB video input interface, and an L/R audio input interface; the smart phone is provided with an MHL interface or an HDMI interface; the conversion The module 231 is respectively connected to the CVBS video input interface or the VGA video input interface or the RGB video input interface and the L/R audio input interface of the vehicle host 210 through data lines;

The conversion module 231 is also connected to the MHL interface or the HDMI interface of the smart phone 210 through a data cable.

In the system of this preferred embodiment, the mouse event, the position coordinates, the screen pixel size data of the smart phone, and the screen coordinate data of the mobile phone are wirelessly established between the smart phone 220 and the vehicle host 210. The way to establish the wireless connection is;

The vehicle host 210 is provided with a first bluetooth module 213 and/or a first WIFI module 214, and the smart phone 220 is provided with a second bluetooth module 221 and/or connected with the first bluetooth module 213 for communication. The first WIFI module 214 communicates with the second WIFI module 222 .

In actual use, the user can choose to establish a wireless connection through the Bluetooth module, or establish a wireless connection through the WIFI module.

In summary, the present invention provides a method and system for displaying and controlling a smart phone by a vehicle-mounted host. The video of the mobile phone is synchronously displayed on the large screen of the vehicle-mounted host, and the audio is played through the vehicle-mounted high-quality audio equipment. It is possible to conveniently perform operations such as dialing/receiving mobile phone calls by controlling the vehicle-mounted host computer; running various application programs of smart phones, usually, the application programs of mobile phones are often updated and upgraded. Therefore, the present invention enriches the navigation of vehicle-mounted electronics. with multimedia content. At the same time, it also improves driving pleasure and safety. The system provided by the invention can be compatible with various mobile phones with MHL or HDMI output.

It should be understood that the application of the present invention is not limited to the above examples, and those skilled in the art can make improvements or transformations according to the above descriptions, and all these improvements and transformations should belong to the protection scope of the appended claims of the present invention.

Claims (8)

1. A method for displaying and controlling a smart phone by a vehicle-mounted host, characterized in that it comprises steps; A. The screen interface of the smart phone is transmitted to the vehicle-mounted host through the audio-video conversion device in the form of a video stream; and the audio data of the smart phone is transmitted to the vehicle-mounted host through the audio-video conversion device in the form of an audio stream; The touch screen of the vehicle-mounted host displays the screen interface of the smart phone, and the audio equipment connected to the vehicle-mounted host plays the audio data; B. Establishing a wireless connection between the vehicle host and the smart phone; C. When the mouse hook software on the vehicle-mounted host monitors that the vehicle-mounted host receives a mouse event input by the user through the touch screen, the mouse hook software acquires the mouse event and the position coordinates corresponding to the mouse event ; D. The mouse hook software pre-receives the smart phone screen pixel size data sent by the smart phone in a wireless manner, and converts the position coordinates corresponding to the mouse event into mobile phone screen coordinates according to the smart phone screen pixel size data data, and the mobile phone screen coordinate data and the mouse event are sent to the smart phone wirelessly, and the smart phone obtains the smart phone screen pixel size data through the application programming interface; E. The smart phone receives the mouse event and the mobile phone screen coordinate data, simulates the mouse event and the mobile phone screen coordinate data to form corresponding operation instructions, and controls the smart phone to perform corresponding operations according to the operation instructions operate; The user controls the application program of the smart phone on the touch screen of the car host, answers/makes calls, browses the Internet, and watches videos. 2. The method for displaying and controlling a smart phone through a vehicle-mounted host according to claim 1, wherein said step A further comprises; phone connection. 3. The method for displaying and controlling a smart phone through the vehicle-mounted host according to claim 1, wherein the step C further comprises, when the mouse hook software does not detect that the vehicle-mounted host receives user input through the touch screen When a mouse event occurs, the mouse hook software continues to monitor. 4. The method for displaying and controlling a smart phone through a vehicle-mounted host according to claim 1, wherein the step B further comprises; Establish a wireless connection with the second Bluetooth module installed on the smart phone through the first Bluetooth module installed on the vehicle host; Alternatively, a wireless connection is established between the first WIFI module installed on the vehicle host and the second WIFI module installed on the smart phone. 5. The method for displaying and controlling a smart phone by a vehicle-mounted host according to claim 4, wherein the first Bluetooth module communicates with the second Bluetooth module through the SPP protocol, and the first WIFI module communicates with the second Bluetooth module through the SPP protocol. The TCP/IP protocol communicates with the second WIFI module. 6. A system based on the method described in any one of claims 1 to 5, comprising: a vehicle-mounted host for being installed on an automobile, and a smart phone connected to the vehicle-mounted host in communication, it also includes a The audio-video conversion device connected to the vehicle-mounted host and the smart phone through a data line, is characterized in that, the vehicle-mounted host is provided with; The touch sensing and display module is used to receive the mouse event input by the user and display the screen interface of the smart phone; The audio and video stream analysis module is used to receive and analyze the audio stream and video stream sent by the smart phone and converted by the audio and video conversion device, and send the analyzed video stream to the touch sensing and display module for display ; The audio stream will be analyzed and sent to the audio equipment connected to the vehicle host for playing; The mouse hook module is used to receive the pixel size data of the smart phone screen sent by the smart phone wirelessly, monitor and obtain the mouse event input by the user through the touch sensing and display module and the position coordinates corresponding to the mouse event, And convert the position coordinates corresponding to the mouse event into mobile phone screen coordinate data according to the pixel size data of the smart phone screen; and send the mouse event and the mobile phone screen coordinate data to the smart phone wirelessly; The smart phone is provided with; Obtaining module, used to obtain smart phone screen pixel size data through application programming interface, and send described smart phone screen pixel size data to the vehicle-mounted host; An analysis module, configured to receive the mouse event and the mobile phone screen coordinate data sent by the vehicle-mounted host, and simulate the mouse event and the mobile phone screen coordinate data to form corresponding operating instructions, and control the smart phone according to The operation instruction performs the corresponding operation; Sending module, for sending the audio stream and the video stream of the smart phone to the vehicle host through the audio and video conversion device; The audio-video conversion device is provided with; The conversion module is used to convert the audio stream and video stream of the smart phone into an audio stream and a video stream consistent with the audio and video input format of the vehicle host according to the audio and video input formats of the vehicle host, and convert the The final audio stream and video stream are input to the vehicle-mounted host; the user controls the application program of the smart phone on the touch-screen of the vehicle-mounted host to answer/make calls, browse the Internet, and watch videos. 7. system according to claim 6, is characterized in that, described vehicle main unit is also provided with first bluetooth module and/or first WIFI module, and described smart mobile phone is provided with and is connected with described first bluetooth module communication. A second Bluetooth module and/or a second WIFI module communicatively connected to the first WIFI module; wherein, The first Bluetooth module communicates with the second Bluetooth module through the SPP protocol, or the first WIFI module communicates with the second WIFI module through the TCP/IP protocol. 8. The system according to claim 6, wherein the vehicle-mounted host is also provided with a CVBS video input interface or a VGA video input interface or an RGB video input interface, and an L/R audio input interface; Has MHL interface or HDMI interface; The conversion module of the audio-video conversion device is respectively connected with the CVBS video input interface or the VGA video input interface or the RGB video input interface of the vehicle-mounted main unit through a data line, and is connected with the L/R audio input interface; It is also connected with the MHL interface or HDMI interface of the smart phone through a data cable.