CN105979224A - Head mount display, video output device and video processing method and system - Google Patents
Head mount display, video output device and video processing method and system Download PDFInfo
- Publication number
- CN105979224A CN105979224A CN201610466833.6A CN201610466833A CN105979224A CN 105979224 A CN105979224 A CN 105979224A CN 201610466833 A CN201610466833 A CN 201610466833A CN 105979224 A CN105979224 A CN 105979224A
- Authority
- CN
- China
- Prior art keywords
- video
- head
- area information
- data
- output device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title abstract description 4
- 238000012545 processing Methods 0.000 claims abstract description 66
- 230000005540 biological transmission Effects 0.000 claims abstract description 53
- 210000005252 bulbus oculi Anatomy 0.000 claims abstract description 43
- 210000003128 head Anatomy 0.000 claims abstract description 43
- 210000001508 eye Anatomy 0.000 claims abstract description 31
- 238000000034 method Methods 0.000 claims description 79
- 230000008569 process Effects 0.000 claims description 45
- 238000007906 compression Methods 0.000 claims description 27
- 230000006835 compression Effects 0.000 claims description 26
- 238000004891 communication Methods 0.000 claims description 18
- 238000005192 partition Methods 0.000 claims description 14
- 230000009467 reduction Effects 0.000 claims description 5
- 230000001066 destructive effect Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000012546 transfer Methods 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000013144 data compression Methods 0.000 description 2
- 241000208340 Araliaceae Species 0.000 description 1
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 description 1
- 235000003140 Panax quinquefolius Nutrition 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000004438 eyesight Effects 0.000 description 1
- 235000008434 ginseng Nutrition 0.000 description 1
- 230000004886 head movement Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000002427 irreversible effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003565 oculomotor Effects 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000003313 weakening effect Effects 0.000 description 1
- 238000013316 zoning Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/167—Position within a video image, e.g. region of interest [ROI]
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Controls And Circuits For Display Device (AREA)
Abstract
The invention discloses a head mount display, a video output device and a video processing method and system. The head mount display comprises an image collection unit which photographs head and eye images of a user in real time; an image processing unit which calculates the eyeball movement track of the user according to the head and eye images, thereby obtaining the information of an important region in a display screen currently watched by eyeballs; a data transmission unit which sends the information of the important region to the video output device, thereby enabling the device to carry out regional compressed coding on a current frame image according to the information of the important region, wherein the data transmission unit also receives the frame image after the regional compressed coding sent by the device; a video decoding unit which carries out regional decoding on the frame image after the regional compressed coding; and a display unit which displays and outputs the decoded frame image, thereby displaying the data in the frame image corresponding to the important region in a high definition mode, wherein the data corresponding to other regions is displayed in a weakened mode. According to the head mount display, the video output device and the video processing method and system, a frame video is divided into regions by use of the watching positions of the eyeballs of the user, and the user experience of the head mount display is optimized.
Description
Technical field
The present invention relates to wear Display Technique field, be specifically related to a kind of head-mounted display, video frequency output sets
Standby and method for processing video frequency, system.
Background technology
At present, head-mounted display achieves development at full speed, head-mounted display (Head Mount Display,
It is called for short HMD) video obtained frame by frame from the picture output device such as smart mobile phone or computer can be shown
Out watch for wearer.Fig. 1 is the transmission of video scene schematic diagram of prior art head-mounted display, ginseng
See Fig. 1, owing to the transmission quantity of video data is relatively big, so needing to be compressed video data processing.
As a example by smart mobile phone, video transmitting procedure includes: mobile phone uses Coding Compression Algorithm to each in video
Frame is compressed coding, then is sent to wear by HDMI connecting line by the frame of video after compressed encoding
Display, head-mounted display shows after being decoded.
Video compressing and encoding method has multiple, on a large scale, can be divided into lossless compression-encoding and damage
Compression.Lossless compression-encoding advantage is: the video after lossless compression-encoding is permissible decoding when
Recovering completely, such as: Huffman encodes, Run-Length Coding, and count coding, shortcoming is the data of data transmission
Amount is the biggest, takies the transfer resource transmission time that is more and that spend longer.And lossy compression method coding
Algorithms most in use has: vector quantization coding, it was predicted that coding, model image coding etc., this compressed encoding is
Irreversible, after compression data and decode after and initial data between there is certain difference, but permissible
Obtain higher compression ratio.
From the foregoing, it will be observed that the video play in the head-mounted display of prior art is all according to the most so
Whole frame carry out lossy compression method or lossless compress after incoming, video transmission efficiency and data integrity cannot
Taking into account, Consumer's Experience is the best.
Summary of the invention
The invention provides a kind of head-mounted display, picture output device and method for processing video frequency, system,
It is all after whole frame carries out lossy compression method or lossless compress to solve broadcasting video in prior art head-mounted display
Incoming, video transmission efficiency and integrity of video cannot be taken into account, the problem that Consumer's Experience is the best.
In order to achieve the above object, the technical scheme is that and be achieved in that:
According to an aspect of the invention, it is provided a kind of head-mounted display, this head-mounted display includes figure
As collecting unit, graphics processing unit, data transmission unit, video decoding unit and display unit:
Image acquisition units, for the head of captured in real-time user and eyes image and by the head of user and
Eyes image is sent to graphics processing unit；
Graphics processing unit, for calculating the fortune of user's eyeball according to the user's head received and eyes image
Dynamic track, obtains the key area information of user's eyeball current fixation display screen, is believed this key area
Breath is sent to data transmission unit；
Data transmission unit, for sending to the video being connected with head-mounted display defeated by key area information
Go out equipment so that the current frame image in video to be output is believed by picture output device according to this key area
Breath carries out subregion compressed encoding process；And, for receiving the subregion pressure that picture output device sends
Two field picture after contracting coded treatment, and this two field picture is sent to video decoding unit；
Video decoding unit, the two field picture after processing the subregion compressed encoding received utilizes pressure
The inverse algorithm reducing the staff code carries out subregion decoding process, and decoded two field picture is sent to display unit；
Display unit, is used for showing the decoded two field picture of output so that with key area in this two field picture
Corresponding data high definition shows, the data reduction display that remaining region is corresponding.
Alternatively, key area is with the coordinate figure of user's eyeball fixes as the center of circle on display screen, with in advance
Measured length is the region of radius.
Alternatively, data transmission unit includes: area information sends subelement and video data receives son list
Unit；
Area information sends subelement, for information transmission in key area be extremely connected with head-mounted display
Picture output device；
Video data receives subelement, for receiving at the subregion compressed encoding that picture output device sends
Two field picture after reason；
Area information sends subelement and video data reception subelement is wireless transport module or wire transmission
Module.
Alternatively, graphics processing unit, it is additionally operable to this key area information is sent to video decoding unit；
Video decoding unit, specifically for the two field picture profit after processing the subregion compressed encoding received
Inverse algorithm with compressed encoding and the key area information from graphics processing unit acquisition carry out subregion
Decoding process.
According to another aspect of the present invention, it is provided that a kind of picture output device, picture output device bag
Include:
Communication unit, for receiving the weight of user's eyeball current fixation display screen that head-mounted display sends
Point area information, is sent to territory, video partition processing unit by this key area information；
Territory, video partition processing unit, is used for the present frame in video to be output according to key area information
Image division is high definition region and atenuator region, and the data in high definition region are not compressed coding or
Carry out lossless compression-encoding process, the data in atenuator region are carried out lossy compression method coded treatment, wherein,
Attach most importance to region corresponding to region in high definition region；
Communication unit, is additionally operable to the two field picture after being processed by subregion compressed encoding and sends to head-mounted display.
Alternatively, picture output device is smart mobile phone or computer, and in this smart mobile phone or computer, storage has
Video data.
Alternatively, communication unit includes: area information receives subelement and video data sends subelement；
Area information receives subelement, for receiving user's eyeball current fixation of head-mounted display transmission
The key area information of display screen；
Video data sends subelement, and the two field picture after being processed by subregion compressed encoding sends to head
Wear display；
Area information receives subelement and video data transmission subelement is wireless communication module or wire communication
Module.
According to a further aspect of the invention, it is provided that a kind of method for processing video frequency, the method includes:
The head of captured in real-time head-mounted display user and eyes image；
Head and the movement locus of eyes image calculating user's eyeball to user, obtains user's eyeball current
Watch the key area information of display screen attentively；
Key area information is sent to the picture output device being connected with head-mounted display so that video is defeated
Go out equipment and the current frame image in video to be output is carried out subregion compression volume according to this key area information
Code processes；And, receive the two field picture after the subregion compressed encoding process that picture output device sends；
Two field picture after processing the subregion compressed encoding received utilizes the inverse algorithm of compressed encoding to carry out
Subregion decoding process；
Two field picture after display output decoding process so that data corresponding with key area in this two field picture
High definition shows, the data reduction display that remaining region is corresponding.
According to a further aspect of the invention, it is provided that a kind of processing system for video, this system includes: as
The head-mounted display of one aspect of the invention, and such as the picture output device of another aspect of the invention；
Wired or wireless data cube computation is set up between head-mounted display and picture output device.
The invention has the beneficial effects as follows: the technical scheme of the embodiment of the present invention, by Real-time Collection user's
Head and eyes image, and calculate user's eyeball current fixation display screen based on the image collected
Key area, is then sent to the video frequency output being connected with head-mounted display by the information of this key area
Equipment so that picture output device can be according to this key area information, to present frame in video to be output
Image carries out subregion compressed encoding: the data of the key area that those correspond to user's viewing are carried out nothing
Damage compression, and the video data in remaining region is carried out lossy compression method.Picture output device is again by subregion
Frame of video after territory processes is sent to head-mounted display, and head-mounted display shows defeated after carrying out subregion decoding
Go out, thus by the lossless compress of video in key area that user is watched or do not compress, it is ensured that regard
The high definition of frequency shows and the non-destructive of data, further through the data acquisition lossy compression method to remaining region, subtracts
Little data volume makes video can be transferred to head-mounted display wirelessly, and improves data and pass
Defeated efficiency, saves the transmission time, optimizes user's experience.
Accompanying drawing explanation
Fig. 1 is head-mounted display and picture output device transmission of video schematic diagram in prior art；
Fig. 2 is the structured flowchart of a kind of head-mounted display of one embodiment of the invention；
Fig. 3 is the subregional schematic diagram of two field picture of one embodiment of the invention；
Fig. 4 is the structured flowchart of the picture output device of one embodiment of the invention；
Fig. 5 is the structured flowchart of the processing system for video of one embodiment of the invention；
Fig. 6 is the schematic flow sheet of the method for processing video frequency of one embodiment of the invention.
Detailed description of the invention
The design concept of the present invention is: be all one whole for the video play in prior art head-mounted display
Frame, a whole frame compression process after incoming, result in the problem that Consumer's Experience is the best.The present invention implements
Example proposes movement locus based on head-mounted display wearer's eyeball and calculates wearer and watch display screen
The key area of curtain, and this key area information is sent to picture output device so that video frequency output sets
It is passed to wear display for after frame of video is carried out subarea processing according to the key area information dynamically obtained
In device, process such that it is able to realize the lossy compression method of data in atenuator region so that video can pass through
Wireless way for transmitting is to head-mounted display, and high definition area video does not processes or lossless compress processes, both
Reduce data volume and also ensure that the integrity of video data in key area simultaneously.
Embodiment one
Fig. 2 is the structured flowchart of a kind of head-mounted display of one embodiment of the invention, sees Fig. 2, should
Head-mounted display 100 includes: image acquisition units 101, graphics processing unit 102, data transmission unit
103, video decoding unit 104 and display unit 105:
Image acquisition units 101, for the head of captured in real-time user and eyes image and by the head of user
Portion and eyes image are sent to graphics processing unit 102；
Graphics processing unit 102, for calculating user's eyeball according to the user's head received and eyes image
Movement locus, obtain the key area information of user's eyeball current fixation display screen, by this emphasis district
Domain information is sent to data transmission unit 103；
Data transmission unit 103, for sending key area information to regarding of being connected with head-mounted display
Frequently outut device so that picture output device to the current frame image in video to be output according to this emphasis district
Domain information carries out subregion compressed encoding process；And, for receiving the subregion that picture output device sends
Two field picture after the compressed encoding process of territory, and this two field picture is sent to video decoding unit 104；
Video decoding unit 104, the two field picture profit after the subregion compressed encoding received is processed
Carry out subregion decoding process by the inverse algorithm of compressed encoding, decoded two field picture is sent to display
Unit 105；
Display unit 105, is used for showing the decoded two field picture of output so that with emphasis in this two field picture
Data high definition in the region that region is corresponding shows, the data in remaining region weaken display.
By the head-mounted display shown in Fig. 2, the head of Real-time Collection wearer and eyes image, and base
Calculate the key area of wearer's current fixation in the image collected, then this key area is believed
Breath is sent to picture output device so that picture output device regards to be transmitted according to this key area information
A two field picture in Pin carries out subregion compressed encoding process and is sent to head-mounted display, by wearing display
Device shows after being decoded.Owing to a two field picture can be carried out territory, dynamic partition compressed encoding thus with existing
There is the processing mode of a whole frame in technology to compare, both can guarantee that video high definition in the key area that user watches
Display and the non-destructive of data, can improve again data transmission efficiency, saves the transmission time.
Embodiment two
Fig. 3 is the subregional schematic diagram of two field picture of one embodiment of the invention.The present embodiment combines figure
Two field picture subregion structural representation shown in 3, decoding and display to this head-mounted display carry out emphasis and say
Bright, other guide sees other embodiments of the present invention.
The present embodiment first passes through in head-mounted display arrange image acquisition units collection user head and
The image of eye.Here image acquisition units can be photographic head, and photographic head may be mounted at head
Wear in the space between the display screen of display and user's eye, by photographic head captured in real-time user's
Head and eyeball obtain real time imaging.
It should be noted that in the concept that ocular movement is followed the tracks of, the main movement of eyeball has three kinds of forms:
Watch attentively, beat and smooth and trail tracking.Research conclusion according to prior art understands oculomotor angle
Scope is about 18 degree, is accomplished by head movement more than 12 degree and completes to coordinate it.So to eye
The when that ball being followed the tracks of, head to be considered and two factors of eyeball.Based on this, the present embodiment utilizes shooting
Head gathers head and the eyes image of user.The original image that photographic head dynamic acquisition is watched to human eye, sends out
Give graphics processing unit.
Original image is processed by graphics processing unit according to tracing of human eye algorithm, calculates user's eyeball
Movement locus, and determine the key area that user's eyeball is watched, then by key area information (such as circle
The coordinate figure of the heart and radius) it is transferred to picture output device so that and picture output device is compiled in video compress
During code, territory, video partition is processed.Graphics processing unit carries out the basic fundamental of human eye sight tracking and can be divided into
Hardware and software.Method based on hardware mainly user brings the special helmet or uses head
Fixed support obtains the track of eyeball, but the method is not convenient and also error is bigger.With software as base
The eye tracking technology of plinth is to obtain eye and the image of face of people first with video camera, is reprocessed by image
After crossing, calculate the track of eyeball, and transfer data to video processing equipment, thus control video
Process and display.I.e. have employed software mode in the present embodiment, graphics processing unit uses image procossing core
Sheet, and according to image processing algorithm and Technical Follow-Up ocular movement.
Here key area is with the coordinate figure of user's eyeball fixes as the center of circle on display screen, with predetermined
The region of a length of radius.In head-mounted display, when user watches display screen, eyeball is corresponding aobvious
The upper position coordinates of display screen curtain, then can observe the empirical value of regional extent according to eyeball and determines one
Individual radius, i.e. can determine that the emphasis viewing areas of eyeball.Here position coordinates is display on-screen pixels
The coordinate of point.The display resolution of such as head-mounted display is 1680 × 1050, i.e. horizontal direction has 1680
Individual pixel, vertical direction has 1050 pixels, then the position coordinates of eyeball fixes can be
The 800th pixel in (800,500), i.e. horizontal direction and the 500th pixel of vertical direction determine
Position coordinates, after being calculated the position coordinates that eyeball is corresponding, in conjunction with the length of radius, the most available
Key area.
Seeing Fig. 3, in the present embodiment, the key area illustrated is the border circular areas of dotted line limit,
Key area is high definition region when two field picture shows, the region except high definition region in two field picture is all
It it is atenuator region.
Owing to display resolution frame sign each with incoming video is consistent, so head-mounted display
After calculating the key area of user's eyeball current fixation, key area information is sent to video frequency output and sets
Standby, a two field picture can be divided into corresponding high definition according to key area information by such picture output device
Region, and using the region outside high definition region as atenuator region, carry out dynamic subarea processing.
In the present embodiment, data transmission unit includes: area information sends subelement and video data connects
Receive subelement；Area information sends subelement, for key area information being sent extremely and head-mounted display
The picture output device connected；Video data receives subelement, for receiving what picture output device sent
Two field picture after the compressed encoding process of subregion；Area information sends subelement and video data receives son list
Unit is wireless transport module or wire transmission module.
It should be noted that due in prior art frame of video be essentially all the process side according to a whole frame
Formula, this mode relative data amount is relatively big, causes the transmission of video approach of existing main flow head-mounted display
It is all by wire transmission mode, as it is shown in figure 1, pass through between picture output device and head-mounted display
HDMI (High Definition Multimedia Interface is called for short HDMI) is even
Wiring connects, and passes to head by HDMI data wire after the video data whole frame compression in picture output device
Wear display.This data transfer mode needs to carry HDMI connecting line, and if in order to improve biography
Defeated efficiency and use whole frame lossy compression method mode be compressed process, then can affect integrity and the stream of video
Smooth degree, and use lossless compress to ensure video data integrity, data transmission efficiency can be sacrificed again
With the transmission time.
Send subelement for area information in this situation, preferably the present embodiment and video data receives
Subelement is wireless transport module, such as, and WiFi wireless transport module.So can with omitted data line,
And by calculating the key area information that user watches attentively, then this key area information is issued and regard
Frequently outut device makes subregion compressed encoding process, it is ensured that video lossless in key area, remaining
Area data lossy compression method.On the one hand, due in a frame data in remaining region by lossy compression method, thus
Substantially reduce data volume so that video wireless transmission is possibly realized.On the other hand, owing to ensure that weight
The non-destructive of some area video, also makes subsequent decoding can be entered by this key area video data in processing
The lossless reduction of row, does not affect the integrity of video.
Video decoding unit is decoded for convenience, and in the present embodiment, graphics processing unit is additionally operable to this
Key area information is sent to video decoding unit, and video decoding unit is specifically for the subregion received
Two field picture after territory compressed encoding processes utilizes the inverse algorithm of compressed encoding and obtains from graphics processing unit
The key area information taken carries out subregion decoding process.
Concrete, picture output device is when carrying out the compressed encoding of a two field picture, according to head-mounted display
The key area information sent carries out territory, dynamic partition compressed encoding, and the two field picture after coding is sent to head
Wear display, the data transmission unit of head-mounted display receive coding after two field picture after be sent to video
Decoding unit.Video decoding unit is when decoding, according to the emphasis district dynamically obtained from graphics processing unit
The coding information that domain information and coded data carry is decoded, and coded data here carries
Coding information may include that high definition area information, and data in high definition region and encryption algorithm thereof are (e.g.,
Huffman encodes), and atenuator region information, in atenuator region, data and encryption algorithm thereof are (such as vector quantity
Change) etc..Owing to video decoding unit has got key area information from graphics processing unit, and encode
The high definition area information of rear data is consistent with key area information, so video decoding unit can be known
Data in high definition region, road are the data that the key area of user's eyeball current fixation is corresponding, then may be used
To use the inverse algorithm of key area data encoding algorithm to be decoded, thus it is corresponding to obtain key area
Video data.
Same, video decoding unit according to the key area information obtained it will also be appreciated that frame figure after Bian Ma
The atenuator region of picture, and be decoded obtaining according to the inverse algorithm of data encoding algorithm in this atenuator region
The data that atenuator region is corresponding, are then sent to the two field picture that decoding obtains display unit and carry out showing defeated
Go out, it is achieved the data high definition in region corresponding with key area in this two field picture shows, remaining region
Data weaken display.
Embodiment three
Fig. 4 is the structured flowchart of the picture output device of one embodiment of the invention, emphasis in the present embodiment
Structure and work process to picture output device illustrate, and other guide sees other realities of the present invention
Execute example.
Seeing Fig. 4, this picture output device 400 includes:
Communication unit 401, for receiving user's eyeball current fixation display screen that head-mounted display sends
Key area information, this key area information is sent to territory, video partition processing unit 402；
Territory, video partition processing unit 402, was used for working as in video to be output according to key area information
Prior image frame is divided into high definition region and atenuator region, and is not compressed the data in high definition region compiling
Code or carry out lossless compression-encoding process, the data in atenuator region are carried out lossy compression method coded treatment,
Wherein, attach most importance to region corresponding to region in high definition region；
Communication unit 401, is additionally operable to the two field picture after being processed by subregion compressed encoding and sends aobvious to wearing
Show device.
In the present embodiment, picture output device 400 is smart mobile phone or computer, this smart mobile phone or computer
Middle storage has video data.
In the present embodiment, communication unit 401 includes: area information receives subelement and video data sends son
Unit；Area information receives subelement, for receiving user's eyeball current fixation that head-mounted display sends
The key area information of display screen；Video data sends subelement, for by subregion compressed encoding
Two field picture after process sends to head-mounted display；Area information receives subelement and video data sends son
Unit is wireless communication module or wire communication module.
Concrete, area information receives subelement and the region letter in the data transmission unit of head-mounted display
Breath sends subelement and connects, it is achieved the transmission of key area information.Video data sends subelement and wears
Video data in the data transmission unit of display receives subelement and connects, it is achieved the transmission of video data.
It addition, the transmission means that area information reception subelement sends subelement with area information should be consistent, example
As, both of which uses wireless transmission method to carry out data transmission.And video data sends subelement and regards
Frequently the transmission means of data receiver subelement should also be as unanimously.
In the present embodiment, the data in high definition region are not compressed by territory, video partition processing unit 402
Encode or carry out lossless compression-encoding process, the data in atenuator region are carried out lossy compression method coded treatment
Time coding standard be H.264.H.264 Video coding advantage is: low bit-rate, high quality graphic, appearance
Wrong ability is strong, network availability strong, and the advantage of its maximum is to have the highest data compression ratio, same
Under conditions of picture quality, compression ratio H.264 is more than 2 times of MPEG-2, is MPEG-4
1.5～2 times.H.264 while there is high compression ratio, also have an image that high-quality is smooth, just because of
To this end, the video data after H.264 standard code compresses, required in network transmission process
Bandwidth is less, the most more economically.
The structure of picture output device as shown in Figure 4 and work process understand, and the video of the present embodiment is defeated
Go out equipment 400 receive head-mounted display by eye follow the tracks of user that is that calculate in real time and that send watch
Key area information after, the territory, two field picture dynamic partition of video to be output is compressed, this two field picture is pressed
(high definition region is corresponding with key area with atenuator region to be divided into high definition region according to key area information
Region, atenuator region is the region in a two field picture beyond high definition region), then, the number to high definition region
According to using lossless compression-encoding algorithm to carry out lossless compression-encoding, or it is not compressed encoding to ensure to regard
The non-destructive of frequency evidence；And the data acquisition for atenuator region processes by lossy compression method, thus reduce data
Transmission quantity, improves efficiency of transmission, saves the transmission time.
It addition, picture output device 400 has H.264 regarding of the highest data compression ratio by employing
Frequency coding, decreases data volume, saves data transfer bandwidth.Further, communication unit 401 can be adopted
Realize video wireless transmission with communication, eliminate the data wires such as HDMI, thus be greatly promoted
Consumer's Experience.
Embodiment four
Fig. 5 is the structured flowchart of the processing system for video of one embodiment of the invention, sees Fig. 5, and this regards
Frequently processing system 500 includes: head-mounted display 100, and picture output device 400, head-mounted display
Wired or wireless data cube computation is set up between 100 and picture output device 400.
Below as a example by setting up wireless data connection between head-mounted display 100 and picture output device 400
Processing system for video is illustrated.
In the present embodiment, the structure of processing system for video 500 is as it is shown in figure 5, picture output device 400
Can be the input equipment such as mobile phone and computer, effect be to provide video frequency output；
Image acquisition units in head-mounted display 100, takes use by wearer's head and eye photo
Carry out eye tracking process later；Here image acquisition units can be photographic head；
The image that image acquisition units is collected by the graphics processing unit in head-mounted display 100 processes
Calculate the key area of wearer's eyeball fixes, issue data transmission unit；
The data transmission unit of head-mounted display 100 includes: area information sends subelement and video data
Receiving subelement, concrete, area information sends the communication unit of subelement and picture output device 400
Middle area information receives subelement and sets up wireless connections, key area information is sent to area information and receives
Subelement；
The area information of picture output device 400 receives subelement by the real-time emphasis of user's eyeball fixes
Area information is sent to territory, video partition processing unit；
After territory, the video partition processing unit of picture output device 400 receives new key area information, root
According to this key area information, to the two field picture zoning that will export, i.e. a two field picture is divided
For high definition region and atenuator region, high definition region is corresponding with key area, atenuator region be in a frame except
The region in high definition region.Then, the data in high definition region are carried out lossless compression-encoding, to atenuator region
Data carry out lossy compression method coding, compressed encoding here uses H.264 standard.Picture output device
Video data after coding is sent to video data transmitting and send son single by territory, the video partition processing unit of 400
Unit, is sent the video data in the subelement data transmission unit with head-mounted display 100 by video data
Receive subelement and set up wireless connections, thus the two field picture after the compressed encoding of subregion is wirelessly transmitted to head
Wear display 100.
After the video data of head-mounted display 100 receives the two field picture that subelement receives, it is sent to video
Decoding unit, video decoding unit is come video solution by the inverse algorithm of video compress after receiving two field picture
Code, is sent to display unit by decoded image.Display unit carries out display output to two field picture.
By the way, the video output system of the offer of the present embodiment, wireless mode can be used to exist
Transmitting video data between picture output device and head-mounted display, is greatly improved Consumer's Experience, and
Video processing when, it is determined by the emphasis viewing areas of user, in emphasis viewing areas, depending on
Frequently Lossless transport, at remaining area video Weakening treatment.So it is possible not only to reach key area video high
Clearly also can guarantee that video compress efficiency of transmission.
It addition, other structures of head-mounted display 100 of including of this processing system for video 500 and work
Process, can be found in the description of previous embodiment one.The video frequency output that this processing system for video 500 includes
Other structures of equipment 400 and work process, can be found in the description of previous embodiment three, the most superfluous
State.
Embodiment five
Fig. 6 is the schematic flow sheet of the method for processing video frequency of one embodiment of the invention, sees Fig. 6, and this regards
Frequently processing method comprises the steps:
Step S610, the head of captured in real-time head-mounted display user and eyes image；
Step S620, head and the movement locus of eyes image calculating user's eyeball to user, used
The key area information of family eyeball current fixation display screen；
Step S630, sends key area information to the picture output device being connected with head-mounted display,
Current frame image in video to be output is carried out point by picture output device according to this key area information
Area compresses coded treatment；And, after receiving the subregion compressed encoding process that picture output device sends
Two field picture；
Step S640, the two field picture after processing the subregion compressed encoding received utilizes the inverse of compressed encoding
Subregion decoding process is carried out to algorithm；
Step S650, the two field picture after display output decoding process so that with key area in this two field picture
Corresponding data high definition shows, the data reduction display that remaining region is corresponding.
It should be noted that this method for processing video frequency of the present embodiment is and the work of aforementioned head-mounted display
Making process corresponding, therefore, in the present embodiment, method for processing video frequency realizes not description in step
Content may refer to the related description in the present invention aforementioned head-mounted display embodiment, repeat no more here.
In sum, the technical scheme of the embodiment of the present invention, by head and the eye of Real-time Collection user
Image, and the key area of user's eyeball current fixation display screen is calculated based on the image collected,
Then the information of this key area is sent to the picture output device being connected with head-mounted display so that
Current frame image in video to be output can be carried out point by picture output device according to this key area information
Area compresses encodes: the data of the key area that those correspond to user's viewing are carried out lossless compress, and
Video data in remaining region is carried out lossy compression method.Picture output device is again by after subarea processing
Frame of video is sent to head-mounted display, and head-mounted display shows output after carrying out subregion decoding, thus logical
Cross in the key area to user's viewing the lossless compress of video or do not compress, it is ensured that the high definition of video shows
Show the non-destructive with data, make further through the data acquisition lossy compression method in remaining region is reduced data volume
Obtain video and can be transferred to head-mounted display wirelessly, and, improve data transmission efficiency, joint
Save the transmission time, optimize user's experience.
The foregoing is only presently preferred embodiments of the present invention, be not intended to limit the protection model of the present invention
Enclose.All any modification, equivalent substitution and improvement etc. made within the spirit and principles in the present invention, all
Comprise within the scope of the present invention.
Claims (9)
1. a head-mounted display, it is characterised in that this head-mounted display includes image acquisition units, figure
As processing unit, data transmission unit, video decoding unit and display unit:
Described image acquisition units, for the head of captured in real-time user and eyes image and by described user
Head and eyes image be sent to graphics processing unit；
Described graphics processing unit, for calculating user's eyeball according to the user's head received and eyes image
Movement locus, obtain the key area information of user's eyeball current fixation display screen, by this emphasis district
Domain information is sent to data transmission unit；
Described data transmission unit, for by described key area information send to described head-mounted display
The picture output device connected so that described picture output device is to the current frame image in video to be output
Subregion compressed encoding process is carried out according to this key area information；And, it is used for receiving described video defeated
Go out the two field picture after the subregion compressed encoding process that equipment sends, and this two field picture is sent to video solution
Code unit；
Described video decoding unit, the two field picture profit after the subregion compressed encoding received is processed
Carry out subregion decoding process by the inverse algorithm of compressed encoding, decoded two field picture is sent to display
Unit；
Described display unit, is used for showing the decoded two field picture of output so that with emphasis in this two field picture
Data high definition corresponding to region shows, data corresponding to remaining region weaken display.
Head-mounted display the most according to claim 1, it is characterised in that described key area is aobvious
With region as the center of circle, with predetermined length as radius of the coordinate figure of user's eyeball fixes on display screen curtain.
Head-mounted display the most according to claim 1 and 2, it is characterised in that described data are transmitted
Unit includes: area information sends subelement and video data receives subelement；
Described area information sends subelement, for sending described key area information to wearing with described
The picture output device that display connects；
Described video data receives subelement, for receiving the subregion pressure that described picture output device sends
Two field picture after contracting coded treatment；
Described area information sends subelement and video data reception subelement is wireless transport module or wired
Transport module.
Head-mounted display the most according to claim 1, it is characterised in that described graphics processing unit,
It is additionally operable to this key area information is sent to described video decoding unit；
Described video decoding unit, specifically for the frame figure after processing the subregion compressed encoding received
As utilizing the inverse algorithm of compressed encoding and entering from the key area information that described graphics processing unit obtains
Row subregion decoding process.
5. a picture output device, it is characterised in that described picture output device includes:
Communication unit, for receiving the weight of user's eyeball current fixation display screen that head-mounted display sends
Point area information, is sent to territory, video partition processing unit by this key area information；
Territory, described video partition processing unit, is used in video to be output according to described key area information
Current frame image be divided into high definition region and atenuator region, and the data in high definition region are not pressed
Reduce the staff code or carry out lossless compression-encoding process, the data in atenuator region are carried out at lossy compression method coding
Reason, wherein, described high definition region is the region that described key area is corresponding；
Described communication unit, is additionally operable to the two field picture after being processed by subregion compressed encoding and sends to described head
Wear display.
Picture output device the most according to claim 5, it is characterised in that described video frequency output sets
Standby for smart mobile phone or computer, in this smart mobile phone or computer, storage has video data.
7. according to the picture output device described in claim 5 or 6, it is characterised in that described communication unit
Unit includes: area information receives subelement and video data sends subelement；
Described area information receives subelement, and the user's eyeball sent for receiving described head-mounted display is worked as
Before the key area information of display screen watched attentively；
Described video data sends subelement, and the two field picture after being processed by subregion compressed encoding sends
To described head-mounted display；
Described area information receive subelement and described video data send subelement be wireless communication module or
Wire communication module.
8. a method for processing video frequency, it is characterised in that the method includes:
The head of captured in real-time head-mounted display user and eyes image；
Head and the movement locus of eyes image calculating user's eyeball to user, obtains user's eyeball current
Watch the key area information of display screen attentively；
Key area information is sent to the picture output device that is connected with head-mounted display so that described regard
Frequently outut device carries out subregion pressure to the current frame image in video to be output according to this key area information
Contracting coded treatment；And, after receiving the subregion compressed encoding process that described picture output device sends
Two field picture；
Two field picture after processing the subregion compressed encoding received utilizes the inverse algorithm of compressed encoding to carry out
Subregion decoding process；
Two field picture after display output decoding process so that data corresponding with key area in this two field picture
High definition shows, the data reduction display that remaining region is corresponding.
9. a processing system for video, it is characterised in that this system includes: as appointed in claim 1-4
One described head-mounted display, and the picture output device as according to any one of claim 5-7；
Wired or wireless data cube computation is set up between described head-mounted display and described picture output device.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610466833.6A CN105979224A (en) | 2016-06-23 | 2016-06-23 | Head mount display, video output device and video processing method and system |
| PCT/CN2016/114042 WO2017219652A1 (en) | 2016-06-23 | 2016-12-31 | Head mounted display, video output device, and video processing method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610466833.6A CN105979224A (en) | 2016-06-23 | 2016-06-23 | Head mount display, video output device and video processing method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105979224A true CN105979224A (en) | 2016-09-28 |
Family
ID=57020541
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610466833.6A Pending CN105979224A (en) | 2016-06-23 | 2016-06-23 | Head mount display, video output device and video processing method and system |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN105979224A (en) |
| WO (1) | WO2017219652A1 (en) |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107333119A (en) * | 2017-06-09 | 2017-11-07 | 歌尔股份有限公司 | The processing method and equipment of a kind of display data |
| WO2017219652A1 (en) * | 2016-06-23 | 2017-12-28 | 青岛歌尔声学科技有限公司 | Head mounted display, video output device, and video processing method and system |
| CN108347557A (en) * | 2017-01-21 | 2018-07-31 | 盯盯拍（东莞）视觉设备有限公司 | Panoramic image capturing device, display device, capturing method, and display method |
| CN108347556A (en) * | 2017-01-21 | 2018-07-31 | 盯盯拍（东莞）视觉设备有限公司 | Panoramic image shooting method, panoramic image display method, panoramic image shooting device, and panoramic image display device |
| WO2019033903A1 (en) * | 2017-08-14 | 2019-02-21 | 歌尔股份有限公司 | Graphics rendering method and device for virtual reality |
| CN110545430A (en) * | 2018-05-28 | 2019-12-06 | 北京松果电子有限公司 | video transmission method and device |
| CN112513719A (en) * | 2018-07-27 | 2021-03-16 | 京瓷株式会社 | Display device, display system, and movable vehicle |
| CN112783319A (en) * | 2019-11-05 | 2021-05-11 | 宏达国际电子股份有限公司 | Display system |
| CN112887702A (en) * | 2021-01-11 | 2021-06-01 | 杭州灵伴科技有限公司 | Near-eye display equipment and camera data transmission method thereof |
| CN113256661A (en) * | 2021-06-23 | 2021-08-13 | 北京蜂巢世纪科技有限公司 | Image processing method, apparatus, device, medium, and program product |
| CN113327489A (en) * | 2021-06-03 | 2021-08-31 | 西安工业大学 | Layered mechanical intelligent braille reader based on machine vision |
| CN114786037A (en) * | 2022-03-17 | 2022-07-22 | 青岛虚拟现实研究院有限公司 | Self-adaptive coding compression method facing VR projection |
| CN114935971A (en) * | 2021-02-05 | 2022-08-23 | 京东方科技集团股份有限公司 | Display driving chip, display device and display driving method |
| TWI802456B (en) * | 2021-06-28 | 2023-05-11 | 見臻科技股份有限公司 | Optical system with improved user experiemce and accurate eye-tracking and related method |
| CN116132431A (en) * | 2023-04-19 | 2023-05-16 | 泰诺尔(北京)科技有限公司 | Data transmission method and system |
| CN116737097A (en) * | 2022-09-30 | 2023-09-12 | 荣耀终端有限公司 | Screen projection image processing method and electronic equipment |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111225228B (en) * | 2020-01-15 | 2022-04-12 | 深圳拙河科技有限公司 | Video live broadcast method, device, equipment and medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103930817A (en) * | 2011-06-20 | 2014-07-16 | 谷歌公司 | Systems and methods for adaptive transmission of data |
| CN104539929A (en) * | 2015-01-20 | 2015-04-22 | 刘宛平 | Three-dimensional image coding method and coding device with motion prediction function |
| CN104767992A (en) * | 2015-04-13 | 2015-07-08 | 北京集创北方科技有限公司 | Head-wearing type display system and image low-bandwidth transmission method |
| GB2526515A (en) * | 2014-03-25 | 2015-12-02 | Jaguar Land Rover Ltd | Image capture system |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN205812229U (en) * | 2016-06-23 | 2016-12-14 | 青岛歌尔声学科技有限公司 | A kind of head-mounted display, picture output device and processing system for video |
| CN105979224A (en) * | 2016-06-23 | 2016-09-28 | 青岛歌尔声学科技有限公司 | Head mount display, video output device and video processing method and system |
-
2016
- 2016-06-23 CN CN201610466833.6A patent/CN105979224A/en active Pending
- 2016-12-31 WO PCT/CN2016/114042 patent/WO2017219652A1/en active Application Filing
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103930817A (en) * | 2011-06-20 | 2014-07-16 | 谷歌公司 | Systems and methods for adaptive transmission of data |
| GB2526515A (en) * | 2014-03-25 | 2015-12-02 | Jaguar Land Rover Ltd | Image capture system |
| CN104539929A (en) * | 2015-01-20 | 2015-04-22 | 刘宛平 | Three-dimensional image coding method and coding device with motion prediction function |
| CN104767992A (en) * | 2015-04-13 | 2015-07-08 | 北京集创北方科技有限公司 | Head-wearing type display system and image low-bandwidth transmission method |
Cited By (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017219652A1 (en) * | 2016-06-23 | 2017-12-28 | 青岛歌尔声学科技有限公司 | Head mounted display, video output device, and video processing method and system |
| CN108347556A (en) * | 2017-01-21 | 2018-07-31 | 盯盯拍（东莞）视觉设备有限公司 | Panoramic image shooting method, panoramic image display method, panoramic image shooting device, and panoramic image display device |
| CN108347557A (en) * | 2017-01-21 | 2018-07-31 | 盯盯拍（东莞）视觉设备有限公司 | Panoramic image capturing device, display device, capturing method, and display method |
| US10620903B2 (en) | 2017-06-09 | 2020-04-14 | Goertek Inc. | Method and device for processing display data |
| CN107333119A (en) * | 2017-06-09 | 2017-11-07 | 歌尔股份有限公司 | The processing method and equipment of a kind of display data |
| US11106422B2 (en) | 2017-06-09 | 2021-08-31 | Goertek Inc. | Method for processing display data |
| WO2019033903A1 (en) * | 2017-08-14 | 2019-02-21 | 歌尔股份有限公司 | Graphics rendering method and device for virtual reality |
| US10859840B2 (en) | 2017-08-14 | 2020-12-08 | Goertek Inc. | Graphics rendering method and apparatus of virtual reality |
| CN110545430A (en) * | 2018-05-28 | 2019-12-06 | 北京松果电子有限公司 | video transmission method and device |
| CN112513719A (en) * | 2018-07-27 | 2021-03-16 | 京瓷株式会社 | Display device, display system, and movable vehicle |
| CN112513719B (en) * | 2018-07-27 | 2023-03-31 | 京瓷株式会社 | Display device, display system, and movable vehicle |
| CN112783319A (en) * | 2019-11-05 | 2021-05-11 | 宏达国际电子股份有限公司 | Display system |
| CN112887702A (en) * | 2021-01-11 | 2021-06-01 | 杭州灵伴科技有限公司 | Near-eye display equipment and camera data transmission method thereof |
| CN114935971A (en) * | 2021-02-05 | 2022-08-23 | 京东方科技集团股份有限公司 | Display driving chip, display device and display driving method |
| CN114935971B (en) * | 2021-02-05 | 2024-08-20 | 京东方科技集团股份有限公司 | Display device and display driving method |
| CN113327489A (en) * | 2021-06-03 | 2021-08-31 | 西安工业大学 | Layered mechanical intelligent braille reader based on machine vision |
| CN113256661A (en) * | 2021-06-23 | 2021-08-13 | 北京蜂巢世纪科技有限公司 | Image processing method, apparatus, device, medium, and program product |
| TWI802456B (en) * | 2021-06-28 | 2023-05-11 | 見臻科技股份有限公司 | Optical system with improved user experiemce and accurate eye-tracking and related method |
| US11726564B2 (en) | 2021-06-28 | 2023-08-15 | Ganzin Technology, Inc. | Optical system providing accurate eye-tracking and related method |
| US12118139B2 (en) | 2021-06-28 | 2024-10-15 | Ganzin Technology, Inc. | Optical system providing accurate eye-tracking and related method |
| CN114786037A (en) * | 2022-03-17 | 2022-07-22 | 青岛虚拟现实研究院有限公司 | Self-adaptive coding compression method facing VR projection |
| CN114786037B (en) * | 2022-03-17 | 2024-04-12 | 青岛虚拟现实研究院有限公司 | VR projection-oriented adaptive coding compression method |
| CN116737097A (en) * | 2022-09-30 | 2023-09-12 | 荣耀终端有限公司 | Screen projection image processing method and electronic equipment |
| CN116737097B (en) * | 2022-09-30 | 2024-05-17 | 荣耀终端有限公司 | Screen projection image processing method and electronic device |
| CN116132431A (en) * | 2023-04-19 | 2023-05-16 | 泰诺尔(北京)科技有限公司 | Data transmission method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2017219652A1 (en) | 2017-12-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105979224A (en) | Head mount display, video output device and video processing method and system | |
| CN111580765B (en) | Screen projection method, screen projection device, storage medium, screen projection equipment and screen projection equipment | |
| CN103460250B (en) | Image procossing based on object of interest | |
| WO2018077142A1 (en) | Panoramic video processing method, device and system | |
| CN106060582B (en) | Video transmission system, method and device | |
| JP3263278B2 (en) | Image compression communication device | |
| US9756328B2 (en) | System, terminal, and method for dynamically adjusting video | |
| US10237548B2 (en) | Video transmission based on independently encoded background updates | |
| CN104539929B (en) | Stereo-image coding method and code device with motion prediction | |
| CN109417624A (en) | Device and method for providing and showing content | |
| KR102166158B1 (en) | Stereoscopic video encoding apparatus, stereoscopic video decoding apparatus and stereoscopic video display system including the same | |
| US11310560B2 (en) | Bitstream merger and extractor | |
| CN106375841A (en) | Wireless screen projection data processing method and device, video data display method and device, and electronic device | |
| CN107211081B (en) | Video transmission based on independent coding of background updates | |
| EP3434021B1 (en) | Method, apparatus and stream of formatting an immersive video for legacy and immersive rendering devices | |
| JP2020115299A (en) | Virtual space information processing device, method and program | |
| CN107332977B (en) | Augmented reality method and augmented reality equipment | |
| CN106534882A (en) | Encoding and decoding method and device for virtual reality panoramic video | |
| CN107396082B (en) | Image data processing method and device | |
| CN114040184A (en) | Image display method, system, storage medium and computer program product | |
| CN205812229U (en) | A kind of head-mounted display, picture output device and processing system for video | |
| CN101001370A (en) | Vedio transmission system and method of mixed with mutiple code and decode | |
| US11457053B2 (en) | Method and system for transmitting video | |
| CN202957933U (en) | 3D video communication device | |
| CN106027991B (en) | Medical video image live broadcast all-in-one |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20160928 |
|
| RJ01 | Rejection of invention patent application after publication |