GB2440384B - Method,system and program product for measuring audio video synchronization using lip and teeth characteristics - Google Patents

Method,system and program product for measuring audio video synchronization using lip and teeth characteristics

Info

Publication number: GB2440384B
Authority: GB; United Kingdom
Prior art keywords: lip; program product; audio video; video synchronization; measuring audio
Prior art date: 2005-04-13
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.): Expired - Fee Related

Application number

GB0622592A

Other versions

GB0622592D0 (en

GB2440384A (en

Inventor

J Carl Cooper

Mirko Dusan Vojnovic

Christopher Smith

Jibanananda Roy

Saurabh Jain

Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)

Pixel Instruments Corp

Original Assignee

Pixel Instruments Corp

Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)

2005-04-13

Filing date

2006-04-13

Publication date

2010-01-13

2005-04-13 Priority claimed from PCT/US2005/012588 external-priority patent/WO2005115014A2/en

2006-04-13 Application filed by Pixel Instruments Corp filed Critical Pixel Instruments Corp

2006-04-13 Priority claimed from PCT/US2006/014023 external-priority patent/WO2006113409A2/en

2006-12-27 Publication of GB0622592D0 publication Critical patent/GB0622592D0/en

2008-01-30 Publication of GB2440384A publication Critical patent/GB2440384A/en

2010-01-13 Application granted granted Critical

2010-01-13 Publication of GB2440384B publication Critical patent/GB2440384B/en

Status Expired - Fee Related legal-status Critical Current

2026-04-13 Anticipated expiration legal-status Critical

Classifications

- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
- G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/02—Feature extraction for speech recognition; Selection of recognition unit
- G10L2015/025—Phonemes, fenemes or fenones being the recognition units

Landscapes

Engineering & Computer Science (AREA)
Health & Medical Sciences (AREA)
Oral & Maxillofacial Surgery (AREA)
Multimedia (AREA)
General Health & Medical Sciences (AREA)
Human Computer Interaction (AREA)
Physics & Mathematics (AREA)
Computer Vision & Pattern Recognition (AREA)
General Physics & Mathematics (AREA)
Theoretical Computer Science (AREA)
Computational Linguistics (AREA)
Signal Processing (AREA)
Audiology, Speech & Language Pathology (AREA)
Acoustics & Sound (AREA)
Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
Television Signal Processing For Recording (AREA)

GB0622592A 2005-04-13 2006-04-13 Method,system and program product for measuring audio video synchronization using lip and teeth characteristics Expired - Fee Related GB2440384B (en)

Applications Claiming Priority (3)

Application Number	Priority Date	Filing Date	Title
PCT/US2005/012588 WO2005115014A2 (en)	2004-05-14	2005-04-13	Method, system, and program product for measuring audio video synchronization
PCT/US2005/041623 WO2007035183A2 (en)	2005-04-13	2005-11-16	Method, system, and program product for measuring audio video synchronization independent of speaker characteristics
PCT/US2006/014023 WO2006113409A2 (en)	2005-04-13	2006-04-13	Method, system, and program product for measuring audio video synchronization using lip and teeth charateristics

Publications (3)

Publication Number	Publication Date
GB0622592D0 GB0622592D0 (en)	2006-12-27
GB2440384A GB2440384A (en)	2008-01-30
GB2440384B true GB2440384B (en)	2010-01-13

Family

ID=37561747

Family Applications (1)

Application Number	Title	Priority Date	Filing Date
GB0622592A Expired - Fee Related GB2440384B (en)	2005-04-13	2006-04-13	Method,system and program product for measuring audio video synchronization using lip and teeth characteristics

Country Status (6)

Country	Link
EP (1)	EP1938622A2 (en)
CN (2)	CN101199207A (en)
AU (1)	AU2005330569A1 (en)
CA (1)	CA2565758A1 (en)
GB (1)	GB2440384B (en)
WO (1)	WO2007035183A2 (en)

Families Citing this family (25)

* Cited by examiner, † Cited by third party
Publication number	Priority date	Publication date	Assignee	Title
US20130297053A1 (en) *	2011-01-17	2013-11-07	Nokia Corporation	Audio scene processing apparatus
US8705812B2 (en) *	2011-06-10	2014-04-22	Amazon Technologies, Inc.	Enhanced face recognition in video
CN105100647A (en) *	2015-07-31	2015-11-25	深圳市金立通信设备有限公司	Subtitle correction method and terminal
CN105512348B (en) *	2016-01-28	2019-03-26	北京旷视科技有限公司	For handling the method and apparatus and search method and device of video and related audio
CN106067989B (en) *	2016-04-28	2022-05-17	江苏大学	A kind of portrait voice and video synchronization calibration device and method
US10997979B2 (en) *	2018-06-21	2021-05-04	Casio Computer Co., Ltd.	Voice recognition device and voice recognition method
CN108924617B (en) *	2018-07-11	2020-09-18	北京大米科技有限公司	Method of synchronizing video data and audio data, storage medium, and electronic device
CN108924646B (en) *	2018-07-18	2021-02-09	北京奇艺世纪科技有限公司	Audio and video synchronization detection method and system
CN109087651B (en) *	2018-09-05	2021-01-19	广州势必可赢网络科技有限公司	A voiceprint identification method, system and device based on video and spectrogram
CN110691204B (en) *	2019-09-09	2021-04-02	苏州臻迪智能科技有限公司	Audio and video processing method and device, electronic equipment and storage medium
CN112653916B (en) *	2019-10-10	2023-08-29	腾讯科技（深圳）有限公司	Method and equipment for synchronously optimizing audio and video
CN113497914B (en) *	2020-03-20	2024-08-30	浙江深象智能科技有限公司	Information determination method and system, electronic device, autonomous mobile device and camera
CN111988654B (en) *	2020-08-31	2022-10-18	维沃移动通信有限公司	Video data alignment method and device and electronic equipment
CN112351273B (en) *	2020-11-04	2022-03-01	新华三大数据技术有限公司	Video playing quality detection method and device
CN114613365A (en) *	2020-12-08	2022-06-10	Tcl商用信息科技(惠州)有限责任公司	A voice acquisition method, computer-readable storage medium and terminal device
CN113242361B (en) *	2021-07-13	2021-09-24	腾讯科技（深圳）有限公司	Video processing method and device and computer readable storage medium
CN114494930B (en) *	2021-09-09	2023-09-22	马上消费金融股份有限公司	Training method and device for voice and image synchronism measurement model
WO2023035969A1 (en) *	2021-09-09	2023-03-16	马上消费金融股份有限公司	Speech and image synchronization measurement method and apparatus, and model training method and apparatus
CN114466178B (en) *	2021-09-09	2025-01-24	马上消费金融股份有限公司	Method and device for measuring synchronization between speech and image
CN114466179B (en) *	2021-09-09	2024-09-06	马上消费金融股份有限公司	Method and device for measuring synchronism of voice and image
CN114089285B (en) *	2022-01-24	2022-05-31	安徽京淮健锐电子科技有限公司	Signal sorting method based on first-order Pulse Repetition Interval (PRI)
CN114550075A (en) *	2022-04-25	2022-05-27	北京华科海讯科技有限公司	Parallel signal processing method and system based on video image recognition
CN115861881A (en) *	2022-11-30	2023-03-28	广东技术师范大学	Sound lip consistency judgment method based on multi-key sound combination score fusion
CN115965724B (en) *	2022-12-26	2023-08-08	华院计算技术(上海)股份有限公司	Image generation method and device, computer readable storage medium and terminal
CN116230003B (en) *	2023-03-09	2024-04-26	北京安捷智合科技有限公司	Audio and video synchronization method and system based on artificial intelligence

Citations (6)

* Cited by examiner, † Cited by third party
Publication number	Priority date	Publication date	Assignee	Title
US4313135A (en) *	1980-07-28	1982-01-26	Cooper J Carl	Method and apparatus for preserving or restoring audio to video synchronization
US4769845A (en) *	1986-04-10	1988-09-06	Kabushiki Kaisha Carrylab	Method of recognizing speech using a lip image
US5387943A (en) *	1992-12-21	1995-02-07	Tektronix, Inc.	Semiautomatic lip sync recovery system
US5572261A (en) *	1995-06-07	1996-11-05	Cooper; J. Carl	Automatic audio to video timing measurement device and method
US5880788A (en) *	1996-03-25	1999-03-09	Interval Research Corporation	Automated synchronization of video image sequences to new soundtracks
US5920842A (en) *	1994-10-12	1999-07-06	Pixel Instruments	Signal synchronization

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number	Priority date	Publication date	Assignee	Title
US4975960A (en) *	1985-06-03	1990-12-04	Petajan Eric D	Electronic facial tracking and detection system and method and apparatus for automated speech recognition
US6829018B2 (en) *	2001-09-17	2004-12-07	Koninklijke Philips Electronics N.V.	Three-dimensional sound creation assisted by visual information

2005
- 2005-11-16 WO PCT/US2005/041623 patent/WO2007035183A2/en active Application Filing
- 2005-11-16 AU AU2005330569A patent/AU2005330569A1/en not_active Abandoned
- 2005-11-16 CN CNA2005800501339A patent/CN101199207A/en active Pending
- 2005-11-16 EP EP05851741A patent/EP1938622A2/en not_active Withdrawn
- 2005-11-16 CA CA002565758A patent/CA2565758A1/en not_active Abandoned
2006
- 2006-04-13 GB GB0622592A patent/GB2440384B/en not_active Expired - Fee Related
- 2006-04-13 CN CNA2006800211843A patent/CN101199208A/en active Pending

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number	Priority date	Publication date	Assignee	Title
US4313135A (en) *	1980-07-28	1982-01-26	Cooper J Carl	Method and apparatus for preserving or restoring audio to video synchronization
US4313135B1 (en) *	1980-07-28	1996-01-02	J Carl Cooper	Method and apparatus for preserving or restoring audio to video
US4769845A (en) *	1986-04-10	1988-09-06	Kabushiki Kaisha Carrylab	Method of recognizing speech using a lip image
US5387943A (en) *	1992-12-21	1995-02-07	Tektronix, Inc.	Semiautomatic lip sync recovery system
US5920842A (en) *	1994-10-12	1999-07-06	Pixel Instruments	Signal synchronization
US5572261A (en) *	1995-06-07	1996-11-05	Cooper; J. Carl	Automatic audio to video timing measurement device and method
US5880788A (en) *	1996-03-25	1999-03-09	Interval Research Corporation	Automated synchronization of video image sequences to new soundtracks

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
Not yet advised *

Also Published As

Publication number	Publication date
AU2005330569A1 (en)	2006-12-07
CN101199207A (en)	2008-06-11
WO2007035183A3 (en)	2007-06-21
GB0622592D0 (en)	2006-12-27
CN101199208A (en)	2008-06-11
EP1938622A2 (en)	2008-07-02
CA2565758A1 (en)	2006-10-13
GB2440384A (en)	2008-01-30
WO2007035183A2 (en)	2007-03-29
AU2005330569A8 (en)	2008-08-07

Publication	Publication Date	Title
GB2440384B (en)	2010-01-13	Method,system and program product for measuring audio video synchronization using lip and teeth characteristics
GB2429889B (en)	2009-04-08	Method, system, and program product for measuring audio video synchronization
EP2008442A4 (en)	2009-08-26	Lip synchronization system and method
EP1875369A4 (en)	2009-11-04	System and method for using product identifiers
EP2189966A4 (en)	2010-10-27	Display unit, method for processing video signal, and program for processing video signal
GB2426164B (en)	2010-05-26	Systems and methods for synchronizing time across networks
PL1938661T3 (en)	2014-10-31	System and method for audio processing
ZA200903481B (en)	2010-04-28	Method, system and computer program product for video insertion
EP1994121B8 (en)	2019-03-13	Improved method and apparatus for producing coke
TWI320665B (en)	2010-02-11	Method and system for audio and video transport
EP2084669A4 (en)	2009-11-11	System and method for cartoon compression
EP2041962A4 (en)	2011-07-27	System and method for home audio and video communication
SG124415A1 (en)	2006-08-30	Method and system to process video effects
IL178549A0 (en)	2007-02-11	System and method for enhanced video selection
EP1915757A4 (en)	2010-01-06	Method for processing audio signal
EP1728150A4 (en)	2012-01-04	System and method for failsoft headend operation
EP2036339A4 (en)	2010-04-21	Method and system for processing digital video
IL198845A0 (en)	2010-02-17	Sustained-release composition and method for producing the same
EP2227902A4 (en)	2011-01-05	Interpolation frame generation apparatus, interpolation frame generation method, and broadcast receiving apparatus
GB2437123B (en)	2011-01-26	Method and apparatus for measuring audio/video sync delay
EP1884121A4 (en)	2010-04-21	A hardware apparatus having video/audio encoding function and multiplexing function, and method thereof
TWI373260B (en)	2012-09-21	System and method for outputting video stream
EP1984826A4 (en)	2010-12-15	Method, system and software product for streaming content
GB0623512D0 (en)	2007-01-03	Method and system for analyzing image differences
GB0423577D0 (en)	2004-11-24	System and method for fingerpringing video

Legal Events

Date	Code	Title	Description
2010-12-29	PCNP	Patent ceased through non-payment of renewal fee	Effective date: 20100413