WO2007110822A1

WO2007110822A1 - Method and apparatus for synchronising recording of multiple cameras

Info

Publication number: WO2007110822A1
Application number: PCT/IB2007/051023
Authority: WO
Inventors: Johannes Weda; Mauro Barbieri
Original assignee: Koninklijke Philips Electronics N.V.
Priority date: 2006-03-28
Filing date: 2007-03-23
Publication date: 2007-10-04

Abstract

A method of capturing data content by a plurality of recorders is disclosed. Time synchronisation of recordings of multiple data recordings is a tedious and time consuming process. Currently available methods provide for time synchronisation by sending a timestamp to trigger the recorders substantially simultaneously. In an improved method of capturing data content by a plurality of recorders disclosed herein, an identifier is received in all the recorders electronically and associated to the recorded data content. The identifier is used in the synchronization of data content.

Description

METHOD AND APPARATUS FOR SYNCHRONISING RECORDING OF MULTIPLE CAMERAS

The invention relates to capturing data content, more particularly to capturing data content by a plurality of recorders.

Capturing refers to the process of receiving a signal from a target in a capturing device. The capturing device is capable of one or more of the following functions: digitizing, manipulating, compressing, storing or transmitting the captured signal. The capturing device can be referred to as recorder. The signal can be an audio or a video signal. The signal may comprise temperature or pressure data. It is also possible to capture other forms of energy, such as infra-red, laser or X-rays. As an example, a video camera can be used to capture audio/video signal. Data content refers to all types of data including audio/video content.

Capturing an event by a number of video cameras is becoming a common practice among people. A plurality of recorders are often used to capture data content of such an event for professional as well as personal recordings. A plurality of recorders can also be used to capture a big scene with multiple events that take place simultaneously such as a carnival. Capturing views of a target from different angles or capturing views of multiple targets can be performed by a plurality of recorders resulting in multiple data recordings. Generally, the multiple data recordings are later edited and combined by a user or a service provider. The user usually searches for some visual or audible cues in the recordings in order to achieve effective synchronization. Synchronizing multiple recordings typically poses a difficult problem because audible or visual cues are difficult to find.

Currently, multiple recordings are synchronized to a common reference clock at the time of recording. One technique comprises transmitting a time stamp to the recorders via a wireless connection to control the recorders. The recorders are then controlled to perform the capturing at the time determined by the time stamp in relation to the common reference clock. Thus the capturing takes place substantially simultaneously. However, the current techniques require prior synchronization of internal clocks of the recorders. Further, the time stamp should take into account the maximum transmission delay in the wireless network. Such requirements increase the complexity of capturing data content.

It is desirable to have an improved method of capturing multiple data content.

Accordingly, in an improved method of capturing data content disclosed herein, each of a plurality of recorders is arranged to capture data content of a respective or common target. An identifier is received electronically in the plurality of recorders. The identifier is associated to the data content of each one of the plurality of recorders. The identifier is used for synchronizing the data content.

It is also desirable to have a recorder capable of capturing data content according to an improved method.

Accordingly, in a recorder for capturing data content disclosed herein, the recorder comprises receiving means for receiving an identifier electronically and processing means for associating the identifier to the captured data content.

It is also desirable to have a remote controller capable of communicating with a plurality of recorders for capturing data content according to an improved method.

Accordingly, in a remote controller capable of communicating with a plurality of recorders disclosed herein, the remote controller comprises generating means for generating an identifier to be associated with the captured data content of the plurality of recorders and transmitting means for transmitting the identifier electronically to the plurality of recorders.

It is also desirable to have a computer program and a medium containing a computer program. The computer program is capable of controlling a recorder for capturing data content in an improved method.

Accordingly, a computer program and a medium containing a computer program containing instructions for implementing an improved method of capturing data content are disclosed herein. The computer program contains instructions for receiving an identifier electronically in a recorder electronically, the recorder arranged to capture a data content of a target and for associating the identifier to the data content of the recorder. These and other aspects will be described in detail hereinafter, by way of example, on the basis of the following embodiments, with reference to the accompanying drawings, wherein:

Fig. 1 schematically shows a scene with a plurality of targets being captured by a plurality of recorders;

Fig. 2 illustrates a method of capturing data content by a plurality of recorders;

Fig. 3 schematically shows a first embodiment of a recorder for capturing data content; Fig. 4 schematically shows a second embodiment of a recorder for capturing data content;

Fig. 5 schematically shows an embodiment of a remote controller for controlling recorders; and

Fig. 6 illustrates a method of synchronizing data content. Corresponding numerals used in the various figures represent corresponding elements in the figures.

Fig. 1 schematically shows a scene 100 with a plurality of targets being recorded by a plurality of recorders. A set of events or targets 160, 170, 180 are shown in the scene 100. A plurality of recorders 110, 120, 130 are arranged to capture the targets 180, 170, 160 either individually or collectively. A remote controller 150 can be optionally arranged to control the recorders 110, 120, 130.

There are various types of data capturing devices, capable of capturing signals and radiation such as light, sound, infra-red, laser or X-rays. The invention is equally applicable to one or more of such data capturing devices. A data capturing device, capable of capturing light and sound from a scene of motion in the form of a sequence of images is a video camera. A video camera can also be called a recorder. However, this assumption does not exclude other types of data capturing devices. Availability of cheaper cameras and availability of cameras in many mobile devices e.g. cell phones have accelerated their widespread personal use. Multiple cameras are often used in professional environments such as stage shows, news collection, surveillance and security applications. Data content captured by a camera can be manipulated, compressed, stored or transmitted. In general, data content provides a more objective and accurate capture of an event's spatial aspects. Data content captured by a plurality of cameras can be edited and combined to obtain a permanent record of an event. The editing can be done by a user or a service provider. There is a need to align the data content in time so that the editing can be carried out efficiently. Although the recorded contents may sometimes contain a time stamp derived from respective internal clocks of the recorders, the internal clocks are not always synchronized. Attempts to synchronize the clocks manually may still result in an error of one second which translates to a mismatch of a large number of e.g. 30 frames in case of video recordings. A large error in synchronisation may adversely affect the editing. Visual and audible cues are often used in synchronization and manual editing of video content. This is a difficult and time consuming process.

Fig. 2 illustrates a method 200 of capturing data content by a plurality of recorders. A plurality of recorders 110, 120, 130 are arranged to capture data content of a respective or common target as explained in the description of Fig. 1. In the first step 210, an identifier 201 is received electronically in the plurality of recorders 110, 120, 130. In the second step 220, the identifier 201 is associated to the recorded content of each one of the plurality of recorders.

The identifier 201 can be a marker or a symbol. The identifier 201 may represent a time source, a counter or a random number. The identifier 201 is chosen such that it is easily identifiable in the recorded content at the time of editing. The identifier 201 can be optionally received from an independent device such as a remote controller, capable of communicating to the plurality of recorders. In another embodiment, the identifier 201 is generated by any one of the plurality of recorders and transmitted electronically to other recorders, besides being locally received by the first recorder. The identifier 201 might be sent at regular or irregular intervals to multiple recorders.

Optionally, the transmission of the identifier can be performed automatically in case of recorders being continuously in use e.g. surveillance cameras. The identifier can be modulated and transmitted through wired or wireless means. Examples of wireless means are radio frequency and infra-red. In the next step 220, the identifier 201 is associated to the recorded content of all the recorders immediately after it is received in the recorders. Data content is generally recorded in frames with delimiters. In one embodiment, the identifier can be associated with the frame number and/or the time of capture. The identifier may be recorded along with the data content e.g. within an image frame of a video sequence. It is also possible to record the identifier separately e.g. in the header file or in a separate file. A pointer to the data content and the identifier can be stored in an association table. The identifier is optionally associated with the immediate next frame of data, without introducing a delay. Thus the identifier is associated with the data content substantially simultaneously in all the recorders. If the data content is transmitted to a central server, the identifier and the association information are also transmitted. The identifier can be encoded and the encoded bits can be included in the compressed data stream. The identifier can be included in the extra information space available in the compressed video and audio data, compressed according to one of the well-known data compression standards e.g. MPEG.

In another embodiment, an identifier can be received after the cameras have started recoding an event. A camera can optionally receive the identifier before capturing an event. In such cases, the identifier is associated with the time of reception noted from the camera's local clock. The association information can be stored in a table for later use. Suppose two cameras A and B are capturing an event. A first identifier ID₁ is transmitted, optionally in the initial phase of capturing. ID₁ is associated with the time of reception at time t_λ in camera A and time t₂ in camera B . The time instances t_λ and t₂ are derived from the internal clocks of the recorders A and B which are not necessarily synchronized. When a third camera C is switched on, another identifier ID₂ can be received in the third camera C at time t₃ and also in both the cameras A and B . Accordingly, the identifier ID₂ can be associated with the time of reception at the respective cameras. Synchronization of data contents captured by the cameras A, B and C can be achieved manually or automatically by time alignment using the time information t_{1 5} t₂ and t₃ . A time difference of the internal clocks of the recorders can be computed from the frame in which the identifier is included. From the time difference, time alignment of data content can be achieved. In case of the identifier being a symbol, time alignment is achieved by coinciding the frames that contain the identifier. Thus the method offers a simple and elegant scheme of synchronisation. The recorders are arranged to receive an identifier only once, optionally in the beginning of the capturing session. Thereafter, the recorders can be operated independently i.e. switched on and off any number of times. Thus the disclosed method offers a relatively greater flexibility of operation of recorders. Fig. 3 schematically shows an embodiment of a recorder for capturing data content. The recorder 300 comprises receiving means 310 and processing means 320. The receiving means 310 is arranged to receive an identifier 301 electronically. The identifier can be received either from one of the recorders or from a remote controller arranged to communicate with the recorders. The identifier 301 may be received in a modulated signal. Accordingly, the receiving means 310 may be arranged to demodulate the signal and to decode the identifier. The processing means 320 is arranged to associate the identifier to the data content. The processing means 320 is optionally arranged to capture the identifier within an image frame or include it as extra information in a compressed data stream. It is also possible to store the identifier and associated frame number or time instant (derived from the local clock of the recorder) in a separate table. Fig. 4 schematically shows a second embodiment of a recorder 400 for capturing data content. The recorder 400 is arranged to include a recorder 300 as explained in the description of Fig. 3, generating means 410 for generating an identifier 411, and transmitting means 420 for transmitting the identifier 411 electronically to other recorders. The identifier 411 can be a symbol or a number. The identifier 411 can be optionally chosen to be a timing information derived from the internal clock of the recorder 400. The transmitting means 420 can be optionally arranged to contain encoding means and modulating means. The output signal 421 of the transmitting means 420 can be a radio -frequency or infra-red signal.

Fig. 5 schematically shows an embodiment of a remote controller for controlling recorders. The remote controller 500 comprises generating means 510 and transmitting means 520. The generating means 510 generates an identifier 511 that can be a symbol or a random number or timing information derived from the internal clock of the remote controller 500. The identifier can be derived from a local clock source or from a random number generator optionally made available within the remote controller 500. The remote controller can be optionally arranged to transmit commands for all the functions e.g. zoom, focus, record etc., of a recorder. The remote controller 500 can be preferably arranged to control the functions of a plurality of recorders.

Fig. 6 illustrates a method of synchronizing data content 600. Data content 601, 602 of a single or multiple targets captured by two recorders are received in respective detecting steps 610, 620. A first identifier 611 and the second identifier 621 are detected from the data contents 601 and 602 respectively. The detection of an identifier can be performed by a number of methods, corresponding to the method by which the identifier was associated to and included in the data content. The possibilities include, but not restricted to inclusion in a separate file, an association table, a header information or within a data frame, typically in an image frame. In addition to the identifier, a frame number or a timestamp associated with the identifier can also be extracted. The frames corresponding to the same identifier are matched in step 630 and time alignment of the two data streams 601 and 602 is achieved in the synchronization step 640.

In another embodiment, the alignment can be verified by an additional step called a verification step. A user can verify the contents manually. The verification can be optionally carried out automatically by well known video content analysis techniques. The verification step is typically required if there is an ambiguity due to one or more of the following instances : (a) identifier is found in two of the three data content; (b) identifier is identical at different instances of time; (c) one of the cameras was switched off for a duration within which the identifier was received. The synchronization step and verification step may be repeated till a satisfactory time alignment is achieved. A time difference between the local clocks of the respective recorders can be computed if respective timestamps are available and the time difference may be conveyed to a subsequent data editing system. The synchronization can also be carried out by a set of computer instructions.

The order described in the embodiments of the disclosed methods is not mandatory. A person skilled in the art may change the order of steps or perform steps concurrently using threading models, multi-processor systems or multiple processes without departing from the disclosed concepts. It should be noted that the above mentioned embodiments illustrate rather than limit the invention and that those skilled in the art will be able to design many alternate embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word comprising does not exclude the presence of elements or steps other than those listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The disclosed method can be implemented by means of hardware comprising several distinct elements and by means of a suitably programmed computer. In the systems claims enumerating several means, several of these means can be embodied by one and the same item of computer readable software or hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Claims

CLAIMS:

1. A method of capturing data content, comprising: providing a plurality of recorders (110, 120, 130), each arranged to capture data content of a respective or common target (180, 170, 160); receiving (210) an identifier (201) in the plurality of recorders electronically; and associating (220) the identifier to the data content of each one of the plurality of recorders.

2. The method of claim 1, wherein receiving (210) the identifier (201) includes: generating the identifier (201) in a remote controller (150); and transmitting the identifier (201) electronically to the plurality of recorders (110, 120, 130).

3. The method of claim 1, wherein receiving (210) the identifier (201) includes: generating the identifier (201) internally in one of the plurality of recorders (110, 120, 130); and transmitting the identifier electronically to the plurality of recorders (110,

120, 130).

4. The method of claim 1, further comprising a step of synchronizing the data content of each one of the plurality of recorders on basis of the respective associated identifiers.

5. A recorder (300) for capturing data content of a target, comprising: receiving means (310) for receiving an identifier (301) electronically; and processing means (320) for associating the identifier (301) to the captured data content.

6. The recorder (400) of claim 5, wherein the recorder (400) includes: generating means (410) for generating an identifier (411); and transmitting means (420) for transmitting the identifier (411) electronically.

7. A remote controller (150, 500) comprising: generating means (510) for generating an identifier (511) to be associated with captured data content of a plurality of recorders (110, 120, 130); and transmitting means (520) for transmitting the identifier (511) electronically to the plurality of recorders (110, 120, 130).

8. The remote controller of claim 7, wherein the identifier (511) is selected from a group comprising (a) an internal time source, (b) a symbol, and (c) a random number.

9. A method (600) of synchronizing captured multiple data content comprising: detecting (610) a first identifier (611) associated with a first data content (601) captured by a first recorder; detecting (620) a second identifier (621) associated with a second data content (602) captured by a second recorder; matching (630) the first identifier (611) with the second identifier (621); and synchronizing (640) the first data content (601) with the second data content (602) on basis of the matching.

10. A computer program comprising : instructions for receiving an identifier in a recorder electronically, the recorder arranged to capture data content of a target; and instructions for associating the identifier to the data content of the recorder.

11. A medium containing a computer program comprising: instructions for receiving an identifier in a recorder electronically, the recorder arranged to capture data content of a target; and instructions for associating the identifier to the data content of the recorder.

12. A computer program comprising: instructions for detecting a first identifier associated with a first data content captured by a first recorder; instructions for detecting a second identifier associated with a second data content captured by a second recorder; instructions for matching the first identifier with the second identifier ; and instructions for synchronizing the first data content with the second data content on basis of the matched identifiers .

13. A medium containing a computer program comprising: instructions for detecting a first identifier associated with a first data content captured by a first recorder; instructions for detecting a second identifier associated with a second data content captured by a second recorder; instructions for matching the first identifier with the second identifier; and instructions for synchronizing the first data content with the second data content on basis of the matched identifiers.