JPH0568241A - Cif image transforming system for video telephone - Google Patents

Abstract

PURPOSE:To attain good conversation by picking up beforehand the image of an objective person in a wide angle, and detecting the face area of the person by picture processing, and generating this area by segmenting it by CIF or QCIF picture elements. CONSTITUTION:In order to generate a CIF or a QCIF picture, the image of the objective person is picked up beforehand in the wide angle, and the face area of the person is detected by the picture processing by a moving area detecting means 103, and center coordinates for segmentation are calculated by a segmenting address calculating means 104, and a rectangular range including the face area of the person is segmented by the CIF or the QCIF picture elements. Then, if a face moves, a segmented range is changed within the image- picked up range in accordance with the movement of the face area, and the face is displayed as following the face area. Thus, the face area never goes out of the picture frame of a camera on account of the movement of the face part of the objective person, and the face are can be transmitted always to an opposite party side, and as the result, the good conversation can be realized through a video telephone.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an NTSC-to-CIF image conversion processing system for a videophone device conforming to the CCITT standard.

[0002]

2. Description of the Related Art Here, the CIF image standard will be described before the description of the conventional conversion processing system from the NTSC to the CIF image in the video telephone apparatus conforming to the CCITT standard. In CCITT, p × 64 kbit /
s Recommended video coding method for audiovisual services (CCITT Rec. H261 "Video Codec for Audio
visual Service at px64kbit / s 1990). Here, a common intermediate format image (CIF; Comon Intermediate Forma) is used in order to absorb the difference in the television signal system of each country.
t) is defined as follows.

In the first format (CIF), the luminance samples are arranged in an orthogonal grid with 352 pixels per line and 288 lines per frame. The samples of the two color difference components are 176 pixels per line and 1
144 lines per frame are arranged in an orthogonal grid. The pixel block boundary of the color difference signal is placed so as to coincide with the pixel block boundary of the luminance signal, and the image area surrounded by these number of pixels has an aspect ratio of 4: 3, and an effective screen of a standard television signal. Matches

The second format (QCIF) is one in which the number of pixels and the number of lines of the above-mentioned CIF are each halved. (The above is an excerpt from the TTC standard JT-H221.) In the videophone device conforming to the above standards, mutual communication can be performed regardless of the difference in standard television systems. The NTSC / CIF conversion processing method in the conventional videophone device based on this standard will be described below.

In a conventional videophone apparatus, the NTSC / CIF conversion processing method for creating a CIF image from the NTSC method, which is a Japanese standard television method, uses a reduction filter for reducing the number of pixels in the vertical and horizontal directions and the number of lines. It was the method used.

In FIG. 7, an NTSC signal is sampled at 13.5 MHz, digital color separation processing is performed, and horizontal direction 704 is performed.
It is an example showing a conventional method of creating an image of 480 lines in the vertical direction of pixels and creating a CIF image from the image. The image of 704 pixels in the horizontal direction and 480 lines in the vertical direction conforms to the standard recommended in CCIR in 1982 as "encoding parameter of studio digital television" (Rec. 601). Here, 1 is an imaging unit, 2 is a low-pass filter (LPF) unit for band limitation, 3
Is an A / D conversion unit, 4 is a digital Y / C separation unit, 5 is a horizontal reduction filter unit, 6 is a non-interlaced conversion unit,
Reference numeral 7 is a vertical reduction filter unit, 8 is an encoding processing unit, and 9 is a network interface processing unit.

The NTSC signal picked up by the image pickup section 1 is band-limited by the LPF section 2, sampled at a sampling frequency of 13.5 MHz by the A / D conversion section 3, and quantized by a quantization bit number of 9 bits. As a result, the number of samples in the horizontal direction is 858 samples and the number of effective pixels is 704 pixels. Digitized NTSC
The signal is separated by the digital Y / C separation unit 4 into component signals of a luminance signal Y and color difference signals C ₁ and C ₂ .
The color-separated component signals are input to the horizontal reduction filter unit. In the concrete processing here, since the number of effective pixels in the horizontal direction is 704 pixels, sub-sampling is performed to ½ and reduction is performed. The non-interlaced conversion unit 6 receives the component signal that is reduced to 1/2 in the horizontal direction.
And the signals of the odd field and the even field are once stored in the field memory. When reading, the interlaced signal is converted into a non-interlaced signal by alternately reading the lines of the odd and even fields. The non-interlaced component signal is input to the vertical reduction filter unit 7 and vertically reduced by a factor of 3/5. As a concrete reduction method, there is a method of simply thinning out 5 lines to 3 lines and a method of multiplying each signal of 5 lines by a weighting factor to convert into 3 lines, but here, the reduction is simply performed by the former method. .

As a result of the above, 1 in each of the horizontal and vertical directions
The component signals reduced to / 2, 3/5 are 352 pixels and 288 lines, respectively, and a CIF image is created, input to the encoding processing unit 8 and subjected to encoding processing,
It is sent to the transmission line via the network interface processing unit 9.

Although the method of creating a CIF image has been described above, a QCIF image can be similarly created by setting the horizontal subsampling to 1/4 and the vertical thinning to 3/10 by the above method.

[0010]

In the above-mentioned conventional method,
Since the number of pixels of CIF or QCIF is small, in order to clearly recognize the facial expression of the other party as a videophone, it must be zoomed up to some degree and imaged and transmitted. If the face part of the subject person moves even a little, it will be out of the image frame of the camera, and the normal facial image will not be transmitted.

The present invention solves the problem that the face area deviates from the image frame of the camera due to movement of the face portion of the subject person, and the human face area is always in the image frame of the camera, and good conversation is ensured. The purpose is to enable.

[0012]

FIG. 1 is a block diagram showing the principle of the present invention. Reference numeral 101 in the figure is A / D conversion processing means, 1
Reference numeral 02 is a frame memory for storing the A / D converted television signal, 103 is a moving area detecting means for detecting a moving area in the television signal,
Reference numeral 104 denotes a cutout address calculating means, which is a means for calculating the cutout address.

[0013]

In order to create a CIF or QCIF image, a person image of a subject is picked up in advance at a wide angle, a human face area is detected by image processing, and the area is cut out with CIF or QCIF pixels. In this case, the cutout range is changed within the imaged range according to the movement of the face area, so that the face area can be always transmitted to the other party. Further, in order to eliminate the distortion of the cut CIF image, a sampling frequency based on the aspect ratio in the horizontal direction and the vertical direction is obtained when the CIF image is cut out, and sampling is performed at the sampling frequency.

In the prior art, CIF or QC
In order to create an IF image, reduction filters in horizontal and vertical directions are used to perform reduction conversion, whereas in the present invention, a face area is detected by image processing from an image picked up in advance at a wide angle, and the detected face area is detected. Rectangular area including
Alternatively, it is created by cutting out with QCIF pixels.
Further, in the conventional technique, 13.
5 MHz or color subcarrier frequency f _SC (3.58 MH
While the A / D conversion is performed at a frequency three or four times that of z), the present invention does not depend on the above frequency because analog color separation is performed, and is suitable for eliminating distortion of a CIF image. Frequency is decided.

[0015]

(Embodiment 1) FIG. 2 is a diagram for explaining a method of creating a CIF image from the NTSC method according to the first embodiment of the present invention. Reference numerals 1, 2, 3, 8 in FIG. Reference numeral 9 corresponds to FIG. 7, 10 is an analog Y / C separation section, 11 is a moving area detection section, 12 is a CIF / QCIF image cutout address calculation section, and 13 is a frame memory section.

An NTSC television signal picked up by the image pickup unit 1 is input to an analog Y / C separation unit 10 and separated into a luminance signal Y and component signals of color difference signals C ₁ and C ₂ . The separated Y, C ₁ and C ₂ signals pass through the LPF unit 2 before the A / D conversion and are band-limited.
The band-limited Y, C ₁ and C ₂ signals are A / D converted by the A / D converter 3. Here, the sampling frequency of the A / D conversion is determined so that the ratio of the physical size of the image in the horizontal direction to the vertical direction when cut out by the CIF or QCIF pixel is 4: 3. That is, it is required that the effective image obtained by removing the horizontal and vertical scanning line blanking periods from the image sampled by the A / D conversion and the image created by the CIF or QCIF pixel have similar shapes. .. Specifically, the horizontal and vertical pixels of the CIF image and the number of lines are 352 pixels each,
When 288 lines, the effective pixels in the horizontal direction and the vertical direction of the video signal to be sampled, and the number of lines are p pixels and 480 lines, respectively, the following relationships are established.

P: 480 = 352: 288 ∴p = 587 Therefore, the number of effective pixels in the horizontal direction is 587 pixels, and when 83% of the horizontal scanning period is the effective period, the number of pixels in the horizontal scanning period is 707 pixels. Become. From this, the sampling frequency for A / D conversion can be determined to be 11.1 MHz. The color difference signals C ₁ and C ₂ are A / D converted at half the luminance signal Y, that is, at 5.56 MHz.

A quadrature type of a CIF or QCIF image including a face area in the moving area detecting section 11 using the Y signal A / D converted with a sampling frequency of 11.1 MHz or 5.56 MHz and a quantization bit number of 8 bits. The center coordinates of the area are determined. At the same time, the Y, C ₁ and C ₂ signals are stored in the frame memory unit 13 every two fields. In addition,
The non-interlaced conversion processing is performed by reading the odd and even fields stored in the frame memory at once. The determined center coordinates of the CIF or QCIF image are stored in the frame memory unit by the CIF / QCIF image cutout address calculation unit 12, Y, C ₁ , C.
_An address cut out from each frame image of the _two signals is calculated, and a CIF or QCIF image is read based on the address. CIF or QCIF read
The image is transmitted to the other party via the encoding processing unit 8 and the network interface processing unit 9.

FIG. 3 is a diagram for explaining in detail the contents of the moving area detecting section 11 of FIG. 2, and shows an example by the background subtraction method using the background reference image. Here, 14 is a background difference calculation unit,
Reference numeral 15 is a background reference image memory unit, 16 is a block processing unit, 17 is a binarization processing unit, 18 is a parietal coordinate calculation unit, 1
Reference numeral 9 denotes a CIF / QCIF image clipping center coordinate calculation unit.

A background reference image that does not include a human figure or a moving object is captured in advance and stored in the background reference image memory unit 15. Next, a normal videophone image including a person is captured, and the captured person image is input to the background difference calculation unit 14. The background difference calculation unit 14 calculates a difference signal from the background reference image for each frame of the input human image. The obtained background difference signal is input to the blocking processing unit 16 in order to reduce the amount of calculation, and is blocked by 16 × 16 pixels or the like, for example. The block image is input to the binarization processing unit 17, binarized with an appropriate threshold value, and the difference between the background reference image and the person image, that is, the region where the person exists is extracted.

FIG. 4 shows a block and binarized difference.
The example of a minute image is shown. Where the point p (x _C, Y_C) Is the person's head
Top coordinates, point q (X_C, Y_C) Is the cutting center coordinate
It In this way, only the person area is extracted as the difference.

The difference image binarized by the binarization processing unit 17 is input to the parietal coordinate calculation unit 18 to calculate parietal coordinates. Although various methods of calculating the parietal region coordinates are conceivable, a method for determining the parietal region coordinates from the person's parietal region, which is the simplest method, will be described here. For example, in the method of detecting the person's crown from the block difference image shown in FIG. 4, the block difference image of FIG. 4 is scanned horizontally from the upper left, and the position at which the difference is detected first is the person's crown coordinate (x _C , y _C ). Wherein the center point of the block, in the case of 2 or more blocks to the average coordinate position of the block and the head top portion horizontal coordinate point x _C If the person parietal horizontal block is one block. The coordinates obtained by the parietal coordinate calculation unit 18 are CI
It is input to the F / QCIF image cutout range center coordinate calculation unit, and the cutout center point for cutting out the CIF or QCIF image is calculated.

Specifically, a suitable block upper part, for example, two blocks upper part from the obtained vertical coordinate y _{C of} the parietal region is set as an upper frame for cutting, and the vertical cutting center coordinate Y _C is obtained from the position of this frame. .. Further, the horizontal coordinate of the top of the head is directly used as the horizontal cut-out center coordinate X _C. Thereby, the center coordinates (X _C , Y _C ) for the CIF or QCIF image cutout range are obtained. The obtained center coordinates are input to the CIF / QCIF image cutout address calculation unit 12 in FIG.

FIG. 5 shows 587 pixels in the horizontal direction and 4 pixels in the vertical direction.
An example is shown in which 352 pixels and 288 lines are cut out from 80 lines, respectively. (Embodiment 2) FIG. 6 shows PAL which is a second embodiment of the present invention.
It is a figure explaining the method of creating a CIF image from the SECAM method. Reference numerals 1, 2, 3, 4, 8 in the figure
Reference numerals 9, 11, 12, and 13 correspond to FIGS. 2 and 7. PAL or SECAM imaged by the imaging unit 1
The system television signal is band-limited by the LPF unit 2 and input to the A / D conversion unit 3. Here, the sampling frequency is determined as in the NTSC system. In case of PAL or SECAM system, the number of effective lines in the vertical direction is 5
Since there are 76 pixels, the following relationship holds when the number of effective pixels in the horizontal direction is p pixels.

P: 576 = 352: 288 ∴p = 704 The number of pixels in the horizontal direction coincides with the number of effective pixels when sampling is performed at 13.5 MHz. Therefore, PAL, S
In case of ECAM system, sampling frequency is 13.5
MHz, and the following processing is the same as in the case of the NTSC system described above.

[0026]

As described above, according to the present invention, C
In order to create an IF or QCIF image, a subject person image is captured in advance at a wide angle, a human face area is detected by image processing, and the center coordinates for clipping are calculated, and a rectangular range including the human face area is divided into CIF or QCIF pixels. When the face is moved, the cutout range is changed according to the movement of the face area within the captured range and the face is displayed following the face area. There is an advantage that the face area can always be transmitted to the other party without being detached, and as a result, a good conversation can be realized via the videophone.

[Brief description of drawings]

FIG. 1 shows a principle configuration diagram of the present invention.

FIG. 2 is a diagram illustrating a method of creating a CIF image from the NTSC method that is the first embodiment of the present invention.

FIG. 3 is a diagram illustrating in detail the content of a moving area detection unit in FIG.

FIG. 4 is a diagram showing an example of a binarized difference image which is made into blocks.

FIG. 5 shows an example of cutting out.

FIG. 6 is a second embodiment of the present invention, PAL or SE.
It is a figure explaining the system which produces a CIF image from a CAM system.

FIG. 7 shows an example of a conventional method in which an NTSC signal is sampled at 13.5 MHz, digital color separation processing is performed, an image of 704 pixels in the horizontal direction and 480 lines in the vertical direction is created, and a CIF image is created from the image. It is a figure.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Imaging unit 2 Low pass filter (LPF) unit for band limitation 3 A / D conversion unit 4 Digital Y / C separation unit 5 Horizontal reduction filter unit 6 Non-interlaced conversion unit 7 Vertical reduction filter unit 8 Encoding processing unit 9 Network interface processing unit 10 Analog Y / C separation unit 11 Moving area detection unit 12 CIF / QCIF image cutout address calculation unit 13 Frame memory unit 14 Background difference calculation unit 15 Background reference image memory unit 16 Blocking processing unit 17 Binarization processing Part 18 Parietal coordinate calculation unit 19 CIF / QCIF image clipping center coordinate calculation unit

Claims (3)

[Claims] 1. A CIF image conversion system for a videophone for obtaining a CIF image from a television signal, processing means for A / D converting the television signal, and storing the A / D converted television signal in a frame memory. Means for detecting a moving area in the television signal, and means for calculating an address for cutting out a rectangular area including the detected moving area from the signal stored in the frame memory. CIF image conversion system for videophones. 2. When the television signal is the NTSC system, the sampling frequency for A / D conversion is 11.1 MH.
The CIF image conversion system for videophone according to claim 1, wherein z is z. 3. The television signal is PAL or SEC.
The videophone CI according to claim 1, wherein in the case of the AM system, the sampling frequency at the time of A / D conversion is 13.5 MHz.
F image conversion method.