GB1605010A - Picture processing apparatus - Google Patents

Description

(54) PICTURE PROCESSING APPARATUS (71) We, MULLARD LIMITED of Abacus House, 33 Gutter Lane, London E.C.2., a British Company do hereby declare the invention, for which we pray that a patent may be granted to us, and the method by which it is to be performed, to be particularly described in and by the following statement: This invention relates to picture processing apparatus capable of extracting significant features from a picture. More particularly the invention relates to apparatus in which the features are extracted by scanning the picture with an array of detectors, individual detectors of which remain in a fixed distribution with respect to one another during the scanning process. Each detector has a field of view adapted to collect radiation from an area of the picture of a size sufficient to allow the required detail of the features to be resolved.

This area will be referred to hereinafter as the picture element. The output of each detector is an analogue quantity, being a voltage or a current, related to the total amount of radiation being emitted by the picture element instantaneously in the field of view of the detector. The detector output corresponds, therefore, to the brightness of the picture element. The picture is essentially a planar distribution of picture elements. The planar distribution may be provided in recorded form, for example as a drawing or a photograph or from a scene reconstructed from a previously recorded and stored set of picture elements.

Alternatively, the planar distribution may be derived, in real time, from a three dimensional scene using a suitable camera which images the scene into planar form. The camera may use radiation emitted from the scene in any convenient region of the electromagnetic spectrum. Reflected visible and infra-red radiation and particularly thermal infra-red radiation may be employed but other portions of the spectrum are not excluded. The invention is directed to the processing of planar pictures however derived.

Patent Specification No. 1605009 (copending application 53350/73; PHB32390), which Specification will be referred to herein as the parent Patent Specification describes and claims such picture processing apparatus for application to a picture comprising a regular two-dimensional array of picture elements each of which is the centre picture element of a plurality of radiating straight lines of contiguous picture elements aligned in radial directions and in which each picture element has a brightness value, said apparatus comprising means for determining the average brightness of one or more picture elements adjacent to said centre picture element in such a radial direction, said average being taken over a number of picture elements small compared to the number of picture elements in the maximum dimension of the picture and excluding the said centre picture element, and means for providing an output signal when the said average brightness differs from the said centre picture element brightness by at least a predetermined threshold amount of the same sign on at least half of the same radial directions.

Any given picture element may be taken as a centre picture element and is alternatively referred to as such hereinafter.

The invention provides a modification of the picture processing apparatus as claimed in claim 1 of the parent Patent Specification for application to a picture comprising regularly spaced rows and regularly spaced transverse columns of picture elements, said modified apparatus comprising means for comparing the brightness of a given picture element with (a) the brightness of a first adjacent picture element in the column containing said given picture element and with (b) the brightness of a second adjacent picture element in the row containing said given picture element, and means for providing an output signal if said first and second picture elements both differ in brightness from said given picture element by at least a predetermined threshold amount of the same sign.

The invention also provides a picture processing apparatus wherein there is provided a line array of detectors parallel to said rows, each detector having a field of view of one picture element and being adapted to produce an output corresponding to the brightness of the picture element in its field of view, and means for scanning said array along said columns of picture elements to determine the brightness values of said given first and second picture elements.

The rows and columns may be interchanged, the length of the line array lying parallel to said columns. The threshold is an amount rather than level. If a scanning array encounters a picture element which differs in brightness from the orthogonally adjacent set of picture elements defined above, the picture element may be a feature in the picture comprising an isolated point set in a darker or lighter background. The picture element may also be at the corner of a rectangular object lying ahead of the scanning array and having one side roughly parallel to the array. However, a bright or dark line or edge which is either parallel or transverse to the array will generally be discriminated against and the binary output not generated. Such a line or edge will generally have picture elements in lines either parallel to or transverse to the array and of more nearly equal brightness and a pair of such picture elements will be less likely to differ by the threshold amount. Lines and edges lying at intermediate angles to the line of the array will be less well discriminated against. The apparatus preferentially extracts point and corner features from a picture as against line and edge features.

The output of each detector may be stored for the duration of one picture element. The previous and present outputs of each detector in the array can then be made available to the processing apparatus. The store used may be a sample-and-hold circuit or may be a delay line. Alternatively, the picture elements may be scanned for a second time by a second array of detectors following behind the first array. Each detector in the second array repeats the scanning of a corresponding detector in the first array. The present and previous outputs are then both made available directly to the processing apparatus without the need for storage.

The spacing between detectors in the array may be chosen to suit the size of features which it is desired to detect. Likewise, the effective spacing between present and previous outputs of a detector may also be chosen at will by choosing the 'hold' period of the sample-and-hold circuit or the delay time of the delay line if storage is used or by choosing the spacing between first and second arrays. Generally, the spacing of detectors along the arrays and the effective spacing between present and previous outputs would be made equal, since, generally most pictures have equal horizontal and vertical resolutions.

The processing apparatus may compare the outputs of adjacent elements in the array to determine whether their difference exceeds the threshold. But the outputs of alternate detectors may be compared so that larger features are detected without loss of resolution for smaller features. In this case the effective separation of the previous outputs would be increased to maintain equality of horizontal and vertical resolutions.

The processing apparatus may provide a separate final output for each detector in one row of the array considered as a centre picture element detector. Alternatively a single final output may be provided when a point or corner has been detected by any one or more of the detectors in one row of the array. The final outputs may be used to make a reconstructed version of the scanned scene showing points only and corners. This reconstructed scene may be viewed separately or superimposed on and in registration with the original scene by well known optical means.

With only two detectors in the array, a feature is detected when either detector output exceeds its own previous output and exceeds the output of the other detector each by the threshold. With three or more detectors, any detector which is not at the ends of the array has two neighbours in the array. The feature detection logic is then that a detector output should exceed its own previous output and exceed the output of either or both of its neighbours, all by the same threshold, for a feature to be detected.

Embodiments of the invention will now be described by way of example with reference to the drawings accompanying the Provisional Specification in which: Figure I shows diagrammatically the signals produced by a line array of three detectors and their delayed outputs.

Figure 2 shows a high gain differential amplifier and resistor network used in the processor.

Figure 3 shows the logic functions carried out by the picture processor in block diagram form.

Figure 4 shows a practical embodiment of the logic functions of Figure 3.

Referring to Figure 1 a line array 1 of three detectors AR, BR and CR is shown, each detector being shown schematically as a rectangle. AR, BR and CR are also: used; subsequently in the description to indicate the signal outputs of these detectors tlie:lettetrR indicating the real-time or immediate output of the detector. The line array 1 i sli scanning vertically upwards as indicated by the arrow 2 across a rectangular array of pit elements P.E. constituting a picture. These picture elements are shown schematicallywasR array of rectangles each equal in area to a detector area. In practice, the rectan lar. arn3y of picture elements may be a continuous distribution of radiation intensity, eachlrectangle indicating the area of the picture contributing to the single output of a detector of rectangular area at an instant of sampling. In this embodiment, the outputs of each detector are stored for a time corresponding to the time taken for a detector to scan from one picture element to an adjacent one. The stored outputs are therefore shown schematically as AS, BS and CS, each corresponding to AR, BR and CR respectively. The outputs AS, BS and CS are therefore the outputs AR, BR and CR respectively, delayed by one sampling period and corresponding to the previous row of picture elements scanned. This has the effect of making available to subsequent circuitry the outputs of a block of six picture elements simultaneously.

Points or corners in the picture are detected by applying the following criteria to the six outputs, which are analogue voltage or current signals. A point or corner is said to exist at detector AR if the signal condition is: AR - AS > T ) ) ( 1 ) and AR - BR > T ) where T is a positive threshold level.

Thus, a point or corner is only said to exist at A if the signal AR exceeds its own delayed output AS by a threshold T and also exceeds the output BR of the adjacent detector in the line array by the same threshold T. Likewise for a point or corner at detector CR: CR - CS > T ) ) -----------------------(2) and CR - BR > T ) For a point or corner at detector BR, two possible adjacent outputs are available, AR or CR. The criteria for detector BR are therefore: BR - BS > T and either BR - AR > T ) ------------------------------(3) or BR-CR > T Thus, in applying these criteria, seven quantities must first be evaluated: AR - AS - T ------------------------------- (4) AR - BR - T -------------------------------- (5) BR - BS - T ------------------------------------- (6) BR-AR-T (7) BR-CR-T (8) CR-CS -T ------------------------------------- (9) CR-BR-T (10) and an indication provided for each if it is greater than zero. Thus a binary signal is generated for each of the seven quantities, having the value '1' if the quantity is greater than zero or the value '0' if the quantity is zero or negative.

Referring to Figure 2 a network of resistors and a high gain differential amplifier is shown, one such arrangement being provided for each quantity to generate the binary signal. As an example, the quantity BR - AR - T is shown evaluated and the corresponding binary output generated. The high gain differential amplifier 3 has a single output terminal 4 which can swing between a limited positive output and a limited negative output.

Amplifier 3 has two input terminals 5 and 6, being the inputs of negative and positive sense respectively. A positive input signal applied to terminal 5 would appear as a greatly amplified negative output at terminal 4 whereas a positive input signal to terminal 6 would appear as an equally amplified positive output at terminal 4. Thus, the effect is to amplify greatly the difference between the signals applied to the inputs 5 and 6. Two equal resistors R are provided at each input terminal. At terminal 6, one resistor is fed with the signal BR, the other being returned to earth or zero signal. Thus, half the BR signal is applied to terminal 6. At terminal 5 one resistor is fed with the AR signal, the other being fed with the threshold level T. Thus half the sum of AR and T is fed to terminal 5. The loss of half the signal amplitudes at terminals 5 and 6 is of no consequence as the gain of the amplifier 3 is so high. The output 4 will be driven to the limiting positive output if BR exceeds AR + T only very slightly. Conversely output 4 will be driven to the limiting negative output if AR + T exceeds BR only ve slightly. The following logic circuits interpret these limiting outputs as a '1' and an 'U' respectively.

Referring to Figure 3, the logic function of the processor is shown schematically. In this processor a single output is provided if any one or more of the criteria (1), (2) and (3) above are satisfied. Thus the final logic gate in the processor is a three-input 'OR' gate 8 which in this embodiment inverts the signal, providing a low voltage or '0' output if any one or more of its three inputs 9, 10 and 11 is a high voltage or '1'. The 'AND' function required for each of the criteria (1), (2) and (3) above is provided by inverting 'AND' gates 12, 14 and 13 respectively. Each of these 'AND' gates has two inputs and provides a high output or '1' only if both inputs are a low voltage or '0'. Consequently for quantities (4), (5), (6), (9) and (10) above, the outputs of the corresponding high gain amplifiers must be at a low voltage or '0' when the corresponding quantities are greater than zero. This inversion of sign when compared to the description of Figure 2 is achieved by reversing the input connections of the high gain amplifiers concerned. For example, in amplifier 15, processing the quantity (4) = AR - AS - T, detector output AR and earth are applied to the resistor pair connected to the negative amplifier input, while the detector output AS and the threshold level T are applied to the resistor pair connected to the positive amplifier input. With this set of amplifier input connections, the output 16 of amplifier 15 will be negative when AR AS - T is positive, which has the effect of a low voltage or '0' on the input of 'AND' gate 12.

For the remaining two quantities (7) and (8) required in criteria 3, an 'OR' function precedes the 'AND' function. The 'OR' gate 17 is also an inverting gate, providing a low voltage or '0' output if either or both of its two inputs, 18 and 19 are a high voltage or '1'.

Consequently the inversion of sign provided in the other five high gain differential amplifiers is not needed and the input connections of Figure 2 are correct for providing the input 18. For input 19 only the term AR is changed to CR in Figure 2. The sign of the logic used at various points in Figure 3 is indicated by bracketed "1" 's and "0" 's at the outputs of amplifiers and gates. Round brackets give the binary value, '1' or '0', when no point or corner has been detected, whereas square brackets give the binary value when a point or corner has been detected.

The output 21 of 'OR' gate 8 is fed to a non-inverting monostable multivibrator 20. When output 21 is a '0' that is when a point or a corner is detected by the array, the multivibrator 20 responds by generating a '0' on output 22 which lasts for the time corresponding to the scanning of one picture element. A '0' on output 22 turns transistor 23 and light-emitting diode 24 on. Thus, a pulse of light is emitted by diode 24 for one picture element time when a corner or point is detected by the vertical scanning of the array of detectors. An image of diode 24 is scanned in synchronism with the array, by means not shown, reconstructing a single vertical line of picture information which shows only corners and points. By optical means not shown, this single line may be viewed superimposed upon and in registration with the original unprocessed picture. A distinctive mark on corners and points satisfying the threshold criteria may thus be provided for an observer.

Referring to Figure 4, a practical circuit embodiment of the logic of Figure 3 is shown. In particular, practical means are shown for generating the delayed outputs AS, BS and CS from the real-time outputs AR, BR and CR respectively. The real-time outputs are each fed to sample-and-hold circuits 25, 26 and 27 respectively. Sample-and-hold circuits are known in the art and have the property of sampling a continuously applied input signal at an instant determined b a control pulse and of storing the value of the input present at the sampling instant and supplying it to an output terminal. The stored value is held and supplied to the output until another control pulse is applied. Then the new value of the sample input supplants the old stored value. Sample-and-hold (SH) circuit 25 is fed with control pulses at terminal (a) which are 10 microseconds long at intervals of 300 microseconds, corresponding to a picture element rate of 3.3 kHz. Thus, at the end of each 300 microsecond hold period, the output 28 of SH circuit 25 is the value AS corrwonding to the real time detector output AR then available. AR and output 28 are fed to:themeiXr network inputs of high gain differential amplifier 29 where the quantity AS + T - w is devaluated as described with reference to Figure 3. Similarly, BR and Cur are fed le SH circuits 26 and 27 respectively. The appropriate selection of the inputs AR, BR and a and the SR circuit outputs 30 and 31 are fed to each of the remaining six high ain diffal amplifiers 32 to 37 inclusive to evaluate the remaining six relations (5) to (10i) irmlusse. The operation of the logic gates 8, 12, 13, 14 and 17 is as described with reference to Figure 3.

The relative timing of the sample-and-hold circuit control pulses and the instant of sampling the output of 'OR' gate 8 are controlled by a 3.3kHz clock pulse source 38 and a monostable multivibrator 41. The output 42 of source 38 consists of a square wave generated by a multivibrator consisting of two inverting logic gates 39 and 40 connected in series with a feed back network composed of capacitor C1 and resistors R1 and R2. The two reversals of phase in the gates 39 and 40 provide overall positive feedback giving an oscillation controlled in frequency at 3.3 kHz by the values of C1, R1 and R2. The output 42 is a square wave of about unity mark-to-space ratio, the positive going portion of which has a duration of 150 microseconds. The sample-and-hold control pulse must be much less than this duration so that the storage time of the sample-and-hold circuits is as near the full 300 microseconds as possible. The monostable multivibrator 41 provides such a shortened pulse in response to the positive-going portion of output 42. The stable state of the monostable 41 is with output 43 at logical '1' or a high positive voltage and output 44 at logical '0' or a low voltage. The positive going onset of output 42 triggers monostable 41 to produce one pulse of width 10 microseconds, controlled in duration by C2 and R3. Output 43 provides a single negative going pulse of this duration which turns transistors T1, T2 and T3 on, supplying simultaneous output current pulses a, b and c respectively to the sample-and-hold circuits 25, 26 and 27 respectively. Output 44 simultaneously supplies a positive going pulse of the same duration to monostable 20, briefly described before with reference to Figure 3.

Provided the output of 'OR' gate is a logical '0', i.e. provided a point or corner has been detected, at the time when pulse 44 arrives, monostable 20 provides a single 300 microsecond duration negative going pulse on output 45, controlled in duration by C3 and R4, to turn light-emitting diode 24 on for one picture element time. The output 44 actually reaches monostable 20 before the sample-and-hold pulses a, b and c reach the sample-and-hold circuits owing to the delay in pulse propagation through transistors T1, T2 and T3. Hence the old stored values in the sample-and-hold circuits are used in the point and corner detection circuits before they are supplanted by the new stored values. These new values are, strictly, the values of the real-time inputs available at the end of the 10 microsecond pulse and consequently they are actually stored for 290 microseconds.

Referring to Figure 4, the value given to the frequency of the clock pulse source 38 of 3.3kHz and the corresponding value of 300 microseconds given to the duration of the pulse generated by multivibrator 20 refer to a particular example of a Figure 4 circuit in which the picture element clock rate is 3.3kHz. This clock rate is determined by the rate at which picture elements are scanned which is settled by other parameters dependent on the purpose to which the picture processor is applied. These values may be adjusted to suit the application. Likewise the value of 10 microseconds given to the sample-and-hold control pulse generated by multivibrator 41 would be adjusted to suit the application. In the example described having a picture element rate of 3.3kHz, the values of the labelled resistors and capacitors are as follows. R1 = 330k, R2 = 680k, R3 = 33k, R4 = 100k, C1 = 390pf, C2 = 390pf and C3 = 3.9nf. The transistors T1, T2, T3 and T4 are all Mullard BCY71. The high gain differential amplifiers 29, 32, 33, 34, 35, 36 and 37 are RCA Type No. CA3060. The gates 12, 13, 14 and 17 are Motorola Type No. MC14001. The gates 8, 39 and 40 are Type No. MC14025. The multivibrators 20 and 41 are Type No. MC14528. The supply voltage +V is 5 volts.

WHAT WE CLAIM IS: 1. A modification of the picture processing apparatus as claimed in claim 1 of the parent Patent Specification for application to a picture comprising regularly spaced rows and regularly spaced transverse columns of picture elements, said modified apparatus comprising means for comparing the brightness of a given picture element with (a) the brightness of a first adjacent picture element in the column containing said given picture element and with (b) the brightness of a second adjacent picture element in the row containing said given picture element, and means for providing an output signal if said first and second picture elements both differ in brightness from said given picture element by at least a predetermined threshold amount of the same sign.

2. Picture processing apparatus as claimed in Claim 1 wherein there is provided a line array of detectors parallel to said rows, each detector having a field of view of one picture

**WARNING** end of DESC field may overlap start of CLMS **.

Claims (8)

**WARNING** start of CLMS field may overlap end of DESC **. microseconds, corresponding to a picture element rate of 3.3 kHz. Thus, at the end of each 300 microsecond hold period, the output 28 of SH circuit 25 is the value AS corrwonding to the real time detector output AR then available. AR and output 28 are fed to:themeiXr network inputs of high gain differential amplifier 29 where the quantity AS + T - w is devaluated as described with reference to Figure 3. Similarly, BR and Cur are fed le SH circuits 26 and 27 respectively. The appropriate selection of the inputs AR, BR and a and the SR circuit outputs 30 and 31 are fed to each of the remaining six high ain diffal amplifiers 32 to 37 inclusive to evaluate the remaining six relations (5) to (10i) irmlusse. The operation of the logic gates 8, 12, 13, 14 and 17 is as described with reference to Figure 3. The relative timing of the sample-and-hold circuit control pulses and the instant of sampling the output of 'OR' gate 8 are controlled by a 3.3kHz clock pulse source 38 and a monostable multivibrator 41. The output 42 of source 38 consists of a square wave generated by a multivibrator consisting of two inverting logic gates 39 and 40 connected in series with a feed back network composed of capacitor C1 and resistors R1 and R2. The two reversals of phase in the gates 39 and 40 provide overall positive feedback giving an oscillation controlled in frequency at 3.3 kHz by the values of C1, R1 and R2. The output 42 is a square wave of about unity mark-to-space ratio, the positive going portion of which has a duration of 150 microseconds. The sample-and-hold control pulse must be much less than this duration so that the storage time of the sample-and-hold circuits is as near the full 300 microseconds as possible. The monostable multivibrator 41 provides such a shortened pulse in response to the positive-going portion of output 42. The stable state of the monostable 41 is with output 43 at logical '1' or a high positive voltage and output 44 at logical '0' or a low voltage. The positive going onset of output 42 triggers monostable 41 to produce one pulse of width 10 microseconds, controlled in duration by C2 and R3. Output 43 provides a single negative going pulse of this duration which turns transistors T1, T2 and T3 on, supplying simultaneous output current pulses a, b and c respectively to the sample-and-hold circuits 25, 26 and 27 respectively. Output 44 simultaneously supplies a positive going pulse of the same duration to monostable 20, briefly described before with reference to Figure 3. Provided the output of 'OR' gate is a logical '0', i.e. provided a point or corner has been detected, at the time when pulse 44 arrives, monostable 20 provides a single 300 microsecond duration negative going pulse on output 45, controlled in duration by C3 and R4, to turn light-emitting diode 24 on for one picture element time. The output 44 actually reaches monostable 20 before the sample-and-hold pulses a, b and c reach the sample-and-hold circuits owing to the delay in pulse propagation through transistors T1, T2 and T3. Hence the old stored values in the sample-and-hold circuits are used in the point and corner detection circuits before they are supplanted by the new stored values. These new values are, strictly, the values of the real-time inputs available at the end of the 10 microsecond pulse and consequently they are actually stored for 290 microseconds. Referring to Figure 4, the value given to the frequency of the clock pulse source 38 of 3.3kHz and the corresponding value of 300 microseconds given to the duration of the pulse generated by multivibrator 20 refer to a particular example of a Figure 4 circuit in which the picture element clock rate is 3.3kHz. This clock rate is determined by the rate at which picture elements are scanned which is settled by other parameters dependent on the purpose to which the picture processor is applied. These values may be adjusted to suit the application. Likewise the value of 10 microseconds given to the sample-and-hold control pulse generated by multivibrator 41 would be adjusted to suit the application. In the example described having a picture element rate of 3.3kHz, the values of the labelled resistors and capacitors are as follows. R1 = 330k, R2 = 680k, R3 = 33k, R4 = 100k, C1 = 390pf, C2 = 390pf and C3 = 3.9nf. The transistors T1, T2, T3 and T4 are all Mullard BCY71. The high gain differential amplifiers 29, 32, 33, 34, 35, 36 and 37 are RCA Type No. CA3060. The gates 12, 13, 14 and 17 are Motorola Type No. MC14001. The gates 8, 39 and 40 are Type No. MC14025. The multivibrators 20 and 41 are Type No. MC14528. The supply voltage +V is 5 volts. WHAT WE CLAIM IS:

1. A modification of the picture processing apparatus as claimed in claim 1 of the parent Patent Specification for application to a picture comprising regularly spaced rows and regularly spaced transverse columns of picture elements, said modified apparatus comprising means for comparing the brightness of a given picture element with (a) the brightness of a first adjacent picture element in the column containing said given picture element and with (b) the brightness of a second adjacent picture element in the row containing said given picture element, and means for providing an output signal if said first and second picture elements both differ in brightness from said given picture element by at least a predetermined threshold amount of the same sign.

2. Picture processing apparatus as claimed in Claim 1 wherein there is provided a line array of detectors parallel to said rows, each detector having a field of view of one picture

element and being adapted to produce an output corresponding to the brightness of the picture element in its field of view, and means for scanning said array along said columns of picture elements to determine the brightness valves of said given, first and second picture elements.

3. Apparatus as claimed in Claim 2 wherein there is provided means for storing the output of each detector for the duration of the scan of one picture element, said stored and direct detector outputs supplying the brightness values of said given, first and second picture elements.

4. Apparatus as claimed in Claim 2 wherein a second line array of detectors is provided parallel and adjacent to said first line array, each detector in the second array being adapted to follow and repeat the scanning of a corresponding detector in the first array, the simultaneously available outputs of both arrays supplying the brightness values of said given, first and second picture elements.

5. Apparatus as claimed in any one of Claims 2, 3 or 4 wherein said line arrays have only three detectors.

6. Apparatus as claimed in any one of Claims 2, 3 or 4 wherein said line arrays have only two detectors.

7. Apparatus substantially as described witha reference to Figures 1, 2 and 3 of the drawings accompanying the Provisional Specification.

8. Apparatus substantially as described with reference to Figures 1, 2 and 4 of the drawings accompanying the Provisional Specification.