JP7278735B2 - Image processing device, image processing method, and program - Google Patents

Description

The present invention relates to technology for processing images acquired by a security camera or the like.

2. Description of the Related Art In recent years, there has been proposed a system for estimating the number of people and the flow of people (crowd analysis processing) for images captured by an imaging device such as a security camera. For example, Patent Document 1 discloses a method of estimating the number of people from the area of a difference image obtained from an input image and a background image. Utilization of such a system makes it possible to detect congestion in public spaces and grasp the flow of people during congestion, and is expected to realize congestion relief during events and appropriate evacuation guidance in the event of a disaster.

On the other hand, it is becoming more and more important to protect the privacy of subjects captured in security camera images. Therefore, it is necessary to appropriately protect the privacy of subjects that make up a crowd. Patent Document 2 discloses a method of updating a background image and superimposing the updated background image on a silhouetted (concealed) foreground image. As a result, although the person's privacy information is not exposed, the person's behavior can be understood, making it possible to use the captured video for the purpose of marketing analysis and service improvement. Further, Patent Document 3 discloses a method of detecting both eyes of a person and applying rectangular mask processing to the eye portions.

JP 2005-242646 A JP 2016-115214 A JP 2009-278325 A

In the case of the above-described conventional method, concealment processing (for example, blurring processing or watermark processing) for realizing privacy protection is uniformly performed on each subject. For this reason, there are cases where privacy processing is insufficient and privacy cannot be properly protected, and conversely, there are cases where privacy processing is excessive and it is not suitable for use in marketing analysis, etc., and appropriate privacy protection is not possible. may not always be realized.

Further, for example, in the technique of Patent Document 3, specific objects such as eyes, face, and head are first detected. learning takes place. However, the detection accuracy of the specific object detector is often degraded when, for example, a plurality of subjects overlap with each other at high density. If the detection accuracy is lowered and the detection of the subject fails, the confidentiality processing is not executed, so there arises a problem that the privacy of the subject is not properly protected.

SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to enable appropriate concealment processing.

According to the present invention, an estimating means for estimating a specific subject included in an image, a detecting means for detecting the specific subject from the image, and a degree of image processing for concealing the subject are set. and output means for outputting a display image obtained by subjecting the estimated subject to anonymization processing according to the set degree of image processing, wherein the display image is the image The output means performs anonymization processing of superimposing the density distribution of the subject estimated by the estimation means on a first region of the image, and further the detection Anonymization processing is performed to superimpose a detection result by means on a second region of the image .

According to the present invention, it is possible to realize appropriate confidentiality processing.

It is a figure which shows an example of the hardware constitutions of the image processing apparatus of embodiment. It is a figure showing an example of functional composition of an image processing device of an embodiment. 4 is a flowchart showing the flow of image processing by the image processing device; It is a figure which shows the setting example of a privacy level. FIG. 4 is a diagram showing an example of small images obtained by dividing an input image; FIG. 4 is an explanatory diagram of an estimator that estimates density distribution and motion vector distribution; FIG. 4 is an explanatory diagram of an example of obtaining a motion vector in an estimator; FIG. 10 is a diagram showing an example of image processing when the privacy level is "maximum"; FIG. 10 is a diagram showing an example of image processing when the privacy level is "high"; FIG. 10 is a diagram showing an example of image processing when the privacy level is "medium"; FIG. 10 is a diagram showing an example of image processing when the privacy level is "low"; FIG. 10 is a diagram showing an example of image processing when the privacy level is "minimum"; FIG. 10 is a diagram showing another example of privacy level setting; FIG. 10 is a diagram showing an example of definitions of operations permitted for each viewing authority of an observer;

Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings. Note that the configurations shown in the following embodiments are merely examples, and the present invention is not limited to the illustrated configurations.
<First Embodiment>
FIG. 1 is a diagram showing an example of a schematic hardware configuration of an image processing apparatus 100 according to the first embodiment.
The image processing apparatus 100 has a CPU 10, a memory 11, a network I/F 12, a display device 13, and an input device 14 as a hardware configuration. The CPU 10 controls the entire image processing apparatus 100 . The memory 11 stores data, programs, etc. that the CPU 10 uses for processing. A network I/F 12 is an interface that connects the image processing apparatus 100 to a network. The display device 13 is a liquid crystal display device or the like, and displays the results of processing by the CPU 10 and the like. The input device 14 is a mouse, buttons, or the like, and inputs user's operations to the image processing apparatus 100 . By the CPU 10 executing image processing and control based on the programs stored in the memory 11, each function of the image processing apparatus 100 shown in FIG. 2 and the processing of the flow chart in FIG.

Each function of the image processing apparatus 100 of this embodiment will be described below with reference to FIG. The image processing apparatus 100 of this embodiment includes an image acquisition unit 201, a privacy level setting unit 202, a background generation unit 203, a moving object extraction unit 204, a number/people flow estimation unit 205, a head detection unit 206, an image processing unit 207, an image An output section 208 is included.

An image captured by a security camera or the like is input to the image acquisition unit 201 . The input image acquired by the image acquisition unit 201 may be an image captured by a security camera or the like, an image recorded in a storage device such as a hard disk, or an image received via a network such as the Internet. It's okay. In the image processing apparatus 100 of the present embodiment, an input image acquired by the image acquisition unit 201 is subjected to image processing for concealing a specific subject (privacy protection for a human subject in the present embodiment). be. The input image acquired by the image acquisition unit 201 is also used as a crowd analysis target. An input image from the image acquisition unit 201 is sent to a background generation unit 203 , a number/people flow estimation unit 205 and a head detection unit 206 .

A background generation unit 203 generates a background image based on the input image acquired by the image acquisition unit 201 . Here, the background image refers to image data of only the background in which the subject is not shown. The background generation unit 203 outputs the generated background image and the input image from the image acquisition unit 201 to the moving object extraction unit 204 .
A moving object extraction unit 204 extracts a portion in which a moving object exists from the input image acquired by the image acquisition unit 201 as a person region. The moving object extraction unit 204 outputs the moving object extraction result and the input image acquired by the image acquisition unit 201 to the image processing unit 207 .
The number of people/people flow estimation unit 205 performs crowd analysis processing using the input image acquired by the image acquisition unit 201, and estimates the number of people and people flow in the input image. The number of people/people flow estimation unit 205 outputs the estimation results of the number of people and people flow to the image processing unit 207 .
The head detection unit 206 performs head detection processing using the input image acquired by the image acquisition unit 201, and estimates the position and size of the human head in the input image. The head detection unit 206 outputs the result of head detection to the image processing unit 207 .

The privacy level setting unit 202 sets the degree of image processing (privacy level) to protect the privacy of the subject. A setting example of the privacy level will be described later. Information about the privacy level set by the privacy level setting unit 202 is sent to the image processing unit 207 .
The image processing unit 207 performs predetermined processing (image processing). Details of the predetermined processing will be described later. An image that has undergone predetermined processing in the image processing unit 207 is sent to the image output unit 208 .
An image output unit 208 outputs the image processed by the image processing unit 207 . The image output from the image output unit 208 is sent to the display device 13 shown in FIG. A display example of the display device 13 will be described later.

FIG. 3 is a flowchart showing an example of image processing by the image processing apparatus 100 of this embodiment. In the following description, processing steps S301 to S308 in the flowchart of FIG. 3 are abbreviated as S301 to S308. The processing of the flowchart in FIG. 3 may be executed by a hardware configuration, or may be partially realized by a software configuration and the rest by a hardware configuration. When processing is executed by a software configuration, it is realized by the CPU 10 or the like executing an image processing program according to the present embodiment stored in the ROM of the memory 11 or the like, for example. The program according to the present embodiment may be prepared in advance in a ROM or the like, read from a detachable semiconductor memory or the like, or downloaded from a network such as the Internet (not shown).

In S301, the image acquisition unit 201 acquires an input image as described above.
Also, in S302, the privacy level setting unit 202 sets the privacy level. The privacy level is set by the supervisor through a user interface installed in the viewer for supervisor displayed on the display device 13, for example.

FIG. 4 shows an example of a supervisor viewer window 400 . A window 400 of the supervisor viewer illustrated in FIG. is displayed. When the observer (user) moves the knob 402 of the slider 401 via the input device 14 , the privacy level setting section 202 sets the privacy level according to the position of the knob 402 . In the vicinity of the slider 401, characters such as "minimum", "low", "middle", "large", and "maximum" are displayed, representing the privacy level in five stages, for example. By moving the knob 402 of the slider 401, the observer can change the privacy level step by step and set the desired level. The privacy level is normally set to a high level (for example, in the initial setting), but in the case of this embodiment, it is possible to set the privacy level to a low level only when an incident or accident occurs. is done.

Also, the supervisor viewer may be configured with user authentication. Then, for example, the privacy level may be set to a low level only when a supervisor with a high viewing authority performs user authentication. In addition, when an incident or accident occurs, all supervisors may be configured to set an arbitrary privacy level regardless of the supervisor's viewing authority.

Returning to FIG. In S<b>303 , the background generation unit 203 generates a background image based on the input image acquired by the image acquisition unit 201 . A background image can be generated by several generation techniques. As an example of a generation method, a method of obtaining an average value or a median value of a plurality of input images obtained by the image obtaining unit 201 at different times and using them as a background image can be cited. As another generation method, there is a method of using an input image acquired by the image acquisition unit 201 at a timing when it is considered that no moving object exists, as a background image. Background image generation techniques are not limited to these techniques, and a plurality of techniques may be combined. In addition, the background generation unit 203 may update the background image in order to follow changes in the background over time. The background image can be updated by periodically regenerating a new background image or replacing only a portion of the background image where no moving object is considered to exist with the latest input image.

Next, in S<b>304 , the moving object extraction unit 204 extracts an area in which the moving object exists from the input image acquired by the image acquisition unit 201 as a person area. Moving objects can be extracted by several extraction methods. An example of an extraction method is a method of extracting a moving object (foreground object) by a background subtraction method that takes a difference between a background image generated by the background generation unit 203 and an input image obtained by the image acquisition unit 201. can be done. As another extraction method, a moving object (foreground object) is extracted by performing an inter-frame difference method or a motion vector detection method on a plurality of input images acquired by the image acquisition unit 201 at different times. be able to. The method of extracting a moving object is not limited to these methods, and multiple methods may be combined.

Next, in S305, the number of people/people flow estimation unit 205 performs crowd analysis processing on the input image acquired by the image acquisition unit 201, and estimates the number of people and people flow in the image. There are several methods for estimating the number of people and the flow of people. An example of the estimation method will be described below.

First, the number/people flow estimation unit 205 divides the input image acquired by the image acquisition unit 201 into a plurality of small images. FIG. 5 illustrates one method of dividing an input image 500 into sub-images. FIG. 5 shows an example in which an input image 500 is divided so that the ratio between the size of each small image and the size of one human body shown in each small image is substantially constant.

Next, the number/people flow estimation unit 205 estimates the number of people and people flow for each small image using an estimator that receives a plurality of temporally continuous small images as input and outputs the crowd density distribution and the movement vector distribution. . Methods for estimating the density distribution and motion vector distribution of a person in an image include, for example, the document "Walach E., Wolf L. (2016) Learning to Count with CNN Boosting. In: Leibe B., Matas J., Sebe N., Welling M. (eds) Computer Vision - ECCV 2016. Lecture Notes in Computer Science, vol 9906. Springer, Cham”. In the literature, a neural network obtained in advance by machine learning is used to obtain the density distribution of people from an image. In the present embodiment, this method is used to pre-learn and estimate a neural network for simultaneously estimating the density distribution and motion vector distribution of a person in an image by inputting two consecutive frames of images.

FIG. 6 is a diagram for explaining the function of an estimator in the number of people/people flow estimating unit 205. As shown in FIG. An image 601 at a time t ₁ and an image 602 at a time t ₂ are input to a neural network 610 in FIG. 6 as small images of two consecutive frames. The neural network 610 simultaneously estimates the population density distribution 621, the horizontal movement vector distribution 622, and the vertical movement vector distribution 623 in the small images of the two consecutive frames. In the example of FIG. 6, in the population density distribution 621 out of the output of the neural network 610, the shaded portion in the figure represents a predetermined area in which a person is estimated, specifically, the area of the person's head. there is Regarding the display of the head position, the area indicating the position of the head may be displayed darker or larger depending on the degree of likelihood of the head. In the movement vector distributions 622 and 623, the darker the shaded portion in the drawing, the greater the amount of movement. The example of FIG. 6 represents an example in which the crowd is moving in the horizontal direction. The vector value is a small value. The population density distribution 621 is designed so that the sum of the values of the density distribution in a certain range is approximately equal to the number of persons existing in that range.

FIG. 7 shows an example of a population density distribution 701, a horizontal movement vector distribution 702, and a vertical movement vector distribution 703 in a small image estimated by the number of people/people flow estimation unit 205. FIG. A motion vector can be extracted by synthesizing the horizontal motion vector distribution 702 and the vertical motion vector distribution 703 . Movement vector distributions 702 and 703 are assumed to have the same weight as the population density distribution 701 . The example of FIG. 7 shows an example in which seven weighted motion vectors are obtained in the small image. Then, the number of people/people flow estimation unit 205 obtains the people flow by performing processing such as clustering on the movement vector.

Returning to FIG. In S306, the head detection unit 206 performs head detection processing using the input image acquired by the image acquisition unit 201, and estimates the position and size of the human head in the input image. Various known pattern recognition and machine learning techniques are used for head detection processing. Here, head detection refers to processing for identifying the position of the head in the human body. For example, the head detection unit 206 extracts a plurality of local patterns by using the matching pattern over the entire image while changing the size of the image, and calculates respective local feature amounts. Then, the head detection unit 206 determines whether it is the head from the sum of the results of weighting the local feature amounts.

In S307, the image processing unit 207 performs predetermined processing on the input image acquired by the image acquisition unit 201 or the background image generated by the background generation unit 203, according to the privacy level set by the privacy level setting unit 202. I do.
An example of an image processing method will be described below in which five levels of privacy levels, "maximum", "large", "middle", "small", and "minimum", are provided in ascending order as shown in FIG.

FIG. 8 is a diagram showing an example of image processing when the privacy level is set to "maximum". FIG. 8 shows an example of the result of image processing that superimposes the information on the number of people and the flow of people generated in S305 on the background image 800 generated in S303. Each frame 802 in FIG. 8 represents an area corresponding to a small image used as a unit for estimating the number of people and the flow of people in S305. The number-of-people information 803 represents information about the estimated number of people in each small image. A numerical value (for example, 0.8, 1.1, etc.) of the number of people information 803 represents the estimated number of people in the small image. The people flow information 804 represents information about the estimated people flow in each small image. The arrow of the people flow information 804 indicates that the direction of the main flow of people is toward the lower right. Text 805 represents the total number of people in the image estimated in S305. In the example of FIG. 8, only the result of analysis processing of the number of people and the flow of people is superimposed on the background image 800, so the observer cannot even know the position or movement of the subject that should be present in the image. By applying the image processing shown in FIG. 8, the observer can obtain the crowd analysis result while protecting the subject's privacy to the maximum.

FIG. 9 is a diagram showing an example of image processing when the privacy level is set to "high". FIG. 9 shows an example of the result of image processing in which a figure 901 representing the population density distribution generated in S305 is superimposed on the head position of the background image 900 generated in S303 by alpha blending. there is The population density distribution in this process appears corresponding to the head of a person as shown in the population density distribution 621 in FIG. An image will be generated that represents only the position and motion of the . Although omitted in FIG. 9, the frame 802, the number of people information 803, and the people flow information 804 may be superimposed as in FIG. This also applies to FIGS. 10 to 12, which will be described later. According to the image processing shown in FIG. 9, it is possible for the observer to visually confirm the position and movement of the subject, while making it difficult for the observer to personally identify the subject.

FIG. 10 is a diagram showing an example of image processing when the privacy level is set to "medium". FIG. 10 shows an example of the result of image processing in which the region of the entire person 1001 extracted in S304 is superimposed on the background image 1000 generated in S303 by alpha blending. This image processing can generate an image representing the silhouettes of the people who make up the crowd. According to the image processing shown in FIG. 10, it is possible for the observer to visually confirm the position, movement, and behavior of the subject, while making it difficult for the observer to personally identify the subject.

FIG. 11 is a diagram showing an example of image processing when the privacy level is set to "low". FIG. 11A is an example of the result of image processing in which a figure 901 representing the population density distribution generated in S305 is superimposed on the head position by alpha blending on the input image 1100 acquired in S301. is shown. By this image processing, the concealment processing is applied only to the subject's head area. According to the image processing shown in FIG. 10, the observer can visually confirm the subject's clothing in addition to the subject's position, movement, and behavior. An effect of confidentiality processing is obtained so that it cannot be browsed.

Note that the estimator of the number of people/people flow estimating unit 205 used in S305 is learned using learning data including crowded images in which people greatly overlap each other. Therefore, the estimator of the number of people/people flow estimator 205 has fewer detection omissions of heads under congestion conditions than the head detector of the head detector 206 . Therefore, in a crowded situation, by using the result of the estimator of the number of people/people flow estimating unit 205, the concealment processing of the subject can be performed more reliably than the case of using the result of the head detector of the head detecting unit 206. It becomes possible to go to

Also, the population density distribution takes a large value near the center of the head, and takes a smaller value as the distance from the center increases. Therefore, simply alpha-blending the graphic 901 indicating the density distribution of the number of people on the input image 1100 may not sufficiently perform the concealment processing on the outer peripheral portion of the head. Therefore, the following methods may be used to more reliably conceal the subject's head. For example, the first method is to increase the weight of the population density distribution when alpha blending is performed. This method can increase the degree of concealment processing based on the density distribution. The second method is to perform alpha blending after binarizing the figure 901 showing the population density distribution. With this method, it is possible to perform the same security processing on the outer peripheral portion of the head as on the vicinity of the center of the head. A third method is to perform alpha blending after dilation processing is applied to the figure 901 showing the population density distribution. As a result, the concealment process can be applied to a wider area centering on the head. Each of these methods may be performed in combination.

Also, the image processing method is not limited to alpha blend processing. As another method of image processing, processing such as mosaic processing or blurring processing may be performed on the input image for areas where the population density distribution is equal to or higher than the threshold.

On the other hand, in a non-crowded situation where people are sparsely present or when a person appears larger than a predetermined size, the estimator of the number of people/people flow estimator 205 is more sensitive than the detector of the head detector 206. There is a tendency that there are many omissions in the detection of the head. Therefore, by using together the result of the detector of the head detection unit 206 in addition to the result of the estimator of the number/people flow estimation unit 205 according to the degree of congestion and the size of the people, the concealment processing of the subject can be further improved. It can be done with certainty.

FIG. 11B shows a diagram 901 representing the population density distribution generated in S305 and a diagram 1104 representing the head detection result generated in S306 on the input image 1100 acquired in S301. An example of the result of image processing superimposed on the head position by processing is shown.

Since the area 1102 has a small depression angle of the camera and is therefore an area where people's heads tend to overlap each other, the number of people/people flow estimating unit 205 misses detection of heads less than the head detecting unit 206 does. There is a tendency. On the other hand, area 1103 has a large depression angle of the camera, and therefore is an area where people's heads are unlikely to overlap each other. tend to be less.

Therefore, in the area 1102, image processing is performed to superimpose the figure 901 representing the population density distribution generated in S305 by alpha blending. On the other hand, in an area 1103, image processing is performed to superimpose a figure 1104 representing the head detection result generated in S306 by alpha blending.

Any method may be used to determine the regions 1102 and 1103 .
The first method is a method in which the user manually and explicitly sets the areas 1102 and 1103 . FIG. 11B shows an example of setting so that the areas 1102 and 1103 do not overlap. On the other hand, by setting areas such that the area 1102 and the area 1103 overlap, the risk of the head being concealed in the overlapping area may be reduced.
A second method is a method of automatically setting the areas 1102 and 1103 . For example, a region 1103 can be defined as a region in which the head is larger than a first predetermined size, and a region 1102 can be defined as a region in which the head is smaller than a second predetermined size.

The size of the head on the image may be explicitly given by the user using an input device such as a mouse. For example, when the user instructs the average size of the head at a plurality of points on the image, the image processing unit 207 calculates the average size of the head at any point on the screen by interpolation. It is possible to estimate

Also, the size of the head on the image may be automatically estimated by statistical processing. For example, head detection processing is performed on the entire screen of a predetermined learning image group, and a set of head frames indicating the position of the head is obtained. Let s be the size of the head frame at coordinates (x, y) on the screen, and assume that s can be represented by x, y and one or more unknown parameters. For example, assume s=ax+by+c. In this example the unknown parameters are a, b and c. An unknown parameter can be obtained by statistical processing such as the least-squares method using a set of head frames acquired from a predetermined group of learning images.

A third method is to automatically set the regions 1102 and 1103 according to the density of the heads. For example, a region 1103 can be defined as a region in which the density of heads is less than or equal to a predetermined value, and a region 1102 can be defined as a region in which the density of heads is greater than or equal to a predetermined value. Head density can be estimated in any manner. For example, the density can be estimated according to the ratio of the number of pixels determined to contain a moving object to the moving object region obtained by background subtraction. Also, for example, motion vector detection can be applied to a predetermined range, and the density can be estimated according to the sum of motion vector lengths.

FIG. 12 is a diagram showing an example of image processing when the privacy level is set to "minimum". When the privacy level is set to "minimum", as shown in FIG. 12, the input image 1200 acquired in S301 is displayed as it is. Therefore, in the case of the example of FIG. 12, the observer can identify the subject 1201 individually.

Returning to FIG. In S<b>308 , the image output unit 208 outputs the image processed by the image processing unit 207 . The output image is viewed by a supervisor through a supervisor viewer or the like shown in FIG.

<Second embodiment>
A second embodiment will be described below. In the second embodiment, a case will be described in which the privacy level setting unit 202 can set the privacy level in more detail. Since the configuration of the image processing apparatus 100 of the second embodiment is the same as that of FIGS. 1 and 2 described above, the illustration thereof is omitted. Also, in the second embodiment, description of the same functions as in the first embodiment will be omitted, and only points different from the first embodiment will be described.
In the case of the second embodiment as well, in S302 of FIG. 3, the privacy level setting unit 202 acquires the privacy level as described above. The privacy level is set by the supervisor through a user interface installed in the viewer for the supervisor, for example.

FIG. 13 shows an example of a supervisor viewer window 1300 in the second embodiment. 13, the frame 1314 is the same as the frame 802 in FIG. 8 described above, the number of people information 1315 is the number of people information 803 in FIG. 8, the people flow information 1316 is the people flow information 804 in FIG. Similarly, it is information indicating the result of estimating the number of people in the entire image. A figure 1312 showing the population density distribution is the same as the figure 901 showing the population density distribution in FIG. In the example of FIG. 13, radio buttons 1301 to 1306 are set at the bottom of the observer viewer. The observer can finely set the contents to be displayed on the observer's viewer by performing inputs to these radio buttons 1301 to 1306 via the input device 14 .

A radio button 1301 is a button for selecting an image to be displayed in the window 1300 of the viewer for supervisor. If radio button 1301 is set to "none", no image is displayed. When the radio button 1301 is set to “background image”, the background image generated by the background generation unit 203 is displayed in the window 1300 . When the radio button 1301 is set to “input image”, the image acquired by the image acquiring unit 201 is displayed in the window 1300 .

A radio button 1302 is a button for selecting a process to be performed on the image displayed in the window 1300 of the viewer for supervisor. When the radio button 1302 is set to “head”, processing is performed to superimpose a figure indicating the population density distribution estimated by the number of people/people flow estimation unit 205 on the image displayed on the window 1300 . When the radio button 1302 is set to “entire person”, the moving object extracted by the moving object extracting unit 204 is regarded as a person, and the image obtained by processing the entire person area is superimposed on the image displayed in the window 1300 . done. When the radio button 1302 is set to "head", a figure 1312 representing the population density distribution is superimposed.

A radio button 1303 is a button for setting whether to display the estimated number of people on the entire screen. If radio button 1303 is set to "yes", text 1313 is displayed.
A radio button 1304 is a button for setting whether or not to superimpose a frame corresponding to each small image divided by the number of people/people flow estimation unit 205 . When radio button 1304 is set to "Yes", frame 1314 is displayed.

A radio button 1305 is a button for setting whether or not to superimpose the number of people estimated for each small image by the number of people/people flow estimation unit 205 . When the radio button 1305 is set to "Yes", the number of persons information 1315 is displayed.
A radio button 1306 is a button for setting whether or not to superimpose the flow of people estimated for each small image by the number of people/flow of people estimation unit 205 . When radio button 1306 is set to "Yes", people flow information 1316 is displayed.
Operations of these radio buttons 1301 to 1306 may be restricted by the viewer's viewing authority.

FIG. 14 shows a case in which five levels of viewing authority for the observer illustrated in FIG. , is an example of a table defining permitted operations for each observer. The privacy level setting unit 202 sets a privacy level according to the viewing authority of these supervisors.
In the case of the table example shown in FIG. 14, for example, a supervisor having viewing authority "maximum" can view all images corresponding to the respective privacy levels shown in FIGS. 8 to 12 described above. Also, for example, a supervisor having viewing authority "high" can view images corresponding to each privacy level shown in FIGS. Unable to browse. 8 to 10, an observer with viewing authority of "medium" can view the images corresponding to the privacy levels shown in FIGS. 8 to 10, but cannot view the images corresponding to the privacy levels shown in FIGS. . A supervisor having viewing authority "small" can view images corresponding to the privacy levels shown in FIGS. 8 and 9, but cannot view images corresponding to the privacy levels shown in FIGS. A supervisor having viewing authority "minimum" can view only images corresponding to the privacy level "maximum" shown in FIG. 8, and cannot view images corresponding to the privacy levels shown in FIGS.

In the second embodiment, FIG. 12 shows an example of using radio buttons as a user interface, but the user interface that can be used is not limited to this. For example, a push button may be arranged instead of the radio button, and the display contents may be toggled each time the push button is clicked.

As described above, according to the image processing apparatus 100 of the first and second embodiments, when the result of crowd analysis using an image of a crowd is displayed to the observer, the viewing Appropriate processing for protecting the privacy of the subject according to authority can be performed. For example, for an observer with low viewing authority, a silhouetted image of the subject's whole body is generated, and for an observer with high viewing authority, only the head area of the subject is masked. Images can be generated, and various image processing can be performed according to viewing authority.
Although the preferred embodiments of the present invention have been described in detail above, the present invention is not limited to such specific embodiments, and various modifications can be made within the scope of the gist of the invention described in the claims.・Changes are possible.

In the above-described embodiment, a person was exemplified as the specific subject and a head was exemplified as the predetermined area, but the subject is not limited to these. The specific subject may be a vehicle other than a person, such as an automobile, and the predetermined area may be, for example, a license plate or a person riding in an automobile. may

A program that implements one or more functions in signal processing according to the present invention can be supplied to a system or device via a network or storage medium, and read and executed by one or more processors of the computer of the system or device. It can be realized by It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

All of the above-described embodiments merely show specific examples for carrying out the present invention, and the technical scope of the present invention should not be construed to be limited by these. That is, the present invention can be embodied in various forms without departing from its technical spirit or main features.

10: CPU, 11: Memory, 13: Display Device, 14: Input Device, 100: Image Processing Device, 201: Image Acquisition Unit, 202: Privacy Level Setting Unit, 203: Background Generation Unit, 204: Moving Object Extraction Unit, 205 : number of people/people flow estimation unit, 206: head detection unit, 207: image processing unit, 208: image output unit

Claims (16)

estimating means for estimating a specific subject included in an image;
detection means for performing detection processing of the specific subject from the image;
setting means for setting the degree of image processing performed to conceal the subject;
output means for outputting a display image obtained by subjecting the estimated subject to anonymization processing according to the set degree of image processing;
has
The display image indicates the number of subjects included in the image,
The output means performs anonymization processing of superimposing the density distribution of the subject estimated by the estimation means on a first region of the image,
The image processing apparatus further performs anonymization processing of superimposing a detection result of the detection means on a second area of the image . The estimating means estimates the density distribution of the subject from the image,
2. The method according to claim 1, wherein the output means outputs the display image in which the density distribution estimated by the estimation means is superimposed on the image based on the set degree of image processing. Image processing device. The output means performs at least one of mosaic processing and blurring processing on an area in which the density of the density distribution estimated by the estimation means is equal to or greater than a threshold based on the set degree of image processing. 3. The image processing apparatus according to claim 2, wherein the image processed by the image processing is output as the display image. the detection means detects a predetermined part of the subject based on a pattern indicating the predetermined part of the specific subject;
4. The display image according to any one of claims 1 to 3, wherein the output means outputs the display image processed so as to hide at least a predetermined portion of the subject according to the set degree of image processing. The image processing device according to . The specific subject is a person,
5. The image processing apparatus according to claim 4 , wherein the predetermined part of the subject is a person's head. 6. The method according to any one of claims 1 to 5, wherein said output means determines at least one of said first area and said second area based on the size of a subject in said image. image processing device. The estimating means estimates the position of a specific subject included in the image,
7. The display image according to any one of claims 1 to 6 , wherein the output means outputs the display image based on the position of the specific subject included in the image estimated by the estimation means. Image processing device. further comprising generating means for generating a background image in which the specific subject is not shown based on the image;
8. The image according to claim 7 , wherein the display image is output by superimposing the position of the subject estimated by the estimation means on the background image based on the set degree of image processing. processing equipment. The output means performs at least one of binarization processing and expansion processing on the position of a specific subject included in the image estimated by the estimation means based on the set degree of image processing. 9. The image processing apparatus according to claim 8 , wherein the image thus processed is output as the display image. The setting means sets a privacy level representing the degree of image processing based on a user's instruction,
10. The image processing apparatus according to claim 1, wherein said output means outputs said display image according to said privacy level. 11. The image processing apparatus according to claim 10 , wherein said setting means sets said privacy level based on an operation on a predetermined user interface. The predetermined user interface is a user interface for setting one of a plurality of stages of the privacy level, or a user interface for setting the content to be displayed after the anonymization process. 12. The image processing apparatus according to claim 11 . The output means outputs an image showing only the position of the subject in the image, an image in which the entire subject in the image is processed, or a predetermined part of the subject in the image, according to the set degree of image processing. 13. The image processing apparatus according to any one of claims 1 to 12 , wherein any one of the displayed images is output as the display image. The estimating means further estimates a motion vector of the subject in the image,
The image processing apparatus according to any one of claims 1 to 13, wherein the display image indicates the estimated movement vector of the subject. an estimation step of estimating a specific subject included in the image;
A detection step of performing detection processing of the specific subject from the image;
a setting step of setting the degree of image processing to be performed to conceal the subject;
an output step of outputting a display image obtained by subjecting the estimated subject to anonymization processing according to the set degree of image processing;
has
The display image indicates the number of subjects included in the image,
In the output step, performing anonymization processing for superimposing the density distribution of the subject estimated in the estimation step on a first region of the image,
An image processing method , further comprising the step of superimposing a detection result of the detection step on a second area of the image for anonymization . A program for causing a computer to function as each means of the image processing apparatus according to any one of claims 1 to 14 .

Images