JP2019159092A - Microscope system, image analysis device and image analysis method - Google Patents

Abstract

To receive an image processing setting and an analysis setting after receiving a user's choice between an individual image and a coupled image.SOLUTION: A setting section receives a choice of either an individual image or a coupled image as an analysis object. A display section displays a map image suggesting imaging positions of a plurality of fluorescent images. The setting section receives designation of the fluorescent image for setting image processing and analysis processing through the map image and receives an image processing setting and an analysis setting when the individual image is selected as the analysis object. The setting section receives designation of the plurality of fluorescent images to be a source of the coupled image through the map image and receives the image processing setting and the analysis setting when the coupled image is selected as the analysis object.SELECTED DRAWING: Figure 4

Description

  The present invention relates to a microscope system, an image analysis apparatus, and an image analysis method.

  An image cytometer that analyzes a large number of images acquired using a microscope in a lump is widely used (Patent Document 1). Image cytometers are sometimes referred to as high content analysis.

  In the image cytometer, the user sets image processing conditions, sets analysis processing conditions, selects an analysis target, executes image processing and analysis processing, and displays and confirms the analysis results. By the way, the low-magnification image has a wide field of view, but the resolution is low and it is not suitable for detailed analysis processing. On the other hand, a high-magnification image has a narrow field of view, but has high resolution and is suitable for detailed analysis processing. Therefore, according to Patent Document 2, a wide-field connected image is generated by connecting a plurality of high-magnification individual images.

JP 2005-525550 A JP 2013-088530 A

  There are several advantages and disadvantages to processing individual images and connected images with an image cytometer. The same cell may appear in a plurality of adjacent individual images, and double counting occurs when the number of cells is counted using the individual image. In addition, the measurement values of the area and shape of the cells located at the end of the individual image deviate from the actual values. In order to solve this problem, if the analysis is performed in consideration of the overlap between adjacent images, the analysis process becomes complicated and the analysis time becomes longer. On the other hand, a connected image is generated by combining a plurality of adjacent individual images so that cells and the like do not overlap. Therefore, it becomes easy to accurately analyze the shape of the cell located at the end of the individual image. It becomes easy to adjust shading from the brightness of a plurality of adjacent individual images. Since the connected image has a wide field of view, it is easy for the user to specify a part of the analysis area. On the other hand, since the connected image has a large image size, the image processing restriction (hardware specifications) may not be satisfied. Even if the restriction is satisfied, if the connected image is used as it is, the analysis time becomes longer. Therefore, when the connected image is reduced, small cells disappear or two individual objects are connected to one object. If image processing conditions and analysis processing conditions are set for individual images and these are applied as they are to the connected image, an analysis result different from the user's intention is obtained. Therefore, an object of the present invention is to accept an image processing setting and an analysis setting after accepting selection of either an individual image or a connected image by a user.

The present invention is, for example,
A stage on which the observation object is placed, irradiation means for irradiating the observation object placed on the stage with excitation light, and fluorescence emitted from the observation object irradiated with the excitation light. An imaging means for acquiring a fluorescence image of the observation object; and a moving means for relatively moving the stage and the imaging means within a predetermined movable range in XY plane coordinates;
Display means for displaying the fluorescence image, analysis means for applying image processing to the fluorescence image acquired by the fluorescence microscope, and executing analysis processing of the fluorescence image to which the image processing is applied, and a plurality of fluorescence An image analysis apparatus comprising: a coupling unit that couples images to generate a coupled image;
A microscope system comprising:
The image analysis device includes:
An individual image that is a fluorescent image acquired by imaging the observation object in a single imaging field of view of the imaging unit and a plurality of fluorescent images that are adjacent to each other and acquired by the imaging unit Receiving means for accepting selection of which of the connected images generated by connecting by the analysis target to be analyzed;
The display means is configured to display a map image that suggests imaging positions of a plurality of fluorescent images,
When the individual image is selected as an analysis target through the reception unit,
The accepting means is
Through the map image displayed on the display means, the designation of the fluorescence image displayed on the display means for accepting the setting of the image processing and the analysis processing is accepted, and the image applied to the fluorescence image The setting of the processing and the setting of the analysis processing for the observation object reflected in the fluorescent image to which the image processing based on the setting of the image processing is applied are received, and the image is displayed through the map image displayed on the display means. Accepting the designation of one or more fluorescent images to be applied to the process and the analysis process;
The analysis means includes the image processing and the setting of the analysis processing based on the setting of the image processing and the setting of the analysis processing received by the reception means for one or more fluorescent images to be applied to the image processing and the analysis processing. Apply the analysis process,
When the connected image is selected as an analysis target through the receiving unit,
The accepting means is
Through the map image displayed on the display means, designation of a plurality of fluorescent images that are the basis of the connected image is received, the connected image generated by the connecting means is displayed on the display means, and the connected image is displayed on the connected image. The map displayed on the display unit, accepting the setting of the image processing to be applied and the setting of the analysis processing for the observation object in the connected image to which the image processing based on the setting of the image processing is applied Accepting designation of one or more observation positions to be applied to the connection process, the image process and the analysis process through an image;
The analysis unit specifies a plurality of fluorescent images that are the basis of the connected image received by the reception unit, the setting of the image processing, and the setting of the analysis processing with respect to the fluorescent image for each observation position. Based on the above, a microscope system characterized by applying the connection processing for generating the connection image, the image processing, and the analysis processing is provided.

  According to the present invention, it is possible to accept an image processing setting and an analysis setting after accepting selection of either an individual image or a connected image by a user.

FIG. 1A and FIG. 1B are perspective views of a microscope that forms the core of the microscope system. FIG. 2 is a block diagram showing main parts constituting the microscope system. FIG. 3 is a block diagram showing various functions realized by the CPU. FIG. 4 is a flowchart showing the image analysis method. FIG. 5 is a flowchart showing details of setting and analysis for individual images. FIG. 6 is a diagram for explaining the setting UI. FIG. 7 is a diagram showing a UI for accepting image processing settings and the like. FIG. 8 is a diagram showing a UI for receiving analysis settings and the like. FIG. 9 is a diagram showing a UI for designating an analysis target. FIG. 10 is a diagram illustrating a UI for displaying the analysis result. FIG. 11 is a flowchart showing details of setting and analysis for a connected image. FIG. 12 is a diagram for explaining the setting UI. FIG. 13 is a diagram illustrating a UI for accepting connection settings. FIG. 14 is a diagram showing a UI for accepting image processing settings for connected images. FIG. 15 is a diagram illustrating a UI for receiving analysis settings and the like for connected images. FIG. 16 is a diagram showing a UI for designating an analysis target. FIG. 17 is a diagram showing a UI for displaying the analysis result.

  Hereinafter, an embodiment of the present invention will be described. The individual embodiments described below will help to understand various concepts, such as the superordinate concept, intermediate concept and subordinate concept of the present invention. Further, the technical scope of the present invention is determined by the scope of the claims, and is not limited by the following individual embodiments.

  1A and 1B are perspective views of the microscope 1 that forms the core of the microscope system. As shown in FIG. 1A, since the microscope 1 uses the casing itself as a dark room, the user does not need to prepare the dark room. When the user (operator) opens the top cover 190, the user (operator) can access the XY stage 6 disposed below the transmission illumination optical system 5. The XY stage 6 is a stage that moves in a direction orthogonal to the optical axis direction of the objective lens. By closing the top cover 190, a dark room is formed. The user can replace the filter cube (color filter or the like) mounted on the filter turret 14 by opening the front cover 191.

  FIG. 2 is a block diagram showing the main parts constituting the microscope system 100. The microscope 1 is controlled by the control device 2 shown in FIG. The control device 2 is, for example, an information processing device (personal computer: PC) in which a control program is installed. That is, the PC may function as the control device 2 of the microscope 1. As described above, the microscope system 100 includes the microscope 1 and the control device 2.

  The microscope 1 is a microscope that can acquire a monochrome image, a color image, and a fluorescence image of the sample 3, but may be a microscope that acquires only one type of these images. A container unit 7 for holding the sample 3 is fixed to the XY stage 6. The sample 3 is sometimes called a specimen, a sample, a specimen, or a workpiece. The container unit 7 includes a container such as a preparation, a dish, or a well, and a container holder that supports the container. The illumination light output from the transmitted illumination light source 4 is applied to the sample 3 through the transmitted illumination optical system 5 including a condenser lens. The transmission illumination optical system 5 may be provided with a mechanical shutter for light shielding. The illumination light from the transmitted illumination light source 4 is used when acquiring a monochrome image or a color image of the sample 3. When performing fluorescence observation on the sample 3, excitation light is output from the fluorescent epi-illumination light source 8. The excitation light passes through the fluorescence epi-illumination optical system 9 and the excitation filter 10. The fluorescent epi-illumination optical system 9 may be provided with a mechanical shutter for shielding light. The excitation filter 10 is a wavelength-selective filter that transmits only the wavelength component serving as excitation light among the light output from the fluorescent epi-illumination light source 8. The excitation light is further reflected by the dichroic mirror 11, passes through the objective lens of the objective lens unit 12, and is irradiated on the sample 3. The dichroic mirror 11 is also a wavelength-selective mirror, which reflects excitation light, but transmits fluorescence emitted by a fluorescent reagent (also called a fluorescent dye or fluorescent dye) added to the sample 3. The objective lens unit 12 includes an electric revolver that is rotated by a motor 13 and a plurality of objective lenses mounted on the electric revolver. The filter turret 14 has four openings, of which three different filter cubes having the excitation filter 10, the dichroic mirror 11, and the absorption filter 16 are attached, but the remaining one opening is nothing. It is not installed. An aperture without a filter is used when acquiring a bright field image. The filter turret 14 is rotated by a motor 15. The absorption filter 16 is a wavelength selective filter that transmits only a necessary wavelength component of the light from the sample 3. The imaging optical system 17 is a lens that images the sample 3 on the imaging surface of the imaging device 18. The color filter 24 is, for example, a liquid crystal tunable filter, and sequentially acquires R, G, and B images by switching the transmission wavelength, and creates a color image by synthesizing them with the image processing unit 19. The color filter is not limited to the liquid crystal tunable filter described above, and a color filter turret that mechanically switches filters having different transmission wavelengths may be disposed. The image processing unit 19 performs various image processing such as amplification and A / D conversion of the image signal output from the imaging device 18 and further performing shading correction to create image data. The control unit 20 controls each unit of the microscope 1 in accordance with instructions from the control device 2. For example, the control unit 20 controls the motor group 21 to move the XY stage 6 in the X-axis direction or the Y-axis direction, or controls the motor 22 to move the objective lens unit 12, sometimes called a Z stage, to the Z-axis. Move it in the direction. By moving the objective lens unit 12 relative to the sample 3 in the Z-axis direction, the in-focus position is changed, and autofocus is executed. The XY stage 6 and the Z stage may have a manual adjustment mechanism. Here, the Z-axis direction is the optical axis direction of the objective lens, and the X-axis direction and the Y-axis direction are directions orthogonal to the Z-axis direction. Moreover, the control part 20 controls the motor 13 according to the instruction | indication from the control apparatus 2, and switches an objective lens (magnification switching). The communication unit 23 is a unit that receives an instruction from the control device 2 and transmits information and image data from the control unit 20 to the control device 2. The control unit 20 is configured by, for example, a microprocessor, CPU, LSI, FPGA, ASIC, or the like. That is, the control unit 20 may be realized by software and its execution means, may be realized only by hardware, or may be realized by mixing the former and the latter.

  In the control device 2, the CPU 30 executes a control program stored in the storage device 31 to control the microscope 1 and causes the display unit 34 to display image data received through the communication interface 32. The communication unit 23 and the communication interface 32 may be connected by a general communication protocol such as USB, IEEE 1394, or LAN, or may be connected by a dedicated communication protocol. The storage device 31 includes a memory such as a ROM and a RAM, a hard disk storage device, and the like. The storage device 31 may include a semiconductor storage device such as an SSD, an optical disk, and a magnetic disk. Further, it may be a memory card used in a consumer digital camera. The operation unit 33 is an example of an operation unit that can be operated by an operator, and is an input device such as a keyboard or a pointing device. The display unit 34 is a liquid crystal display device or the like, and provides a user interface (UI) for setting control parameters for controlling the microscope 1 and a UI for displaying an analysis result (still image or moving image).

  The microscope system 100 can execute Z-stack image capturing and time-lapse capturing. The Z stack is a technique for acquiring a plurality of images (Z stack images) by imaging the sample 3 while changing the distance in the optical axis direction (Z axis direction) between the sample 3 and the objective lens. That is, the Z stack image refers to a plurality of images (layer images / slice images) of the sample 3 acquired at each in-focus position while shifting the in-focus position by moving the Z stage little by little. In general, the sample 3 has many thicknesses, and it becomes easy to acquire an image focused on the sample 3 by acquiring a plurality of images by shifting the focusing position little by little in the Z-axis direction. The number of images acquired by one Z stack is called the Z stack number or the Z stack number. Note that depth composition is also possible by the Z stack. Time-lapse imaging refers to an imaging method that acquires one or more images at regular time intervals. Sample 3 such as a microorganism grows or changes during observation. Therefore, time-lapse imaging is effective in knowing the progress.

[various settings]
In general, a wide-field image has a low resolution due to a low magnification. Accordingly, the control device 2 generates a connected image with a wide field of view and a high resolution by connecting a plurality of individual images acquired at a high magnification. However, there are cases where an individual image is preferable depending on the sample 3 and analysis contents, and there are cases where a connected image is preferable. Therefore, the CPU 30 receives from the user the selection of whether to perform analysis using individual images or to set using connected images.

  FIG. 3 is a block diagram showing various functions realized by the CPU 30. Some or all of these functions may be realized by a hardware circuit such as an FPGA or an ASIC. The imaging control unit 401 images the sample 3 by controlling the microscope 1 using control parameters set in advance by the user. The control parameter is included in the setting data 433. The imaging control unit 401 stores the image data 431 received from the microscope 1 in the storage device 31. The imaging control unit 401 transmits a control signal for moving the XY stage 6 and the objective lens unit 12 (Z stage) to the microscope 1. The imaging control unit 401 sets the objective lens designated by the user as an objective lens for observation by driving the motor 13 and rotating a revolver equipped with a plurality of objective lenses. The display control unit 440 controls display of the user interface.

  The setting unit 402 creates setting data 433 including control parameters for controlling the microscope 1 in accordance with an instruction input from the operation unit 33 and stores the setting data 433 in the storage device 31. The selection unit 403 selects either an individual image or a connected image as an analysis target in accordance with an instruction input from the operation unit 33. The well receiving unit 404 receives from the operation unit 33 designation of a well to be analyzed among a plurality of wells in the container unit 7 placed on the XY stage 6. The imaging position receiving unit 405 receives an imaging position to be analyzed from the operation unit 33 among a plurality of imaging positions in the well. The image processing setting reception unit 406 receives image processing settings applied to individual images and image processing settings applied to linked images from the operation unit 33. The analysis setting receiving unit 407 receives from the operation unit 33 analysis settings applied to individual images and analysis settings applied to linked images. The connection setting reception unit 408 receives connection settings necessary for generating a connection image from the operation unit 33.

  The image processing unit 410 performs image processing necessary for analyzing the sample 3 on the image data 431. The connection unit 411 generates a connected image by connecting a plurality of individual images according to the connection setting. The Z stack unit 412 generates a plurality of image groups (Z stack images) each having a different Z coordinate. The Z stack unit 412 may perform depth synthesis for each individual image. For example, the Z stack unit 412 may combine a plurality of images (Z stack images) each having a different Z coordinate to generate one image (depth combined image) focused on the entire surface. The connecting unit 411 may connect a plurality of depth-combined individual images.

  The analysis unit 420 reads the analysis target image data 431 subjected to image processing by the image processing unit 410, executes analysis processing, and causes the display unit 34 to display the analysis result through the display control unit 440. For example, the image processing unit 410 binarizes the connected image to enhance the outline of the observation object (eg, cell), and the counting unit 421 counts the number of observation objects from the binarized image.

  The XY stage 6 of the microscope 1 shown in FIGS. 1 and 2 can move in a direction perpendicular to the optical axis of the objective lens of the objective lens unit 12 (that is, the XY direction). The control unit 20 detects and manages the coordinates of the XY stage 6. For example, when the control unit 20 receives an imaging instruction (trigger signal) from the imaging control unit 401, the control unit 20 controls the imaging device 18 to perform imaging of the sample 3, and the image processing unit 19 outputs the image data of the XY stage 6. Imaging coordinate data indicating coordinates may be added. The imaging coordinate data may be separated from the image data 431 and stored in the storage device 31 as metadata.

[Analysis method including setting processing]
FIG. 4 is a flowchart showing the image analysis method.

  In S <b> 1, the CPU 30 (setting unit 402) displays a user interface on the display unit 34.

  In S <b> 2, the CPU 30 (selection unit 403) determines whether an individual image is selected as an analysis target from the individual images and the connected images based on a user operation on the user interface. When the individual image is selected, the CPU 30 proceeds to S3. When the connected image is selected, the CPU 30 proceeds to S4.

  In S3, the CPU 30 (setting unit 402, image processing unit 410, and analysis unit 420) receives settings for individual images, and executes image processing and analysis processing based on the received settings. Thereafter, the CPU 30 proceeds to S5.

  In S <b> 4, the CPU 30 (setting unit 402, image processing unit 410 and analysis unit 420) accepts settings for linked images, and executes image processing and analysis processing based on the accepted settings. Thereafter, the CPU 30 proceeds to S5.

  In S <b> 5, the CPU 30 (analysis unit 420) displays the analysis result on the display unit 34 through the display control unit 440.

Individual Image Analysis FIG. 5 is a flowchart showing details of setting and analysis for an individual image. This process corresponds to S3.

  In S <b> 11, the CPU 30 (setting unit 402) displays a setting UI for performing settings necessary for individual image analysis on the display unit 34.

  In S12, the CPU 30 (well reception unit 404) receives the designation of the well.

  In S13, the CPU 30 (imaging position accepting unit 405) accepts designation of an imaging position within the designated well.

  In S <b> 14, the CPU 30 (setting unit 402) displays the setting individual image on the display unit 34.

  FIG. 6 is a diagram for explaining the setting UI 700. The setting UI 700 corresponds to the processes from S11 to S14. The setting UI 700 has a setting tab 701 for individual images and a setting tab 702 for linked images. The selection unit 403 determines whether an individual image has been selected by the user by recognizing the tab clicked by the pointer 708 among the setting tabs 701 and 702. The setting tab 701 has a well designation part 704, an imaging position designation part 706, and an image display area 709. The well designation unit 704 and the imaging position designation unit 706 may be referred to as a map image 703. The well designation unit 704 is an object that imitates the container unit 7. The well designating part 704 has six well parts 705. The well reception unit 404 recognizes the well part 705 clicked by the pointer 708 among the six well parts 705. The six well portions 705 correspond to the six wells in the container unit 7. The imaging position designation unit 706 corresponds to the well portion 705 designated by the well designation unit 704 and accepts designation of the imaging position 707 of the individual image. In this example, the rectangle indicating the imaging position 707 corresponds to one individual image (fluorescence image). That is, one rectangle corresponds to a single field of view for a high-magnification objective lens. The user clicks the imaging position 707 with the pointer 708 to specify the imaging position used for image processing setting and analysis setting. In this example, the imaging position in the center of the well is designated. The imaging control unit 401 converts the imaging position designated by the imaging position designation unit 706 within the well designated by the well designation unit 704 into coordinate data of the XY stage 6, and moves the XY stage 6 to the coordinate data The control unit 20 is instructed to execute imaging. The imaging control unit 401 may have a conversion table that converts imaging positions into XY coordinate data. The control unit 20 moves the XY stage 6 in accordance with this instruction, causes the imaging device 18 to perform imaging, and outputs the acquired fluorescent image data to the control device 2. The imaging control unit 401 stores the received image data in the storage device 31. In addition, when the image data of the designated imaging position has already been acquired, such image data acquisition processing may be omitted. The setting unit 402 displays a fluorescent image based on the image data at the designated imaging position in the image display area 709. The user determines an imaging position to be used for image processing setting and analysis setting while checking the fluorescent image.

  In S15, the CPU 30 (image processing setting reception unit 406, analysis setting reception unit 407) receives image processing settings and analysis settings applied to individual images.

  FIG. 7 shows a UI 800 for accepting image processing settings and the like. A button 801 is a button for starting image processing setting. A button 802 is a button for starting analysis setting. A button 803 is a button for starting analysis processing. A button 804 is a button for displaying the analysis result. An image display area 805 is an area for displaying an image before image processing is applied. An image display area 806 is an area for displaying an image after image processing is applied. A button 807 is a button 807 for performing detailed setting of image processing. The image processing setting accepting unit 406 is activated when a button 807 is pressed, and accepts various image processing settings. For example, the image processing settings include parameters relating to image brightness, black balance, presence / absence of an edge enhancement filter, presence / absence of a deconvolution filter, and the like. The image processing setting reception unit 406 stores the image processing setting in the setting data 433. The image processing unit 410 applies image processing to the fluorescent image at the designated imaging position in accordance with the image processing setting, and displays the fluorescent image to which image processing has been applied in the image display area 806. The buttons 801 to 804 may be replaced with a display object indicating whether the current process is image processing setting, analysis setting, analysis execution, or result display. In this case, a switching button (for example, a “next” button for proceeding to the next process or a “return” button for returning to the previous process) for switching the image data at the designated imaging position is displayed on the UI 800. It may be provided.

  FIG. 8 shows a UI 800 for accepting analysis settings and the like. When the button 802 is operated by the pointer 708, the setting unit 402 displays the UI 800 shown in FIG. The analysis setting reception unit 407 displays the fluorescence image before the analysis setting is applied in the image display area 815. That is, the image displayed in the image display area 815 is a fluorescent image to which image processing is applied. The analysis setting reception unit 407 receives an analysis setting when the button 807 is operated. For example, when a cell count that counts the number of cells (cells) included in the sample 3 is designated as the analysis process, the analysis setting reception unit 407 receives a threshold value for binarizing the fluorescence image. When color separation is adopted as the analysis processing, the analysis setting reception unit 407 receives settings necessary for color separation. By performing color separation, the fluorescent image is separated into a plurality of colors, and the area for each color is calculated. When the detailed setting of the analysis process is completed, the analysis setting reception unit 407 stores the analysis setting in the setting data 433. The analysis unit 420 executes analysis processing according to the analysis setting. The analysis unit 420 displays the fluorescence image to which the analysis setting is applied in the image display area 816. In this example, a fluorescence image binarized for cell counting is displayed in the image display area 816.

  In S16, the CPU 30 (well reception unit 404, imaging position reception unit 405) receives designation of a well to be analyzed and designation of an imaging position in the well. In S12 and S13, a well and an imaging position for designating a setting image are accepted, but in S16, one or more wells and one or more imaging positions to be actually analyzed are designated.

  In S <b> 17, the CPU 30 (the image processing unit 410) applies image processing based on the designated image processing setting to the fluorescent image at the imaging position in the well designated as the analysis target.

  In S <b> 18, the CPU 30 (analysis unit 420) applies analysis processing based on the designated analysis setting to the fluorescence image at the imaging position in the well designated as the analysis target.

  FIG. 9 shows a UI 800 for designating an analysis target. When the button 803 is operated by the pointer 708, the setting unit 402 displays the UI 800 shown in FIG. The well reception unit 404 receives a well corresponding to one or more well portions 705 clicked by the pointer 708 as an analysis target. In this example, two wells are designated. The imaging position accepting unit 405 accepts designation of an imaging position to be analyzed for each designated well. In this example, two imaging positions are designated as analysis targets. The image processing unit 410 instructs the imaging control unit 401 to image at the specified imaging position. The imaging control unit 401 converts the imaging position designated by the imaging position designation unit 706 into coordinate data of the XY stage 6, and instructs the control unit 20 to perform imaging by moving the XY stage 6 to the coordinate data. . The control unit 20 moves the XY stage 6 in accordance with this instruction, causes the imaging device 18 to perform imaging, and outputs the acquired fluorescent image data to the control device 2. The imaging control unit 401 stores the received image data in the storage device 31. Note that when the image data at the designated imaging position has been acquired, the image processing unit 410 may simply read the image data from the storage device 31. The image processing unit 410 applies image processing corresponding to the image processing setting to the image data. Further, the analysis unit 420 executes an analysis process corresponding to the analysis setting. The analysis unit 420 displays the fluorescent image at the imaging position last clicked in the display area 836. Image processing and analysis processing are applied to this fluorescent image. Thus, common image processing settings and common analysis settings are applied to a plurality of fluorescent images (individual images).

  FIG. 10 shows a UI 800 for displaying the analysis result. When the button 804 is operated by the pointer 708, the analysis unit 420 displays the UI 800 illustrated in FIG. 10 on the display unit 34. The first result display area 831 is an area for displaying the analysis result in a graph format. Graph formats include histograms and bar graphs. The second result display area 832 is an area for displaying an analysis result (for example, a cell count value) for each well.

Linked Image Analysis FIG. 11 is a flowchart showing details of setting and analyzing a linked image. This process corresponds to S4.

  In S <b> 21, the CPU 30 (setting unit 402) displays a setting UI for performing settings necessary for individual image analysis on the display unit 34.

  In S22, the CPU 30 (well reception unit 404) receives the designation of the well.

  FIG. 12 is a diagram for explaining the setting UI 700. In addition, the same referential mark is provided to the already demonstrated part, The description is used. A setting tab 702 of the setting UI 700 is a tab for performing settings related to the linked image. A setting tab 702 corresponds to the processing from S21 to S22. When the setting tab 702 is clicked with the pointer 708, the selection unit 403 determines that the connected image has been selected by the user. The well reception unit 404 recognizes the well part 705 clicked by the pointer 708 among the six well parts 705. The imaging position designation unit 706 corresponds to the well portion 705 designated by the well designation unit 704, and accepts designation of the imaging positions 707 of a plurality of individual images for forming a connected image. In this example, nine imaging positions (individual images) included in a rectangle drawn by the pointer 708 are designated. The connection setting accepting unit 408 designates the designated nine imaging positions to the imaging control unit 401 and instructs the imaging of nine fluorescent images. When the image data at the designated imaging position has been acquired, the connection setting reception unit 408 may simply read out the image data from the storage device 31. As a result, the image data of nine fluorescent images is stored in the storage device 31. The connection setting reception unit 408 may generate nine connected images by connecting nine fluorescent images and display them in the image display area 709.

  In S23, the CPU 30 (connection setting reception unit 408) receives the connection setting.

  FIG. 13 is a diagram illustrating a UI 900 for accepting connection settings. The image display area 901 is an area for displaying a plurality of fluorescent images designated through the imaging position designation unit 706. The connection setting reception unit 408 may exclude the fluorescence image clicked by the pointer 708 from the plurality of fluorescence images displayed in the image display area 901 from the connection image. As a result, fluorescent images that reduce the accuracy of analysis are excluded in advance. The shading correction setting unit 902 receives designation of whether to enable shading correction for a plurality of fluorescent images when generating a connected image. Further, when the shading correction has a plurality of correction modes, the shading correction setting unit 902 may accept selection of the correction mode. The image quality selection unit 903 is a UI for selecting whether to enable information compression processing. Since the connected image is formed from a plurality of high-magnification images, the information amount of the connected image tends to increase. Therefore, by applying information compression, the information amount of the connected image is reduced. However, the information compression may erase a small observation object included in the connected image. When non-compression is selected, a small observation object included in the connected image is maintained. The position correction setting unit 904 sets whether to correct the position of each fluorescent image when connecting a plurality of fluorescent images. When the position correction is set to be effective, the connecting unit 411 corrects the positions of the plurality of fluorescent images by paying attention to the similarity between the end regions of the plurality of adjacent fluorescent images. A plurality of adjacent fluorescent images may show the same cell or the like. Therefore, the connecting unit 411 corrects the positions of the plurality of fluorescent images in the connected image so that the same cell does not become a plurality of cells in the connected image.

  The connection setting reception unit 408 stores the connection setting designated through the UI 900 in the setting data 433. When the connection start button 905 is clicked with the pointer 708, the CPU 30 proceeds to S24.

  In S <b> 24, the CPU 30 (connecting unit 411) generates a single connected image by connecting a plurality of fluorescent images based on the connection setting, and stores them in the storage device 31. Thereby, a connected image for image processing setting and analysis setting is completed.

  In S <b> 25, the CPU 30 (image processing setting reception unit 406) receives an image processing setting for a connected image.

  FIG. 14 shows a UI 800 for accepting image processing settings for connected images. 14 is different from FIG. 7 in that a connected image before image processing is displayed in the image display area 805 and a connected image after image processing is displayed in the image display area 806. A button 801 is a button for starting image processing setting for a connected image. The image processing setting accepting unit 406 is activated when the button 807 is pressed, and accepts various image processing settings for connected images. For example, the image processing settings include parameters relating to image brightness, black balance, presence / absence of an edge enhancement filter, presence / absence of a deconvolution filter, and the like. The image processing setting reception unit 406 stores the image processing setting in the setting data 433. The image processing unit 410 applies image processing to the connected image according to the image processing setting, and displays the connected image to which the image processing is applied in the image display area 806. Thereby, the user can confirm whether the content of the image processing setting is appropriate.

  In S <b> 26, the CPU 30 (analysis setting reception unit 407) receives analysis settings for the connected images.

  FIG. 15 shows a UI 800 for accepting analysis settings for connected images. When the button 802 is operated by the pointer 708, the setting unit 402 displays the UI 800 shown in FIG. 15 is different from FIG. 8 in that a connected image before analysis is displayed in the image display area 815 and a connected image after analysis is displayed in the image display area 816. When the button 807 is operated, the analysis setting reception unit 407 receives an analysis setting for a connected image. For example, when a cell count that counts the number of cells (cells) included in the sample 3 is designated as the analysis process, the analysis setting reception unit 407 receives a threshold value for binarizing the connected image. When the detailed setting of the analysis processing is completed, the analysis setting receiving unit 407 stores the analysis setting for the linked image in the setting data 433. The analysis unit 420 executes analysis processing according to the analysis setting. The analysis unit 420 displays the connected image to which the analysis setting is applied in the image display area 816. When the button 803 for instructing execution of analysis is clicked, the CPU 30 proceeds to S27.

  In S27, the CPU 30 (well reception unit 404) receives designation of a well to be analyzed.

  FIG. 16 shows a UI 800 for designating an analysis target. 16 is different from FIG. 9 in that a connected image formed by connecting a plurality of fluorescent images included in the connected range specified by the imaging position specifying unit 706 is displayed in the image display area 826. When the button 803 is operated by the pointer 708, the setting unit 402 displays the UI 800 shown in FIG. The well reception unit 404 receives a well corresponding to one or more well portions 705 clicked by the pointer 708 as an analysis target. The imaging position designation unit 706 may accept designation of a connection range for each well. Alternatively, the connection range may be common among a plurality of wells.

In S <b> 28, the CPU 30 (connecting unit 411) generates a connected image for each well by connecting a plurality of fluorescent images in the well designated as the analysis target. The connection unit 411 passes the position of each well and the imaging position of each fluorescent image in the connection setting to the imaging control unit 401. The imaging control unit 401 converts the position of each well and the imaging position of each fluorescent image in the connection setting into coordinate data of the XY stage 6, and passes the coordinate data to the control unit 20. The control unit 20 causes the imaging device 18 to capture a fluorescent image while moving the XY stage 6 according to the coordinate data. As a result, image data of a plurality of fluorescent images is stored in the storage device 31. Note that when the image data at the designated imaging position has been acquired, the connecting unit 411 may simply read out the image data from the storage device 31. The connecting unit 411 reads the image data of a plurality of fluorescent images for each well, connects the plurality of fluorescent images based on the connection setting, and generates a connected image. The image data of the connected image is stored in the storage device 31. In S29, the CPU 30 (image processing unit 410) executes image processing on each connected image according to the common image processing setting.

  In S30, the CPU 30 (analysis unit 420) executes an analysis process on each connected image according to the common analysis setting. Thereafter, the CPU 30 proceeds to S5.

  FIG. 17 shows a UI 800 for displaying the analysis result. When the button 804 is operated by the pointer 708, the analysis unit 420 displays the UI 800 illustrated in FIG. 17 on the display unit 34. The first result display area 831 is an area for displaying the analysis result of the connected image in a graph format. Graph formats include histograms and bar graphs. The second result display area 832 is an area for displaying the analysis result (for example, cell count value) of the connected image for each well. A connected image is displayed in the image display area 816.

<Summary>
The microscope 1 is an example of a fluorescence microscope. Sample 3 is an example of an observation object. The XY stage 6 is an example of a stage on which an observation object is placed. The fluorescent epi-illumination light source 8 is an example of an irradiating unit that irradiates an observation target placed on a stage with excitation light. The imaging device 18 is an example of an imaging unit that receives fluorescence emitted from an observation object irradiated with excitation light and acquires a fluorescence image of the observation object. The motor group 21 is an example of a moving unit that relatively moves the stage and the imaging unit within a predetermined movable range in the XY plane coordinates.

  The control device 2 is an example of an image analysis device. The display unit 34 is an example of a display unit that displays a fluorescent image. The CPU 30 (the image processing unit 410 and the analysis unit 420 is an example of an analysis unit that applies image processing to a fluorescence image acquired by a fluorescence microscope and executes analysis processing of the fluorescence image to which the image processing is applied. The connecting unit 411 is an example of a connecting unit that connects a plurality of fluorescent images to generate a connected image.

  The operation unit 33 and the setting unit 402 are an example of a reception unit that receives selection of which of an individual image and a connected image is to be analyzed. An individual image is an image acquired by imaging an observation object in a single imaging field of imaging means. A connected image is an image generated by connecting together a plurality of fluorescent images acquired by the image pickup means and having adjacent image pickup positions by a connection process. The display unit 34 and the display control unit 440 are configured to display a map image that suggests imaging positions of a plurality of fluorescent images. FIG. 6 shows an example of the map image.

  When an individual image is selected as an analysis target through the accepting unit, the setting unit 402 designates a fluorescent image displayed on the display unit for setting image processing and analysis processing through the map image displayed on the display unit. Accept. Furthermore, the setting unit 402 sets image processing to be applied to the fluorescence image and analysis processing to the observation object that is reflected in the fluorescence image to which image processing based on the image processing setting is applied. Accept. Further, the setting unit 402 accepts designation of one or more fluorescent images to be applied to image processing and analysis processing through the map image displayed on the display unit. The analysis unit 420 and the image processing unit 410 are configured to perform image processing and analysis processing based on image processing settings and analysis processing settings received by the reception unit for one or more fluorescent images to be applied to the image processing and analysis processing. Apply.

  When the connected image is selected as an analysis target through the accepting unit, the setting unit 402 accepts designation of a plurality of fluorescent images that are the basis of the connected image through the map image displayed on the display unit. The setting unit 402 displays the connected image generated by the connecting unit on the display unit, and sets the image processing applied to the connected image and the connected image to which the image processing based on the image processing setting is applied. The setting of the analysis processing for the observed observation object is accepted. Furthermore, the setting unit 402 receives designation of one or more observation positions to be applied to the connection process and the image process and the analysis process through the map image displayed on the display unit. In the analysis unit 420 and the image processing unit 410, the analysis unit specifies a plurality of fluorescent images that are the basis of the connected image received by the reception unit, sets image processing, and the fluorescence image for each observation position. Based on the setting of analysis processing, connection processing, image processing, and analysis processing for generating a connection image are applied.

  As shown in FIG. 6 and the like, the map image 703 includes an image (well portion 705) indicating the positions of a plurality of wells that accommodate different observation objects. The well reception unit 404 of the setting unit 402 receives designation of a well to be analyzed among a plurality of wells. Since the image imitating the container unit 7 is displayed on the display unit 34 in this manner, the user can easily specify his intended well.

  As shown in FIG. 6 and the like, the map image 703 may include an image (imaging position designation unit 706) indicating a plurality of imaging positions in a single well. The setting unit 402 receives designation of an imaging position to be analyzed among a plurality of imaging positions. As described above, since a plurality of imaging positions in a single well are imitated, an image is displayed, so that the user can easily specify the intended imaging position.

  As shown in FIG. 6 and the like, the imaging position 707 may be indicated by a rectangle corresponding to one fluorescent image. Since the fluorescent image is a rectangular image, the user will easily understand the imaging position 707 and a single visual field range (range of individual images) from the rectangle.

  Although examples of analysis results are shown in FIGS. 10 and 17, the display unit 34 may display the analysis results for each designated well. For example, when the observation conditions of the sample 3 are different for each well, the user can easily identify the analysis result for each well.

  As illustrated in FIG. 13, the connection setting reception unit 408 may receive a selection as to whether or not to perform image information compression in the connection process. As a result, the user can select whether to give priority to the large amount of information or to give priority to the small burden on the storage capacity of the storage device 31.

  As shown in FIG. 5, when an individual image is selected as an analysis target, the setting unit 402 accepts designation of a well used for setting of image processing and setting of analysis processing in S12, and in S13 The designation of a plurality of imaging positions is accepted, and a fluorescent image corresponding to the imaging position is displayed on the display means in S14. Further, the setting unit 402 receives image processing settings through the fluorescent image displayed on the display unit in S15, and receives analysis processing settings through the fluorescent image displayed on the display unit to which image processing is applied. Accepts designation of a well to be subjected to image processing and analysis processing. When the individual image is selected in this way, the user can determine the image processing setting and the analysis setting according to the setting route prepared for the individual image.

  As shown in FIG. 11, when a connected image is selected as an analysis target, the setting unit 402 accepts designation of wells used for setting of connection processing, setting of image processing, and setting of analysis processing in S22. In step S23, designation of a plurality of imaging positions in the well may be received in order to specify a plurality of fluorescent images to be connected by the connecting process. The setting unit 402 causes the display unit to display a connected image generated by connecting the plurality of fluorescent images specified according to the designation of the plurality of imaging positions in S24. The setting unit 402 receives image processing settings through the connected image displayed on the display unit in S25, receives analysis processing settings through the connected image applied to the image processing displayed on the display unit in S26, and in S27. You may receive designation | designated of the well used as the object of a connection process, an image process, and an analysis process. When a connected image is selected in this way, the user can determine an image processing setting and an analysis setting according to a setting route prepared for the connected image.

Claims (10)

A stage on which the observation object is placed, irradiation means for irradiating the observation object placed on the stage with excitation light, and fluorescence emitted from the observation object irradiated with the excitation light. An imaging means for acquiring a fluorescence image of the observation object; and a moving means for relatively moving the stage and the imaging means within a predetermined movable range in XY plane coordinates;
Display means for displaying the fluorescence image, analysis means for applying image processing to the fluorescence image acquired by the fluorescence microscope, and executing analysis processing of the fluorescence image to which the image processing is applied, and a plurality of fluorescence An image analysis apparatus comprising: a coupling unit that couples images to generate a coupled image;
A microscope system comprising:
The image analysis device includes:
An individual image that is a fluorescent image acquired by imaging the observation object in a single imaging field of view of the imaging unit and a plurality of fluorescent images that are adjacent to each other and acquired by the imaging unit Receiving means for accepting selection of which of the connected images generated by connecting by the analysis target to be analyzed;
The display means is configured to display a map image that suggests imaging positions of a plurality of fluorescent images,
When the individual image is selected as an analysis target through the reception unit,
The accepting means is
Through the map image displayed on the display means, the designation of the fluorescence image displayed on the display means for accepting the setting of the image processing and the analysis processing is accepted, and the image applied to the fluorescence image The setting of the processing and the setting of the analysis processing for the observation object reflected in the fluorescent image to which the image processing based on the setting of the image processing is applied are received, and the image is displayed through the map image displayed on the display means. Accepting the designation of one or more fluorescent images to be applied to the process and the analysis process;
The analysis means includes the image processing and the setting of the analysis processing based on the setting of the image processing and the setting of the analysis processing received by the reception means for one or more fluorescent images to be applied to the image processing and the analysis processing. Apply the analysis process,
When the connected image is selected as an analysis target through the receiving unit,
The accepting means is
Through the map image displayed on the display means, designation of a plurality of fluorescent images that are the basis of the connected image is received, the connected image generated by the connecting means is displayed on the display means, and the connected image is displayed on the connected image. The map displayed on the display unit, accepting the setting of the image processing to be applied and the setting of the analysis processing for the observation object in the connected image to which the image processing based on the setting of the image processing is applied Accepting designation of one or more observation positions to be applied to the connection process, the image process and the analysis process through an image;
The analysis unit specifies a plurality of fluorescent images that are the basis of the connected image received by the reception unit, the setting of the image processing, and the setting of the analysis processing with respect to the fluorescent image for each observation position. And applying the connection processing for generating the connection image, the image processing, and the analysis processing. The map image includes an image showing the positions of a plurality of wells each containing different observation objects,
The microscope system according to claim 1, wherein the receiving unit receives designation of a well to be analyzed among the plurality of wells. The map image includes an image showing a plurality of imaging positions in a single well,
The microscope system according to claim 2, wherein the accepting unit accepts designation of an imaging position to be analyzed among the plurality of imaging positions.   The microscope system according to claim 3, wherein the imaging position is indicated by a rectangle corresponding to one fluorescent image.   The microscope system according to any one of claims 2 to 4, wherein the display means displays an analysis result for each designated well.   The microscope system according to any one of claims 1 to 5, wherein the reception unit receives a selection as to whether or not to perform image information compression in the connection process. The accepting means is
When the individual image is selected as an analysis target,
Accepting the designation of the well used for the setting of the image processing and the setting of the analysis processing,
Accepting designation of a plurality of imaging positions in the well;
Displaying the fluorescent image corresponding to the imaging position on the display means;
Accept image processing settings through the fluorescent image displayed on the display means,
Accepting the setting of analysis processing through the fluorescence image to which the image processing is applied, displayed on the display means,
The microscope system according to any one of claims 1 to 6, wherein designation of a well to be subjected to the image processing and the analysis processing is received. The accepting means is
When the connected image is selected as an analysis target,
Accepting the designation of wells used for the setting of the connection processing, the setting of the image processing and the setting of the analysis processing,
Accepting designation of a plurality of imaging positions in the well in order to identify a plurality of fluorescent images linked by the linkage process;
Displaying the connected image generated by connecting the plurality of fluorescent images specified according to the designation of the plurality of imaging positions on the display means;
Accept image processing settings through the connected image displayed on the display means,
Accepting the setting of the analysis processing through the connected image to which the image processing is applied, displayed on the display means;
The microscope system according to any one of claims 1 to 7, wherein designation of a well to be a target of the connection process, the image process, and the analysis process is received. A stage on which the observation object is placed, irradiation means for irradiating the observation object placed on the stage with excitation light, and fluorescence emitted from the observation object irradiated with the excitation light. Acquired by a fluorescence microscope having imaging means for acquiring a fluorescence image of the observation object and moving means for relatively moving the stage and the imaging means within a predetermined movable range in XY plane coordinates. An image analysis apparatus for analyzing a fluorescent image,
Display means for displaying the fluorescent image;
Analyzing means for applying image processing to the fluorescence image acquired by the fluorescence microscope, and executing analysis processing of the fluorescence image to which the image processing is applied,
An individual image that is a fluorescent image acquired by imaging the observation object in a single imaging field of view of the imaging unit and a plurality of fluorescent images that are adjacent to each other and acquired by the imaging unit Accepting means for accepting selection of which of the connected images generated by concatenating by the analysis target,
Connecting means for connecting the plurality of fluorescent images to generate the connected image;
Have
The display means is configured to display a map image that suggests imaging positions of a plurality of fluorescent images,
When the individual image is selected as an analysis target through the reception unit,
The accepting means is
Through the map image displayed on the display means, the designation of the fluorescence image displayed on the display means for accepting the setting of the image processing and the analysis processing is accepted, and the image applied to the fluorescence image The setting of the processing and the setting of the analysis processing for the observation object reflected in the fluorescent image to which the image processing based on the setting of the image processing is applied are received, and the image is displayed through the map image displayed on the display means. Accepting the designation of one or more fluorescent images to be applied to the process and the analysis process;
The analysis means includes the image processing and the setting of the analysis processing based on the setting of the image processing and the setting of the analysis processing received by the reception means for one or more fluorescent images to be applied to the image processing and the analysis processing. Apply the analysis process,
When the connected image is selected as an analysis target through the receiving unit,
The accepting means is
Through the map image displayed on the display means, designation of a plurality of fluorescent images that are the basis of the connected image is received, the connected image generated by the connecting means is displayed on the display means, and the connected image is displayed on the connected image. The map displayed on the display unit, accepting the setting of the image processing to be applied and the setting of the analysis processing for the observation object in the connected image to which the image processing based on the setting of the image processing is applied Accepting designation of one or more observation positions to be applied to the connection process, the image process and the analysis process through an image;
The analysis unit specifies a plurality of fluorescent images that are the basis of the connected image received by the reception unit, the setting of the image processing, and the setting of the analysis processing with respect to the fluorescent image for each observation position. Based on the above, an image analysis apparatus characterized by applying the connection processing for generating the connection image, the image processing, and the analysis processing. A stage on which the observation object is placed, irradiation means for irradiating the observation object placed on the stage with excitation light, and fluorescence emitted from the observation object irradiated with the excitation light. Acquired by a fluorescence microscope having imaging means for acquiring a fluorescence image of the observation object and moving means for relatively moving the stage and the imaging means within a predetermined movable range in XY plane coordinates. An image analysis method for analyzing a fluorescent image,
An individual image that is a fluorescent image acquired by imaging the observation object in a single imaging field of view of the imaging unit and a plurality of fluorescent images that are adjacent to each other and acquired by the imaging unit An accepting step for accepting selection of which of the connected images generated by connecting with the analysis target is to be analyzed;
Displaying a map image suggesting imaging positions of a plurality of fluorescent images on a display means;
Applying an image process to the fluorescence image acquired by the fluorescence microscope, and performing an analysis process of the fluorescence image to which the image process is applied;
Have
In the reception step, when the individual image is selected as an analysis target,
Through the map image displayed on the display means, the designation of the fluorescence image displayed on the display means for accepting the setting of the image processing and the analysis processing is accepted, and the image applied to the fluorescence image The setting of the processing and the setting of the analysis processing for the observation object reflected in the fluorescent image to which the image processing based on the setting of the image processing is applied are received, and the image is displayed through the map image displayed on the display means. Receiving a designation of one or more fluorescent images to be applied to the process and the analysis process,
The analysis step includes the image processing and the analysis processing setting based on the image processing setting and the analysis processing setting received in the reception step for one or more fluorescent images to be applied to the image processing and the analysis processing. Having a process of applying analysis processing;
In the receiving step, when the connected image is selected as an analysis target,
Through the map image displayed on the display means, designation of a plurality of fluorescent images that are the basis of the connected image is received, the connected image is displayed on the display means, and the image processing applied to the connected image is performed. Accepting setting and setting of analysis processing for the observation object reflected in the connected image to which image processing based on the setting of the image processing is applied, and through the map image displayed on the display means, Receiving a designation of one or more observation positions to be applied to the image processing and the analysis processing,
In the analysis step, for the fluorescence image for each observation position, designation of a plurality of fluorescence images that are the basis of the connected image received in the reception step, setting of the image processing, and setting of the analysis processing And a process of applying the connection processing for generating the connection image, the image processing, and the analysis processing.