JP5826561B2 - Microscope system, specimen image generation method and program - Google Patents

Description

  The present invention relates to a microscope system that acquires and displays a specimen image obtained by imaging a specimen using a microscope, a specimen image generation method, and a program.

  When a specimen is observed using a microscope, the range that can be observed at a time is mainly determined by the magnification of the objective lens. For example, if the magnification of the objective lens is increased, a high-definition image can be obtained, but the observation range is narrowed. Therefore, while moving the field of view using an electric stage, etc., a plurality of images are taken and pasted (or combined) to create a wide-field and high-resolution microscope image for pathological diagnosis, etc. A system (hereinafter referred to as a virtual microscope system) that is utilized in the field is known.

  For example, in Patent Documents 1 and 2, the range of a sample is divided into small sections, and the images obtained by photographing the sample portions corresponding to the respective small sections with a high-resolution objective lens are joined together to expand the range. A microscope system for constructing an image of the entire field of view and high resolution specimen is disclosed. Hereinafter, the microscope constructed in this way is referred to as a virtual slide (VS) image. Similarly, Patent Documents 3 and 4 disclose a microscope system that creates a microscope image with a wide field of view and high resolution.

  In this way, the VS image obtained by imaging the entire specimen with high definition can be browsed regardless of location and time if there is a personal computer and a network environment. For this reason, it is beginning to be used for medical students' pathology training and remote consultation between pathologists. Note that the VS image is also referred to as Whole Slide Imaging.

  By the way, in the pathological observation, various staining methods and observation methods are used according to the observation object and purpose. For example, as morphological observation staining for observing the morphology of tissues or cells, non-such as hematoxylin and eosin staining (hereinafter referred to as “HE staining”) using two pigments of hematoxylin and eosin, Papanicolaou staining (Pap staining), etc. Fluorescent staining is known. Bright field observation using an optical microscope is performed on a specimen that has been subjected to such morphological observation staining.

  In pathological observation, for the purpose of complementing morphological diagnosis based on morphological information, the target is stained with molecular target staining to confirm the expression of molecular information and functions such as abnormal expression of target molecules (specific genes and proteins) Molecular pathological tests may be performed to diagnose abnormalities. In this case, for example, the sample is fluorescently labeled (stained) by IHC (immunohistochemistry) method, ICC (immunocytochemistry) method, ISH (in situ hybridization) method, etc., and fluorescence observation is performed, Enzyme labeling and bright field observation.

  Further, in pathological observation, diagnosis may be performed by using the above-described method together. For example, it is possible to diagnose morphological abnormalities by bright field observation of specimens subjected to HE staining, etc., and to diagnose functional abnormalities such as abnormal molecular expression by fluorescence observation (FISH method (fluorescence in situ hybridization, fluorescent antibody method)). is there.

  In recent years, a treatment called a molecular target therapy using a therapeutic agent (antibody therapeutic agent) that acts on a specific molecule as a target has been performed as a treatment for cancer and the like, and a therapeutic effect and an effect of reducing side effects are expected. ing. In this molecular target therapy, an antibody therapeutic drug that targets a molecule (antigen protein) specifically expressed in cancer cells is used. When such molecular target therapy is performed, prior to the treatment, for example, the IHC method is used to observe whether or not an antigen serving as a target molecule of an antibody therapeutic drug is expressed on the cell surface, that is, on the cell membrane. Patient selection is made.

  For example, in breast cancer treatment, targeted treatment using an antibody therapeutic agent is progressing, and the disease type (cell subtype) is changed to “Luminal B” or “Luminal” according to the expression pattern of a plurality of target molecules in the tumor site. Classification is made into four types called “A”, “HER2 disease”, and “Basal like”, and based on this classification, a basic treatment is selected. Specifically, it depends on the presence or absence of expression on the cell nucleus of estrogen receptor (hereinafter abbreviated as “ER”) and progesterone receptor (hereinafter abbreviated as “PgR”), which are hormone receptors. Judge whether the cancer grows sexually or not, and select whether or not to apply endocrine therapy (hormone therapy). In addition, whether to apply trastuzumab (Herceptin (registered trademark)), which is an anti-HER2 antibody preparation, is selected depending on whether or not the HER2 receptor (hereinafter abbreviated as “HER2”) is expressed on the cell membrane. In a so-called triple negative breast cancer (TNBC) in which none of ER, PgR, and HER2 is expressed, chemotherapy is mainly performed.

  Thus, when visualizing a plurality of target molecules, fluorescence observation is useful, but on the other hand, there is a problem that a fluorescent specimen cannot be stored for a long period of time. In this regard, VS imaging of a specimen with a virtual microscope system also leads to the recording and maintenance of the basis of diagnosis and eliminates the need for consideration of problems related to long-term preservation of specimens.

  As a technique related to a VS image, Patent Document 5 discloses that a normal staining capable of bright field observation and a fluorescent label for a target molecule are applied to the same slide specimen, and a target molecule in an arbitrary region designated on the bright field VS image. Is disclosed as a fluorescence VS image.

Japanese Patent Laid-Open No. 9-281405 Japanese Patent Laid-Open No. 10-333056 JP 2006-343573 A JP-T-2002-514319 JP 2009-14939 A

  However, in fluorescence observation, since the field of view is darker than in bright field observation, the photographing time per shot becomes longer. Therefore, it usually takes several hours or more to convert the entire specimen into a VS image. In addition, the examination area required in the molecular pathological diagnosis is a limited area mainly such as a tumor part, and the normal part is often outside the examination target. For this reason, when the tumor part exists only in a very small region of the whole specimen, not only is a wasteful time spent acquiring the VS image, but also an unnecessary increase in the image capacity is caused. Furthermore, since the fluorescently stained specimen is colorless and transparent in the unexcited state, the fluorescent specimen in the unexcited state can be used even if the position of the tumor is identified by bright field observation on the specimen that has undergone normal morphological observation staining. It is very difficult to confirm the position and selectively image the tumor on the specimen.

  With respect to this point, in Patent Document 5 described above, since the region for performing fluorescence observation is designated on the bright field VS image, the processing time for creating the fluorescence VS image can be shortened. However, Patent Document 5 does not mention the difference in focus position (focus position) between bright field observation and fluorescence observation for each target molecule.

  In order to acquire the focus position in fluorescence observation, for example, it is conceivable to use the focus position in the bright field image as it is. However, in this case, the focus position may be shifted in the fluorescent image. Alternatively, it is conceivable that the autofocus process is performed every time in each shot of the fluorescent image. However, as described above, since the field of view during the fluorescence observation is dark, the autofocus process also takes time. For this reason, eventually, it takes a long time to generate the fluorescent VS image.

  The present invention has been made in view of the above, and performs fluorescence observation of a target molecule on a multi-stained specimen in which non-fluorescent staining capable of bright field observation and a fluorescent label for a target molecule are applied on the same slide specimen. It is an object of the present invention to provide a microscope system, a specimen image generation method, and a specimen image generation program that can accurately detect the in-focus position in a short time.

In order to solve the above-described problems and achieve the object, the microscope system according to the present invention is configured to place a microscope objective lens and specimen capable of bright field observation and fluorescence observation in a direction perpendicular to the optical axis of the objective lens. One or more cell components constituting a cell in a microscope system for acquiring a plurality of microscope image groups by relatively moving and generating a virtual slide image obtained by connecting the plurality of microscope image groups to each other Is used to control the acquisition of a bright-field virtual slide image in which the cell components are displayed by bright-field observation of a specimen stained with a non-fluorescent dye and the target molecule to be detected is labeled with the fluorescent dye. A visual virtual slide image acquisition control means and a fluorescence virtual slide image corresponding to the target molecule are obtained by fluorescence observation on the specimen. A fluorescent virtual slide image acquisition control means for controlling the Tokusuru, the fluorescent acquisition subject to regions of the virtual image from the plurality of small compartments divided in a grid pattern, bright field image used in obtaining the bright field image A plurality of small sections for actually measuring the in-focus position are extracted, and a first for further actual measurement of the in-focus position for the fluorescent image used when acquiring the fluorescent image from the extracted small sections. A plurality of small sections are extracted, the in-focus positions for the bright field image and the fluorescent image in each first small section are measured to calculate the difference between the two in-focus positions, and all the first small sections are extracted. The average value and the standard deviation of the difference in the section are calculated, and the first small section among the small sections for actually measuring the in-focus position for the bright field image when the standard deviation is equal to or smaller than a predetermined value. In the second sub-compartment other than Kicking wherein the focus position of the fluorescent image, further comprising a said second focus position acquiring means for calculating from the focus position for the bright field image that is actually measured for the small compartments and the average value And

  In the microscope system, the focus position acquisition unit extracts a plurality of extraction points for actually measuring the first focus position from within the sample, and measures the first focus positions at the plurality of extraction points by actual measurement. And acquiring the second in-focus position in at least a part of the plurality of extraction points by actual measurement.

  In the microscope system, when the focus position acquisition unit acquires the second focus position at each of the plurality of extraction points, a predetermined offset from the first focus position at the same extraction point. It is characterized by using.

  The microscope system further includes an optical cube switching unit that switches an optical cube selectively inserted into an optical path according to the specimen observation method, and the focus position acquisition unit acquires the first focus position. In this case, the first in-focus position is obtained by actual measurement in a state where the optical cube for observing the fluorescent image is inserted in the optical path.

  In the microscope system, the fluorescence virtual slide image acquisition control unit sets a plurality of coordinate values in the optical axis direction with respect to the first focus position, and acquires a plurality of fluorescence images at the plurality of coordinate values, respectively. Control is performed for each of a plurality of small sections obtained by dividing a plane orthogonal to the optical axis, and a plurality of fluorescences corresponding to the small sections adjacent to each other are arranged for each arrangement order of the plurality of coordinate values. A feature is that a plurality of fluorescent virtual images are created by connecting images to each other.

  The microscope system further includes image synthesizing means for creating a synthesized image obtained by superimposing the bright field virtual slide image and the fluorescence virtual slide image.

  In the microscope system, the specimen is labeled with two or more types of fluorescent dyes corresponding to two or more types of target molecules, respectively, and the fluorescence virtual slide image acquisition control means A fluorescence virtual slide image is acquired.

  The microscope system may further include an area setting unit that sets an area from which the fluorescent virtual slide image is to be acquired, using the bright field virtual slide image.

In the microscope system, the cell component is a cell nucleus.
In the above microscope system, the cell component is stained with hematoxylin.

A specimen image generation method according to the present invention includes a microscope objective lens and specimen capable of bright field observation and fluorescence observation, which are moved relative to each other in a direction perpendicular to the optical axis of the objective lens, and a plurality of microscope image groups. And a specimen image generation method for generating a virtual slide image obtained by connecting the plurality of microscope image groups to each other, and one or more cell components constituting the cell are stained with a non-fluorescent dye, and Bright field virtual slide image acquisition control step for performing control to acquire a bright field virtual slide image in which the cell components are displayed by bright field observation on a specimen in which a target molecule to be detected is labeled with a fluorescent dye, and A fluorescence virtual slider that performs control to acquire a fluorescence virtual slide image corresponding to the target molecule by fluorescence observation of the specimen. And de image acquisition control step, wherein the acquisition target and a region of the fluorescent virtual image from the plurality of small compartments divided in a grid pattern, the focus position for the bright field image used in obtaining the bright field image A plurality of small sections for actual measurement are extracted, and a plurality of first subsections are further extracted from the extracted small sections for further actual measurement of the focus position for the fluorescent image used when acquiring the fluorescent image. , Measuring the in-focus positions for the bright field image and the fluorescent image in each first small section to calculate a difference between the two in-focus positions, and averaging the differences in all the first sub-sections When a value and a standard deviation are calculated, and the standard deviation is equal to or smaller than a predetermined value, a second small block other than the first small block among the small blocks for actually measuring the focus position for the bright field image is calculated. The in-focus position for the fluorescent image in the compartment Characterized in that it comprises a focus position acquiring step of calculating from an in-focus position for actually measured the bright field image with respect to the second subsection and the average value.

The sample image generation program according to the present invention is configured to move a microscope objective lens and a specimen capable of bright field observation and fluorescence observation relative to each other in a direction perpendicular to the optical axis of the objective lens, thereby obtaining a plurality of microscope image groups. And a sample image generation program that causes a computer to execute a process of generating a virtual slide image obtained by connecting the plurality of microscope image groups to each other, wherein one or more cell components constituting the cell are non-fluorescent dyes Bright field virtual slide image that performs control to obtain a bright field virtual slide image in which the cell components are displayed by bright field observation on a sample that is stained by the above and the target molecule to be detected is labeled with a fluorescent dye A fluorescence virtual slide corresponding to the target molecule by an acquisition control step and fluorescence observation on the specimen A fluorescent virtual slide image acquisition control step performs control to acquire the image, from among the fluorescent plurality of small sections the acquisition target and a region is divided into lattice-like virtual image, and is used to acquire the bright-field image To extract a plurality of small sections for measuring the in-focus position for the bright field image, and to further measure the in-focus position for the fluorescent image used when acquiring the fluorescent image from the extracted small sections A plurality of first subsections are extracted, the in-focus positions for the bright field image and the fluorescent image in each first subsection are measured, and the difference between the two in-focus positions is calculated. An average value and a standard deviation of the differences in one small section are calculated, and when the standard deviation is a predetermined value or less, the first section among the small sections for actually measuring the in-focus position for the bright field image. 2nd other than a small parcel of The focus position of the fluorescent image, include a said second focus position acquisition step of calculating from the measured in-focus position for the bright field image relative to the small section to the average value in subsection It is characterized by.

  According to the present invention, a first in-focus position used in bright field observation is acquired, and a second in-focus position used in fluorescence observation is acquired using the first in-focus position. As a result, it is possible to accurately detect the in-focus position in the fluorescence observation of the multiple stained specimen in a short time.

FIG. 1 is a diagram illustrating a configuration example of the entire microscope system according to Embodiment 1 of the present invention. FIG. 2 is a block diagram showing a configuration example of the host system shown in FIG. FIG. 3 is a flowchart showing the operation of the microscope system shown in FIG. FIG. 4 is a top view showing the slide glass on which the specimen is arranged. FIG. 5 is a schematic diagram showing a sample area detected from the sample search area. FIG. 6A is a schematic diagram showing small sections and focus position extraction points obtained by dividing the region of interest into a grid. FIG. 6B is a schematic diagram showing small sections and focus position extraction points obtained by dividing the region of interest into a grid. FIG. 7 is a diagram illustrating an example of a focus map. FIG. 8 is a schematic diagram illustrating a data configuration example of an image file. FIG. 9 is a flowchart showing focus map creation processing. FIG. 10 is a flowchart showing processing for actually measuring the in-focus position at point B. FIG. 11 is a flowchart illustrating focus map creation processing according to the second embodiment. FIG. 12 is a schematic diagram illustrating a configuration of a three-dimensional image created in the third embodiment. FIG. 13 is a flowchart illustrating a fluorescence image acquisition process according to the third embodiment.

Hereinafter, embodiments according to the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to these embodiments.
The microscope system according to the present invention records and displays a multiple-stained pathological tissue specimen or cytological specimen as a digital image with a wide field of view and high definition using a microscope. More specifically, the microscope system according to the present invention uses a specimen in which morphological observation staining and molecular target staining are multiply stained for observation and diagnosis.

  Below, the case where this invention is applied to the microscope system (virtual microscope system) which produces | generates a virtual slide image is demonstrated to an example. A virtual slide image (hereinafter also referred to as a VS image) refers to moving the objective lens of the microscope and the specimen in a direction perpendicular to the optical axis of the objective lens by moving an electric stage on which the specimen is placed. The image is generated by connecting a plurality of images obtained by capturing the specimen for each part while shifting the visual field range.

(Embodiment 1)
FIG. 1 is a diagram illustrating a configuration example of the entire microscope system according to the first embodiment.
A microscope system 1 shown in FIG. 1 includes a microscope apparatus 10, a host system 20 connected to the microscope apparatus 10 so as to be able to transmit and receive data, and a microscope controller 30 that controls each part of the microscope apparatus 10 under the control of the host system 20. A television (TV) camera controller 40 connected to the host system 20; and a television (TV) camera 50 attached to the microscope apparatus 10 and operating under the control of the TV camera controller 40.

  The microscope apparatus 10 includes an electric stage 11 on which a slide glass 100 on which a specimen S is placed, a microscope body 12 having a substantially U-shape in side view and supporting the electric stage 11, and a bottom portion of the microscope body 12. A transmission illumination optical system 13 for transmitting and illuminating the specimen S; an epi-illumination optical system 14 for illuminating the specimen S by epi-illumination; and a microscope body 12 that is rotatably provided. The revolver 15 to which the objective lens 151 is attached, the optical cube switching unit 16 for switching a plurality of optical cubes 161 selectively inserted in the optical path L of the observation light of the sample S, and the microscope body 12 are mounted on the upper part. The lens barrel 17 is provided. In the following description, it is assumed that the optical axis direction of the objective lens 151a shown in FIG. 1 is the Z direction, and the plane orthogonal to the Z direction is the XY plane. In the Z coordinate, the direction in which the sample surface (the sample placement surface of the electric stage 11) is separated from the objective lens 151 is defined as a negative direction, and the direction approaching the sample surface is defined as a positive direction.

  The electric stage 11 is driven by a motor 32 that operates under the control of the stage XY drive control unit 31 and a motor 34 that operates under the control of the stage Z drive control unit 33, and in the XY plane and in the Z direction. Move freely to each. The electric stage 11 has an origin position detection function (not shown) by an origin sensor, and can set coordinates for each part of the slide glass 100 (specimen S) placed on the electric stage 11.

  The transmission illumination optical system 13 includes a transmission illumination light source 130, a collector lens 131 that collects illumination light emitted from the transmission illumination light source 130 (hereinafter referred to as transmission illumination light), a transmission filter unit 132, and a transmission shutter. 133, a transmission field stop 134, a transmission aperture stop 135, a bending mirror 136 that bends the optical path L1 of the transmitted illumination light along the optical axis of the objective lens 151a, a condenser optical element unit 137, and a top lens unit 138. Is provided.

  The epi-illumination optical system 14 includes an epi-illumination light source 140, a collector lens 141 that collects illumination light emitted from the epi-illumination light source 140 (hereinafter referred to as epi-illumination light), an epi-illumination filter unit 142, and an epi-illumination shutter. 143, an epi-illumination field stop 144, and an epi-illumination aperture stop 145.

  The revolver 15 is equipped with a plurality of objective lenses 151a, 151b, ... (collectively referred to as objective lenses 151) having different magnifications (observation magnifications). The objective lens 151 is selectively switched according to the rotation of the revolver 15 and inserted into the observation optical path L, and is used for observing the sample S. FIG. 1 shows a state in which the objective lens 151a is inserted in the observation optical path L.

  The revolver 15 includes an objective lens 151 having a relatively low magnification (for example, 2 × and 4 ×), an objective lens (hereinafter referred to as a low magnification objective lens), and a high magnification (a magnification that is higher than that of the low magnification objective lens). For example, at least one objective lens (hereinafter referred to as a high magnification objective lens) of 10 times, 20 times, and 40 times is held. Note that the low magnification and the high magnification are examples, and it is sufficient that at least one magnification is higher than the other magnification.

  The optical cube switching unit 16 holds a plurality of optical cubes 161a, 161b,... (These are also collectively referred to as an optical cube 161), and selects one of the optical cubes 161 according to the microscopic method. It is inserted into the observation optical path L (more specifically, the position where the observation optical path L and the optical path L2 of the epi-illumination light intersect).

  Each optical cube 161 includes an excitation filter that transmits light having the excitation wavelength of the fluorescent dye that stains the specimen S, an absorption filter that transmits light having a wavelength including the fluorescence wavelengths of all the fluorescent dyes that stain the specimen S, and a dichroic. The mirror is unitized. The optical cube 161 inserted in the observation optical path L reflects the excitation wavelength component of the fluorescent dye that stains the sample S out of the light emitted from the epi-illumination light source 140 and guides it in the direction of the sample S. Fluorescence wavelength component emitted from the specimen S is transmitted in the reflected light from the light and guided in the direction of the lens barrel 17.

  The lens barrel 17 is provided with an eyepiece, and the binocular unit 171 for visually observing the sample image of the sample S, the TV camera 50, and the observation light from the sample S on the binocular unit 171 side and the TV camera 50 side And a beam splitter 172 that branches into a line.

  Each part of the microscope apparatus 10 is electrically driven and operates under the control of the microscope controller 30. In addition, it is also possible for a user to operate each part of the microscope apparatus 10 manually.

  The host system 20 connects a CPU, a video board, a main storage device such as a main memory, an external storage device such as a hard disk and various storage media, a communication device, an output device such as a display device and a printing device, an input device, and each unit. Alternatively, it is realized by a known hardware configuration (for example, a general-purpose computer such as a workstation or a personal computer) provided with an interface device for connecting an external input.

  FIG. 2 is a block diagram showing the configuration of the host system 20. As shown in FIG. 2, the host system 20 includes an input unit 21, a display unit 22, an image data recording unit 23, a shooting coordinate recording unit 24, a program recording unit 25, and a video input from the TV camera 50. A video board 26 that processes signals and a control unit 27 that controls the operation of each of these units and the entire microscope system are provided. In addition, although not shown, the host system 20 includes an interface unit that manages the exchange of various data with each unit constituting the microscope system shown in FIG.

The input unit 21 is realized by an input device such as a mouse or a keyboard. The input unit 21 is used when a user inputs various information and commands to the host system 20.
The display unit 22 is realized by a display device such as an LCD or an EL display.

  The image data recording unit 23, the photographing coordinate recording unit 24, and the program recording unit 25 are, for example, various IC memories such as ROM and RAM such as update-recordable flash memory, built-in or external hard disk, or CD-ROM. This is realized by an information recording medium and a reading device thereof. The image data recording unit 23 records an image obtained by photographing the sample S with the TV camera 50 (hereinafter also referred to as an observation image). The photographing coordinate recording unit 24 records the XYZ coordinates when photographing the observation image of the sample S. The program recording unit 25 records various control programs for controlling the operation of the microscope system 1, data used during the execution of these control programs, and the like.

  The control unit 27 is realized, for example, by reading various control programs recorded in the program recording unit 25 into a CPU (Central Processing Unit) and executing them. The control unit 27 includes a fluorescence VS image area setting unit 271, a bright field VS imaging control unit 272 as a bright field VS image acquisition control unit, a fluorescence VS imaging control unit 273 as a fluorescence VS image acquisition control unit, and a focus position. The acquisition unit 274, the VS image creation unit 275, and the image composition unit 276 generate various control signals for controlling the operation of the microscope apparatus 10 and output the generated control signals to the microscope controller 30. In addition, the control unit 27 transmits a control signal to the TV camera controller 40 to cause the TV camera 50 to perform shooting.

  The fluorescence VS image region setting unit 271 sets a region on the specimen S (hereinafter also referred to as a region of interest) that is a target for acquiring a fluorescence VS image, in accordance with an input signal from the input unit 21 by a user operation.

  The bright-field VS imaging control unit 272 irradiates the specimen S with transmitted illumination light by the transmitted illumination optical system 13 and forms one or more cell components (cell nuclei, etc., which constitute cells stained with a non-fluorescent dye. A series of operations for acquiring a bright-field image corresponding to a cell component is also controlled.

  The fluorescence VS imaging control unit 273 controls a series of operations for irradiating the specimen S with epi-illumination light by the epi-illumination optical system 14 and acquiring a fluorescent image corresponding to the fluorescently labeled target molecule.

  The focus position acquisition unit 274 acquires the focus position when acquiring the bright field image and the fluorescence image of the sample S by actual measurement or calculation. More specifically, the focus position acquisition unit 274 acquires a focus position for a bright field image, and acquires a focus position for a fluorescent image using the focus position. The focus position is actually measured by evaluating the contrast of the image captured by the TV camera 50 for each Z coordinate while the focus position acquisition unit 274 moves the electric stage 11 in the Z direction via the microscope controller 30. Is called. This focusing operation is also called a video AF function.

The VS image creation unit 275 creates a bright-field VS image by connecting a plurality of bright-field images obtained by partially imaging the region of interest of the sample S with the microscope device 10 and also partially within the region of interest of the sample S. A plurality of fluorescent images picked up automatically are connected to create a fluorescent VS image.
The image composition unit 276 creates a composite image in which the bright field VS image and the fluorescence VS image are superimposed.

  The microscope controller 30 controls the operation of the entire microscope apparatus 10 according to a control signal from the host system 20. Specifically, the microscope controller 30 controls each part of the microscope apparatus 10 such as changing the spectroscopic method (bright field observation / fluorescence observation), dimming the transmitted illumination light source 130 and the incident illumination light source 140, Detection of the state of each part of the current microscope apparatus 10 and transmission to the host system 20 are executed. Further, the microscope controller 30 outputs a control signal for controlling the movement of the electric stage 11 to the stage XY drive control unit 31 and the stage Z drive control unit 33.

  The TV camera controller 40 controls the operation of the TV camera 50 according to a control signal from the host system 20. Specifically, automatic gain control ON / OFF, gain setting, automatic exposure control ON / OFF, exposure time setting, and the like are executed for the TV camera 50.

  The TV camera 50 includes an image sensor such as a CCD or a CMOS that forms a sample image (specifically, a visual field range of the objective lens 151), images the sample image, and outputs image data of the sample image to the host system 20. .

  Next, the operation of the microscope system 1 according to the first embodiment will be described. FIG. 3 is a flowchart showing the operation of the microscope system 1 according to the first embodiment. The operation of the microscope apparatus 10 described below is basically automatically performed through the microscope controller 30 under the control of the host system 20, but may be manually operated by the user.

First, in step S10, the microscope system 1 sets the observation method to bright field observation, and performs insertion / removal control of various optical members to and from the optical path in order to execute the transmission bright field observation method for each part of the microscope apparatus 10. Do. Accordingly, the epi-illumination shutter 143 is inserted into the optical path L2 for epi-illumination, the transmission shutter 133 on the optical path L1 for trans-illumination is opened, and the optical cube 161 for bright-field observation is selected and inserted into the observation optical path L. Is done. Thereby, the transmitted illumination light is irradiated onto the specimen S.
In subsequent step S11, the microscope apparatus 10 inserts a low magnification (for example, 4 times) objective lens 151 into the optical path.

In step S12, the microscope system 1 acquires a low-resolution bright-field image corresponding to the entire specimen S.
FIG. 4 is a top view showing the entire slide glass 100 on which the specimen S is arranged. In the following description, as an example of the specimen S, a breast cancer tissue specimen subjected to multiple staining is used. As multiple staining, hematoxylin staining is performed on cell nuclei as morphological observation staining, anti-ER antibody recognizing estrogen receptor (hereinafter also simply referred to as ER) is fluorescently labeled with Alexa555, and HER2 receptor is received. It is assumed that the anti-HER2 antibody that recognizes the body (hereinafter also simply referred to as HER2) is fluorescently labeled with Cy5. In the first embodiment, bright field (cell nucleus) observation and fluorescence observation are performed on such a specimen S. The label LB describes information related to the sample S.

  For such a sample S, the microscope apparatus 10 projects a predetermined sample search region R1 (for example, a region of 25 mm in length and 50 mm in width) on the slide glass 100 onto the TV camera 50. Is divided into a plurality of sections according to the imaging region. Note that the size of the imaging region (vertical × horizontal length) is determined by the magnification of the objective lens 151 inserted on the observation optical path L. Then, the microscope system 1 moves the electric stage 11 in the X and Y directions, sequentially captures each of these sections with the TV camera 50, and acquires image data of a plurality of microscope images (bright field images). The host system 20 generates an entire image of the specimen search region R1 by combining the plurality of microscope images with each other. Image data corresponding to the entire image is recorded in the image data recording unit 23 as a slide glass entire image file.

  In step S13, the microscope apparatus 10 rotates the revolver 15 and replaces the objective lens 151 with a high magnification (for example, 20 times). The reason why the lens is replaced with a high-magnification lens is to acquire a high-resolution cell nucleus image by bright field observation and a target molecule image by fluorescence observation.

  In step S14, the microscope system 1 sets a target region to be subjected to high-resolution bright-field observation and fluorescence observation in the specimen search region R1, and measures a focus position in the target region (hereinafter referred to as a focus position). (Referred to as a focus position extraction point).

  More specifically, the microscope system 1 automatically detects the sample region R2 including the sample S from the entire image of the sample search region R1 acquired in step S12 and displays it on the display unit 22. FIG. 5 is a schematic diagram showing a sample region R2 that is automatically detected. A known image processing method is used as an automatic detection method for the sample region R2.

  When the user designates rectangular attention regions R3 and R4 using the input unit 21 (for example, a pointing device such as a mouse) on the screen on which such a sample region R2 is displayed, the host system 20 displays the input unit 21. The coordinates of the attention areas R3 and R4 are temporarily stored in the imaging coordinate recording unit 24 in accordance with the input signal from. As the attention areas R3 and R4, an area including a part where the user visually recognizes an abnormality, such as a tumor part, is usually designated.

  The microscope system 1 may display the entire sample search region R1 on the display unit 22 when the user selects the attention regions R3 and R4, or displays the sample search region R1 on the display unit 22. In the above, the sample region R2 may be visually designated by the user.

  Further, as shown in FIGS. 6A and 6B, the microscope system 1 sets a plurality of small sections P by dividing each region of interest R3, R4 into a lattice shape. Each small section P corresponds to an imaging area of one shot projected on the TV camera 50, and the size (vertical × horizontal length) of each small section P is determined by the magnification of the objective lens 151. Therefore, the size of the small section P is smaller than one section captured in step S12.

Then, the microscope system 1 extracts a plurality of small sections (focus position extraction points) FP for actually measuring the in-focus position from these small sections P.
Here, if it is attempted to obtain the focus position for all the small sections P by actual measurement, it takes a very long time. Therefore, in the first embodiment, a small section that samples (actually measures) the focus position is automatically extracted from all of the small sections P, and the other small sections are calculated based on the sampled values. Ask. Note that the automatic extraction method of the focus position extraction point may be arbitrary, may be random, or may be regular (for example, every third section).

  The automatically extracted focus position extraction point FP is used as a focus position measurement point when a bright field image is acquired. Further, the focus position extraction point for the fluorescent image is preferably further narrowed down from the focus position extraction point FP for the bright field image. This is because fluorescence observation is basically performed in a dark field and imaging takes time, and the focus position measurement process requires several times to several tens of times longer than bright field observation. Therefore, in the first embodiment, the overall processing time is shortened by reducing the number of focus position extraction points for fluorescent images. The method for extracting the focus position extraction point for the fluorescent image from the focus position extraction point FP for the bright field image may be arbitrary, may be extracted randomly, or may be extracted by regular thinning. . Hereinafter, among the focus position extraction points FP extracted for the bright field image, the points extracted as the focus position extraction points for the fluorescent image are referred to as points A, and the points that are not extracted are referred to as points B.

  In subsequent step S15, the microscope system 1 creates a focus map for a bright field image and a fluorescence image for each region of interest R3 and R4. More specifically, after a focus map for a bright field image is measured and calculated to create a focus map, a focus position for a fluorescent image is measured and calculated based on the focus position for a bright field image. This is performed for each target molecule (ER, HER2), and focus maps for ER and HER2 are created. FIG. 7 is a diagram illustrating an example of a focus map. As shown in FIG. 7, the focus map M includes coordinate numbers ((001, 001), (002, 001),...) Assigned to the small sections P in the attention areas R3 and R4, and the small sections P. And coordinate information (stage coordinates) of the electric stage 11 at the time of imaging. Each piece of coordinate information includes an X coordinate and a Y coordinate of the electric stage 11 when the small section P is aligned with the optical axis of the objective lens 151, and a Z coordinate representing the in-focus position. Such a focus map M is recorded in the imaging coordinate recording unit 24. Detailed processing for creating the focus map will be described later.

  In step S16, the microscope system 1 operates the microscope apparatus 10 based on the focus map M, and acquires a high-resolution bright field image and fluorescent image corresponding to the attention areas R3 and R4 using the high-magnification objective lens 151. .

  More specifically, the microscope apparatus 10 moves the electric stage 11 according to the coordinate information registered in the focus map, and adjusts the focus position for each small section P. Then, the transmitted illumination optical system 13 and the epi-illumination optical system 14 are sequentially operated to cause the TV camera 50 to capture the bright field image, the ER image, and the HER2 image of each small section P. The host system 20 combines the image input from the TV camera 50 with the image of the adjacent small section P for each type of image (bright field image and various fluorescent images). By repeating such image input and combining processing for all the small sections P registered in the focus map M, a wide-field and high-definition entire image (VS image) of the attention areas R3 and R4 is created. The Image files corresponding to these VS images are stored in the image data recording unit 23. When recording the VS image in the image data recording unit 23, the VS image may be compressed by a known compression algorithm such as JPEG or JPEG2000.

  FIG. 8 is a schematic diagram illustrating a data configuration example of an image file stored in the image data recording unit 23. As shown in FIG. 8, the image file 6 includes incidental information 61, whole specimen image data 62, and attention area designation information 63.

  The auxiliary information 61 includes information such as a slide number 611, an imaging magnification 612 of the entire slide glass image (that is, the specimen search region R1), the number of fluorescent dyes (m) 613, and fluorescent dye # 1 information to fluorescent dye #m information 614. . The slide number is an identification number arbitrarily assigned to the slide glass 100. This identification number is input to the host system 20 by reading a barcode printed on the label LB (see FIG. 4) with a barcode reader (not shown), for example. Further, the imaging magnification of the entire slide glass is the magnification of the low-magnification objective lens 151 used when the specimen search region R1 is imaged in step S12. The number of fluorescent dyes (m) is the number of types of fluorescent dyes used for staining the specimen S. In the first embodiment, there are two types of Alexa 555 and Cy5 (m = 2). The fluorescent dye # 1 information to the fluorescent dye #m information include information on each used fluorescent dye.

  The whole specimen image data 62 includes image data of a low resolution image corresponding to the specimen search region R1 acquired in step S12.

  The attention area designation information 63 includes information such as the attention area designation number (n) 631 and the attention area # 1 information to the attention area #n information 632. The attention area designation number (n) 631 is the number of areas designated by the user as the attention area from the sample area R2, and in the first embodiment, there are two attention areas R3 and R4 (n = 2).

  Each attention area #j information 632 (j = 1, 2,..., N) includes imaging information 632a, focus map data 632b, and image data 632c. The imaging information 632a includes an imaging magnification 632a-1, a scan start stage X coordinate 632a-2, a scan start stage Y coordinate 632a-3, a pixel number 632a-4 in the X direction, and a pixel number 632a in the Y direction. Information such as -5 is included. Among these, the imaging magnification is the magnification of the high-power objective lens 151 used when imaging the attention regions R3 and R4 in step S16. Further, the scan start stage X coordinate and Y coordinate are position coordinates at which imaging of the small section P in the attention areas R3 and R4 is started. Usually, the position of the upper left small section P of each of the attention areas R3 and R4 is (See FIGS. 6A and 6B). The number of pixels in the X direction and the Y direction is the number of pixels of the attention area image included in each side of the attention areas R3 and R4 in the X direction and the Y direction.

  The focus map data 632b includes information on the focus map M created in step S15. The image data 632c is image data corresponding to the VS images (bright field (cell nucleus) image, ER image, and HER2 image) of each region of interest R3 and R4 acquired in step S16.

  In step S <b> 17, the microscope system 1 displays a VS image of the sample S on the display unit (monitor) 22 based on the image file stored in the image data recording unit 23. At this time, the host system 20 assigns, for example, the B (blue) channel, the ER image, for example, the G (green) channel, and the HER2 image, for example, the R (red) channel to the bright field (cell nucleus) image, and superimposes these images. To display. Alternatively, the host system 20 may display an image in which a desired fluorescent image is superimposed on a bright field image in accordance with an input signal input by the user via the input unit 21. The channel assigned to each image can be set as desired by the user.

Next, the focus map creation process in step S15 will be described in detail. FIG. 9 is a flowchart showing focus map creation processing.
First, a focus map for a bright field image is created. That is, in step S101, the microscope system 1 measures the in-focus position at each focus position extraction point FP extracted in step S14. More specifically, the microscope apparatus 10 moves the electric stage 11 in the XY plane so that the focus position extraction point FP is aligned with the optical axis of the objective lens 151. Subsequently, the host system 20 receives an input of an image captured by the TV camera 50 while moving the electric stage 11 in the Z direction, and obtains an in-focus position by evaluating the brightness and contrast of the image. The focus position coordinates of the focus position extraction points FP actually measured in this way are recorded in the focus map M.

In subsequent step S102, the host system 20 calculates the in-focus position of each small section P that has not been extracted as the focus position extraction point FP by performing an interpolation operation based on the in-focus position actually measured in the vicinity. The calculated focus position coordinates of each small section P are recorded in the focus map M.
The focus map M for a bright field image with high resolution is completed by the above processing. The focus map M is stored in the imaging coordinate recording unit 24.

  Subsequently, the process proceeds to a process for creating a focus map for a fluorescent image. In the first embodiment, in order to observe the expression of two types of proteins, ER and HER2, focus maps for ER and HER2 (that is, a total of four) are created for each region of interest R3 and R4. .

  In step S103, the microscope system 1 switches the observation method to fluorescence observation that matches the target molecule to be observed. Accordingly, the transmission shutter 133 is inserted into the optical path L1 for transmission illumination, the epi-illumination shutter 143 on the optical path L2 for epi-illumination is opened, and an optical cube (for example, for ER) 161 for observing the target molecule is selected. Inserted into the observation optical path L. Thereby, the sample S is irradiated with excitation light corresponding to the target molecule.

In subsequent step S104, the microscope system 1 measures the in-focus positions at all the points A extracted for the fluorescent image among the focus position extraction points FP. More specifically, the microscope apparatus 10 moves the electric stage 11 in the XY plane so that the point A is aligned with the optical axis of the objective lens 151. Subsequently, the host system 20 receives an input of an image captured by the TV camera 50 while moving the electric stage 11 in the Z direction, and obtains an in-focus position by evaluating the brightness and contrast of the image. At this time, the microscope controller 30 moves the electric stage 11 within a predetermined range (for example, within a range that takes into account the thickness of the sample S) around the focus position for the bright field image actually measured in step S101. It is better to move up and down in the Z direction. Thereby, it is possible to measure the speedup of the in-focus position measurement process.
The focus position coordinates of each point A acquired in this way are recorded in the focus map M.

  In step S <b> 105, the host system 20 calculates the next statistic from the measured value of the in-focus position at point A. That is, the difference between the focus position for the bright field image measured in step S101 for a certain point A and the focus position for the fluorescence image measured in step S104 for the corresponding point A of the XY coordinates is calculated. Then, an average value μ and a standard deviation δ of the differences at all points A are calculated.

When the standard deviation δ is equal to or smaller than a predetermined value (for example, ½ of the depth of field of the objective lens 151 in use) (step S106: Yes), the host system 20 determines from the in-focus position for the bright field image. It is determined that the in-focus position for the fluorescent image can be estimated, and the in-focus positions for the fluorescent image at all points B are determined by offset (step S107). Specifically, the value Z _BF −μ calculated from the focus position coordinate Z _BF for the bright field image and the average value μ of the differences is set as the focus position coordinate for the fluorescent image. The focus position coordinates of the point B calculated in this way are recorded in the focus map M.

  On the other hand, when the standard deviation δ is larger than the predetermined value (step S106: No), the microscope system 1 measures the in-focus positions for fluorescent images at all points B (step S108). The actually measured in-focus position coordinates of the point B are recorded in the focus map M.

FIG. 10 is a flowchart showing processing for actually measuring the in-focus position at point B.
First, in step S121, the microscope system 1 determines whether or not there is a small section P in which the focus position for the fluorescent image is actually measured in the vicinity of each point B (for example, within five sections). The vicinity range may be variable according to the areas of the attention regions R3 and R4. When there is a small section P whose focus position is actually measured in the vicinity (step S121: Yes), the microscope system 1 sets the focus position coordinate of the nearby small section P as the Z coordinate of the point B, and 1 fluorescence image is acquired (step S122).

  In subsequent step S123, the microscope system 1 performs in-focus evaluation of the first fluorescent image. The focus evaluation may be performed by various known methods. For example, the focus evaluation according to the specimen state can be performed by comparing the brightness and contrast of the first fluorescent image with the average value of the focus evaluation calculation (evaluation of brightness and contrast) executed in step S104. it can.

  As a result of the focus evaluation in step S123, when it is determined that the first fluorescent image is in focus (step S123: Yes), the host system 20 captures the Z coordinate ( That is, the same value as the focus position coordinate actually measured in the vicinity) is recorded in the focus map M as the focus position coordinate of the point B (step S133).

  When it is determined that the first fluorescent image is not in focus (step S123: No), or when there is no small section P in which the focus position is actually measured in the vicinity of the point B (step S121: No). The process proceeds to step S124.

In step S124, the microscope system 1 uses the acquired focus position coordinates Z _BF for actually measured bright field image in the point B, the average value of the difference calculated in the coordinate value Z _BF and step S105 mu The value Z _BF −μ is calculated. Then, the second coordinate image at the point B is acquired by setting the Z coordinate to the value Z _BF −μ.
In subsequent step S125, the microscope system 1 performs focusing evaluation of the second fluorescent image. Details of the focus evaluation are the same as in step S123.

As a result of the focus evaluation in step S125, when it is determined that the second fluorescent image is in focus (step S125: Yes), the host system 20 uses this value Z _BF −μ as the focus of the point B. The position coordinates are recorded in the focus map M (step S133).

On the other hand, when it is determined that the second fluorescent image is not in focus (step S125: No), the host system 20 determines the focus position coordinate Z _BF for the bright field image, the average value μ of the difference, and the standard. The value Z _BF −μ−δ is calculated using the deviation δ. Then, the Z coordinate is set to the value Z _BF −μ−δ, and the third fluorescence image at the point B is acquired (step S126).

  In step S127, the host system 20 compares the in-focus state between the second fluorescent image and the third fluorescent image. In FIG. 10, a larger inequality sign indicates that the in-focus state is better.

If towards the third fluorescent image, focus state is better (step S127: Yes), the microscope system 1, the Z-coordinate of the motorized stage 11 values Z _BF -μ~Z _BF -μ-2δ While moving within the range, the focus position at point B is searched (step S128). Thereafter, the process proceeds to step S133, and the search result is recorded in the focus map M as the in-focus position coordinates at the point B.

On the other hand, when the second fluorescent image is in a better focused state (step S127: No), the microscope system 1 sets the Z coordinate to the value Z _BF −μ + δ and sets the fourth coordinate at the point B. Is acquired (step S129).

In step S130, the host system 20 compares the in-focus state between the second fluorescent image and the fourth fluorescent image. When towards the fourth fluorescent image, focus state is better (step S130: Yes), the microscope system 1, the Z-coordinate of the motorized stage 11 values Z _BF -μ~Z _BF -μ + 2δ While moving within the range, the focus position at point B is searched (step S131). Thereafter, the process proceeds to step S133, and the search result is recorded in the focus map M as the focus position coordinates of the point B.

On the other hand, when the in-focus state is better in the second fluorescent image (step S130: No), the microscope system 1 sets the Z coordinate of the electric stage 11 to the values Z _BF −μ−δ to Z _BF −. While moving in the range of μ + δ, the in-focus position at point B is searched (step S132). Thereafter, the process proceeds to step S133, and the search result is recorded in the focus map M as the focus position coordinates of the point B.
After these steps S121 to S133 are performed for all the points B, the processing returns to the main routine.

  Referring to FIG. 9 again, in step S109, the host system 20 determines the focus position of the small section P other than the focus position extraction point FP (points A and B) as the focus position coordinates of the nearby focus position extraction point FP. Is obtained by interpolation based on the above. The calculated focus position coordinates of each small section P are recorded in the focus map M.

  When there is another target molecule (for example, HER2) to be observed (step S110: Yes), the process returns to step S103. On the other hand, when there is no other target molecule to be observed (step S110: No), the process returns to the main routine.

  By these steps S103 to S110, the focus map M for each target molecule (for example, for ER and HER2) is completed. These focus maps M are stored in the imaging coordinate recording unit 24.

In step S126, when determining the search range of the in-focus position, the moving direction (plus direction / minus direction) from the reference coordinate value Z _BF −μ is determined using the standard deviation δ of the difference. However, this determination may be performed using a prescribed value such as the depth of field of the objective lens. In steps S128 and S131, the search range for the in-focus position is 2δ. However, in order to further improve the detection accuracy of the in-focus position, the search may be performed in the range of 3δ.

In addition, when a fluorescent image is acquired in steps S124, S126, and S129, a predetermined light amount or contrast cannot be obtained at any of the coordinate values Z _BF −μ, Z _BF −μ−δ, and Z _BF −μ + δ. Alternatively, it may be determined that the target molecule does not exist, and a value indicating “no focus position” may be recorded in the focus map M. Further, if the focus position Z _BF for bright field image is determined to be "no focus position", it may be set to "no focus position" without requiring the focus position of the fluorescent image.

  In step S16, when the bright-field VS image representing the cell nucleus and the fluorescence VS image representing various target molecules are superimposed, hematoxylin (cell nucleus), ER, and HER2 are assigned to the RGB channels and the composite display is performed. For example, as disclosed in Japanese Patent Application Laid-Open No. 2010-134195, by converting each dye that dyes the specimen S into a dye having a virtual spectrum and performing synthesis, the RGB color system can be displayed. Four or more types of multiple stained specimens that are not present may be combined and displayed.

  As described above, in the first embodiment, when a cell component (cell nucleus) is stained with a non-fluorescent dye and a multi-stained specimen in which a target molecule to be detected is labeled with the fluorescent dye is observed, a low resolution is obtained. After selecting a region on the bright field image acquired by bright field observation of the above, a bright field VS image is created from the bright field image acquired by high-resolution bright field observation for the selected region, and by fluorescence observation A fluorescence VS image is created from the acquired fluorescence image. At that time, since the in-focus position used for acquiring the fluorescent image is determined using the in-focus position used in acquiring the bright field image, the in-focus position adjusting speed for the fluorescent image can be improved. . As a result, it is possible to create a fluorescent VS image with high accuracy in a shorter time than before, and to reduce fading of the specimen due to excitation light.

  Further, according to the first embodiment, the focus position coordinates for the fluorescent image are obtained based on the distribution (statistics) of the difference between the focus position coordinates for the bright field image and the focus position coordinates for the fluorescent image. Since the method is determined, the expression of the target molecule in the specimen can be detected efficiently and accurately.

  Further, according to the first embodiment, since the bright field VS image corresponding to the entire specimen S acquired by the bright field observation with low resolution is displayed on the screen, the user can easily select a region such as a tumor part. it can. For this reason, selection mistakes that can occur when the user selects an area while viewing only the fluorescent image can be suppressed. In addition, since the time for the user to search for the target part can be shortened, fading of the specimen can be prevented. In addition, even if the person is not an expert in the fluorescence microscope, the area can be easily selected.

  Further, according to the first embodiment, since the fluorescence VS image obtained by fluorescence observation of the target molecule is superimposed on the bright field VS image obtained by observing the cell nucleus in the bright field, the entire image of the specimen can be easily grasped. At the same time, the cell nucleus image can be used as a landmark to identify a cell expressing the target molecule.

  Moreover, according to Embodiment 1, it is possible to display expression evaluations of a plurality of types of target molecules in a lesion or the like, classified by type on a cell-by-cell basis. Further, by specifically displaying the cells by type, it is possible to easily check the distribution of cells by type. Therefore, it becomes possible to digitize the expression level of the target molecule and use it for selection of a treatment method, prediction of prognosis, and the like. Furthermore, since the relationship between the expression of multiple types of target molecules and the morphological information can be quantified and compared, the cell type can be estimated from the morphological information such as cell nuclei obtained by HE staining. .

(Embodiment 2)
Next, a second embodiment of the present invention will be described.
The overall configuration and operation of the microscope system according to the second embodiment are the same as those of the first embodiment, and the focus map creation process (step S15 in FIG. 3) is different from that of the first embodiment. More specifically, in the first embodiment, a focus map for a bright field image is first created, and a focus map for a fluorescence image is created using the focus map. In contrast, in the second embodiment, in-focus positions for the bright field image and the fluorescent image are acquired for each focus position extraction point extracted in step S14. Thereby, the movement of the electric stage 11 and the number of times of switching of the optical cube 161 are minimized, and the time for creating the focus map is shortened.

  FIG. 11 is a flowchart illustrating focus map creation processing according to the second embodiment. In the following description, as an example, three types of focus maps used for acquiring a bright field image, an ER fluorescence image, and a HER2 fluorescence image are created.

  First, in step S201, the microscope system 1 selects and inserts the optical cube 161 for the ER fluorescence image into the observation optical path L. At this time, in order to acquire a fluorescence image of the ER antibody labeled with Alexa 555 dye, the optical cube 161 having a high visible light transmittance in the blue to green wavelength region (wavelength: 450 nm to 570 nm) is selected. In addition, a specific optical cube 161 for a fluorescent dye labeled with an antibody with a high expression level may be selected.

  In step S202, the microscope controller 30 moves the electric stage 11 in the XY direction via the stage XY drive control unit 31 so that the focus position extraction point FP extracted in step S14 matches the optical axis of the objective lens 151. Move.

In subsequent step S203, the microscope system 1 switches the illumination on the specimen S to transmitted illumination light. That is, the epi-illumination shutter 143 is inserted into the epi-illumination light path L2, and the translucent shutter 133 inserted in the trans-illumination optical path L1 is opened to irradiate the specimen S with the trans-illumination light.
In step S204, the microscope system 1 measures the focus position of the bright field image and records the focus position coordinates in the focus map M for the bright field image.

In subsequent step S205, the microscope system 1 switches the illumination on the specimen S to epi-illumination light. That is, the transmissive shutter 133 is inserted into the optical path L1 for transmissive illumination, the epi-illumination shutter 143 inserted into the optical path L2 for epi-illumination is opened, and the optical cube 161 for ER fluorescence observation is inserted into the observation optical path L. The specimen S is irradiated with epi-illumination light.
In step S206, the microscope system 1 measures the in-focus position of the ER fluorescence image and records the in-focus position coordinates in the focus map M for the ER fluorescence image.

  In subsequent step S207, the microscope system 1 obtains the in-focus position for the HER2 fluorescent image and records the in-focus position coordinates in the focus map M for the HER2 fluorescent image. The in-focus position coordinates may be obtained by adding a predetermined offset value to the in-focus position coordinates actually measured for the ER fluorescent image and / or the bright field image, or may be obtained by actual measurement. In the actual measurement, the optical cube 161 is switched to one for HER2 fluorescence observation.

  As the offset value used here, for example, a shift of the in-focus position of the HER2 fluorescent image with respect to the in-focus position coordinates measured for the ER fluorescent image and / or the bright field image using one region of the reference specimen or the specimen S May be obtained by an experiment in advance, and the value of the deviation may be used. At this time, it is also possible to obtain in-focus position shifts at a plurality of points and use an average value thereof as an offset value.

  In step S208, the host system 20 determines whether or not in-focus positions have been acquired for all the focus position extraction points FP extracted in step S14 (step S208). If the in-focus positions have been acquired for all the focus position extraction points FP (step S208: Yes), the process proceeds to step S209. On the other hand, when there is a focus position extraction point FP for which the in-focus position has not yet been acquired (step S208: No), the process returns to step S202.

  In this case, in step S204 after the repetition, the in-focus position of the bright field image may be measured while the optical cube 161 for fluorescence observation is inserted in the observation optical path L. Thereby, the switching frequency of the optical cube 161 in the optical cube switching unit 16 can be reduced. Further, when the in-focus position of the HER2 fluorescent image is obtained by actual measurement in step S207, the order of step S206 and step S207 may be alternately switched. That is, processing is executed in the order of steps S206 → S207 for a certain focus position extraction point FP, and processing is executed in order of steps S207 → S206 for the next focus position extraction point FP. Thereby, the switching frequency of the optical cube 161 for ER and HER2 can be further reduced.

  In step S209, the host system 20 obtains the in-focus position of the small section P other than the focus position extraction point FP by performing an interpolation operation based on the in-focus position coordinates of the focus position extraction point FP existing in the vicinity. Thereby, three types of focus maps M for the bright field image, the ER fluorescence image, and the HER2 fluorescence image are completed. Thereafter, the process returns to the main routine.

  As described above, according to the second embodiment, when the in-focus position at the focus position extraction point FP is obtained, the number of times of movement of the electric stage 11 and the number of times of switching of the optical cube 161 can be reduced. It is possible to efficiently create types of focus maps.

(Embodiment 3)
Next, a third embodiment of the present invention will be described.
The configuration and operation of the entire microscope system according to the third embodiment are the same as those in the first embodiment, and focus map creation processing (step S15 in FIG. 3) and VS image display processing (step S17) are performed in the first embodiment. Is different. More specifically, in the third embodiment, as shown in FIG. 12, one bright field VS image G _BF is created, and the in-focus position coordinate Z _BF on the bright field VS image G _BF is used as a reference. The focus position coordinates (focus position coordinates Z _ER for _ER , focus position coordinates Z HER2 for _HER2 ) are obtained for each target molecule (in this embodiment, for ER and HER2). And in each in-focus position coordinate Z _ER , Z _HER2 , and their surrounding coordinates Z _{ER (k)} , Z _{HER2 (k)} (k = 1, 2,..., 1 ′, 2 ′,...), Fluorescence VS images (fluorescence VS images G _ER and G _{ER (k) for ER} and fluorescence VS images G _HER2 and G _{HER2 (k)} for HER2 ₎ corresponding to each target molecule are created. Furthermore, a three-dimensional fluorescence VS image is constructed for each target molecule by superimposing and synthesizing these fluorescence VS images for each target molecule. In FIG. 12, each VS image G _BF , G _ER , G _{ER (k)} , G _HER2 , and G _{HER2 (k)} is represented by a bar associated with the Z coordinate axis. In FIG. 12, the focus position coordinates (Z coordinates) in the VS images G _BF , G _ER , G _{ER (k)} , G _HER2 , and G _{HER2 (k)} are all assumed to be equal. Although shown, these Z coordinates are acquired individually by actual measurement or calculation, and therefore are not all equal in reality and are three-dimensionally configured.

Next, the operation of the microscope system 1 in the third embodiment will be described.
The focus map creation process for high-resolution bright-field images and the bright-field image _GBF acquisition process are the same as those described in the first embodiment (see FIGS. 3 and 9).

On the other hand, the fluorescence image is acquired as follows. FIG. 13 is a flowchart illustrating a fluorescence image acquisition process according to the third embodiment.
First, in step S301, the host system 20 acquires the focus position coordinates Z _BF for bright field image at each small section P in the region of interest R3, R4.

In subsequent step S302, the host system 20, a predetermined offset value to the focus position coordinates Z _BF for bright field image [alpha] 1, the Z-coordinate value Z _ER and Z _HER2 plus each [alpha] 2, fluorescence VS image for ER G _ER, and set as an in-focus center position of the fluorescence VS image G _HER2 for HER2 to create a focus map. As in the first embodiment, the offset values α1 and α2 are obtained by combining the ER fluorescence image and the HER2 fluorescence image with respect to the focus position coordinates actually measured for the bright field image using one region of the reference sample or the sample S. The deviation of the focal position may be obtained by experiments in advance, and the value of those deviations may be used.

In step S303, the microscope system 1 determines, based on the focus map created in step S302, the focusing center position Z _ER for each small section P and the position Z _ER ± Δ that is separated from the focus center position by a predetermined interval Δ in the vertical direction. , Z _ER ± 2Δ,... Are acquired by the TV camera 50. Here, the interval Δ is a value set in advance based on the focal depth of the objective lens 151. Similarly, the microscope system 1 uses the TV camera 50 to display the HER2 fluorescent image at the in-focus center position Z _HER2 and the positions Z _HER2 ± Δ, Z _HER2 ± 2Δ,. get.

  At this time, it is preferable to move the electric stage 11 in the order of the arrangement of the Z coordinates for acquiring the fluorescence image. Thereby, the movement time of the electric stage 11 can be shortened. In addition, when a fluorescence image is acquired for a certain small section P in the order of the Z coordinate, it is preferable to acquire a fluorescent image for the adjacent small section P in the reverse order of the Z coordinate. Thereby, the movement time of the electric stage 11 can be further shortened.

In the subsequent step S304, the host system 20 creates a plurality of fluorescent VS images G _ER and G _{ER (k)} by connecting the ER fluorescent images corresponding to the adjacent small sections P for each Z coordinate arrangement order. To do. The HER2 fluorescence images are similarly connected to create a plurality of fluorescence VS images G _HER2 and G _{HER2 (k)} .

Furthermore, the host system 20 superimposes the created fluorescent VS image group on the bright field image so that the Z coordinate arrangement order is obtained for each target molecule. Thereby, an image in which the target molecule is three-dimensionally displayed on the cell component (cell nucleus or the like) is generated. Specifically, bright field VS image G and the fluorescence VS image G _ER and 3-dimensional composite image by superimposing the G _{ER (k)} for the ER _BF, fluorescence VS for HER2 bright field VS image G _BF A three-dimensional synthesized image _obtained by superimposing the images G _HER2 and G _{HER2 (k)} is generated.

When a bright-field image and a fluorescent image are combined and displayed, only the Z-coordinate pixel having the highest luminance pixel on the same XY coordinate for each of the three-dimensional fluorescent image for ER and the three-dimensional fluorescent image for HER2. By selecting, it is also possible to display a bright field image, an ER fluorescence image, and a HER2 fluorescence image by converting them into two-dimensional images.
For details of a method for acquiring a three-dimensional wide-field and high-definition image, see Japanese Patent Application Laid-Open No. 2006-343573.

  As described above, according to the third embodiment, it is possible to generate an image in which target molecules are three-dimensionally synthesized and displayed on cell components such as cell nuclei. Further, according to the third embodiment, since it is not necessary to actually measure the calculation of the in-focus position for the fluorescence image, the processing time can be shortened and a three-dimensional VS image can be created at high speed. Furthermore, when fluorescent labeling is performed with a plurality of genes as targets, for example, as in a multi-color FISH specimen, the expression of a plurality of types of genes located at different Z coordinates in the cell nucleus can be imaged without exception, It is possible to reduce oversight at the time of diagnosis.

  In Embodiments 1 to 3 described above, three types of multiple staining were performed in which one type of non-fluorescent staining and two types of fluorescent staining were combined. However, the present embodiment includes non-fluorescent staining. The present invention can be applied to multiple stained specimens subjected to at least two types of staining methods.

  Further, the present invention is not limited to the above-described embodiments as they are, and various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiments. For example, some components may be excluded from all the components shown in the embodiment. Or you may form combining the component shown in different embodiment suitably.

DESCRIPTION OF SYMBOLS 1 Microscope system 10 Microscope apparatus 11 Electric stage 12 Microscope main body 13 Transmission illumination optical system 130 Transmission illumination light source 131 Collector lens 132 Transmission filter unit 133 Transmission shutter 134 Transmission field stop 135 Transmission aperture stop 136 Bending mirror 137 Condenser optical element unit 138 Top lens unit 14 Epi-illumination optical system 140 Epi-illumination light source 141 Collector lens 142 Epi-illumination filter unit 143 Epi-illumination shutter 144 Epi-illumination field stop 145 Epi-illumination aperture stop 15 Revolver 151, 151a, 151b Objective lens 16 Optical cube switching unit 161, 161a , 161b Optical cube 17 Lens barrel 171 Binocular unit 172 Beam splitter 20 Host system 21 Input unit 22 Display unit 23 Image data recording unit 24 Shooting coordinate recording unit 25 Program recording unit 26 Video board 27 Control unit 271 Fluorescence VS image region setting unit 272 Bright field VS imaging control unit 273 Fluorescence VS imaging control unit 274 Focus position acquisition unit 275 VS image creation unit 276 Image composition unit 30 Microscope controller 31 Stage XY drive controller 32, 34 Motor 33 Stage Z drive controller 40 TV camera controller 50 TV camera 100 Slide glass

Claims (10)

A microscope objective lens and specimen capable of bright field observation and fluorescence observation are relatively moved in a direction perpendicular to the optical axis of the objective lens to obtain a plurality of microscope image groups, and the plurality of microscope images In a microscope system that generates a virtual slide image that connects groups together,
One or more cell components constituting a cell are stained with a non-fluorescent dye, and a bright field display of the cell component is displayed by bright field observation on a specimen in which a target molecule to be detected is labeled with a fluorescent dye. Bright-field virtual slide image acquisition control means for performing control for acquiring a visual-field virtual slide image;
Fluorescence virtual slide image acquisition control means for performing control to acquire a fluorescence virtual slide image corresponding to the target molecule by fluorescence observation on the specimen;
A small section for actually measuring the in-focus position for the bright field image used when acquiring the bright field image from among the plurality of small sections obtained by dividing the region to be acquired of the fluorescence virtual image into a lattice shape. A plurality of first subsections are extracted from the plurality of extracted subsections, and a plurality of first subsections for further actual measurement of the in-focus position for the fluorescence image used when acquiring the fluorescence image are obtained. The in-focus position for the bright field image and the fluorescent image in the above are measured to calculate the difference between the two in-focus positions, and the average value and the standard deviation of the difference in all the first small sections are calculated. When the standard deviation is equal to or smaller than a predetermined value, the fluorescent image in the second small section other than the first small section among the small sections for actually measuring the focus position for the bright field image is used. The in-focus position is A focus position acquiring means for calculating from said average value and focus position for has been the bright field image,
A microscope system comprising: Further comprising an optical cube switching unit for switching an optical cube selectively inserted in the optical path according to the specimen observation method;
The focus position acquiring unit, when obtaining the in-focus position for the bright field image, the focus position for the bright field image optical cube for the fluorescent image observation in a state inserted in the optical path The microscope system according to claim 1 , wherein the microscope system is acquired by actual measurement. The fluorescence virtual slide image acquisition control means sets a plurality of coordinate values in the optical axis direction with reference to the in-focus position for the bright field image and acquires a plurality of fluorescence images at the plurality of coordinate values, respectively. the area subjected to control to perform for each of the plurality of small sections divided in a grid pattern, each array order of the plurality of coordinate values, tied to fluorescence between images that corresponds to the subdivision between you adjacent The microscope system according to claim 1, wherein a plurality of fluorescent virtual images are created by stitching together. The microscope system according to any one of claims 1 to 3, further comprising an image synthesizing means for creating a composite image by superimposing the said fluorescent virtual slide image and the bright field virtual slide image. The specimen is labeled with two or more fluorescent dyes corresponding to two or more types of target molecules,
The microscope system according to any one of claims 1 to 4 , wherein the fluorescence virtual slide image acquisition control unit acquires the fluorescence virtual slide image for each type of the target molecule. Using the bright field virtual slide image, among the sample, any one of claims 1 to 5, further comprising a region setting unit that sets an area to be acquisition target of the fluorescent virtual slide image The microscope system described in 1. The microscope system according to any one of claims 1 to 6, wherein the cell component which is a cell nucleus. The microscope system according to claim 7 , wherein the cell component is stained with hematoxylin. A microscope objective lens and specimen capable of bright field observation and fluorescence observation are relatively moved in a direction perpendicular to the optical axis of the objective lens to obtain a plurality of microscope image groups, and the plurality of microscope images In a sample image generation method for generating a virtual slide image in which groups are connected to each other,
One or more cell components constituting a cell are stained with a non-fluorescent dye, and a bright field display of the cell component is displayed by bright field observation on a specimen in which a target molecule to be detected is labeled with a fluorescent dye. Brightfield virtual slide image acquisition control step for performing control to acquire a visual field virtual slide image;
A fluorescence virtual slide image acquisition control step for performing control to acquire a fluorescence virtual slide image corresponding to the target molecule by fluorescence observation on the specimen;
A small section for actually measuring the in-focus position for the bright field image used when acquiring the bright field image from among the plurality of small sections obtained by dividing the region to be acquired of the fluorescence virtual image into a lattice shape. A plurality of first subsections are extracted from the plurality of extracted subsections, and a plurality of first subsections for further actual measurement of the in-focus position for the fluorescence image used when acquiring the fluorescence image are obtained. The in-focus position for the bright field image and the fluorescent image in the above are measured to calculate the difference between the two in-focus positions, and the average value and the standard deviation of the difference in all the first small sections are calculated. When the standard deviation is equal to or smaller than a predetermined value, the fluorescent image in the second small section other than the first small section among the small sections for actually measuring the focus position for the bright field image is used. The in-focus position is A focus position acquiring step of calculating from the has been the mean value and focus position for the bright field image,
A specimen image generation method comprising: A microscope objective lens and specimen capable of bright field observation and fluorescence observation are relatively moved in a direction perpendicular to the optical axis of the objective lens to obtain a plurality of microscope image groups, and the plurality of microscope images In a sample image generation program for causing a computer to execute a process of generating a virtual slide image in which groups are connected to each other,
One or more cell components constituting a cell are stained with a non-fluorescent dye, and a bright field display of the cell component is displayed by bright field observation on a specimen in which a target molecule to be detected is labeled with a fluorescent dye. Brightfield virtual slide image acquisition control step for performing control to acquire a visual field virtual slide image;
A fluorescence virtual slide image acquisition control step for performing control to acquire a fluorescence virtual slide image corresponding to the target molecule by fluorescence observation on the specimen;
A small section for actually measuring the in-focus position for the bright field image used when acquiring the bright field image from among the plurality of small sections obtained by dividing the region to be acquired of the fluorescence virtual image into a lattice shape. A plurality of first subsections are extracted from the plurality of extracted subsections, and a plurality of first subsections for further actual measurement of the in-focus position for the fluorescence image used when acquiring the fluorescence image are obtained. The in-focus position for the bright field image and the fluorescent image in the above are measured to calculate the difference between the two in-focus positions, and the average value and the standard deviation of the difference in all the first small sections are calculated. When the standard deviation is equal to or smaller than a predetermined value, the fluorescent image in the second small section other than the first small section among the small sections for actually measuring the focus position for the bright field image is used. The in-focus position is A focus position acquiring step of calculating from the has been the mean value and focus position for the bright field image,
A specimen image generation program characterized by comprising:

Images