JP2009162708A - Image processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily compare time variations of a plurality of samples. <P>SOLUTION: In an incubation observation device, a plurality of observation image groups obtained by imaging a cell at an observation position of a predetermined vessel at a predetermined time interval are recorded. When some of these observation image groups are selected by a user, a multi-point display screen 331 is displayed on the incubation observation device. A plurality of display regions 341 for displaying the observation image groups are arranged on the multi-point display screen 331. Respective observation images of the selected observation image groups are displayed in order in time series are displayed on respective display regions 341. The present invention can be applied to a cell incubation observation device. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing apparatus, and more particularly, to an image processing apparatus suitable for displaying an observation image obtained by time-lapse observation.

  In experiments using cultured cells, especially the observation of the effects of anticancer drugs on cancer cells and the study of primary culture cell culture methods, containers such as multiple petri dishes and well plates are used. Changes are compared.

  That is, one petri dish or one well of a well plate is used as one sample, and cells as specimens to be observed are uniformly spread on each of the plurality of samples. Then, a different drug is added to each sample in which cells are seeded, or the same drug is added at different concentrations, and the cells in each sample are observed in a time-lapse manner.

  For example, some conventional cell culture apparatuses have a function of performing time-lapse observation for each sample and capturing an observation image of cells in the sample at each time (see, for example, Patent Document 1). According to this cell culture device, the user can grasp how the cells change with time by displaying the observation images of the desired samples at each time in order and comparing them.

JP 2002-277754 A

  However, with the above-described technique, the user can know the temporal change of the specimen in one desired sample, but compares the temporal changes of the specimens in a plurality of samples having different observation conditions. It was difficult. For example, the user must operate the cell culture device to display the observation image in order for each sample and compare each specimen, so when the observation image of the specimen of a predetermined sample is displayed, It was not possible to confirm the condition of other samples.

  The present invention has been made in view of such a situation, and makes it possible to easily compare temporal changes of specimens of a plurality of samples.

  The image processing apparatus according to the present invention is a selection unit that selects an observation image group composed of a plurality of observation images with different imaging times obtained by imaging a specimen to be observed at a predetermined observation position of a container at a predetermined position. The observation image groups are respectively displayed at display positions corresponding to the positional relationship of the observation objects so that the observation images of the selected plurality of observation image groups are displayed side by side for each observation image group. Display control means for sequentially displaying the observed images in time series.

  According to the present invention, temporal changes of a plurality of specimens can be compared more easily.

  Embodiments to which the present invention is applied will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration example of an embodiment of a culture observation apparatus to which the present invention is applied.

  The culture observation apparatus 11 is, for example, a cell culture observation apparatus that cultures cells that are specimens to be observed and time-lapse observations of the cells. The culture observation apparatus 11 includes a container holding unit 21, a stage 22, an observation unit 23, a recording unit 24, an input unit 25, a control unit 26, a display control unit 27, a display unit 28, a moving image generation unit 29, and a communication unit 30. Composed.

  The container holding unit 21 includes a cell culture chamber and holds a container containing cells to be observed. For example, the container held in the container holding unit 21 is a petri dish or a well plate, and the container holding unit 21 holds these containers in a culture chamber maintained at a predetermined temperature and humidity, and the cells in the container Is cultured.

  In addition, the container holding unit 21 places a predetermined container among the plurality of containers held in the culture chamber on the stage 22 or is placed on the stage 22 according to an instruction from the control unit 26. The stored container is stored in the culture chamber provided in the container holding unit 21.

  The stage 22 is driven in accordance with an instruction from the control unit 26 to move the container placed on the stage 22. The observation unit 23 illuminates the cells in the container placed on the stage 22 with illumination light, forms an image of the observation light from the cells in the container, a camera that captures an image of the cells, and the like. And taking an observation image of the cells in the container in accordance with an instruction from the control unit 26.

  The recording unit 24 records an observation image captured by the observation unit 23, various information such as cell observation conditions, a program executed by the control unit 26, and the like. The input unit 25 includes a mouse, a button, a touch panel, and the like, and supplies a signal corresponding to a user operation to the control unit 26.

  The control unit 26 controls the entire culture observation apparatus 11. For example, the control unit 26 controls the container holding unit 21, the stage 22, and the observation unit 23 to capture the observation images of the cells in the container held by the container holding unit 21 at predetermined time intervals, An observation image is acquired from the observation unit 23 and recorded in the recording unit 24. In addition, the control unit 26 instructs the display control unit 27 to display the observation image recorded in the recording unit 24, or a moving image moving image file in which a plurality of observation images are displayed simultaneously on the moving image generation unit 29. Or instructing the generation of.

  Further, the control unit 26 includes a reproduction position adjustment unit 41. The reproduction position adjustment unit 41 generates reproduction start position information indicating the first captured image when the observation images are displayed in time series in order of the imaging time according to the signal from the input unit 25.

  The display control unit 27 controls display of an image on the display unit 28 in accordance with an instruction from the control unit 26. For example, the display control unit 27 supplies the observation image recorded in the recording unit 24 to the display unit 28 for display. The display unit 28 includes, for example, a liquid crystal display, and displays an image supplied from the display control unit 27.

  The moving image generation unit 29 acquires a plurality of observation images from the display control unit 27 according to an instruction from the control unit 26, and generates a moving image file of one moving image in which the acquired observation images are displayed side by side. To do. The communication unit 30 communicates with other apparatuses connected to the culture observation apparatus 11 via a communication network, and exchanges various information with the other apparatuses.

  By the way, the culture observation apparatus 11 holds a plurality of containers, and cells are cultured in these containers. For example, the user can operate the input unit 25 to designate a predetermined observation position of a desired container and cell observation conditions, thereby allowing time-lapse observation of the cell at the designated observation position.

  Here, assuming that a petri dish as a container or one well of a well plate is referred to as one sample, first, the user operates the input unit 25 to select a sample of a container containing cells to be observed. In addition, if necessary, a desired position in the sample is designated as an observation position.

  Next, the user operates the input unit 25, and as observation conditions, for example, a time interval (interval) for capturing an observation image of a cell, an observation start time, an observation end time, an observation method such as fluorescence observation or phase difference observation, observation, etc. Specify the magnification of the objective lens used for.

  As described above, when the observation position and the observation condition are designated by the user, the control unit 26 controls each unit of the culture observation apparatus 11 based on the signal supplied from the input unit 25 according to the user's operation. The observation is performed under the designated observation condition at the designated observation position (sample).

  That is, the container holding unit 21 moves the container specified by the control unit 26 and places it on the stage 22, and the stage 22 has an observation position of the container on the stage 22 within the field of view of the camera as the observation unit 23. So as to enter the position specified by the control unit 26. Moreover, the observation part 23 images the observation image of the cell in an observation position at the time interval designated by the user based on the control of the control part 26.

  When the observation image captured by the observation unit 23 is supplied from the observation unit 23 to the control unit 26, the control unit 26 displays the sample name and observation position of the designated sample, the observation image supplied from the observation unit 23, The observation conditions input from the input unit 25 and input by the user are supplied to the recording unit 24. Then, the control unit 26 causes the recording unit 24 to record the sample name, observation position, observation image, and observation condition supplied to the recording unit 24 as sample information that is information related to one sample subjected to time-lapse observation.

  At this time, the control unit 26, as necessary, from the container holding unit 21 to other observation conditions of the sample (cell), more specifically, information related to cell culture, for example, the name of the specimen to be observed, the temperature in the culture chamber, Information such as the humidity in the culture chamber, the reagent added to the sample, the addition time of the reagent, and the concentration of the reagent is acquired and supplied to the recording unit 24 to be recorded as sample information.

  Further, the control unit 26 executes a predetermined analysis program as necessary to perform an analysis on the observation image from the observation unit 23, supplies the analysis result to the recording unit 24, and records it as sample information. Let For example, by analyzing the observation image, information such as the number of cells confirmed in the observation image, the temporal change in the intensity of fluorescence from the cells, and the cell growth rate can be obtained.

  Further, the control unit 26 supplies the recording unit 24 with comments and the like, which are supplied from the input unit 25 and input by the user at the time-lapse observation, and records them as sample information. Therefore, when time-lapse observation of a designated sample is performed, the recording unit 24 stores information such as a sample name, an observation position, an observation image group including observation images at each time, observation conditions, analysis results, and input comments. Are recorded as sample information in association with each other.

  As described above, when the user operates the input unit 25 to instruct time-lapse observation of a predetermined sample, more specifically, a cell in the sample, the time-lapse observation of the specified sample is performed, and the sample information of the sample is recorded. Is done. In addition, when time-lapse observation, for example, when cells are observed by both observation methods of phase difference observation and fluorescence observation, an observation image group obtained by phase difference observation and an observation image group obtained by fluorescence observation Are associated with each other and recorded as one sample information. An observation image group obtained by phase difference observation and a plurality of observation image groups obtained by fluorescence observation, for example, a plurality of fluorescence observation image groups having different wavelengths are associated with each other and recorded as one sample information. May be.

  When the sample information of each sample is accumulated in the recording unit 24 in this way, the user designates observation image groups of some sample information recorded, and the observation images of those observation image groups are It is possible to display the observation image groups side by side in a time series on one display screen. Hereinafter, a display method in which a plurality of observation images are arranged on one display screen and the observation images are displayed in time series is also referred to as multipoint display.

  Next, a multipoint display process, which is a process in which the culture observation apparatus 11 displays an observation image in multipoint, will be described with reference to the flowcharts of FIGS. 2 and 3.

  In step S <b> 11, the control unit 26 selects a container according to a signal from the input unit 25. That is, when the multipoint display process is started, the display control unit 27 controls the display unit 28 in accordance with an instruction from the control unit 26, and a list of containers containing cells that are being cultured in the culture observation apparatus 11. Is displayed on the display unit 28.

  Thereby, for example, the container list screen 61 shown in FIG. On the container list screen 61, buttons indicating the containers stored in the culture chamber of the culture observation apparatus 11, more specifically, buttons indicating the positions where the containers are arranged are arranged in the same arrangement as the containers in the culture chamber. Yes. Among these buttons, buttons 71-1 to 71-8 indicating containers in which cells are cultured are displayed in different colors from the other buttons. The buttons 71-1 to 71-8 display characters identifying the container such as the name of the container input by the user, the name of the experiment performed using the container, and the experiment start date. Yes. Hereinafter, the buttons 71-1 to 71-8 are simply referred to as buttons 71 when it is not necessary to individually distinguish them.

  In addition, a function button 72 that is operated when various functions are executed is displayed at the lower left in the figure of the container list screen 61.

  When the user operates the input unit 25 and selects the button 71 while the container list screen 61 is displayed, the control unit 26 selects a container specified by the button 71 according to the user's operation. The display control unit 27 is instructed to select and display a screen showing details of the container. Then, the display control unit 27 displays the screen illustrated in FIG. 5 on the display unit 28 in accordance with an instruction from the control unit 26, for example.

  In the screen shown in FIG. 5, an image of the container specified by the button 71 selected in the center is displayed. In the example of FIG. 5, a well plate having six circular wells as a container is displayed. Further, a button 101 for switching the screen displayed on the display unit 28 to the container list screen 61 displayed immediately before is provided at the lower left in the figure of the screen. , A button 102 for displaying a list of displayed container samples is provided.

  When the user operates the input unit 25 and selects the button 102, the control unit 26 instructs the display control unit 27 to display a list of samples. Then, the display control unit 27 causes the display unit 28 to display a list of the sample of containers selected according to the instruction from the control unit 26.

  Thereby, for example, a sample list screen 131 shown in FIG. 6 is displayed on the display unit 28. The sample list screen 131 includes numbers 1 to 6 indicating the samples provided in the selected container, that is, the six wells of the well plate displayed in FIG. 5, and sample names 141-1 to 141-1 for specifying the samples. Sample names 141-6 are displayed in the vertical direction in the figure. Each of the sample names 141-1 to 141-6 is a button displaying a sample name in more detail, and is operated when a sample is selected. Hereinafter, the sample names 141-1 to 141-6 are simply referred to as sample names 141 when it is not necessary to individually distinguish them.

  In addition, the sample list screen 131 is provided with check boxes 142-1 to 142-6 for selecting each sample on the left side of the sample name 141 in the drawing. When these check boxes 142-1 to 142-6 are operated and a check symbol is displayed, a sample specified by the corresponding sample name 141-1 to sample name 141-6 is selected. Hereinafter, when it is not necessary to individually distinguish each of the check boxes 142-1 to 142-6, they are simply referred to as check boxes 142.

  The user operates the input unit 25 to operate a desired one of the check boxes 142 to select a corresponding sample, and further presses a button 143 provided on the left side of the sample list screen 131. When operated, a list of observation images of the selected sample can be displayed. In other words, when the button 143 is operated, the control unit 26 causes the display control unit 27 to display the observation image captured at the end of the selected sample, that is, the observation image with the latest imaging time, according to the user's operation. Instruct the display. Then, in accordance with an instruction from the control unit 26, the display control unit 27 displays the observation images captured at the end of the selected sample side by side on the display unit 28.

  Further, the user can display check symbols in all the check boxes 142 and select samples corresponding to the check boxes 142 by operating the button 144 on which the characters “select all” are displayed.

  As described above, when the button 102 is operated while the screen shown in FIG. 5 is displayed, the sample list screen 131 for selecting the sample of the container is displayed. In order to display the sample list screen 131, it is not always necessary to select the button 71 indicating a container on the container list screen 61. For example, by operating the function button 72 on the container list screen 61, The sample list screen 131 can also be displayed by performing a search.

  That is, when the function button 72 is operated, the control unit 26 instructs the display control unit 27 to display a function list screen in accordance with a signal from the input unit 25. Then, the display control unit 27 causes the display unit 28 to display, for example, a function list screen 171 shown in FIG. 7 in response to an instruction from the control unit 26.

  The function list screen 171 is provided with buttons for executing various functions and buttons 181 operated when the function list screen 171 is closed. For example, a search button 182 for searching for a container that satisfies a predetermined condition is provided as a button for executing various functions. When the user operates the input unit 25 and selects (operates) the search button 182, the control unit 26 displays a display of a search condition input screen for inputting a search condition when searching for containers according to the user's operation. The controller 27 is instructed. Then, the display control unit 27 causes the display unit 28 to display, for example, a search condition input screen 211 illustrated in FIG. 8 in response to an instruction from the control unit 26.

  The search condition input screen 211 is provided with a button for selecting a cultivator, an input field for inputting a sample name, and a button operated when selecting the type of cell put in the container. The search condition input screen 211 includes a pull-down box for designating the culture start date of the cells in the sample, a button operated when selecting the type of container, and a button operated when specifying other conditions. Is provided.

  The user operates the input unit 25 and operates these pull-down boxes and buttons to input search conditions.

  Further, the search condition input screen 211 is provided with a button 221 that is operated when the search is executed and a button 222 that is operated when the search condition input screen 211 is closed. For example, when the user operates the input unit 25 to input the search condition on the search condition input screen 211 and further operates the button 221, the control unit 26 is recorded in the input search condition and the recording unit 24. Based on the sample information, a container satisfying the input search condition is retrieved from the containers stored in the container holding unit 21. Then, the control unit 26 instructs the display control unit 27 to display a sample list screen 131 showing a list of samples of the containers specified by the search. As a result, the sample list screen 131 of containers satisfying the search condition is displayed on the display unit 28.

  As described above, when the search condition input screen 211 is displayed and a container satisfying the input search condition is searched, the control unit 26 selects a container obtained as a result of the search in step S11.

  Returning to the description of the flowchart of FIG. 2, when a container is selected in step S11 and the sample list screen 131 is displayed, in step S12, the control unit 26 selects a sample of the container.

  For example, when the user operates the input unit 25 and selects one sample name 141 from the sample list screen 131, the control unit 26 selects a sample specified by the sample name 141 selected according to the user's operation. select.

  More specifically, in step S12, the control unit 26 selects not only the sample but also the observation position in the sample, the magnification of the objective lens when the observation position is observed, the observation method, and the like. That is, when the sample name 141 is selected, the control unit 26 instructs the display control unit 27 to display a culture history screen on which the cell culture history of the sample specified by the selected sample name 141 is displayed.

  The display control unit 27 acquires sample information of the instructed sample from the recording unit 24 in accordance with an instruction from the control unit 26, and causes the display unit 28 to display a culture history screen based on the acquired sample information. For example, when the sample name 141-1 is selected, the culture history screen 251 shown in FIG.

  In the upper part of the culture history screen 251, the sample name, the name of the cell in the sample, the name of the incubator are displayed, and an image representing the selected container is also displayed. In the example of FIG. 9, an image of a well plate having six wells is displayed as an image representing a container in the upper right in the drawing, and among the six wells, wells as samples selected by the user are other It is displayed in a different color from the wells.

  Furthermore, in the culture history screen 251, display columns 261-1 to 261-5 in which information about cells in the sample are displayed for each imaging time of an observation image captured during time-lapse observation are arranged in the vertical direction in the figure. Is provided. Further, in the display columns 261-1 to 261-5, detailed display buttons 262-1 to detailed display buttons 262-5 that are operated when displaying the details of the information about the cells in the sample are provided. .

  Hereinafter, when it is not necessary to individually distinguish each of the display fields 261-1 to 261-5, they are simply referred to as display fields 261, and the detail display buttons 262-1 to 262-5 are respectively referred to as display fields 261. When it is not necessary to distinguish between them, the details display button 262 is simply referred to.

  In the culture history screen 251, in the state before the detail display button 262 in the display field 261 is operated, only the detail display button 262, the observation image capturing time, and the name of the incubator are displayed in the display field 261. . When the detail display button 262 is operated, detailed information such as an observation image is further displayed in the display field 261.

  In the example of FIG. 9, only the detail display button 262, the imaging time, and the name of the incubator are displayed in the display fields 261-2 to 261-5. For example, in the display column 261-2, characters “2007 / Nov / 01 11:10” indicating the imaging time are displayed, and characters “BioStationCT Admin” indicating the incubator name are displayed.

  The display column 261-1 is in a state where the detail display button 262-1 is operated. The display column 261-1 includes the detail display button 262 and the imaging time “2007 / Nov / 01 11:20”. In addition to the incubator name “BioStationCT Admin”, detailed information such as observation images is also displayed.

  That is, in the display column 261-1, a sub-display column 264 that displays the image 263 of the selected container and information for each observation position in the sample at the imaging time “2007 / Nov / 01 11:20”. Is displayed. Furthermore, in the figure in the sub display column 264, on the left side, an image of a circular well is displayed as an image of the selected sample, and a point indicating the observation position is displayed at the center of the well of the image. Has been.

  Further, on the right side of the drawing in the sub display column 264, an observation image 265 obtained by imaging the observation position indicated by the sample image at the imaging time displayed on the upper side in the drawing in the display column 261. -1 to observation image 265-4 are displayed. In each of the observation images 265-1 to 265-4, characters “4 ×”, “10 ×”, “20 ×”, and “FL” indicating the observation conditions are displayed on the upper side.

  For example, the character “4 ×” indicates that the observation image 265-1 below the character is an observation image when the cells are subjected to phase difference observation using an objective lens having a magnification of 4. Further, the character “FL” indicates that the observation image 265-4 below the character is an observation image when the cells are observed with fluorescence. Thus, in the sub display column 264, observation images having the same observation position and imaging time and different observation conditions are arranged and displayed.

  Furthermore, in each figure of the observation image 265-1 to the observation image 265-4, an enlargement button 266-1 to an enlargement button 266-4 that are operated when the images are enlarged and displayed are displayed on the lower side. It is displayed. Hereinafter, when it is not necessary to individually distinguish each of the observation images 265-1 to 265-4, they are simply referred to as observation images 265, and each of the enlargement buttons 266-1 to 266-4 is individually designated. When it is not necessary to distinguish, it is simply referred to as an enlarge button 266.

  Note that only one sub display field 264 is displayed in the display field 261-1. However, when observation images are captured at a plurality of observation positions at the imaging time displayed in the display field 261-1. Are displayed side by side in the sub-display columns for those observation positions. Furthermore, in the state shown in FIG. 9, when the detail display button 262-1 is operated, the display of the image 263 and the sub display field 264 disappears from the display field 261-1, and the imaging time, Only the detail display button 262-1 and the incubator name are displayed.

  In this way, when the culture history screen 251 is displayed and the user operates the input unit 25 and selects the enlargement button 266, the control unit 26 displays the observation image 265 corresponding to the enlargement button 266 in accordance with the user's operation. select.

  For example, when the enlarge button 266-1 is selected, the control unit 26 selects a sample specified by the sample name displayed on the culture history screen 251 among the samples of the container selected in the process of step S11. Further, an observation image group when the phase difference is observed with a 4 × objective lens with the center in the sample as the observation position is selected. That is, the control unit 26 selects an observation image group including the observation image 265-1 among the observation image groups included in the sample information recorded in the recording unit 24.

  Thus, in the process of step S12, the control unit 26 selects not only the sample but also the observation position and the observation condition in the sample. As a result, an observation image group obtained by time-lapse observation of the observation position under a predetermined observation condition with the position in the sample of the predetermined container as the observation position is selected.

  Returning to the description of the flowchart of FIG. 2, when a sample is selected by the process of step S12, that is, when the enlarge button 266 in the culture history screen 251 is selected, in step S13, the display unit 28 selects the selected enlargement. An observation image corresponding to the button 266 is displayed.

  For example, when the enlarge button 266-1 is selected, the control unit 26 causes the display control unit 27 to enlarge and display the observation image 265-1 corresponding to the enlarge button 266-1 in accordance with a user operation. The display control unit 27 causes the display unit 28 to display an observation image display screen on which the observation image 265-1 is enlarged in accordance with an instruction from the control unit 26.

  Thereby, for example, an observation image display screen 291 shown in FIG. 10 is displayed on the display unit 28. In the display area 301 of the observation image display screen 291, an observation image corresponding to the enlargement button 266 is enlarged and displayed on the right side of the display area 301, the sample name, the observation position in the sample, and the imaging time of the observation image. The culture conditions (observation conditions) of the cells to be observed are displayed.

  The observation image display screen 291 is provided with a button group 302 that is operated when controlling the reproduction of the observation image group. For example, the user operates the button group 302 to cause the display area 301 to sequentially display the observation image group including the observation image 265 corresponding to the selected enlargement button 266 in time series. As a result, the user can determine whether or not the observation image is to be an image to be displayed in multipoint while viewing the enlarged observation image.

  Further, the observation image display screen 291 is provided with a multipoint button 303, a button 304, tabs 305-1 to 305-4, and an overlap button 306. A multipoint button 303 is operated when selecting an observation image displayed in the display area 301, more specifically, an observation image group including the observation image as an observation image group for multipoint display. It is operated when the observation image display screen 291 is closed.

  The tabs 305-1 to 305-4 are operated when switching the display of the observation image in the display area 301. That is, when there are other observation image groups associated with the observation image group included in the observation image 265 corresponding to the enlargement button 266, the user selects one of the tabs 305-1 to 305-4. Thus, any one of the observation image groups associated with each other can be displayed in the display area 301.

  Here, the observation image group associated with each other means an observation image group in which the observation target and the imaging time of each observation image are the same and only the observation conditions are different. For example, an observation image group consisting of observation images obtained by phase-difference observation of cells, and each observation image of the observation image group has the same imaging time, and from observation images obtained by fluorescence observation of the same cells The observed image group is an observed image group associated with each other.

  Characters “Ph”, “Ch1”, “Ch2”, and “Ch3” are displayed on each of the tabs 305-1 to 305-4. When the tab 305-1 is selected, the display area 301 is displayed. Displays an observation image obtained by phase difference observation. When any of the tabs 305-2 to 305-4 is selected, an observation image obtained by fluorescence observation is displayed in the display area 301. Here, the observation image displayed when each of the tabs 305-2 to 305-4 is selected is an observation image having a different wavelength of the captured fluorescence.

  Furthermore, the overlap button 306 is operated when two observation images among the observation images associated with each other are displayed in a superimposed manner on the display area 301. For example, when the tab 305-1 is selected and an observation image obtained by phase contrast observation is displayed in the display area 301, the user operates the overlap button 306 to select the tab 305-2. In the display area 301, an observation image obtained by fluorescence observation is superimposed (synthesized) and displayed on an observation image obtained by phase difference observation.

  Returning to the description of the flowchart of FIG. 2, when an observation image is displayed in the display area 301 of the observation image display screen 291, in step S <b> 14, the control unit 26 multiplies the observation image group including the displayed observation image. It is determined whether or not an observation image group for point display is used.

  For example, when the observation image display screen 291 is displayed, when the multipoint button 303 is selected (operated), it is determined that the observation image group is for multipoint display, and when the button 304 is operated, the multipoint button 303 is operated. It is determined that the observation image group for point display is not used.

  When the button 304 is operated and it is determined in step S14 that the observation image group for multipoint display is not selected, the display control unit 27 controls the display unit 28 in accordance with an instruction from the control unit 26 to display an observation image. The screen 291 is closed. And a process returns to step S12 and the process mentioned above is repeated.

  For example, when the observation image display screen 291 is closed, the display on the display unit 28 is switched to the culture history screen 251, and the user operates the enlarge button 266 to reselect the sample. In addition, the user can close the culture history screen 251 to display the container list screen 61 by operating the input unit 25, and can select a container again. In this case, the process returns to step S11, and the control unit 26 selects a container in accordance with a user operation.

  On the other hand, when it is determined in step S14 that the observation image group is for multipoint display, in step S15, the control unit 26 selects the observation image group of the observation image displayed in the display area 301 of the observation image display screen 291. Select as a group of observation images for multipoint display. In other words, the control unit 26 sets an observation image group including the observation image 265 corresponding to the selected enlargement button 266 as an observation image group for multipoint display.

  In step S16, the display unit 28 displays a multipoint display screen. That is, the control unit 26 instructs the display control unit 27 to display a multipoint display screen on which observation images of a plurality of observation images for multipoint display are displayed side by side. Then, the display control unit 27 acquires sample information from the recording unit 24 as necessary in accordance with an instruction from the control unit 26, and causes the display unit 28 to display a multipoint display screen using the acquired sample information.

  As a result, for example, a multipoint display screen 331 shown in FIG. 11 is displayed on the display unit 28. The multipoint display screen 331 is provided with display areas 341-1 to 341-6 where observation images for multipoint display are displayed side by side, and the display areas 341-1 to 341-6 are arranged. The observation images of different observation image groups are displayed on the screen.

  In the example of FIG. 11, since only one observation image group for multipoint display is selected, no observation image is displayed in the display area 341-2 to display area 341-6, and the display area 341- The observation image is displayed only on 1. Hereinafter, when it is not necessary to distinguish the display areas 341-1 to 341-6 from each other, they are simply referred to as display areas 341.

  Further, the user can select an observation image group for multipoint display to be displayed in the selected display area 341 by operating the input unit 25 and selecting the display area 341. For example, when the display area 341-2 is selected and an observation image group for multipoint display is designated, an observation image of the newly selected observation image group is displayed in the display area 341-2. Thus, on the multipoint display screen 331, the user can display a desired observation image in an arbitrary display area 341. Therefore, a plurality of observation images can be displayed side by side in the user's preferred arrangement.

  Furthermore, the multipoint display screen 331 is provided with a button group 342 that is operated when controlling the display of the observation image in the display area 341. By operating the button group 342, the user can display the observation images in the time series in the display area 341, or can stop, pause, or fast-forward the display of the observation images in the time series. .

  Furthermore, a seek bar 343 is provided on the lower side of the display area 341 in the multipoint display screen 331, and the seek bar 343 is positioned at a position corresponding to the imaging time of observation images displayed in time series. A knob 343a is displayed.

  For example, when the observation images constituting the observation image group are displayed in the display area 341 in order from the oldest imaging time, the knob 343a is displayed on the seek bar 343 when the first observation image of the observation image group is displayed. It is located at the left end in the figure. The knob 343a moves to the right in the drawing as time passes, and reaches the right end in the drawing on the seek bar 343 when the last observation image in the observation image group is displayed. The user operates the input unit 25 to move the knob 343a, so that an observation image at a desired imaging time in the observation image group can be displayed in the display area 341.

  Further, in the drawing of the seek bar 343 in the multi-point display screen 331, a composite setting button 344-1 to a composite setting button 344-4, a button 345, a playback position adjustment button 346, and a download button 347 are provided on the lower side. .

  The combination setting button 344-1 to the combination setting button 344-4 are operated when an observation image displayed in the display area 341 is further displayed with another observation image associated with the observation image. Characters “Ph”, “Ch1”, “Ch2”, and “Ch3” are displayed on the combination setting buttons 344-1 to 344-4, and when the combination setting button 344-1 is selected. In the display area 341, an observation image obtained by phase difference observation is displayed. When the combination setting button 344-2 to the combination setting button 344-4 are selected, an observation image obtained by fluorescence observation is displayed in the display area 341. Here, an observation image displayed when each of the combination setting button 344-2 to the combination setting button 344-4 is selected is an observation image having a different captured fluorescence wavelength.

  For example, when the observation image group used for multipoint display is an observation image group obtained by phase difference observation, the combination setting button 344-1 is in a previously selected state. In this state, when the user further selects the combination setting button 344-2 and the combination setting button 344-3, the combination setting button 344 is further displayed on the display area 341 with respect to the observation image obtained by the phase difference observation. -2 and the synthesis setting button 344-3, the observation images obtained by the fluorescence observation are superimposed and displayed.

  That is, in the display area 341, an image obtained by combining the observation image obtained by the phase difference observation and the two observation images obtained by the fluorescence observation is sequentially displayed in time series. Become. Hereinafter, when it is not necessary to distinguish each of the combination setting button 344-1 to the combination setting button 344-4, they are simply referred to as a combination setting button 344.

  The button 345 is operated when the multipoint display screen 331 is closed, and the reproduction position adjustment button 346 is operated when changing the first observation image among the observation images displayed in time series in the display area 341. Is done.

  For example, when the reproduction position adjustment button 346 is operated and a predetermined observation image is selected as the first observation image from the observation image group displayed in the display region 341-1, the display region 341-1 is selected. Observation images are displayed in chronological order from the observed images. At this time, an observation image whose imaging time is older than the selected observation image is not displayed in the display area 341-1.

  The download button 347 is operated when generating a moving image file of one moving image in which an observation image is displayed in time series in each of the display areas 341.

  In addition to the observation image, an entire image of the container selected by the user, that is, an image displaying a plurality of samples provided in the container is displayed in the display area 341 of the multipoint display screen 331. You may do it.

  Returning to the description of the flowchart of FIG. 2, when the multipoint display screen 331 is displayed in step S <b> 16, in step S <b> 17, the control unit 26 determines whether or not to select a new observation image group for multipoint display. To do. For example, when the display area 341 is selected by the user and the selection of the observation image group for multipoint display is instructed, it is determined that the observation image group for multipoint display is selected.

  If it is determined in step S17 that an observation image group for multipoint display is selected, the process returns to step S13, and a new observation image group for multipoint display is selected. That is, the display control unit 27 switches the display on the display unit 28 from the multipoint display screen 331 to the observation image display screen 291 in accordance with an instruction from the control unit 26. When the user operates the button 304, the display control unit 27 further switches the display on the display unit 28 from the observation image display screen 291 to the culture history screen 251 in accordance with an instruction from the control unit 26.

  The user operates the input unit 25 to select the enlargement button 266 from the culture history screen 251, and further selects the multipoint button 303 on the observation image display screen 291 newly displayed thereby, thereby the control unit 26. To select an observation image group for multipoint display.

  On the other hand, when it is determined in step S17 that the observation image group for multipoint display is not selected, in step S18, the control unit 26 determines whether or not to adjust the reproduction start position. For example, when the playback position adjustment button 346 is operated by the user while the multipoint display screen 331 is displayed, it is determined that the playback start position is adjusted.

  If it is determined in step S18 that the reproduction start position is not adjusted, the process proceeds to step S20.

  On the other hand, when it is determined in step S18 that the reproduction start position is adjusted, in step S19, the reproduction position adjustment unit 41 displays the observation image group in time series according to the user's operation. Select the first observation image. Then, the reproduction position adjustment unit 41 generates reproduction start position information indicating the head observation image and the display area 341 in which the observation image is displayed.

  For example, the observation image group G1 displayed in the display area 341-1 includes observation images captured at a predetermined time interval from the time before addition of the reagent to the sample, and is displayed in the display area 341-2. Assume that the image group G2 includes observation images captured at predetermined time intervals immediately after the reagent is added to the sample. In this case, when multipoint display is performed, an observation image from the time before the addition of the reagent is displayed in the display area 341-1, and an observation image from the time immediately after the addition of the reagent is displayed in the display area 341-2. Is displayed.

  However, in order to compare temporal changes of these two samples (cells displayed in the observation image), the observation image group G1 displayed in the display region 341-1 is also observed at the time immediately after the addition of the reagent. It is desirable that the display starts from the image. Therefore, for example, the user operates the reproduction position adjustment button 346, specifies the display area 341-1, and displays the observation image at the time immediately after the reagent is added to the sample among the observation images of the observation image group G1. The reproduction start position of the observation image group G1 is adjusted by selecting it as the first observation image.

  For example, when the playback position adjustment button 346 is operated and the display area 341-1 is specified, the display on the display unit 28 displays only the observation images of the observation image group displayed in the display area 341-1 in time series. Switch to the displayed screen. Then, the user designates the first observation image while operating the seek bar displayed on the screen to display the observation image at each imaging time.

  The reproduction position adjustment unit 41 selects an observation image designated by the user as the first observation image to be displayed in time series among the observation images of the observation image group G1 according to the user's operation, Then, reproduction start position information indicating the display area 341 in which the observation image is displayed is generated. Thereby, the user can adjust the reproduction start position of the observation image group with a simple operation and more easily compare temporal changes of cells in the plurality of samples.

  In particular, when an image indicating the timing of addition of the reagent, a comment, and the like are displayed together with the observation image, the user can easily adjust the reproduction start position of the observation image group.

  If the reproduction start position is adjusted in step S19, or if it is determined in step S18 that the reproduction start position is not adjusted, in step S20, the control unit 26 determines whether to set the observation image composition display. Determine whether. For example, when the composite setting button 344 is operated by the user, it is determined that the composite display is set.

  If it is determined in step S20 that composite display is not set, the process proceeds to step S22. On the other hand, if it is determined in step S20 that setting of composite display is to be performed, in step S21, the control unit 26 generates composite setting information indicating an observation image group to be displayed superimposed (synthesized) on the display area 341. .

  That is, the user designates a display area 341 to be set for composite display, and operates the composite setting button 344 so that observation image groups associated with each other to be displayed in the display area 341 are selected. Then, the control unit 26 generates composition setting information indicating the observation image group selected by the composition setting button 344 and the display area 341 in which the observation image group is displayed in accordance with the user operation.

  When the composite setting information is generated in step S21 or it is determined in step S20 that the composite display is not set, in step S22, the control unit 26 displays the time series of the observation images in the multipoint display. Whether or not is instructed is determined. For example, when the user operates the button group 342 to instruct the time-series display of the observation image, it is determined that the time-series display of the observation image is instructed.

  If it is determined in step S22 that the time-series display of the observation image is not instructed, the process proceeds to step S24. On the other hand, when it is determined in step S22 that display of the observation image in time series is instructed, in step S23, the display unit 28 displays the observation images in time series in each display area 341.

  That is, the control unit 26 supplies the reproduction start position information and the composite setting information to the display control unit 27 and also displays the display region 341 of the observation image group selected as the observation image group for multipoint display by the display control unit 27. Instruct to display in time series. Then, based on the reproduction start position information and the composition setting information from the control unit 26, the display control unit 27 displays the observation images of the selected observation image group in each of the display areas 341-1 to 341-6. Each one is displayed in chronological order.

  At this time, with respect to the display area 341 indicated by the reproduction start position information, the display control unit 27 starts the observation image indicated by the reproduction start position information, and the observation images at subsequent imaging times are sequentially arranged in time series. The display of the observation image in the display area 341 is controlled so as to be displayed. Further, the display control unit 27 displays the display area 341 indicated by the combination setting information so that the observation images of the plurality of observation image groups indicated by the combination setting information are combined and displayed on the display area 341. The display of the observation image on 341 is controlled.

  Thereby, for example, the observation image obtained by the phase difference observation and the observation image obtained by the fluorescence observation are synthesized and displayed in the display region 341-1. In this way, the observation image obtained by the phase difference observation and the observation image obtained by the fluorescence observation are overlapped and simultaneously displayed, so that the user can easily determine from which part of the cell the fluorescence is expressed. I can grasp it. In addition, the user can easily grasp how much cells are proliferating and how much fluorescence is expressed from the site of each cell.

  Furthermore, at the time of multipoint display, the display control unit 27 displays each of the observation images captured when the same time has elapsed from the respective imaging times of the first observation image of the observation image group displayed in the display area 341. The display of the observation image is controlled so that is simultaneously displayed.

  Specifically, for example, the observation image group G21 displayed in the display area 341-1 is composed of observation images captured at a time interval t, and the observation image group G22 displayed in the display area 341-2 is a time interval. It is assumed that the image consists of an observation image captured at 2t. At this time, the display control unit 27 displays the top observation image of the observation image group G21 in the display area 341-1 and displays the top observation image of the observation image group G22 in the display area 341-2.

  The display control unit 27 switches the display in the display area 341-1 from the first observation image of the observation image group G21 to the second observation image when a predetermined time elapses, and when a predetermined time elapses, The display in the display area 341-1 is switched to the third observation image in the observation image group G21. For the display area 341-2, the display control unit 27 displays the first observation image of the observation image group G 22 in the display area 341-2 until immediately before the third observation image is displayed in the display area 341-1. Keep it displayed. Then, the display control unit 27 displays the second observation image of the observation image group G22 on the display area 341-2 at the timing when the third observation image is displayed on the display area 341-1.

  Here, the third observation image of the observation image group G21 is an observation image captured when the time 2t has elapsed from the imaging time of the first observation image, and the second observation image of the observation image group G22 is The observation image captured when the time 2t has elapsed from the imaging time of the first observation image. In this way, by allowing the observation images captured when the same time has elapsed from the imaging time of the first observation image displayed in each display area 341 to be displayed simultaneously, the user can Time changes of cells in multiple samples can be easily compared.

  When the observation image is displayed in time series in the display area 341, the display control unit 27 acquires sample information from the recording unit 24 as necessary, and the observation conditions and analysis results included in the acquired sample information, the user's You may make it display a comment etc. with an observation image.

  When it is determined in step S23 that the observation image is displayed in multipoint or in step S22 that the time-series display of the observation image is not instructed, in step S24, the control unit 26 generates a moving image file. It is determined whether or not an instruction has been given. For example, when the user operates the download button 347, it is determined that generation of a moving image file has been instructed.

  If it is determined in step S24 that generation of a moving image file has not been instructed, the process proceeds to step S26. On the other hand, when it is determined in step S24 that generation of a moving image file has been instructed, in step S25, the moving image generation unit 29 generates a moving image file.

  That is, when the download button 347 is operated to instruct generation of a moving image file, the control unit 26 instructs the moving image generation unit 29 to generate a moving image file. Then, the moving image generation unit 29 acquires image data for displaying the multipoint display screen 331 and an observation image group to be displayed in the display area 341 from the display control unit 27 in accordance with an instruction from the control unit 26. Then, the moving image generation unit 29 displays the observation images of the corresponding observation image group in time series in each of the display areas 341 of the multipoint display screen 331 based on the acquired image data and the observation image group. A moving image file of one moving image is generated.

  In this moving image, for example, as shown in FIG. 12, an image in which each of the observation images is displayed in each of the display areas 341 of the multipoint display screen 331 is an image for one frame constituting the moving image. Yes. In other words, the moving image generation unit 29 generates a moving image moving image file in which the display of the observation image in the display area 341 is switched and the multipoint display screens 331 at each time are arranged in time series.

  Therefore, when the moving image of the moving image file generated by the moving image generating unit 29 is reproduced and displayed on the display unit 28, the display unit 28 displays the moving image file on the display unit 28 during multipoint display in the process of step S23. The same screen as that to be displayed is displayed.

  However, in multipoint display, each of the observation images constituting each of the observation image groups is displayed in time series in each of the display areas 341 based on the observation image group. Therefore, for example, when the button 345 is operated and the multipoint display screen 331 is once closed, the user again displays the same observation image in order to display the same observation image in the same display area 341. The image group must be reselected.

  On the other hand, when the moving image generation unit 29 generates a moving image moving image file in which the image of the multipoint display screen 331 at each time is an image of one frame, the user reproduces the moving image. Thus, it is possible to easily and repeatedly view images displayed in time series with the same combination of observation images. In addition, the user does not have to reselect the observation image group one by one.

  Even when the moving image of the generated moving image file is played back, a plurality of observation images are displayed superimposed on the display area 341, as in the case of multipoint display in the process of step S23, or an observation image is displayed. At the same time, observation conditions, analysis results, comments, and the like are displayed, or are displayed in order from the observation image selected as the first observation image. In addition to the moving image, such information may be added to the moving image so that observation conditions, analysis results, comments, and the like can be displayed.

  In this way, the moving image file generated by the moving image generation unit 29 is supplied to the control unit 26 and is recorded in the recording unit 24 by the control unit 26 or is transmitted to the culture observation apparatus 11 via the communication unit 30. Or sent to other connected devices.

  If a moving image file is generated in step S25 or if it is determined in step S24 that generation of a moving image file has not been instructed, in step S26, the control unit 26 performs a process of displaying the observation image in a multipoint manner. It is determined whether or not to end. For example, when the button 345 of the multipoint display screen 331 is operated, it is determined that the process is ended.

  If it is determined in step S26 that the process is not terminated, the process returns to step S17, and the above-described process is repeated.

  On the other hand, when it is determined in step S26 that the process is to be terminated, the control unit 26 controls the display control unit 27 to close the multipoint display screen 331, and the multipoint display process is terminated.

  As described above, the culture observation apparatus 11 displays the observation images of the plurality of observation image groups side by side on the multipoint display screen 331 in time series. Thus, by displaying each of the plurality of observation images side by side in time series, the user can easily compare temporal changes of a plurality of different specimens (cells).

  In the above description, it has been described that the user operates the culture observation apparatus 11 to cause the culture observation apparatus 11 to execute various processes such as a multipoint display process. However, the user performs, for example, a communication network. The culture observation apparatus 11 may be caused to execute processing by operating an apparatus such as a personal computer connected to the culture observation apparatus 11. In such a case, for example, the culture observation apparatus 11 performs various processes based on the remote control of the personal computer, and transmits the processing results to the personal computer. The personal computer receives and displays the processing result transmitted from the culture observation apparatus 11.

  In addition, the sample information is recorded not only in the culture observation apparatus 11 but also in an apparatus such as a personal computer or a microscope, and these apparatuses are caused to execute the multipoint display process described with reference to FIGS. May be.

  Furthermore, the series of processes described above can be executed by hardware or can be executed by software. When the above-described series of processing is executed by software, a program to be executed by the control unit 26 for performing the series of processing is recorded in the recording unit 24 in advance or connected to the culture observation apparatus 11. It can be installed in the recording unit 24 of the culture observation apparatus 11 from an external device such as a server.

  The program for executing the above-described series of processing is performed by using a culture observation device via a wired or wireless communication medium such as a local area network, the Internet, or digital satellite broadcasting via an interface such as a router or a modem as necessary. 11 may be installed.

  The program executed by the computer such as the control unit 26 may be a program that is processed in time series in the order described in this specification, or in parallel or when a call is made. It may be a program that performs processing at a necessary timing.

  Furthermore, the embodiments of the present invention are not limited to the above-described embodiments, and various modifications can be made without departing from the gist of the present invention.

It is a block diagram which shows the structural example of one Embodiment of the culture observation apparatus to which this invention is applied. It is a flowchart explaining a multipoint display process. It is a flowchart explaining a multipoint display process. It is a figure which shows the example of a display of a container list screen. It is a figure which shows the example of a display of the screen which shows the selected container. It is a figure which shows the example of a display of a sample list screen. It is a figure which shows the example of a display of a function list screen. It is a figure which shows the example of a display of a search condition input screen. It is a figure which shows the example of a display of a culture history screen. It is a figure which shows the example of a display of an observation image display screen. It is a figure which shows the example of a display of a multipoint display screen. It is a figure explaining a moving image file.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 11 Culture observation apparatus, 21 Container holding part, 23 Observation part, 24 Recording part, 26 Control part, 27 Display control part, 28 Display part, 29 Moving image generation part, 30 Communication part, 41 Playback position adjustment part, 251 Culture history Screen, 331 Multipoint display screen

Claims (5)

Selection means for selecting an observation image group composed of a plurality of observation images with different imaging times obtained by imaging a specimen to be observed at a predetermined observation position of a container at a predetermined position;
The observation image group is observed at the display position corresponding to the positional relationship of the observation target so that the observation images of the selected plurality of the observation image groups are arranged and displayed for each observation image group. An image processing apparatus comprising: display control means for sequentially displaying images in time series. Based on the selected plurality of observation image groups, moving image generation means for generating one moving image in which each observation image of the corresponding observation image group is displayed in time series at each of the display positions. The image processing apparatus according to claim 1, further comprising: The image processing apparatus according to claim 1, further comprising reproduction position adjustment means for selecting a head observation image for starting time-series display at the display position from the selected observation image group. The display control means superimposes and displays an observation image of the corresponding observation image group and an observation image of another observation image group associated with the observation image group on the display position. The image processing apparatus according to claim 1. The image processing apparatus according to claim 1, wherein the display control unit displays information related to observation of the specimen associated with the observation image at the display position together with the observation image.

Images