GB2368221A - Camera apparatus having both omnidirectional and normal view imaging modes. - Google Patents

Abstract

Imaging apparatus 2 is fitted to a camera 1 and allows both a normal, direct field of view via a imaging window 3 and/or an omnidirectional or panospherical field of view via a fisheye lens system 20. The field of view imaged by camera 1 is selected by rotation of a mirror 7, which serves to block the direct view via window 3 when panoramic imaging is required. Further embodiments include a dual fisheye lens arrangement, for example as shown in Figure 5. Also disclosed is a panoramic imaging system (Figure 19) utilising planar surfaces (40) to collect light from a wide area before it is directed, via a lens and mirror assembly (41-46) to a camera. As shown in Figure 23, the device of Figure 4 may use this planar panoramic imager. Further embodiments use dual omnidirectional lens systems as shown in Figure 27 onward.

Description

OMNIDIRECTONAL IMAGING ATTACHMENT This invention relates generally to omnidirectional imaging of a panoramic scene. One aspect of the invention provides the option of a panospherical field of view as well as the normal field of view. Another aspect of the invention provides both panospherical and normal fields of view continuously. A further aspect of the invention relates to an

omnidirectional image capturing device which employs planar reflecting surfaces.

These different aspects of the invention can be used independently or in combination.

The first aspect of this invention can be used with an imaging device, such as a CCD camera to provide the option of a panospherical field of view, as well as the normal field of view and it can facilitate rapid selective viewing of a part of a panoramic scene to enable a user to view and preferably to zoom into the area of interest without optical distortion. It can be used, for example, in the field of surveillance, where it can greatly increase the amount of information captured by a camera without re-locating the camera. However, this aspect of the invention can be applied more broadly wherever the normal field of view of the lens system of a camera needs to be supplemented by omnidirectional image information.

The second aspect of the invention can be used to provide continuously both image information from a panospherical and normal fields of view. This is also useful in the field of surveillance, especially where continuous information needs to be recorded in

a surveillance zone. It also enables a user to observe an area of interest and preferably zoom into the area in the normal field of view. The third aspect of the invention relates to the use of planar reflecting surfaces (in contrast to the use of wide angle lenses or mirrors, such as fisheye lenses or parabolic convex reflectors) for producing a panospherical field of view. This second aspect can be used broadly wherever there is a need to capture omnidirectional image information.

Reference herein to my co-pending GB Patent Applications is intended to include any of the following: GB 0006396. 6"Imaging Apparatus" GB 0018017.4"Imaging Apparatus"

GB 0019850. 7"Imaging Apparatus" GB 0021433. 8"Omnidirectional Imaging Apparatus" GB 0023786. 7"Omnidirectional Imaging Apparatus" References herein to a"panospherical field of view"should be taken to include a field of view extending over a hemisphere, besides a sphere itself, and any field of view which is major part of either such field of view. For example, a known fisheye or panoramic lens can provide a hemispherical field of view of a panoramic scene. Some parts of the hemispherical image may be very distorted due to optical effects near the edges of such wide angle lenses. When back-to-back fisheye lenses are used to provide a spherical field of view, this distortion is worse and there is also a need to seam the two hemispherical images together. Parabolic convex mirrors can be used to provide similar hemispherical and spherical fields of view. In some of my co-pending GB Patent Applications, I describe planar reflecting surfaces which can be used to achieve the same effects. Accordingly, any suitable means can be employed to capture omnidirectional information (in the first and second aspects of the invention).

Generally speaking, optical information captured over at least a major part of a

hemisphere provides for 180 x 3600 viewing, whereas that captured over the major part of a sphere is for 360 x 360 viewing.

Whilst embodiments of the invention can be described with reference to surveillance equipment, this is by way of example only, because the principles of the invention can be used by those skilled in the art in other fields of use.

In the field of surveillance, a video camera is usually located in a position where it can be panned, tilted and rotated in order to aim at a point in the surveillance zone where some activity needs to be observed. The camera is also fitted with zoom so that, after scanning around the total zone of surveillance, to watch activity occurring anywhere in that zone, an operator can zoom in to observe the activity more closely. The scanning zone could cover, for example, a large car park whereby a camera scans around to watch for any suspicious activity and then, if such activity is seen in a smaller area of the zone, the camera is aimed at this area and zoomed in to observe the activity more closely (e. g. someone attacking a car door). Whilst such equipment has proved to be of considerable advantage to those involved with security, they have the

disadvantage that, during a standby phase, the camera may be pointing in the wrong direction in order to observe initially some suspicious activity.

Another problem involved with surveillance, is that permission is usually necessary before erecting a camera on a mast, so there is often a conflict of interest between those wanting more cameras for better security and those refusing permission to instal say another camera for reasons of privacy. This can be a problem where say one camera has only a very limited field of view and it is necessary to try to find a suitable location for another camera, besides the additional costs involved in its installation and operation.

The first aspect of the present invention seeks to solve these and other problems in the surveillance field, by providing apparatus which can, for example, be simply attached to an existing camera in order to improve its image capture. For example, this may take the form of an adapter which can be fitted to the lens system of the existing camera. Alternatively, the camera can effectively incorporate the adapter, in which

case it can be used as a new camera to replace an existing conventional camera. However, this first aspect of the invention has other advantages for 180 x 360 or 360 x 360 image gathering as will be more apparent from the following description of background art.

US-A-6002430 discloses a 360 image gathering system using back to back fisheye lenses attached to a camera with computer processed output of a navigable plane image. This appears to have been developed specifically for the Film Entertainment and

Internet industry where constant 3600 image gathering is required for constant image mapping and continuous interactive image navigation by a remote user. There is no facility for an undistorted telephoto enlargement of a selected area and no facility to retract plane mirrors or correcting optics.

US-A-6118474 discloses a separate camera mirror and parabolic catadioptric reflector

system to achieve the separate use of Omnidirectional viewing. However it requires the camera to be moved rather than a retracting plane mirror or prism. The system is not compact and is not practical for outdoor use as it does not address the problems cause by rain. Further it does not show a capability of 360 image gathering on a vertical axis. Where some provision is made for optical zoom, this would merely amplify any

optically reflected aberrations if an optical zoom were applied to the convex parabolic reflector shown in this reference.

The problems of resolution for the purpose of enlargement in catadioptric reflector system employing parabolic, hyperbolic, or ellipsoidal reflectors is covered in an Internet publication entitled Analysis of a Defocus Blur, by Simon Baker in paper published on Columbia Universities website. US-A-6118474 attempts to resolve resolution problems by use of telecentric methods to filter out a substantial portion of principle rays that are not orthographically reflected it does not provide a perfect solution to defocus blur for the purpose of enlargement. However much this may assist in digital scaling, it does not resolve the ability of the system to produce an optical enlargement, because any enlargement of the camera's view would merely increase the size of the central area until one is only left with an enlarged image reflection of the camera lens itself, and its optical anomalies.

US-A-5185667 and US-A-5359363 advocate the use of Fisheye lenses to replace PTZ systems for camera navigation, both of these methods suffer from the inability to perform an optical zoom in a selected area, nor is the method seen as an omnidirectional attachment.

Computer mapping of the resulting panospherical images is described in my copending GB Applications. Computer mapping from spherical or panospherical means to a plane image is also well described in the textbook Space Image Processing, e. g. where Julio Sanchez mentions that skilled workers at NASA have developed over 30 algorithms to translate a spherical image to a flat plane view. Sanchez describes Haversine Solutions and in a chapter entitled Map Projections, projecting a sphere into a developable surface, such as Cylindrical, conical and plane. Furthermore he refers to the mapping of cylindrical projections into regular, traverse and oblique formats which have been in common usage for producing world maps since 1569. In A Simplified Approach to Image Processing, Randy Crane describes computer-processing methods of spatial transform such as Affine and bilinear transforms that can remap images to the flat plane and preserve perspective. The use of Cartesian look up tables to assist in real time processing is also described in my co-pending GB Applications. Reference is also made to US-A-5563650.

US-A-4012126 and US-A-484801 show hyperbolic reflectors which attempt to resolve Petzvtal curvatures introduced by such reflectors by specific lens groupings on a 360 horizontal and up to 1800 vertical axis, The annular image recorded is intended for projection back through the assembly to artificially negate the distortion effects to the viewer on a spherical projection screen. There is no facility for an optical zoom or computer mapping nor does it provide for a 360'Vertical image gathering.

None of the above noted prior art teaches a satisfactory method of tracking a moving object in a panoramic scene (or surveillance zone), since there will either be some distortion in parts of the panoramic image, which will mean that the object cannot be clearly seen in these parts, or there will be other limitations on the nature or quality of the image information available.

The first aspect of the invention solves these prior art problems of improving image quality by enabling an operator to use an omnidirectional imaging device to track or to detect visually or through computer analysis, targets moving within the entire horizontal and vertical field of view. The operator may then switch to standard zoom or telephoto lens to make an accurate undistorted identification. Hence, the invention can be embodied so that an omnidirectional lens or mirror need not be removed to resume normal lens and camera function. It can also be employed to resolve problems of poor image quality of omnidirectional devices and the inability to scale or zoom either optically or digitally on a selected area without granularity or without exaggerating severe spherical or chromatic or other optical aberrations inherent within such systems. The capability to scale or zoom or enlarge is important in relation to

surveillance, security and defence applications. However in Sport or Film and Entertainment applications it would save the use of a separate camera to provide close up, tracking or zoom enlargements.

The first aspect of the invention provides apparatus for use with a camera having a zoom lens system with a normal (limited) field of view. The apparatus can provide an optional panospherical field of view, and it can be embodied either as an attachment for

the camera, or as part of the camera itself. The apparatus includes : (a) panospherical image capturing means for capturing omnidirectional information from around a panoramic scene in

an omnidirectional viewing mode ; (b) optical means for capturing image information from part of the panoramic scene in a normal viewing mode; and (c) selecting means for selecting either the omnidirectional, or the normal viewing mode, by causing light from either the panospherical means (a), or the optical means (b) to be incident on an image plane of the camera.

An advantage of the latter apparatus is that the optical device or camera can be simply switched from omnidirectional to normal viewing, or vice versa, where part of a scene can be viewed (preferably by zoom) without distortion introduced by omnidirectional image gathering. This is made possible by having both the panospherical and the optical image capturing means available on the same camera so that, for example, a user can switch from one to the other by causing various means to move, such as a mirror which (in one position) reflects an omnidirectional image from say a fisheye lens onto the image plane of the camera, or (in another position) enables the image plane from the camera to receive an image in a normal field of view of a standard, or preferably standard zoom lens system. Thus, the omnidirectional image can provide continuous panoramic scene viewing, (where some spherical distortion may exist but is tolerable), whilst the normal field of view provides a non-distorted image in an area of interest within the zone of the panoramic scene.

According to the second aspect of the invention, these omnidirectional and normal fields of view are available at the same time. In this case there is no need for moveable

means to divert images from one'lens system or another to cast either an omnidirectional image or a normal image onto a single image plane of a camera.

Instead, there are two image planes respectively for omnidirectional and normal fields of view. A practical example of this can employ a divided screen display where (e. g. ) the full frame shows a panospherical image in a flat viewing plane, and an inset vignette shows the normal field of view (e. g. zoom).

Apart from increasing-the field of view, both the first and second aspects of the invention can provide for a fail safe system should there be any servo motor failure in

cameras directed remotely by a mechanical Pan Tilt and Zoom (PTZ) system. Should a partial or full failure in any of the servo guidance motors or a communications failure occur the operator can still observe through the omnioptics the complete field of view if the PTZ system leaves the camera looking the wrong way. Although this still has limited scaling. It is very useful in a security situation where cameras are often

controlled remotely via wireless links, and jamming of camera guidance communications has been known to occur. The cameras internal electronics can be programmed to switch to omnidirectional automatically when the PTZ guidance signals are lost.

According to the third aspect of the invention, the panospherical imaging device comprises a plurality of planar reflecting surfaces arranged to reflect light from a panoramic scene into corresponding light guiding means; optical means for combining image information received from the light guiding means so as to provide a composite image of at least an annular section of the panoramic scene.

The planar surfaces are preferably sides of a pyramid, the sides being triangular, quadrilateral or other shapes.

Preferably, each light guiding means is arranged to view an area which not only includes the respective part of the planar reflecting surface, but also a portion of adjacent reflecting surfaces, whereby overlapping optical image information is captured for the purposes of joining images from the adjacent planar surfaces when producing the composite image.

Embodiments of the invention in general can also resolve problems caused by environmental conditions such as rain and dust and also lens flare. For example, the means to provide a large field of view result in optics having a large exposed and

complex portal areas which require protection. Without some form of environmental protection these devices would be of no use. Whilst the portals can be protected to some degree by using known chemical compounds to assist in water removal, they could be rendered useless in heavy downpours or snow or in desert conditions. In preferred embodiments of these aspects of the invention, means are provided for the camera devices to be continually positively pressurised by low-pressure air, e. g. from a compressor which will also supply means to treat the air for temperature and

humidity. This prevents dust or water ingress ; prevents the build up condensation ; avoids flexing of the reflective components ; and reduces the complexity of seals.

Embodiments of the invention can also convert e. g. a CCD camera to a pure 3600 boy 3600 motion detection system by utilising known methods of computer image analysis. For example, the system described in the publication Handbook of Image and Video Processing by Al Bovik detailed in chapter 3.10 entitled Motion Detection and Estimation. This removes the requirement for a separate device and lens system.

The omnidirectional means, i. e. the lens systems or reflector systems can be mounted on the top and bottom of a housing or in a side by side configuration, to suit individual operational conditions. In an embodiment of the invention, this can be achieved by a locking ring on the adapter mounting plate and by remounting the front housing containing the frontal protection window and wiper assembly.

A suitable omnidirectional optical means is a fish-eye lens and this can be used, back

to-back, as shown in US-A-4256373, but other lens assemblies of 1800 field of view +/-may be utilised instead. US-A-563177S is an example of a catadioptric system that attempts to achieve wide angles of view utilising Cassgrain effect principles and is another suitable fitting for the invention. Other suitable lens include (but not exclusively) The Nikon Nikkor 6. 3 mm f2. 8, The Pentax Fisheye 17mm F4 or The Ziess F-Distagon 16mm F2. 8. Further suitable examples can be see in Merte USP 2126126 Muller USP 4647161 or designs by Myamoto Josa. The Optical assembly described in US-A-5627675 could also be modified for use.

Any wide angle lens, Dioptric or catadioptric or cataptric especially those that reduce the length or depth of the back workings are preferable as the ideal embodiment attempts to bring the lens assemblies as close together as possible to reduce any blind area in the centre of the field of view.

My co pending GB application No. 0019850.7 resolves the problems of lens flare or defraction from the large lens or reflector surfaces by the use of FOLED or TOLED or transparent LCD coatings, on portal covers using dynamic shading and this can be a feature of embodiments of the invention.

Embodiments of the invention can be employed to provide for a retractable lens relay or correction system to enable stock cameras and lenses to be used in conjunction. Where such cameras have a macro or close focusing capability the retractable relay or correction lens may be replaced by a ground glass screen to project the reflected scenes.

Retracting prisms as well as retracting plane mirrors can be used to select either the normal or omnidirectional viewing mode and the choice will suit varying optical embodiments, or a more robust assembly, but use of prisms increases the size of the back workings requiring more space for retraction.

The use of a single retracting beam splitter rather than two mirrors is also an option in a more simplified approach, but it is a larger solid component and needs more space when retracted, besides creating a larger distance between the two lenses or reflectors.

A dual lens or reflector configuration can include a mechanism to cover an aperture on the opposite side of the retracted component, thereby preventing stray light from entering the assembly during normal lens function.

Some embodiments of the invention use drive means for moving the image plane of the camera, so that (in one position) the image plane receives a focused omnidirectional image, and (in another position), the image plane receives a focused image from the normal field of view, (preferably from a zoom lens system). This can greatly simplify the mechanisms required for moving any of the optical means when the same image plane is shared by both a panospherical lens and the standard (e. g. zoom) lens systems.

Preferably, in the first aspect of the invention, the selecting means includes a beam splitter and inhibiting means which enable the beam splitter to reflect light or to pass light to the image plane of the camera, depending on whether the ommidirectional or the normal viewing mode is selected.

For general environmental protection the devices utilise a combination of chemical water repellent coatings, wiper blades and centrifugal methods. On the fisheye arrangements no wiper are shown. Preferred arrangements utilise centrifugal and water repellent coatings only, as the use of wipers restricts the field of view. Wipers shown on all omni portal coverings are as optional for severe environmental conditions.

A front portal screen or window can have varying angles from 90'to 45 with wiper fitting to suit varying operating condition. A preferred embodiment has a glass window parallel to the image plane. In heavy rain or certain covert applications where stray reflections from the glass may be undesirable a 450 embodiment may be preferred.

All mirrors and reflectors are first surface reflectors, e. g. preferably using vacuum deposited materials and coatings.

For use in surveillance of a zone in which the apparatus captures image information from a panoramic scene, the invention comprises an imaging device having a zoom lens system, means for mounting the imaging device on an axis of rotation so that it can be rotated to a selected azimuthal position, whereby the zoom lens system can capture image information from an area in the panoramic scene within its field of view, the imaging device further including panospherical image capturing means for capturing an omnidirectional image from said scene, and selecting means which are operable so as to capture either image information from the zoom lens system of the imaging device, or from the panospherical image capturing means, or both, depending on a selected mode of operation.

The latter apparatus preferably includes image processing and displaying means for processing the image information from the camera and the panospherical image capturing means to enable respective images to be displayed on a flat viewing plane.

An advantage of the latter apparatus is that whichever radial direction the imaging device is pointed to observe and zoom into the detail in the normal field of view, the panospherical image capturing means will continuously capture omnidirectional information around the scene. This could be an all round panoramic image or a part of the scene. Alternatively, or in addition, the azimuthal position required for viewing a particular area in the zone can be determined with respect to the panoramic scene seen by the panospherical image capturing means. For example, where the panospherical image is in the form of a disc or discs, and where activity is seen in a radial position around the disc (s), the image display means can provide azimuthal position data and/or drive signals to cause drive means to pan the imaging device about the axis so that it looks in the right direction for tilting and zooming onto the activity.

This could be done by observing the area on the disc as it rotates to a reference

position, i. e to pan the imaging device correspondingly.

Embodiments of the invention will now be described with reference to some of the accompanying schematic drawings, in which: Figs 1 and 2 are respectively elevational and plan views of a prior art system in which a TV camera views an area A in a surveillance zone Z, Fig. 3 is an elevational view of a camera modified in accordance with an embodiment of the invention, the camera being mounted on a mast in the zone Z for 3600 horizontal and up to 1800 vertical movement; Fig. 4 is a sectional elevation view through a conventional camera and an adapter for providing an optional omnidirectional view for 3600 horizontal x 1800 vertical viewing; Fig. 5 is a cross-section in plan view of a similar embodiment but with two fish eye lenses for 3600 x 3600 vertical viewing with side by side hemispherical views; Fig. 6 is an embodiment similar to Fig. 5 but intended for 3600 horizontal x 360 vertical viewing top and bottom hemispherical views; Fig. 7 is another similar embodiment which uses prisms; Fig. 8 is a sectional elevation through an embodiment which employs a primary convex parabolic reflecting surface for providing an omnidirectional view and a secondary plane or concave secondary reflector for reflecting light through an aperture in the convex reflector to the mirror system; Fig. 9 shows a similar arrangement with top and bottom primary convex hyperbolic reflectors and secondary planar or curved reflectors for 3600 vertical axis viewing; Fig. 10 shows another embodiment for 3600 vertical axis viewing with top and bottom omnidirectional devices each having a primary convex reflector and a lens system;

Fig. 11 shows a simplified embodiment where a single convex reflector reflects light directly onto a moveable mirror ; Fig. 12 shows an embodiment where the lens system (la) has been separated from the CCD imaging part (lb) of the CCD device (la, lb) normally found together in the camera 1;

Figs. 13-18 show embodiments with generally similar features to those of corresponding preceding embodiments except that the lens system (la) is shown separated from the CCD imaging part (lob) of the CCD device normally used in combination (1 a, lb) ; Fig. 19 shows a first simplified embodiment of an omnidirectional imaging device which employs planar reflecting surfaces (and no fisheye lens, nor convex paraboloid reflectors); Fig. 20-20e are respectively cross-sectional elevation and plan views of a similar device using planar reflectors but also having Goertz prisms; a plan view showing image segment in the eight-sided pyramid planar reflector arrangement; a view of sectors in an upper lens system; and a composite assembly of image segments in a flat viewing plane; Fig. 21 shows a similar embodiment but with a lens assembly for receiving light from an upper segment of a panoramic scene via the upper lens system; Fig. 22-Fig. 25 show embodiments including the planar mirror arrangement for omnidirectional viewing; Fig. 26 shows an arrangement with moveable CCD devices;

Figs. 28-30 show low-cost arrangements with manually movable mirrors ; Figs. 31-32 show an embodiment having a worm drive for advancing and withdrawing a CCD so as to provide focusing in the omnidirectional and normal viewing modes ;

Fig. 33 represents different embodiments where two cameras are used, one for omnidirectional viewing and receiving light from a solid V-shaped reflector, the other camera being used for the normal field of view; and Fig. 34-40 show other embodiments which employ a stationary beam splitter.

The drawings are not to scale and are provided only to show principles of the invention. Focal lengths or focal plane distances from lenses or reflectors are not shown in correct scale and reference is made to the prior art as mentioned herein and elsewhere in order to provide the details for the optical constructions. For example, US-A-4012126 shows more details of an arrangement which is similar to that shown in Fig. 8, except that the prior art reference does not deal with the processing of image signals from a CCD in order to derive a montage of different segments of panoramic scene in a flat viewing plane. Instead, this US reference deals with a device simply used for collecting light from an omnidirectional field of view, passing it through a lens system and projecting it on a hemispherical viewing plane, whereby spherical distortion is not a problem. (It will also be noted that the transparent outer cover described and illustrated in US-A-4012126 should have a parabolic shape in order to avoid introducing effects due to coma or a stigmatism).

Most lens assemblies are not detailed, references can be made to Modem Optical Engineering by Warren Smith, or any of the authors other books listed above. Further reference can be found in Visual Instrumentation by Pantaziz Mouroulis or Practical Computer aided lens design by Gregory Hallock Smith. Such lens configurations would be chosen to minimise the Petzval curvature introduced by the reflectors in the catadioptric embodiments.

Lens, mirror and prisms mountings are not fully detailed references can be found in publications such as Design and Mounting Of Prisms and Small Mirrors in Optical Instruments by Paul Yader. Seals and bearings and other supplementary fittings are also not described.

Turning now to the drawings, Figs. 1 and 2 show a prior art system, where a conventional TV camera C is mounted on a post P, usually high enough to enable the camera to look down on to an area A within a much larger surveillance zone Z (shown

circular in Fig. 2). Whilst the camera C can be tilted and panned so that it can look at any part of the zone Z, it must be controlled by the security personnel who need to use initiative in moving the camera to observe activity somewhere in zone Z.

Camera C is replaced by an imaging device having an omnidirectional field of view (e. g. a camera with a fisheye lens which looks downwardly on zone Z), whilst this would capture image information continually from zone Z, it is difficult to spot some activity occurring in a small area of this zone. The fisheye lens and CCD combination provides image information, for example, in the form of a circular image, which needs to be processed, by known means, to assist in removing spherical distortion before assembling a flat image from different parts of the panoramic scene (see for example my co-pending UK Patent Applications). Therefore, such an omnidirectional imaging device needs to be used in conjunction with a conventional camera, which can be aimed

and zoomed onto a much smaller selected area within the zone. However, no means are then available to aid the conventional camera to point in the right direction when some activity is observed in a particular area in the zone and the conventional camera C independently since it will often be looking in the wrong direction. There is also the disadvantage of cost if both the conventional camera system, as well as an omnidirectional imaging unit are used for surveillance of the same zone.

Fig. 3 shows a camera or imaging device 1, modified in accordance with the invention by having a demountable adapter 2 which provides either a normal limited field of view N, or an omnidirectional field of view O. The drawing is schematic because the camera 1 would normally be tilted downwardly so that the normal field of view will see an area A on the"floor"of the zone Z, and the omnidirectional field of view will see the zone Z. This will be more apparent with regard to the following more detailed description of Fig. 4.

Referring to Fig. 4, this shows the conventional CCD camera or imaging device 1 with the demountable adapter 2 attached thereto by a screw coupling 4. Camera 1 has CCD imaging means la, lb, which includes a standard lens system la with a normal field of view, preferably having a zoom facility (and optionally a wide angle view) and a CCD device lb which converts optical image information into video signals. The "normal field of view"is more limited than that of the omnidirectional field of view.

By suitably processing digital image signals from the CCD device lb, the normal field of view can be observed on a flat viewing image plane (on the screen of a monitor).

As there is reduced or little distortion of the image, it can be viewed with a standard (or non-standard) aspect ratio. However, the fisheye lens 20 will introduce spherical distortion. While the omnidirectional field of view can also be observed on a flat image plane of the monitor, the initial image from the fisheye lens may be disc shaped (since it circumferentially depicts a panoramic scene). This needs to be developed first into e. g. a cylindrical or annular image where spherical distortion is reduced as far as possible. This annular image can then be developed into a continuous strip whereby parts of the strip image can then be montaged into a suitable aspect ratio for viewing on the monitor.

The lens system la focuses image information entering window 3 (along axis a-a shown in chain and dot line) onto the sensitive screen of the CCD lb, for generating the video signals, which are processed (as known in the art), to show (on the monitor) the image from the normal field of view. The lens and CCD combination la, lb can also receive light from the omnidirectional field of view of the fisheye lens system 20 and mirror 7, whereby video signals are processed to reduce spherical distortion before viewing the panoramic scene on the monitor. The camera can be used to select either a normal field of view, or an omnidirectional field of view by tilting the mirror 7 as will now be described in more detail below.

The adapter 2 comprises a planar mirror 7, mounted on an opaque backing plate 8, which can be tilted about pivot 9 so that it can be positioned either at 450 with respect to axis a-a (as shown in the drawing), or it can be folded down into a storage position (not shown) where the leading edge 7e of the mirror and backing plate abuts a recess 14 in the comer of housing 15. When the mirror 7 is in the folded or stored position, the barrel 16 is clear, device 17 also then being in a retracted position (see below). When the barrel 16 is clear, light enters window 3 and passes unimpeded towards the lens system la of the lens/CCD combination la, lb. (As the barrel 16 provides a rather elongated entry port to the CCD device la, lb, it can restrict the normal field of view, but in embodiments described below, the lens system la is moved along the axis a-a so as to occupy position, for example, between window 3 and mirror 7.) Window 3 is provided with a wiper blade 5, driven by motor 5 in housing 5a in order

to provide a clear view in rainy weather. Mirror 7 and plate 8 are fast with pulley 10, which is connected by belt 11 to pulley 12, which is driven by motor 13 in order to flip mirror 7 into its omnidirectional or stored position. Other drive means can be employed such as cogs and wheels, etc. , or this can also be accomplished by some mechanical linkage operated by hand.

Device 17 can be any one or more of (i) a relay or correcting lens system; (ii) a ground glass screen; or (iii) a CCD device. Device 17 can be raised as shown in Fig. 4 (across the barrel 16) or lowered into a storage position within housing 15, by any suitable drive means, such as rack and pinion 17,18 which includes a belt drive from a motor to the toothed pinion.

In the omnidirectional mode, light from a panoramic scene enters transparent dome 24 over a wide angle (as indicated by arrows 21a, 21b) and is received by the fisheye lens system 20 whereby light from the lens system is incident on mirror 7 and thereby reflected onto device 17. As indicated above, this can be a relay or correcting lens system so that the omnidirectional image information is correctly focused onto the lens system la of the lens and CCD combination la, lb. Alternatively, if device 17 is a ground glass screen, then a focused image is received on this screen, whereby lens system la can focus an image of the screen on the CCD surface of lb for the same purpose. Alternatively, device 17 may be a CCD which receives a focused image from mirror 7. Clearly, the choice of device 17 will depend on the design of the optics and the application. For example, the prior art mentioned herein can be used to design suitable optics within the concepts of the invention.

The apparatus is equipped with means (not shown) for providing a stream of heated dried pressurised air via tube 22 to the chamber 23 in which the lens system 20 is mounted. This not only prevents dust or water ingress, but also deals with condensation and prevents movement of optical components (due to steady temperature control) which can otherwise flex and cause defocusing. It will also be appreciated that the various components of the adapter are connected together with seals to keep out dust and rain, but these need not be so complex when pressurised air is continuously pumped into the adapter interior, since this will keep dust and moisture out of the system. As shown in Fig. 4, the omnidirectional image is that which is obtained by looking downwardly on the panoramic scene. Hemispherically shaped transparent

dome 24 is designed to avoid flare and optical aberration, so that light from the panoramic scene passes through the shield 24 without any problem. The dome 24 is coupled, via circular rack and pinion 25, to motor 26 so that the dome 24 can be spun to throw off rain by centrifugal force.

Where the lens system la of the camera 1 includes a zoom facility, this can be used, in the normal field of view, to zoom into part of the area seen by the camera. In the omnidirectional field of view, if the CCD device Ibis of high resolution, parts of the panoramic image can be inspected more closely by scaling techniques used when processing the digital signals, so as to enlarge a portion of the image on the CCD.

These digital processing techniques are well known in the art. Moreover, as described below with reference to Fig. 5, where two fisheye lens systems are used back-to-back

in order to provide 3600 x 3600 viewing, known processing techniques can be used for seaming together images from the upper and lower (or side by side) hemispherical fields of view.

Instead of providing CCD devices and video signals which are processed for display on a monitor, the camera can be modified for use with film, e. g. to produce either still or moving images, in which case known film transport mechanisms are employed.

Fig. 5 shows an embodiment in which similar components have been used, the main difference being the inclusion of another fisheye lens assembly 20a, the former fisheye lens assembly being shown as 20b. This arrangement is intended to provide two hemispherical fields of view, one on either side of (e. g.) a vertical mast. The adapter 2 also comprises two mirrors 7a, 7b, which can be pivoted towards each other, in order to join at an apex on the optical axis of barrel 16 and lens/CCD device la, lb. Also devices 17a, 17b (similar to any of those described above with regard to device 17) can be moved out of storage positions in housings 15a, 15b, so as to meet on the optical axis (as shown) where they act as relay lenses, ground glass screens, or CCD devices according to requirements. Panospherical image information in the form of video signal, can be processed by known means in order to transform images to remove spherical distortions and also to provide seams between the hemispherical images whereby a spherical image can be constructed from the optical information captured by both fisheye lens systems. When used in the normal field of view, the mirror 7a, 7b are tilted into their storage positions (similar to that shown in Fig. 4) and devices 17a, 17b

are retracted into their housings 15a, 15b. The lens/CCD combination la, lb then has an unrestricted view of the window 3.

Fig. 6 is an embodiment with generally similar features to that of Fig. 5 except that this arrangement is intended for use when viewing upper and lower hemispheres of a panoramic scene. A motor 5 provides a drive for a wiper blade 6 for weather protection.

Fig. 7 shows an embodiment similar to that of Fig. 5, except that prisms 30a, 30b replace planar reflector mirrors 7a, 7b. Whilst prisms provide improved reflections, more space is required within the cylindrical housing 2a, 2b to accommodate movement when the prisms are pivoted from the positions shown in the drawing, to storage positions. In other words, the side housings 2a, 2b need to be longer.

Fig. 8 illustrates an adapter which is based on using a panospherical image capturing

system similar to that disclosed in USO-A-4012126 (Rosendahl). This replaces the fisheye lens and it is capable of capturing image information over a very wide angle of view. This panospherical image capturing means comprises hyperbolic mirrors 30, 31 arranged opposite one another and having confocal surfaces as explained in US-A4012126. The primary and convex hyperbolic mirror 30 reflects light from the panoramic scene onto the concave hyperbolic surface 31, which in turn reflects light into a lens assembly 32, which is also described in detail in US-A-4012126. This lens system provides a strong positive contribution to Petzval curvature of the system which assists in correcting image aberrations and provides correction for axial colour and colour magnification. However, this prior art system employs a camera or projector positioned to receive light directly from the lens assembly 32, since it does not deal with CCD imaging and the processing of video signals to provide transformations to remove spherical distortions. Instead, Rosendahl records an annular image intended for projection back through the lens assembly 32 artificially to negate distortion effects to the viewer on a spherical projection screen. There is no facility for optical zoom or computer mapping, nor provision for 3600 degree vertical image gathering.

One of the problems faced by the invention is to provide panospherical images which are transformed into flat viewing planes without spherical distortions and this is not addressed by Rosendahl.

Referring again to Fig. 8, convex parabolic primary and secondary reflectors may also be used, depending on the intended operational parameters. A transparent cover 33 has outer and inner surfaces formed symmetrically with respect to the near focal point of the primary mirror 30 to allow primary rays to pass normally through the surfaces of

the cover 33 without introducing a stigmatism or coma. The concave secondary reflector 31 may also, in some applications, be a plain reflector. (see for example, my co-pending GB Patent Applications). A wiper blade 34 can be moved into contact with the surface of the transparent cover 33, so that, when the cover is rotated by drive system 25,26, the blade clears raindrops. Alternatively, the cover may be spun so as to remove rain by centrifugal force.

As an alternative to using the teachings in US-A-4012126 (Rosendahl) use can be made of the panospherical imaging device disclosed in US-A-44844801 (Cox).

Fig. 9 shows an embodiment based on that shown in Fig. 8, where two of the panospherical imaging devices 33a, 33b, are provided for 3600 x 3600 viewing, These are arranged on opposite sides of the barrel 16 of the adapter.

In Fig. 8, mirror 8 can be flipped up or down, (by the drive system including motor 13) between the position shown in the drawing and a storage position where the barrel 16 is clear. Device 17 can also be moved up into the position shown, or retracted into a storage position. In Fig. 9, planar mirrors 7a, 7b, can be rotated into the positions shown in the drawing, where they meet at an apex, or rotated into storage positions (not shown). Likewise, devices 17a, 17b, can be extended or retracted.

The embodiment of Fig. 10 is of much simpler construction since it dispenses with a secondary reflector and can use a more simple lens system. In this embodiment, the primary convex reflectors 36a, 36b reflect light from the panoramic scene directly onto the respective lens systems 37a, 37b, which directs light onto respective mirrors 7a, 7b. These mirrors 7a, 7b, can be moved into the position shown, or retracted into storage positions as before. Also, device 17 can be moved into the position shown, or retracted into a storage position.

Fig. 11 shows a more simplified arrangement for 180 x 360 image capture and it employs a convex reflector 36 which reflects light directly onto mirror 7, which in turn

reflects light onto device 17. Mirror 7 is movable from the position shown in the drawing, to a storage position, and device 17 can be retracted into housing 15 by drive system 18, in order to provide the normal field of view. Weather protection is provided by a transparent dome section which is designed so as to follow the symmetry of the reflector 36 thereby reducing coma and/or a stigmatism (similar to that described above with reference to Figs. 8 and 9).

In the embodiment of Fig. 12, the components are similar to those shown in the embodiment of Fig. 4 except for the main difference of separating the lens system la from the CCD device lb. The lens system la preferably includes zoom and is situated between the mirror 7 and the window 3. This is closer to the inlet aperture of the adapter and hence provides a better"normal"field of view. In the position shown in the drawing, the mirror 7 is flipped up so as to reflect light from the fisheye lens system 20 onto the CCD device lb. In this case, device 17 (similar to that described above) is shown in its storage position, since it may or may not be required in either the normal or omnidirectional mode of operation. For example, it may be a relay or correcting lens which is raised across the barrel 16 in order to correct for focusing in the normal field of view, when mirror 7 is in its storage position. Alternatively, it might be a ground glass screen or a CCD which receives image information from the fisheye lens when the mirror 7 is in the position shown.

The embodiment of Fig. 13 is similar to that of Fig. 5, except for separating the lens system la from the CCD device lb. This has the same advantages as described above with reference to Fig. 12. The operation of this embodiment will be clear from preceding embodiments, except that devices 17a and 17b are shown in their storage positions, since they can be extended across the barrel 16 in order to act as relay or correcting lenses when light is received from the lens system la and the mirrors 7a and 7b are in their storage positions.

Similarly, Fig. 14 shows an embodiment like that of Fig. 8, except for separating the lens system la from the CCD device Ib as shown in the drawing. Otherwise, the operation will be clear by analogy with the preceding embodiments where the lens and CCD combination la, lb, is separated.

Fig. 15 is another embodiment in which the lens system la is separated from the CCD

device lb, but otherwise similar to the embodiment of Fig. 9. However, it will also be noticed that device 17a is in the storage position, whilst device 17b is extended across the path between the mirrors 7a, 7b, and the CCD device lb. These could be different relay or correcting lens systems which are used for focusing image information on the CCD surface, lb due to the difference between the optics of the omnidirectional and normal fields of view. In the embodiment of Fig. 16, the lens assembly la is also separated from the CCD device Ib (as in Fig. 15) but this embodiment is similar to that shown in Fig. 10.

Likewise, the embodiment of Fig. 17 shows lens assembly la separated from CCD device lb, but otherwise similar to the embodiment of Fig. 11. However, device 17 is shown in the stored position because it could be a relay or correcting lens which is lowered across the CCD device lb when the mirror 7 is down and the device is used with the normal field of view. A lens assembly 39 may also be provided for correcting focus where light is reflected from the convex reflector 36 onto mirror 7 and then onto the CCD lb.

The embodiments of Figs. 19-25 will now be described but it will be noted that they each include a panospherical image capturing device which employs six, and preferably eight inclined planar reflecting surfaces 40. This device can be used with the preceding aspect of the invention, but can also be used independently. Before describing the device in detail, reference is made to US-A-5016109 and US-A-6115176. Both of these references employ four and eight-sided pyramids used as reflectors, but require multiple cameras positioned over the mirror surfaces. The use of multiple cameras is an expensive and cumbersome arrangement and is not practical in respect of some of the aspects of this invention, which require an image of the scene to be relayed to the centre of the structure. Moreover, both of these references describe unsuitable image mapping techniques.

Generally speaking, both fisheye and parabolic reflectors create maximum distortion or focal aberrations at the periphery or edge of the field of view. As this periphery is utilised for 360 x 360 viewing, it is the desirable section of the panoramic scene and this aspect of the invention can utilise plain or flat mirrors to image that part of the scene which will otherwise be distorted using the prior art wide angle lenses or parabolic reflectors.

Referring to the embodiment of Fig. 19, light from the surrounding panoramic scene is incident on each of six planar reflecting surfaces 40. These planar surfaces 40 are at 450 to the optical axis. Six outriggers 47 are arranged above the planar surfaces and each of these contains a relay lens 41 and a mirror arrangement or right angled prism 42,43, for turning the light back towards the centre of the structure into a lens 44, which slightly enlarges the image seen by the respective planar mirror and which contains vanes or light guides which crop or mask part of the overlap of the panoramic scene viewed by adjacent planar mirrors. The resulting image is then transmitted through relay and field lenses 45,46, to below the structure where the image

information is used by the camera device (as described in more detail below). It is to be noted that the upper part 47'is opaque, since it represents the outrigger 47 which is seen at the side of the cross-sectional view and that the light path through the lens 44' is that of the outrigger seen behind the cross-sectional view.

In the embodiment of Fig. 20a, the planar mirrors 42,43, are replaced by modified Goertz prisms 50. It will again be noted that the prism 50'is seen end on, compared with the side view of prisms 50, in the drawing. This prism arrangement allows a much more compact profile which is useful when using a central lens arrangement 53 (as described below with reference to Fig. 21). Apart from the prisms, the construction of the device shown in Fig. 20a will be apparent from the embodiment of Fig. 19.

Fig. 20b is a plan view of the arrangement shown in Fig. 20a. Here it can be seen that the outriggers 47 extend over part of the incline planar mirrors 40, i. e. looking down on the same. The lenses 41 (see Fig. 20a) have a field of view which not only covers most of the planar mirror 40 over which they are located, but also a portion of the adjacent planar mirrors, one on each side. This overlap is used in optically assisting in composing the omnidirectional image in the planar viewing field (as described below).

Fig. 20c-20e shows how the panoramic image is composed in the flat viewing plane.

Fig. 20c is similar to the plan view of Fig. 20b, except for showing regions which have been numbered and lettered. The numbers 1-8 represent the respective parts of the scene reflected by the planar mirrors 40. The letters A-H represent sectors in an upper part"9"of the arrangement shown in the plan view of the Fig. 20c. This will include a lens 53, as shown in Fig. 21, which provides a wide angle view of the upper part of

the scene and directs light downwardly through the lens system 45, 46, towards, e. g. a CCD device, which also receives the light from the planar reflectors via lenses 44.

Fig. 20d shows these optical image sectors and segments 1-8 and A-H in more detail.

Optical image sectors and segments enable a composite image to be composed at least partly within the camera (in contrast to the digital signal transformation techniques mentioned by Nalwa in US 6115176). This facilitates the processing of the video signals in order to display the omnidirectional image in the flat viewing plane (as disclosed for example, in my co-pending GB Application 0021433. 8). Light guides, reflecting surfaces, lenses, montaging baffles and/or masking matrixes can be used to compose the optical image sectors and segments on the face of a common CCD, for example, so as to position the respective parts of the panoramic scene side by side, during the process of assembling the composite image for viewing in the flat viewing plane. Some processing of the digital image signals can also be carried out to create a cropped image in the flat viewing plane (if this is required). This will be simplified by any of the optical composing steps carried out within the camera. However, in some cases, it may not be necessary to create an image in the flat viewing plane, especially for movement detection purposes. Fig. 20e shows how the sectors and segments are composed side by side (optically and/or by known digital transformation techniques) after removing distortions, so as to provide a matrix of columns and rows of rectangular image portions A-H and 1-8 corresponding with those in Fig. 20c. Where digital processing is required known techniques can be used in order to achieve this.

With further reference to Fig. 21, a wide angle lens 53 can be inserted in the centre of the outriggers 47 to view the central 900 area and this is composed into the central ring or circle of Fig. 20d.

Figs. 22 shows how the arrangement of Fig. 21 can be used in a 3600 x 180 camera arrangement. Similar components are being given similar reference numerals. In Fig. 22, lens system la and CCD device Ib are located together, and device 17 is in the raised position, e. g. to act as a relay lens for the omnidirectional device. When the mirror is stored, device 17 is also stored, since the lens/CCD combination la, lb, has a normal field of view. Fig. 23 shows a similar embodiment, but the lens system lb is separated from the CCD device la and device 17 is lowered into its stored position

and a correction lens 55 is employed for correcting the focus of the omnidirectional imaging device. In the normal field of view, the mirror is stored and device 17 is raised into its operating position to correct the focusing. Other parts of the construction and operation of the embodiments of Figs. 22 and 23 will be clear from the preceding description.

Fig. 24 shows the planar reflecting device used in a 360 x 3600 panoramic imaging embodiment which otherwise employs folding mirrors 7a, 7b and devices 17a, 17b, besides the lens and CCD combination la, lb. A similar embodiment is shown in Fig.

25, except the lens system la is separated from the CCD device lb to obtain a better

"normal"field of view. However, it will also be noted that other relay or correcting lenses 57, 58 are provided. Lens 57 is fixed across the optical path to the CCD device lb, whereas lens 58 is attached to the backing plate of mirror 7a so that, when the mirrors 7a, 7b, are moved into their stored positions, the lens 58 is positioned across the optical path between the lens system la and the CCD device 1 b so that it cooperates with lens 57 to correct for focusing.

In either Fig. 24 or 25, the devices 17a, 17b, can be any of the devices mentioned above. Moreover, they may provide different prescriptions for the omnidirectional field of view, i. e. either device 17a or 17b being moved into the optical path between the mirrors and the CCD device lb. However, they may also be part of the focusing correction in the normal field of view when the mirrors are in their stored positions.

Fig. 26a shows a detailed modification where mirrors 7a, 7b, are in their operating positions for reflecting light from omnidirectional imaging devices onto a combination of devices 17c, 17d. Device 17c can be, for example, a high resolution CCD which is used for enlarging or scaling up a portion of the scene in the omnidirectional field of view. Device 17d can be an infrared CCD which is useful for detecting movement in a surveillance zone, e. g. by detecting movement of a thermal body such as an intruder.

Devices 17c and 17d may be used in combination or separately. Also, device 17a can be moved from its storage position into an operational position to provide other facilities, such as correcting focus, etc. In this modification, the lens system la is adjacent the CCD device lb, but they may also be separated as described above with respect to other embodiments.

Fig. 26b shows an arrangement where device 17a is positioned (as shown) across the optical path to the lens/CCD combination la, lb (or CCD lb) in order to correct focusing of the omnidirectional image capturing means and wherein the device 17b is raised into the same operating position (after storing device 17a) to correct for focusing in the normal field of view.

Fig. 27 illustrates an embodiment in which back-to-back fisheye lens systems are used similar to those in the embodiments of Figs. 5 and 6. These may be mounted for side to side, or top to bottom hemispherical views of a panoramic scene and may be suspended from a mast 70, or located on top of a mast, depending on how the apparatus is to be used. In this case, folding mirrors 7a, 7b, can be moved manually into their operating position (as shown in the drawing) or into stored positions where they move from being inclined to being vertical. Access is available through an inspection plate (71) which can also be removed in order to provide an unimpeded optical path to device 17. This can be a CCD for converting panospherical image information into video signals, or it could be a film holder, either for still or motion photography. Device 17 can also be selectively adjusted or replaced via access through the inspection cover 71 and after folding back mirrors 7a, 7b. This embodiment also enables a lens system to be fitted (manually) at the location of the inspection cover 71, or just inside the recess defined by the internal walls 72, whereby the apparatus can be used for capturing image information from a normal field of view (i. e. looking downwardly from device 17).

Generally speaking, Fig. 27 provides a low-cost alternative, especially where inexpensive panospherical image capturing means can be used, such as inexpensive fisheye lenses, or other devices which will serve the same purpose. For example, Fig.

28 shows a modification where the fisheye lenses are replaced by the omnidirectional image capturing means shown in Fig. 9. In this case, the inspection cover 71 is located in the position otherwise occupied by window 3. Fig. 9 shows another modification, based on the embodiment of Fig. 10 and also having an inspection cover 71. Fig. 30 shows a modification with the planar reflecting omnidirectional devices, similar to the embodiment of Fig. 24. This also has an inspection cover 71.

Figs. 31 and 32 illustrate embodiments which employ back-to-back fisheye lens systems 20a, 20b (similar to the embodiments of Figs. 5 and 6), but where the lens/CCD combination la, 1 b, has been further modified to simplify the construction. In this case, the lens system la is permanently located between the window 3 and the movable mirrors 7a, 7b. These mirrors are shown in the omnidirectional viewing position in

Fig. 31, which also shows CCD device lb in a retracted position at which the omnidirectional image information reflected from the mirrors is focused on the surface of the CCD device. The CCD lb is mounted on a backing plate 71, which is threadably coupled to a screw 72 that can be rotated by motor 73. Motor 73 is operable in forward and reverse modes whereby screw 72 can be rotated either clockwise, or anticlockwise to move the CCD device lb and backing plate 71 either forwards or backwards so as to correct for the focal distance of either the omnidirectional (e. g. fisheye) lens 20a, 20b or the standard lens la.

Fig. 32 shows the normal field of view, i. e. where mirrors 7a, 7b, have been pivoted into their storage positions and where CCD lb and backing plate 71 have been advanced towards the window 3. Hence, the CCD device lb can be moved to the correct position for receiving a focused image from the lens system la in the normal field of view.

The embodiments of Figs. 31 and 32 can greatly simplify the construction of the apparatus, since it disposes with device 17 and its drive systems, as well as simplifying the structure with regard to the use of relay and correcting lens systems.

Fig. 33 shows an embodiment where back-to-back fisheye lenses are also used for panospherical image gathering, but in this case, the mechanisms are comparatively fixed for omnidirectional and normal fields of view. For example, the lens system la has a normal field of view via window 3 and is arranged to focus images on CCD 17e.

Mirrors 7a, 7b are fixed in the positions shown, in order to reflect light from the backto-back fisheye lens systems 20a, 20b onto CCD device lb. This embodiment provides a low-cost alternative, i. e. where mechanisms for moving various components within the camera device are no longer required. It also has the advantage of enabling continuous image capture from both the normal field of view (3, la, 17e) as well as the omnidirectional field of view (20a, 7a, 20b, 7b, lb).

Another alternative is to provide panospherical image capturing means (e. g. a fisheye lens) and a standard lens on a movable member, such as a slide or turret, whereby either one may be moved into position for giving a normal or omnidirectional field of view to the same image plane of the camera. Such a device could be housed in a waterproof transparent enclosure which is pressurised with warm dry air.

Figure 34 shows a modification of the embodiment shown in Figure 8, the main difference being that a beam-splitter 80 is used instead of movable planar mirror 8.

This has the important advantage of avoiding the need to move the mirror 8 and hence avoiding the need for servomotors, which suffer from wear due to continued use. The beam splitter can be any device which enables image information to be gathered on each of the incident light paths of the optics for the omnidirectional and normal (or standard) fields of view and which combines them on a common incident path to the imaging system (CCD). An example of a beam splitter is found in US 5140151 (Weiner). Preferably a beam-splitting cube (as illustrated in Fig. 34) can be used, but it can be a semi-silvered transparent substrate, such as a partially reflective/partially transparent mirror. For example, dichroic cube beam-splitters having two precision right angle prisms cemented together, with an appropriate interference coating on the surface for reducing absorption loss, can be used.

Alternatively, pellicle beam-splitters can be used, wherein the pellicles are very thin covered cellulose membranes bonded to lapped aluminium frames. Further details of such beam-splitting devices are provided in the catalogue of Edmund Optical Company as available, for example, from www. edmondoptics. com., the contents of which are incorporated by cross-reference. These and other beam splitters in the state of the art can be components in cameras, or attachments, which embody the invention. These are characterised in that they can be fixed, yet still provide the facility to gather omnidirectional and normal images. In the case of the cube 80, light is either reflected from the semi-silvered surface (i. e. ray path 81,82), or it passes straight through the semi-silvered surface (i. e. ray path 83, 82). Thus, as shown in Figure 34, the panospherical image captured by convex mirror 30, which is then reflected from the planar or curved mirror31, is directed through lens system 32 onto the beam-splitting cube 80 where it is reflected onto the imaging device lb, whereas light in the normal field of view passes through (zoom) lens system la, and then directly through the beam-splitting cube 80, where it is received by the same CCD device lb. The focussing of the optics enables these images to be imaged on the focal plane of the camera. This would result in two simultaneous images on the focal plane and whilst these could be separated by using differently polarised light for each view and suitable filters, or other optical or digital processing means for separating the images for viewing either separately or simultaneously the panospherical or partial scene, it is preferred to include further means which optically switches either image on or off on

the camera's focal plane. This can be achieved by inhibiting or blanking out the unwanted field of view by using stops, irises, shields, or filters to blank out one or other image. For example, in Fig. 34, the lens systems la and 32 are each provided with an iris diaphragm 84a, 84b and either can be"stopped down"so that the diameter of the hole ("the stop") is reduced to nil, whereby the entire circular interleaving components effectively block light passing to the semi-silvered beam splitter 80.

Where circular iris diaphragms are not provided, iris components or simple shields having other shapes (such as rectangular or rectilinear, square, and crosshatching interleaving components such as overlapping blind formations) can be used. The Edmund Optical Company (see above cross-reference describes circular neutral density filters of simple, variable and stepped construction; 3 M Light Control film; polarising filters and variable density filters which can all be used for these purposes.

As irises have moving mechanical parts, these are subject to wear, and they can be avoided by using, for example, some of these controllable filters, and devices such as a transparent LCD (TLCD) or similar display material such as, but not exclusively, transparent organic light emitting diodes (TOLED), which can generate a 100 per cent black mask for either the normal or omnidirectional field of view. These can be called "dynamic"or"LCD blinds". The use of stops or irises can thus effectively switch the field of view on the beam splitter from omnidirectional to normal or vice versa, and can remove any ghost or overlaying duplicate image from the unwanted field of view before it is received by the CCD imaging device lob..

With the iris 84a stopped down, and iris 84b open, the semi silvered cube 80 will allow 100per cent reflectance of the omnidirectional field of view. With iris 84b stopped down and iris 84a open, light passes directly through the cube 80. As a further alternative, a shield or stop can be switched or moved either to block, or to pass light, but this has the disadvantage of requiring a movable part.

A controllable device is, for example, a TLCD, which a display having a matrix of pixels that can be switched either to pass, or to block all red, green and blue light.

The matrix display then appears either transparent or black respectively (since switching the red, green and blue light effectively passes white light, or blocks all light).

Use of polarising materials to reduce or to pass the light transmitted to the semi silvered beam splitter 80 can include polarising filters used in pairs or individually rotated about an axis to control the light output from either the omnidirectional or normal field of view. Such devices are also available from Edmund Optical Company as noted above.

Stepped or linear density filters can be used in a variable filter translation mount, or circular variable density filters can also be used to mask or blind the undesired optic (and these are also available from the Edmund Optical Company) or to control light output.

As shown in Fig 34, the devices 84a and 84b, for blanking out one or other lens system la or 32, can be simply attached to the lens system, but preferably, they are combined

with a filter or shield built into the semi-silvered beam splitting cube or glass sheet. This is shown in Figs. 35 and 36, where a TLCD or mechanically operated filter or shield 90b is incorporated onto the 45 degree surface of the prism 91 (or glass sheet with semi silvered surface, not shown), and/or a second filter or shield 90a is attached to the horizontal surface. When the filter 90b on the 45 degree surface is activated to mask the horizontal beam, this results in a mirror finish which assist in diverting the vertical or lower beam through 90 degrees to the image sensor. Conversely, when the filter 90a on the horizontal surface is activated, and the 45 degree surface deactivated, the 45 degree surface would be clear so that the horizontal ray would penetrate to the sensor, while the lower optic would be masked. Both lens systems for the omnidirectional and normal fields of view also have an iris or a TLCD blind 84a, 84b. An iris can be operated by a servo, and the lens may have one anyway. These TLCD blinds could act as a means for controlling the exposure, as disclosed in my copending International Application No. PCT/GB01/01115) and could save the cost of a mechanical iris.

In practice, a TLCD blind could have a visible matrix (or dot screen) when it is transparent and in focus to the sensor, so it would best be positioned to avoid being in sharp focus on the sensor.

These devices which employ a beam splitter, can be embodied in the other arrangements which employ movable elements, such as mirrors, for example in the

embodiments described above with regard to Figs 7. 12, 17 and 22.

Alternatively, the image information gathered in the omnidirectional and normal fields of view can be directed to respective separate autonomous image sensors, whereby some form of switching enables either the panoramic scene or the partial scene to be selected for display on a viewing screen. This can also be achieved by multiplexing and/or by selective processing of the information in order to display either the panoramic or partial scene. The same advantage is still obtained in using separate optics for the image information gathered in the omnidirectional and normal fields of view, even though it is directed to respective image sensors, because better image resolution of part of the panoramic scene is achieved with (say) an ordinary zoom lens, compared to scaling up a portion of the image information gathered by the omnidirectional optics. A simple way to achieve this alternative, is to use either respective CCD imaging chips for the omnidirectional and normal fields of view, or to use respective parts of a common CCD imaging chip for the same purpose. It is then only necessary to switch from one to the others so as to view the required scene. This could be a lower cost alternative than using means for moving mirrors or other elements, depending on the cost of the chip.

Figs 37-40 show a similar modification to previous embodiments, i. e. where the beam splitter 80 and inhibiting means 84a, 84b, 84c (such as irises, blinds or TLCD's are used to effect selectively the passage of light through, or reflection of light from the semi silvered surface of th beam splitter, as explained above with respect to Figs 34-36.

Whilst various embodiments have been described in detail, the components used in these embodiments may be interchanged, in some cases, in order to modify the

examples described. Moreover, other changes can be made within the broad concepts of the various aspects of this invention.

Claims (41)

1. CLAIMS 1. Apparatus for providing an optional panospherical field of view in a camera having a lens system with a normal field of view, being provided either in the form of an adapter, or as part of a camera; the apparatus including: (a) panospherical image capturing means for capturing omnidirectional information from around the panoramic scene in an omnidirectional viewing mode; (b) optical means for capturing image information from part of the panoramic scene in a normal viewing mode; and (c) selecting means for selecting either the omnidirectional, or the normal viewing mode, by causing light from either the panospherical image capturing means, or the optical means with the normal field of view, to be incident on an image plane of the camera.
2. 2. Apparatus according to claim 1, wherein the selecting means is operable so that: in the omnidirectional viewing mode, only the omnidirectional image information from the panospherical image capturing means is incident on the image plane of the camera; and in the normal viewing mode, only the image information from the optical means with the normal field of view is passed to the image plane of the camera.
3. 3. Apparatus according to claim 2, wherein the selecting means includes reflecting means which reflect omnidirectional image information from the panospherical means to the image plane of the camera and which also prevent the image information from the optical means, with the normal field of view, from reaching the image plane of the camera.
4. 4. Apparatus according to claim 2, wherein the selecting means includes reflecting means which, in the normal viewing mode, passes the image information from the

   optical means to the image plane of the camera, and which prevent the omnidirectional image information from reaching the image plane of the camera.
5. 5. Apparatus according to claims 3 or 4, wherein the reflecting means includes one or more reflectors movable between first and second positions in which the respective information is either reflected onto, or prevented from reaching the image plane of the camera.
6. 6. Apparatus according to claim 1, including a dual operating mode in which omnidirectional information, as well as image information from the optical means with the normal field of view are passed to the image plane of the camera and further including means for distinguishing between the respective image information captured by the panospherical and optical means.
7. 7. Apparatus according to any preceding claim wherein the camera includes a movable element defining the image plane of the camera, and drive means for moving said element between positions where either omnidirectional information or information from the optical means with the normal field of view, is in focus on the image plane of the camera.
8. 8. Apparatus according to claim 7, wherein said movable element includes a CCD device.
9. 9. Apparatus according to claim 8, wherein said movable element is coupled to reflecting means, whereby movement of said element results in either information from the normal field of view, or omnidirectional image information being reflected from said image plane of the camera.
10. 10. Apparatus according to any preceding claim, wherein said optical means comprises a barrel portion which can either be demountably attached, or is fixed to the camera, said barrel portion having a window with means for dispersing atmospheric drops or particles.
11. 11. Apparatus according to claim 10, wherein said barrel portion has at least one branch portion in which respective panospherical image capturing means is mounted.
12. 12. Apparatus according to any preceding claim, including means for maintaining optical surfaces clean and dry.
13. 13. Apparatus according to claim 12, wherein said means for maintaining optical surfaces clean and dry includes any one of wiper blades; centrifugally driven surfaces; and/or means for pressurising the apparatus with heated dry air.
14. 14. Apparatus according to any preceding claim, including a screen for receiving an omnidirectional image from the panospherical means, whereby an image on the screen means can then be picked up on the image plane of the camera.
15. 15. Apparatus according to any preceding claim, including a CCD device for capturing the respective image information in either the omnidirectional or normal viewing modes.
16. 16. Apparatus according to any of claims 1-14, including an auxiliary imaging means, which can be moved into or out of an optical path to the image plane of the camera, for receiving either the omnidirectional information, or the optical information from the normal field of view so as to provide auxiliary image information.
17. 17. Apparatus according to any preceding claim, wherein the panospherical image capturing means includes back-to-back hemispherical image capturing means.
18. 18. Apparatus according to claim 17, further including one or more reflecting surfaces which move into a position in order to reflect respective hemispherical images onto the image plane of the camera, and into a different position to enable the image plane to see the normal field of view.
19. 19. Apparatus according to any preceding claim, wherein said panospherical image capturing means includes any one or more of fish eye lenses, convex reflectors, or a plurality of planar reflectors.
20. 20. Apparatus according to any preceding claim, wherein said image plane also provides a support for film whereby the camera can be used for still or motion

    photography.
21. 21. Surveillance apparatus for capturing image information from a panoramic scene, comprising an imaging device having a standard and preferably zoom lens system; means for mounting the imaging device on an axis of rotation so that it can be rotated to a selected azimuthal position, whereby the zoom lens system can capture image information from an area in the panoramic scene within its field of view, the imaging device further including panospherical image capturing means for capturing an omnidirectional image from said scene; and selecting means which are operable so as to capture either image information from the standard lens system of the imaging device, or from the panospherical image capturing means, or both, depending on a selected mode of operation.
22. 22. Apparatus according to claim 21, wherein the imaging device includes means for converting optical information into video signals, and further including image processing and display means for displaying either the image information from the zoom lens, or the omnidirectional information from the panospherical image capturing means.
23. 23. Apparatus according to claim 22, further including means coupled between the imaging device and the image display means for providing azimuthal position data as the imaging device is rotated about said axis, whereby activity observed in an area within the panospherical scene can be viewed by rotating the optical device to the respective azimuthal position.
24. 24. Apparatus according to claim 23, wherein the imaging device is coupled to the processing and display means so that the imaging device can be automatically guided to observe activity in an area within the panospherical image.
25. 25. Apparatus according to any of claims 21-24, wherein motion sensing means provides pan, tilt and/or elevation data for controlling the position of the imaging device.
26. 26. Apparatus according to any of claims 21-25 wherein the panospherical image capturing means includes optical means for capturing hemispherical images, and

    wherein the standard lens system has its normal field of view between the means for capturing the hemispherical images.
27. 27. Apparatus according to claim 26, wherein the hemispherical image means are mounted either side by side to provide omnidirectional information on each side of a central zone, the standard lens then having said central zone as its normal field of view, or one above the other to omnidirectional information from regions above and below a scene, the standard lens then having a peripheral part of the regions as its normal field of view.
28. 28. Apparatus for providing a panospherical field of view and a normal field of view, the apparatus being provided either in the form of an adapter, or as part of a camera, the apparatus including: (a) panospherical image capturing means for capturing omnidirectional information from around a panoramic scene and for supplying it to a first image plane; (b) optical means for capturing image information from part of the panoramic scene in a normal viewing mode and having a second imaging plane; whereby both the omnidirectional information, as well as the information from the normal field of view is available from both the first and second imaging planes.
29. 29. A panospherical imaging device comprising a plurality of planar reflecting surfaces arranged to reflect light from a panoramic scene into corresponding light guiding means; optical means for combining image information received from the light guiding means so as to provide a composite image of at least an annular section of the panoramic scene.
30. 30. Apparatus according to claim 29, wherein the planar reflecting surfaces are

    sides of a pyramiddal structure, said sides being triangular or quadilateral in shape and inclined with respect of the annular section of the scene.
31. 31. Apparatus according to claims 29 or 30, wherein each light guiding means is arranged to view an area including at least a the respective part of the planar reflecting surface, and preferably also a portion of adjacent planar reflecting surfaces, whereby overlapping optical information is captured for the purpose of joining images from the adjacent planar surfaces, when producing the composite image.
32. 32. Apparatus according to any of claim 29-31, wherein the light guiding means comprise planar reflecting surfaces or prisms.
33. 33. Apparatus according to any of claims 29-32, wherein there are at least six planar reflecting surfaces for each pyramid and preferably eight.
34. 34. Apparatus according to any of claims 29-33, including a standard lens system located centrally of the planar reflecting surfaces at the apex of a pyramid for receiving light directly incident from the other part of the panoramic scene than the annular section.
35. 35. Apparatus according to claim 34, wherein the lens system is situated at the apex of a pyramid structure for directing light inwardly of the structure.
36. 36. Apparatus according to any of claims 1-4, wherein the selecting means includes a beam splitter which either passes or reflects image information from either the normal or omnidirectional field of view to the image plane of the camera, and inhibiting means which either (a) operates either to prevent the image information from reaching, or to pass the image information to the beam splitter, from either the omnidirectional or normal field of view, or (b) operates so as to cause the beam splitter to reflect or to pass the image information from either the omnidirectional or normal field of view.
37. 37. Apparatus according to claim 36, wherein the inhibiting means includes respective stops, filters, irises or blinds which block at least a major part of the image information incident on, or passing through either the panospherical, or optical image capturing means, whereby the beam splitter either passes, or reflects incident light onto the image plane of the camera.
38. 38. Apparatus according to claims 36 or 37, wherein the beam splitter includes a semi reflective surface which passes and reflects image information from the normal or omnidirectional field of view, or vice versa, to the image plane of the camera.
39. 39. Apparatus according to claims 38, wherein the inhibiting means includes means on, or proximate to the semi reflective surface, for selectively preventing the image information from reaching the beam splitter according to whether the omnidirectional or normal field of view is selected.
40. 40. Apparatus according to any of claims 36 to 39, wherein the inhibiting means includes a transparent display material having a matrix or mask of optically active elements that can be switched (a) either cause the beam splitter to pass or to reflect the image information from the normal or omnidirectional field of view, or (b) to block or to pass the image information from the omnidirectional or normal field of view.
41. 41. Apparatus according to any of claims 1-4, wherein respective image sensing means are provided for the image information from the normal or omnidirectional fields of view and wherein the selecting means includes display means, and means for switching, or for processing the information to enable either field of view to be displayed.