GB2633401A - Vehicle steering rack control based on steering input device image data - Google Patents

Abstract

A vehicle image processing system is provided. Image signals, including image data of a steering input device 101 of a vehicle (such as a steering wheel) are received from vehicle image sensors (typically cameras); this may include the position of a driver’s hands on a steering wheel. A position of steering input device 101 is determined based on the received image data. A signal indicating the determined position of the steering input device is output to a steering rack control system to control a steering rack 21 of the vehicle. Also provided are a vehicle steering rack control system, a vehicle, a method, and computer readable instructions.

Description

VEHICLE STEERING RACK CONTROL BASED ON STEERING INPUT DEVICE IMAGE DATA

TECHNICAL FIELD

The present disclosure relates to vehicle steering rack control based on steering input device image data.

Aspects of the invention relate to a vehicle image processing system, to a vehicle steering rack control system, to a vehicle system, to a vehicle, to a method, and to computer readable instructions.

BACKGROUND

A vehicle such as a car has a steering input device -typically, a steering handwheel -which is actuated, e.g. rotated, by a driver of the vehicle to control a steering angle of steered road wheels of the vehicle. Conventionally, a mechanical connection is provided between the steering handwheel and a steering rack that controls the position of the steered road wheels. However, in so-called 'steer-by-wire' systems, this interface is instead provided by electrical signals between a steering handwheel control module and a steering rack control module. In particular, the steering input or request of the vehicle driver is monitored by the steering handwheel control module, e.g. via sensors measuring the driver's actuation of the steering handwheel, and sent to the steering rack control module, which determines a steering rack demand signal for controlling the steering rack in accordance with the driver's request.

In order to ensure continued availability of the steer-by-wire system, a level of redundancy in the system may be provided. For instance, duplication of electrical components, e.g. sensors, in the steer-by-wire system may be provided in order to protect against failure of one or more such components. However, duplication of components in this manner can add bulk, cost and complexity to the system. In addition, such duplication may also not be of help in the case of a failure in which duplicate components both fail, e.g. a loss of power.

It is an aim of the present invention to address one or more of the disadvantages associated with the prior art.

SUMMARY OF THE INVENTION

Aspects and embodiments of the invention provide a vehicle image processing system, a vehicle steering rack control system, a system, a vehicle, a method, and computer readable instructions as claimed in the appended claims According to an aspect of the present invention there is provided an image processing system for a vehicle.

The image processing system comprises one or more processors collectively configured to: receive one or more image signals from one or more image sensors of the vehicle, the one or more image signals including image data of a steering input device of the vehicle; determine, in dependence on the received image data of the steering input device, a position of the steering input device; and, output a signal indicative of the determined position of the steering input device. The signal may be output to a steering rack control system for controlling a steering rack of the vehicle.

The position of the steering device may include an instantaneous position of the steering device and/or a change in position (e.g. a 'delta) of the steering device, e.g. relative to a previous frame of the image data.

The present invention is advantageous in that it provides a technique for controlling a steering rack of a vehicle that may be used in conjunction with, or instead of, existing steering rack control methods. This allows for more accurate and/or reliable control of the vehicle steering rack. Advantageously, a steering intention of a driver may be determined based on image signals indicative of a steering input device position, with control of the steering rack then being based on the determined driver intention. Determination of driver intention in this manner may be beneficial when alternative approaches for determining driver intention are unavailable or are less accurate than desired. The invention beneficially uses image sensors that may already be present in a vehicle.

In examples in which the invention is used as an alternative to existing methods for controlling a vehicle steering rack, the invention provides a level of redundancy to maintain steering rack operation in case of a fault occurring in the existing method. The invention beneficially provides such redundancy without the need for additional hardware such as duplicate/secondary sensors or other power elements. In a steer-by-wire vehicle steering system, the existing method may involve a driver steering intention being measured by sensors monitoring driver input to the vehicle steering device, and then a steering device controller / control module transmitting the detected driver intention to the steering rack controller / control module. A fault in the existing method may occur in the event of a failure of one or more steering input device sensors and/or a communication error between the steering device controller and the steering rack controller, for instance.

The image processing system comprises one or more controllers collectively comprising at least one electronic processor having an electrical input for receiving an input signal; and at least one memory device electrically coupled to the at least one electronic processor and having instructions stored therein; and wherein the at least one electronic processor is configured to access the at least one memory device and execute the instructions thereon so as to: receive the one or more image signals from the one or more image sensors; determine the position of the steering input device; and, output the signal indicative of the determined position of the steering input device to the steering rack control system for controlling the steering rack of the vehicle.

In an embodiment, to determine the position of the steering input device the image processing system is configured to determine a change in position of the steering input device by comparing a current frame of the received image data relative to a previous frame of the received image data. Beneficially, this provides substantially real-time updates to the steering input device position.

In an embodiment, the received one or more image signals include image data of one or more hands of a driver of the vehicle. The image processing system may be configured to determine the position of the steering input device in dependence on the received image data of the driver's hands. The position of the steering device may include an instantaneous position of the steering device and/or a change in position (e.g. a 'delta') of the steering device, e.g. relative to a previous frame of the image data. Beneficially, the position of the driver's hands can be used as an additional indication of steering input position to increase accuracy. For instance, this may be beneficial in a case in which a part of the steering input device is obstructed from view of the image sensor(s), e.g. by the driver's hands, such that it is not possible or less accurate to determine steering input device position based on image data of the steering input device itself The image processing system may be configured to determine, in dependence on the received image data of the driver's hands, whether at least one of the driver's hands is on the steering input device. The image processing system may be configured to determine that the position of the steering input device is unchanged relative to a previously-determined steering input device position if it is determined that neither of the driver's hands are on the steering input device. Beneficially, this removes the need for further image processing when it can more readily be deuced that no positional change of the steering device has occurred.

In an embodiment, the image processing system is configured to determine a change in position of at least one of the driver's hands on the steering input device relative to a previous frame of the received image data.

In an embodiment, the image processing system is configured to determine the position of the steering input device in dependence on the determined change in position of at least one of the driver's hands. Beneficially, a change in position of the driver's hands may be used as a proxy for -or additional information to assist the determination of-the position of the steering input device.

In an embodiment, the image processing system is configured to determine that the position of the steering input device is unchanged relative to a previously-determined steering input device position if it is determined that there is no change in the position of the driver's hands. Beneficially, this uses the fact that typically the driver's hands will need to move in order to effect a positional change of the steering input device, and to remove the need for further image processing in the case in which no such hand movement is detected.

In an embodiment, the image processing system is configured to determine a rate of change of position of the steering input device in dependence on the determined steering input device position at two or more successive frames of the received image data. The output signal may be indicative of the determined rate of change of position of the steering input device. Beneficially, this additional parameter may be used to provide a more accurate indication of the driver's steering intention for controlling the vehicle.

In an embodiment, the steering input device is a steering handwheel of the vehicle. The determined steering input device position may be an angular position of the steering handwheel. In other embodiments, other suitable steering input devices may be used, e.g. a joystick.

According to another aspect of the invention, there is provided a system comprising the image processing system defined above, and comprising a steering rack control system for controlling a steering rack of the vehicle. The steering rack control system comprises one or more processors collectively configured to: receive the output signal from the image processing system; determine a steering rack position demand in dependence on the received output signal; and, output a control signal to control the steering rack in accordance with the determined steering rack demand. This advantageously allows the steering rack control system to determine a steering rack position demand based on image data of the steering input device, which may provide an alternative or additional source of information on which to determine a driver's steering intention for the vehicle. This may be beneficial if one or other sources of information for such a use are not available. Alternatively, this may be beneficial for increasing the accuracy of the steering rack position demand as a reflection of the driver's steering intention.

In an embodiment, the steering rack control system may be configured to receive a steering signal from a steering input device system of the vehicle. To determine the steering rack position demand the steering rack control system may be configured to: select one of the steering signal and the output signal as a selected signal, and determine the steering rack position demand in dependence on the selected signal. In this way, redundancy of the system (in a steer-by-wire system, for instance) may be increased as if the signal from one source, e.g. the steering input device system, is unavailable or faulty, then the signal from the other source the image processing system -may instead be used for the determination of steering rack position demand.

The steering rack control system may be configured to: receive an indication of whether there is an error associated with the steering signal; and, select the output signal from the image processing system as the selected signal when the indication of an error is received. In this way, the output signal from the image processing system can beneficially be used as a backup signal for providing an indication of the driver's steering intention when a primary signal -from the steering device control system -is unavailable or faulty.

According to another aspect of the present invention there is provided a steering rack control system for controlling a steering rack of a vehicle. The steering rack control system comprises one or more processors collectively configured to: receive a steering signal from a steering input device system of the vehicle, the steering signal being indicative of a position, and optionally a rate of change of position, of the steering input device determined based on steering input device sensor signals received by the steering input device system; receive the output signal from the image processing system defined above; select one of the steering signal and the output signal as a selected signal, and determine a steering rack position demand in dependence on the selected signal; and, output a control signal to control the steering rack in accordance with the determined steering rack demand.

According to an aspect of the present invention there is provided a vehicle comprising the image processing system defined above. According to an aspect of the present invention there is provided a vehicle comprising the system defined above. According to an aspect of the present invention there is provided a vehicle comprising the steering rack control system defined above.

According to another aspect of the present invention there is provided a method for a vehicle, comprising: receiving one or more image signals from one or more image sensors of the vehicle, the one or more image signals including image data of a steering input device of the vehicle; determining, in dependence on the received image data of the steering input device, a position of the steering input device; and, outputting a signal indicative of the determined position of the steering input device to a steering rack control system for controlling a steering rack of the vehicle.

According to another aspect of the present invention there is provided computer readable instructions which, when executed by a computer, are arranged to perform the method defined above.

Within the scope of this application it is expressly intended that the various aspects, embodiments, examples and alternatives set out in the preceding paragraphs, in the claims and/or in the following description and drawings, and in particular the individual features thereof, may be taken independently or in any combination. That is, all embodiments and/or features of any embodiment can be combined in any way and/or combination, unless such features are incompatible. The applicant reserves the right to change any originally filed claim or file any new claim accordingly, including the right to amend any originally filed claim to depend from and/or incorporate any feature of any other claim although not originally claimed in that manner.

BRIEF DESCRIPTION OF THE DRAWINGS

One or more embodiments of the invention will now be described, by way of example only, with reference to the accompanying drawings, in which: Figure 1 shows a schematic illustration of a vehicle in accordance with an aspect of the invention; Figure 2 shows a schematic top view of the vehicle of Figure 1, including components of a steer-by-wire system of the vehicle, and an image processing system in accordance with an aspect of the invention; Figure 3 schematically illustrates components of the image processing system of Figure 2; Figure 4 schematically illustrates a control scheme implemented by the image processing system of Figure 2; Figure 5 schematically illustrates a control scheme implemented by a steering rack control module of the steerby-wire system of Figure 2; and, Figure 6 summarises the step of a method performed by the image processing system of Figure 2 in accordance with an aspect of the invention.

DETAILED DESCRIPTION

Figure 1 shows a vehicle 10 in the form of a car. The vehicle 10 includes a steering input device 101 in the form of a steering handwheel (or, simply, steering wheel). In a known manner, the steering wheel 101 is positioned in front of a driver's seat of the vehicle 10 adjacent a dashboard of the vehicle 10. The steering wheel 101 is rotatable/actuatable by the driver so as to steer the vehicle 101. Specifically, rotation of the steering wheel 101 causes a change in steering angle of steered road wheels 102 of the vehicle 10. In the described example, the vehicle 10 has four road wheels, including two each at the front and the rear of the vehicle 10. In the described example, the front wheels 102 are the steered wheels of the vehicle 10, whereas the steering angle of the rear wheels 103 does not change. In different examples, the rear wheel may be steered wheels, either in addition to, or instead of, the front wheels. While the vehicle 10 illustrated in Figure 1 is a left-hand drive vehicle with the steering wheel 101 on the left-hand side of the vehicle cabin 104, it will be appreciated that the disclosure provided herein is also applicable to a right-hand drive vehicle.

With additional reference to Figure 2, the vehicle steering is controlled or initiated by a so-called 'steer-by-wire' (SBW) system 20. Unlike in conventional (front) steering systems, there is no mechanical linkage between the steering wheel 101 and a steering rack 21 of the vehicle 10. The SBW system 20 includes a steering input device (steering handwheel/wheel) control system/module 201 and a steering rack control system/module 202.

The control modules 201, 202 are in communication with one another via electrical signals transmitted along electrical wiring 203 of the SBW system 20. In alternative examples, the control modules 201, 202 may be in wireless communication with one another.

The steering wheel control module 201 is configured to receive signals indicative of rotation/actuation of the steering wheel 101. In particular, the SBW system 20 includes one or more torque and/or angle sensors 204 for monitoring or measuring how the steering wheel 101 is being moved (by the vehicle driver). The steering wheel control module 201 is configured to receive monitored sensor data collected by the torque/angle sensor(s) 204 via electrical signals 205. The steering wheel control module 201 may determine one or more (steering) parameters indicative of steering wheel actuation based on the received sensor signals 205. For instance, these may include one or more of: measured rotation angle/position of the steering wheel 101; a rate of change of rotation angle/position of the steering wheel 101; and, a diagnostic status of the steering wheel control system 201 and/or sensor(s) 204. The steering wheel rotation angle/position may in this context be an instantaneous position of the steering wheel 101 or it may be a change in position of the steering wheel 101, for instance since a previous sampling time/instant of the sensor(s) 204.

The steering rack control module 202 is configured to determine a steering rack demand signal based on the received signals 204 from the steering wheel control module 201. The steering rack control module 202 is then configured to output a control signal 206 to control the steering rack 21 of the vehicle 10 in accordance with the determined steering rack demand. In particular, the control signal 206 may be output to a steering rack actuator that actuates the steering rack 21 in accordance with the steering rack demand. The steering rack demand may be determined based on one or more of the above-mentioned parameters, e.g. steering wheel rotation angle/position. In alternative embodiments, the signals 204 received by the steering rack control module 202 include the measured sensor output 205 received by the steering wheel control module 201 from the steering wheel sensor(s) 204, and one or more of the steering parameters are then determined by the steering rack control module 202. The steering rack control module 202 may determine different parameters either in addition to, or as an alternative to, the above-mentioned steering parameters. For instance, these may include a measured position of the steering rack 21, e.g. using one or more appropriate position sensors, and/or a diagnostic status of the steering rack control module 202.

With continuing reference to Figures 1 and 2, and with additional reference to Figure 3, the vehicle 10 includes an image processing system 30 comprising a controller 301. The controller 301 comprises processing means 302 and memory means 303. The processing means 302 may be one or more electronic processing devices which operably execute computer-readable instructions. The memory means 303 may be one or more memory devices. The memory means 303 is electrically coupled to the processing means 302. The memory means 303 is configured to store instructions, and the processing means 302 is configured to access the memory means 303 and execute the instructions stored thereon.

The controller 301 comprises an input means 304 and an output means 305. The input means 305 may comprise an electrical input of the controller 301. The output means 305 may comprise an electrical output of the controller 301. The input 304 is arranged to receive an image signal 31 from an image sensor, as discussed in greater detail below. The image signal 31 is an electrical signal which is indicative of image data that includes, inter alia, the vehicle steering wheel 101. The output 305 is arranged to output a signal 32 that is indicative of signal indicative of a determined position of the steering wheel 101 to the steering rack control system 202. The processing means 302 may comprise relatively powerful computing devices. The processing means 302 is configured to perform image-image video processing to determine a change ('delta') between successive frames of image data received at the input 304. The image processing system 30 may for instance include a higher level of processing power than the steering wheel control module 201 and/or the steering rack control module 202. The image processing system hardware may be existing hardware of the vehicle 10 that is used for different purposes, e.g. for autonomous driving functions of the vehicle.

The vehicle 10 includes one or more image sensors 105 in the form of cameras. The or each camera collects image data from inside the vehicle cabin 104. In this way, each camera may be referred to as an interior-facing camera of the vehicle 10. Each camera may be positioned in the vehicle cabin 104. Each camera is arranged to receive image data that is indicative of a position of the vehicle steering wheel 101. As such, each camera may be positioned to be directed at an area that includes the steering wheel 101, and may be positioned in the vicinity of the steering wheel 101. In the embodiment illustrated in Figures 1 and 2, one camera 105 is shown. The indicated camera is positioned above a rear view mirror 106 of the vehicle 10, adjacent to a windscreen of the vehicle 10. The camera 105 is arranged such that it may collect image data of the steering wheel 101, and optionally the vehicle driver when they are in the vehicle driver seat 107. It will be understood that in different embodiments there may be interior-facing cameras positioned at different locations for collecting image data of the steering wheel 101, optionally in addition to, or as an alternative to, the camera above the rear-view mirror 106. This could include cameras positioned in one or more of the following positions: within an instrument cluster area of the vehicle 10; above the vehicle driver seat 107, e.g. in a roof (lining) of the vehicle 10; and/or, to a side of the driver seat 107, e.g. in a central console unit, in a central armrest beside the driver seat 107, in a driver's door 108 of the vehicle 10, and/or in a side mirror 109 of the vehicle 10.

The image processing system 30 is configured to determine a steering intention or request of the vehicle driver based on image data received from the image sensor(s) 105, and to communicate the determined driver steering intention to the steering rack control module 202. Figure 4 illustrates processing blocks/modules implemented by the image processing system 30 in accordance with one embodiment of the invention. At block 401, the image processing system receives image data collected by the image sensor(s) 105. As mentioned above, the collected image data is reflective of a position of the steering wheel 101, and so the collected image data will typically include (at least a portion of) the steering wheel 101 and optionally at least part of the vehicle driver, e.g. one or more of their hands. As such, the image sensor(s) 105 may be regarded as driver-facing cameras. The image sensor(s) 105 may be configured to substantially continuously collect image data. The sensor(s) 105 may typically sample image data at a certain sampling rate/frequency, e.g. N samples per second, where N is any suitable positive integer. Each sample may comprise a frame of image data, such that the image processing system 30 receives N frame of image data per second.

At block 402, the image processing system 30 analyses each frame of image data received from the image sensor(s) 105. The system 30 identifies the steering wheel 101 in a frame of image data, and then determines the position of the steering wheel 101, i.e. the angular or rotational position of the steering wheel 101, in the image frame. This may be performed in any suitable way. For instance, the system 30 may be configured to identify and track the position of one or more defined points/portions of the steering wheel 101 in order to determine an overall rotational position of the steering wheel 101. This may vary depending on the specific steering wheel under consideration as different vehicles may have steering wheels of different designs each having specific aspects or features that may be tracked for the present purposes. For instance, the steering wheel 101 may have three arms/prongs each extending from a central boss/portion of the steering wheel 101 to an outer ring/rim of the steering wheel. The arms are spaced apart from one another by a defined/known amount, and are positioned about the circumference of the steering wheel 101 for a certain, known steering angle (rotational position of the steering wheel 101).

In one example, therefore, the image data from the image sensor(s) 105 may be analysed to identify certain defined points/portions of the steering wheel 101, e.g. one or more arms of the steering wheel 101, in an image frame, and then determine the position of the defined points/portions relative to other, fixed portions of the vehicle appearing in the image frame, e.g. a steering column, a head-up display, etc, in order to determine the rotational position of the steering wheel 101 in the image frame. It will be understood that any other suitable features of the steering wheel may be used to determine its rotational position, e.g. differences in surface colour and/or texture at different parts of the steering wheel.

In the described embodiment, at block 402 the image processing system 30 also analyses the received image data to identify one or more of the vehicle driver's hands in an image frame, and then determines the position of the driver's hand(s), specifically relative to the steering wheel 101. The system 30 may use known, defined features of hands to identify one or more hands in the received image frame under consideration. The system 30 then determines a position of one or more of the driver's hands relative to the steering wheel 101.

In the above, known image processing techniques may be used to determine the rotational position of the steering wheel and/or the position of the driver's hand(s) relative to the steering wheel. It will be understood that any suitable image processing techniques may be used. As an illustrative example, the techniques described in Chapter 3 of "The Essential Guide to Video Processing", by Alan C. Bovik (2009), may be used for the present purposes.

In the described example, at block 403 the image processing system 30 is configured to determine whether at least one of the driver's hands are on -i.e. in contact with -the steering wheel 101. The determination is based on the determination of the position of the driver's hands relative to the steering wheel 101 in block 402. If it is determined that neither of the driver's hands are present on (in contact with) the steering wheel 101 in the current image frame of image data under consideration, then the system 30 determines at block 404 that no change in the rotational position of the steering wheel 101 has occurred since a determined steering wheel rotational position in a previous image frame. This is based on an assumption that at least one of the driver's hands needs to be in contact with the steering wheel 101 in order to actuate/move/rotate the steering wheel 101 to a different rotational/angular position. As the image sensor will typically be sampling image data several times per second, this assumption also relies on the fact that there would not be sufficient time between image frames for the driver to move the steering wheel 101 by a significant amount and then also remove their hands from the steering wheel 101. This determination beneficially means that further processing to determine a change in position of the steering wheel and/or driver's hands since a previous such positional determination, e.g. at a previous time step at which the image data was sampled, is not needed for the current image frame under consideration.

If it is determined at block 403 that at least one of the driver's hands is in contact with the steering wheel 101 in the current image frame, then at block 405 the image processing system 30 is configured to determine whether the position of the driver's hand(s) is different in the current image frame relative to a previous image frame. This may be performed by determining a difference between the determined position of the driver's hands in these successive image frames, optionally with reference to a threshold change. If there is determined to be no change in the driver's hand(s) position then the system 30 again determines at block 404 that no change in the rotational position of the steering wheel 101 has occurred since the determined steering wheel rotational position in the previous image frame. This assumption relies on the fact that there would not be sufficient time between successively received image frames for the driver to move the steering wheel using their hand(s) and then return their hand(s) to the starting/previous position.

If it is determined at block 405 that the position of the driver's hand(s) is different from the previous image frame, then the system 30 determines that the steering wheel 101 has changed position since the previous image frame. In such a case, at block 406 the image processing system 30 determines a change in position (delta position) of the steering wheel 101. This may be performed by comparing the determined/evaluated position of the steering wheel 101 in the current image frame (block 402) with the determined/evaluated position of the steering wheel 101 in the previous image frame. In the described example, at block 407 the image processing system 30 determines a change in position (delta position) of the driver's hand(s). Again, this may be performed by comparing the determined/evaluated position of the driver's hand(s) in the current image frame (block 402) with the determined/evaluated position of the driver's hand(s) in the previous image fra me.

The positional changes or deltas determined at blocks 406, 407 may be fused together at block 408 to determine an overall rotational or angular position of the steering wheel 101. In some embodiments, the determined delta of the steering wheel position at block 406 may be sufficient to determine the overall steering wheel position. However, there may be instances where one or more of the points/portions of the steering used to determine the position or delta of the steering wheel is obscured in a particular image frame under consideration, e.g. by the driver, as a result bad lighting, etc. In such cases, the determined steering wheel position or delta may be less accurate or it may even not be possible to make the determination in the event of significant obstruction. It can therefore be beneficial to use the driver's hand(s) position or delta to supplement, or even replace, the steering wheel position or delta in the determination of overall steering wheel rotational or angular position at block 408 in order to increase the accuracy of the overall determination across different scenarios.

In the described example, at block 409 the image processing system 30 determines a rate of change of the rotational or angular position of the steering wheel 101. This may be calculated in a known manner based on the determination of the overall steering wheel position at block 408 at two or more successive frames of received image data.

At block 410 the image processing system 30 is configured to communicate the determined steering wheel position and rate of change of position to the steering rack control module 202. In particular, this is in the form of the output signal 32 at the output 305 of the image processing system 30. The output signal 32 is in the form of an electrical signal, which may be a wired or wireless signal. For instance, the output signal 32 may be transmitted along a CAN bus of the vehicle 10. In the described embodiment, the output signal 32 is transmitted to the steering rack control module 202 irrespective of whether a change of position has occurred or not. That is, if it is determined at block 404 that no positional change has occurred, then this is communicated to the steering rack control module 202 via the output signal 32. On the other hand, if a positional change has occurred, then the calculations at blocks 408, 409 are communicated to the steering rack control module 202 via the output signal 32.

In the described embodiment, the steering rack control module 202 is configured to receive the output signal 32 from the image processing system 30 and to determine a steering rack position demand based on the received output signal 32. The output signal 32 may be regarded as the driver's steering intention or request for controlling the vehicle steering, and so the steering rack position demand is determined so that the steering rack 21 is controlled to reflect the driver's intention, i.e. the steering rack is controlled such that the vehicle steering is adjusted in accordance with the driver's intention/input. Specifically, the steering rack control module 202 is configured to transmit a control signal to a steering rack actuator that actuates the steering rack, the control signal being for controlling the steering rack actuator to actuate the steering rack 21 in accordance with the determined steering rack position demand.

Figure 5 schematically illustrates control blocks of the steering rack control module 202 in an embodiment in which the steering rack control module 202 can receive an indication of driver steering input/intention from multiple sources, namely, the steering wheel control module 201 and the image processing system 30. Whereas in the previous embodiment, the output signal 32 from the image processing system 30 is used to determine the steering rack position demand, in the present embodiment the steering wheel control module 201 makes a determination as to which received signal to use to determine the steering rack position demand.

In the described example, the steering rack control module 202 is configured to receive the output signal 32 from the image processing system 30. The output signal 32 may be received substantially continuously. In addition, the steering rack control module 202 is configured to receive a steering signal 51 from the steering wheel control module 201. The steering signal 51 is indicative of a rotational or angular position, and optionally a rate of change of said position, of the steering wheel 101 as determined by the steering wheel control module 201.

In the described example, the steering wheel control module 201 is configured to receive sensor data from one or more sensors of the vehicle 10 that measure or monitor movement of the steering wheel 101. As mentioned above, these sensors are in the form of torque sensors or angle sensors for measuring a rotation angle through which the steering wheel 101 is actuated. The torque/angle sensors may typically sample data at a certain sampling rate/frequency, e.g. N samples per second, where N is any suitable positive integer, with the steering wheel control module 201 then receiving N samples per second from the torque/angle sensors.

The sampling rate of the torque/angle sensors may be the same as, or different from, the sampling rate of the image sensor(s) 105. The steering wheel control module 201 determines/calculates a steering angle/position being requested by the vehicle driver based on the received torque/angle sensor data. This may be performed in any suitable manner, and may involve an analogue-to-digital conversion, as well as scaling and offset calculations. The steering wheel angle may in some examples be a change in steering wheel angle, e.g. relative to a previous time step or sampling point. In the described example, the steering wheel control module 201 also determines/calculates a rate of change of the steering angle/position being requested by the vehicle driver. Again, this may be performed in any suitable manner and may be based on the determined steering angle at two or more time steps or sampling points. The steering wheel control module 201 is then configured to send/transmit the determined steering angle, and optionally rate of change of steering angle, to the steering rack control module 202 as the steering signal 51.

Still referring to Figure 5, the steering rack control module 202 is configured to select one of the received output signal 32 and the received steering signal 51, and to determine/calculate the steering rack position demand based on the selected signal at control block 52. The selection may be performed as described below. The steering rack control module 202 is then configured to output a control signal 53 to control the steering rack 21 in accordance with the determined steering rack demand, in particular by controlling the steering rack actuator.

In the described example, the steering rack control module 202 is configured to receive a further signal from the steering wheel control module 201, namely, an error state signal (or diagnostic status signal) 54. The error state signal 54 provides an indication of whether the steering signal 51 from the steering wheel control module 201 accurately reflects the driver's steering intention/input, or whether it is inaccurate or includes an error. In order to determine the error state of the steering signal 51, or the steering wheel control module 201 more generally, the steering wheel control module 201 is configured to perform one or more checks on the torque/angle sensor data being received by the steering wheel control module 201. For instance, the control module 201 may check/monitor that sensor data from the torque/angle sensors is indeed being received, in order to check that none of the sensors are faulty. The control module 201 may check that the sensor data being received is within certain bounds that such data would be expected to be in. The control module 201 may also cross-check the data received from different sensors -if more than one sensor is being used -to determine that the received data is consistent. Based on the various checks of the sensor(s) and/or sensor circuits, the steering wheel control module 201 determines whether there is an error associated with the sensor data, and therefore the steering wheel position being calculated by the steering wheel control module 201 based on the sensor data. The steering wheel control module 201 is then configured to communicate to the steering rack control module 202 via the error state signal 54 whether there is determined to be an error in the operation of the steering wheel control module 201 such that the steering signal 51 is faulty, inaccurate or absent.

In the described example, if the error state signal 54 received by the steering rack control module 202 indicates at control block 55 that the input received from the steering wheel control module 201 -i.e. the steering signal 51 -does not include an error, then the steering rack control module 202 is configured to select the steering signal 51 from the steering wheel control module 201 at control block 56 as the signal to be used to calculate the steering rack position demand at block 52. On the other hand, if the error state signal 54 received by the steering rack control module 202 indicates at control block 55 that the input received from the steering wheel control module 201 does include an error, then the steering rack control module 202 is configured to select (or switch to) the output signal 32 from the image processing system 30 at control block 57 as the signal to be used to calculate the steering rack position demand at block 52. If one or both of the steering signal 51 and/or the error state signal 54 is not received by the steering rack control module 202 -e.g. because of a power failure of the steering wheel control module 201 and/or the communication channel between the control modules 201, 201 -then the control module 202 may be configured to select the output signal 32 as the selected signal to calculate steering rack position demand. The determination at control block 55 may be made substantially constantly so that intermittent loss of communication with the steering wheel control module 201 may be replaced with the steering intention from the output signal 32 from the image processing system 30. In different embodiments, the steering signal 51 and the output signal 32 may be combined and/or fused together to provide an overall driver steering intention signal, and the steering rack position demand is then calculated based on this overall signal.

Figure 6 shows the steps of a method 1000 according to an embodiment of the invention. The method is for implementation in a vehicle, such as the vehicle 10 in Figures 1 and 2. The method 1000 may be performed by the image processing system 30 illustrated in Figure 3. In particular, the memory 303 may comprise computer-readable instructions which, when executed by the processor 302, causes the processor 302 to perform the method 1000. At step 1100, the method 1000 involves receiving one or more image signals from one or more image sensors 105 of the vehicle 10. The image signal(s) includes image data of a steering input device, e.g. a steering handwheel 101, of the vehicle 101. At step 1200, the method 1000 involves determining a position of the steering input device 101. The determination is performed in dependence on the received image data of the steering input device 101. The position may refer to an instantaneous position of the steering input device or may refer to a change in position of the steering input device, e.g. a change in position between successive frames of received image data. In an example in which the steering input device is the steering wheel 101, the position may refer to a rotational or angular position of the steering wheel, e.g. relative to an initial or default rotational or angular position. At step 1300, the method 1000 involves outputting a signal 32 indicative of the determined position of the steering input device 101 to the steering rack control system 202 for controlling the steering rack 21 of the vehicle 10. In particular, a position, e.g. steering angle, of the steering rack 21 may be controlled (by a steering rack actuator) in accordance with a steering rack demand determined by the steering rack control system 202 in dependence on a driver steering request or intention deduced from the determined position of the steering input device 101.

It will be appreciated that various changes and modifications can be made to the present invention without departing from the scope of the present application.

The following references may be used in to replace the referenced numerals in the appended figures.

Table 1

Figure 4 4000 Driver facing camera takes N-images per second Figure 4 4002 Each image is evaluated for: position of hand(s); position of hand-wheel Figure 4 4004 Hand(s) present on steering wheel Figure 4 4006 Position different from previous step Figure 4 4008 No change to previous steering wheel position Figure 4 4010 Calculate delta position of each hand Figure 4 4012 Calculate delta position of handwheel Figure 4 4014 Calculated overall steering hand-wheel angular position Figure 4 4016 Calculate rate of change of steering hand-wheel angular position Figure 4 4018 Communicate angular position and rate of change to steering control Figure 4 4020 Camera and image processing system Figure 5 5000 Use input from steering hand-wheel Figure 5 5002 Steering hand-wheel input ok Figure 5 5004 Calculate steering rack position demand Figure 5 5006 Use input from camera / image processing Figure 5 5008 Steering rack position calculation

Claims (15)

1. CLAIMS1. An image processing system for a vehicle, the image processing system comprising one or more processors collectively configured to: receive one or more image signals from one or more image sensors of the vehicle, the one or more image signals including image data of a steering input device of the vehicle; determine, in dependence on the received image data of the steering input device, a position of the steering input device; and, output a signal indicative of the determined position of the steering input device to a steering rack control system for controlling a steering rack of the vehicle.
2. 2. An image processing system according to Claim 1, wherein to determine the position of the steering input device the image processing system is configured to determine a change in position of the steering input device by comparing a current frame of the received image data relative to a previous frame of the received image data.
3. 3. An image processing system according to Claim 1 or Claim 2, wherein the received one or more image signals include image data of one or more hands of a driver of the vehicle, and wherein the image processing system is configured to determine the position of the steering input device in dependence on the received image data of the driver's hands.
4. 4. An image processing system according to Claim 3, the image processing system being configured to: determine, in dependence on the received image data of the driver's hands, whether at least one of the driver's hands is on the steering input device; and, determine that the position of the steering input device is unchanged relative to a previously-determined steering input device position if it is determined that neither of the driver's hands are on the steering input device.
5. 5. An image processing system according to Claim 3 or Claim 4, wherein the image processing system is configured to determine a change in position of at least one of the driver's hands on the steering input device relative to a previous frame of the received image data, and wherein the image processing system is configured to determine the position of the steering input device in dependence on the determined change in position of at least one of the driver's hands.
6. 6. An image processing system according to Claim 5, wherein the image processing system is configured to determine that the position of the steering input device is unchanged relative to a previously-determined steering input device position if it is determined that there is no change in the position of the driver's hands.
7. 7. An image processing system according to any previous claim, wherein the image processing system is configured to determine a rate of change of position of the steering input device in dependence on the determined steering input device position at two or more successive frames of the received image data, and wherein the output signal is indicative of the determined rate of change of position of the steering input device.
8. 8. An image processing system according to any previous claim, wherein the steering input device is a steering handwheel of the vehicle, and wherein the determined steering input device position is an angular position of the steering handwheel.
9. 9. A system comprising the image processing system of any of Claims 1 to 8, the system comprising a steering rack control system for controlling a steering rack of the vehicle, the steering rack control system comprising one or more processors collectively configured to: receive the output signal from the image processing system; determine a steering rack position demand in dependence on the received output signal; and, output a control signal to control the steering rack in accordance with the determined steering rack demand.
10. 10. A system according to Claim 9, the steering rack control system being configured to receive a steering signal from a steering input device system of the vehicle, and wherein to determine the steering rack position demand the steering rack control system is configured to: select one of the steering signal and the output signal as a selected signal, and determine the steering rack position demand in dependence on the selected signal.
11. 11. A system according to Claim 10, the steering rack control system being configured to: receive an indication of whether there is an error associated with the steering signal; and, select the output signal from the image processing system as the selected signal when the indication of an error is received.
12. 12. A steering rack control system for controlling a steering rack of a vehicle, the steering rack control system comprising one or more processors collectively configured to: receive a steering signal from a steering input device system of the vehicle, the steering signal being indicative of a position, and optionally a rate of change of position, of the steering input device determined based on steering input device sensor signals received by the steering input device system; receive the output signal from the image processing system of any of Claims 1 to 8; select one of the steering signal and the output signal as a selected signal, and determine a steering rack position demand in dependence on the selected signal; and, output a control signal to control the steering rack in accordance with the determined steering rack demand
13. 13. A vehicle comprising the image processing system of any of Claims 1 to 8, the system of any of Claims 9 to 11, or the steering rack control system of Claim 12.
14. 14. A method for a vehicle, comprising: receiving one or more image signals from one or more image sensors of the vehicle, the one or more image signals including image data of a steering input device of the vehicle; determining, in dependence on the received image data of the steering input device, a position of the steering input device; and, outputting a signal indicative of the determined position of the steering input device to a steering rack control system for controlling a steering rack of the vehicle.
15. 15. Computer readable instructions which, when executed by a computer, are arranged to perform a method according to Claim 14.