104 may execute image processing in the form of live person image recognition to determine whether images within the captured scene include images of enrolled persons authorized to control or actuate ignition, steering, and propulsion components of vehicle 102.

Cameras 105, 115, and 125 can operate to detect electromagnetic radiation in a range of wavelengths. For example, cameras 105, 115, and 125 can detect visible light, infrared radiation, ultraviolet light, or some range of wavelengths including visible, infrared, and/or ultraviolet light. For example, cameras 105, 115, and 125 can include image sensors such as charge-coupled devices (CCD), active-pixel sensors such as complementary metal-oxide semiconductor (CMOS) sensors, etc. Cameras 105 and 125 can be of sufficient resolution to digitize scenes that include facial images of live persons positioned, for example, at up to 2.0 meters from a door of vehicle 102. Camera 115 can be of sufficient resolution to digitize scenes including facial images of persons positioned in a driver's position of vehicle 102, to the right of the driver's position, and positioned in the rear seat of the vehicle.

Exemplary System Operations

FIG. 2 depicts a top view 200 of a group of individuals attempting to enter vehicle 102. As seen in the example of FIG. 2, camera 105 is mounted on a vertical support structure of vehicle 102. Camera 105 is thus capable of capturing images of persons 210, 215, 220, 225, 230, and 235 positioned within field-of-view 205 of the camera, including persons who may be below or above average or typical sizes of adult males and females. Accordingly, as part of a process to enter vehicle 102, person 215 may initiate a live person image recognition process in which camera 105 captures a scene containing at least partial images of persons 215-235. However, as part of the live person image recognition process, camera 105 may capture a scene that includes images of persons 210-235. Accordingly, image processing programming of computer 104 may attempt to perform live person image recognition by extracting facial and/or other parameters of persons 210-235, all of whom may be at least partially visible to camera 105. In an example, image processing to extract facial and/or other parameters of persons 210-235 may utilize any suitable facial recognition algorithm to identify facial landmarks to distinguish a live person's face from other objects. Facial recognition algorithms may determine a distance between the eyes of a live person, a distance from the forehead to the chin of a live person, a distance between the nose and the mouth of a live person, depth of a live person's eye sockets, shape of cheek bones, and/or contour of the lips, cars, and chin of persons 210-235. Such distances, geometrical factors (and/or ratios between these) of the faces of persons 210-235 may be compared with corresponding features and/or geometrical factors of an enrolled person to determine whether the distances and/or geometrical factors deviate by a threshold amount, e.g., less than 5%, less than 10%, etc. Deviation by less than the threshold amount can result in computer 104 determining that one or more of persons 210-235 corresponds to an enrolled individual. Alternatively, or in addition, consistency between facial and/or other features of one or more of persons 210-235 and those of an enrolled person may be determined utilizing other suitable techniques, such as a histogram of oriented gradients, and/or techniques that utilize a convolutional neural network. Accordingly, programming of computer 104 may, for example, sequentially compare the extracted facial parameters and/or other parameters, e.g., the shoulders, of persons 210-235 to determine whether the parameters of the captured image of at least one of the persons corresponds to the facial and/or other parameters of an enrolled person authorized to control or actuate a door lock of vehicle 102.

In an example, programming of computer 104 can apply a cropping operation to a captured scene within field-of-view 205 prior to initiating a facial recognition process. Thus, for example, programming of computer 104 can crop images that include facial and/or other features of persons 210-235. In an example, computer 104 may crop images present in the captured scene corresponding to the largest face visible to camera 105. In this context, the largest area of facial features within field-of-view 205 of camera 105 means the face and/or other features determined by computer 104 to include the largest number of pixels in any direction. Accordingly, as seen in the example of FIG. 2, responsive to person 215 including facial features larger than those of person 210 (a child), image processing programming of computer 104 may execute a cropping operation to provide an image that includes the facial features of person 210. A cropping operation executed by computer 104 may be followed by applying a facial recognition technique to determine whether person 215 represents an enrolled person authorized to actuate or control an unlocking mechanism of a door lock of vehicle 102.

In another example, programming of computer 104 may crop images present in the captured scene to remove portions outside of an area of the face of a person positioned within at a central portion with respect to camera 105. In this context, a central portion with respect to camera 105 corresponds to an area intersected by optical axis 250 of camera 105. Accordingly, as seen in the example of FIG. 2, responsive to person 215 being positioned along optical axis 250 of camera 105, a cropping operation can remove images of the captured scene outside of cropped image 217. Removal of images of the captured scene outside of cropped image 217 can queue image processing programming of computer 104 to apply a facial recognition technique to the cropped image to determine whether person 215 represents an enrolled person authorized to actuate or control an unlocking mechanism of a door lock of vehicle 102.

In another example, computer 104 may perform facial recognition in response to applying, independently of one another, two or more cropping operations to determine whether a person positioned within field-of-view 205 of camera 105 is authorized to control or actuate, for example, an unlocking mechanism of a door of vehicle 102. For example, image processing programming of computer 104 can execute a cropping operation to provide an image that includes the largest area of facial features in a captured scene, and, in addition, to remove portions outside of an area of the facial features positioned within the central portion of the scene with respect to camera 105. Computer 104 may then independently apply a facial recognition process to each cropped image and compute a weighted sum of scores that result from such operations. In an example, a facial recognition algorithm may compute a score, which may be expressed in the form of a probability, e.g., between 0% and 100%, that facial features of a person correspond to those of an enrolled person. Application of two or more facial recognition processes, applied independently to each of two or more cropped images of a scene, may enhance the facial recognition capability of the programming of computer 104. In computing a weighted sum, programming of computer 104 may assign an increased weight to a computed score in response to a facial recognition process determining that parameters extracted from an image correspond or at least approximate, e.g., within 10%, 5%, etc., to those of a recent, or most recent, enrolled person that has accessed vehicle 102. Thus, for example, a first facial recognition process may result in a computed score that indicates an 80% probability that the facial features of a person correspond to the most recent enrolled person to have accessed vehicle 102. A second facial recognition process may result in a computed score that indicates a 90% probability that the facial features of the person correspond to those of a person other than the most recently enrolled person to access the vehicle. Programming of computer 104 may then apply a weighting factor greater than 1.0, e.g., 1.1, 1.2, 1.25, etc., so as to add to the probability score of the most recently enrolled person. Application of a weighting factor greater than 1.0 may operate to predispose the facial recognition process to select the most recent enrolled person to have accessed vehicle 102 over other persons.

In an example, weighting factors may operate to prioritize a cropped image of a scene that includes the largest area of facial features over a cropped image of a scene that includes facial features of a person positioned along optical axis 250 of a camera (e.g., 105, 125). In an example, a cropped image of a scene that includes facial features of a person positioned along optical axis 250 of a camera (e.g., 105, 125) may be prioritized over a cropped image that includes facial features of a most recent user of vehicle 102. In an example, a cropped image of a scene that includes facial features of a most recent user of vehicle 102 may be prioritized over a cropped image of a person having the least-obstructed shoulders. In an example, a cropped image of a scene of a person having the least-obstructed shoulders may be prioritized over a cropped image of a scene of a person having a head pose angle directed away from a camera (e.g., 105, 125). In another example, programming of computer 104 can apply additional cropping operations to form cropped images that include the facial features of all persons within field-of-view 205, e.g., persons 210, 215, 220, 225, 230, and 235 to form cropped images 212, 217, 222, 227, 232, and 237. Cropping operations may be followed by image processing of computer 104 applying a facial recognition technique to determine whether any person, e.g., 210-235, within field-of-view 205, represents an enrolled person authorized to actuate or control an unlocking mechanism of a door lock of vehicle 102. In an example such cropping operations, applied to multiple persons in a single scene captured via a camera 105, bring about a single increment toward computer 104 implementing a lockout counter of vehicle 102.

FIG. 3 depicts scene 300 within field-of-view 305 of dashboard-mounted camera 115 of vehicle 102. Dashboard-mounted camera 115 is capable of capturing images of person 310, seated in the driver's position of vehicle 102, and images of persons 315 and 320, who are seated in the rear seat of vehicle 102. Person 310 and persons 315 and 320 can represent enrolled persons authorized to access one or more components of vehicle 102. Thus, for example, person 310 seated in the driver's position of vehicle 102, may be permitted to control or actuate ignition and/or propulsion of vehicle 102 (e.g., component 1), controlling an infotainment component of vehicle 102 (e.g., component 2), and controlling a climate control setting of vehicle 102 (e.g., component 3), to actuate or adjust a driver's seat position, etc. Persons seated in the rear seat of vehicle 102 may be permitted to control or actuate, for example, fewer vehicle components than person 310 such as, for example an infotainment component of vehicle 102 (e.g., component 2) and a climate control setting (e.g., component 3) of vehicle 102.

However, prior to executing facial recognition to identify persons 310, 315, and 320, programming of computer 104 may execute a cropping operation based on the positions of persons 310, 315, and 320. Responsive to a cropping operation, computer 104 may determine whether the person, e.g., 310, 315 and 320, represents an enrolled person authorized to control or actuate a vehicle component.

In the example of FIG. 3, responsive to detecting persons 310, 315, and 320 being seated within vehicle 102, computer 104 may execute cropping operations to determine which of the persons represents the person located in the driver's position of the vehicle. In an example, image processing programming of computer 104 may perform a cropping operation to determine the person having the least-obstructed shoulders in field-of-view 305. Accordingly, for example, image processing programming of computer 104 may execute a cropping operation to remove areas outside of the facial features, neck, and shoulders of each of persons 310, 315, and 320. Thus, in a first cropping operation, indicated via cropped image 322, programming of computer 104 may determine that portions of person 320, seated in the rear seat of vehicle 102, appears to have a significant portion of both right and left shoulders obstructed from the view of camera 115. Similarly, in a second cropping operation, resulting in cropped image 317, programming of computer 104 may determine that portions of person 315 appear to have a right shoulder visible while the left shoulder is obstructed from the view of camera 115. Similarly, in a third cropping operation, indicated via cropped image rectangle 312, programming of computer 104 may determine that both right and left shoulders of person 310 are visible within the view of camera 115.

Accordingly, such as in the example of FIG. 3, person 310 may be determined to be seated in the driver's position of vehicle 102 responsive to programming of computer 104 determining that person 310 presents the least-obstructed shoulders of the persons within the field-of-view of camera 115. In addition, the head pose of person 310, oriented in a forward direction relative vehicle 102, may assist in determining that person 310 is seated in the driver's position.

In an example, programming of computer 104 determining that person 310 is seated in the driver's position of vehicle 102, the computer may execute a facial recognition technique to determine whether person 310 represents an enrolled person permitted to operate vehicle 102. In response to programming of computer 104 determining that person 310 represents a person enrolled as an authorized operator of the vehicle, computer 104 may permit the person to control or actuate ignition and/or propulsion of vehicle 102 (e.g., component 1), controlling an infotainment component of vehicle 102 (e.g., component 2), controlling a climate control setting of vehicle 102 (e.g., component 3), controlling a seat adjustment, etc. Conversely, in response to programming of computer 104 determining that person 310 represents a person not enrolled as an authorized operator the vehicle, computer 104 may deny person 310 from controlling or actuating propulsion. Persons seated in the rear seat of vehicle 102 may be permitted to control, for example, fewer vehicle components than person 310 such as, for example an infotainment component of vehicle 102 (e.g., component 2) and a climate control setting (e.g., component 3) of vehicle 102.

In addition, responsive to programming of computer 104 determining that persons 315 and 320 do not represent the person having the least-obstructed shoulders, persons 315 and 320 may be permitted to control a smaller number of vehicle components than those that can be controlled by person 310. Thus, for example, persons 315 and 320 may be permitted to control an infotainment component of vehicle 102 (e.g., component 2) and/or a climate control setting of vehicle 102 (e.g., component 3).

In addition to a cropping operation executed by computer 104 determining an person having the least-obstructed shoulders in a view of camera 115, a cropping operation may involve removal of a portion of the captured image outside of an area that includes a largest area of facial features of a person. For example, in place of (or in addition to) computer 104 determining both right and left shoulders of person 310 are visible to camera 115, computer 104 may crop a captured image outside of the facial features of persons 310, 315, and 320. Image processing programming of computer 104 may then determine that person 310 presents the largest area of facial features with respect to persons 315 and 320. Accordingly, computer 104 may then execute a facial recognition technique to determine whether person 310 is enrolled as an authorized operator of vehicle 102. Computer 104 may then independently apply a facial recognition process to each cropped image to compute a weighted sum that results from such operations. Application of two or more facial recognition processes, applied independently to each of two or more cropped images, may enhance the facial recognition capability of the programming of computer 104.

In response to a facial recognition technique being unsuccessful in recognizing any of persons 310, 315, and 320 as enrolled operators of vehicle 102, programming of computer 104 can increment a lockout counter, which may indicate that, for example, none of persons 310, 315, and 320 is enrolled as operators of vehicle 102. In an example, responsive to a lockout counter incrementing beyond a predetermined number, e.g., 2, 3, 4, 5, etc., ignition of vehicle 102 may be disabled for a period of time, e.g., 5 minutes, 10 minutes, 15 minutes, 30 minutes, 1 hour, 2 hours, etc.

FIG. 4 is a process flow diagram of an example process 400 for controlling or actuating a system, e.g., a vehicle, via live person image recognition. For example, the memory of computer 104 stores executable instructions for performing the steps of process 400, and/or such programming could be implemented in other computing devices, some other non-vehicle computer, and/or other computing devices of vehicle 102. Example process 400 can be utilized to provide controlling or actuating of components of a vehicle via live person image recognition in response to persons attempting to control or actuate a door unlocking mechanism, ignition of the vehicle, or to control other vehicle functions, such as an infotainment component, a climate control component, or another vehicle component. Although process 400 is described as a vehicle-based process, it is to be understood that process 400 could be applied in the context of controlling any other system via live person image recognition, such as controlling a security system to gain access to a building or to gain access to an area within a building, controlling a retail payment system, access or control over machinery, etc.

As a general overview of vehicle-based process 400, vehicle computer 104, or a computer utilized in an application to control another system via live person image recognition, can execute programming to receive a captured scene that includes cropped images of facial and/or other features of one or more persons. In an example, computer 104 may select a cropping operation to be applied to images of facial and/or other features of persons present in a scene captured by a vehicle camera. The cropping operation can be based on a component of the vehicle that the person is attempting to control or actuate. For example, based on a person attempting to gain entry to vehicle 102, programming of computer 104 can select a cropping operation that directs facial recognition processing toward the area of the scene occupied by the largest face in the scene, thereby guiding facial recognition processes towards the direction of the person closest to a camera positioned near a door of vehicle 102 (e.g., camera 105, 125). In another example, programming of computer 104 can select a cropping operation that directs facial recognition processing toward a person positioned along an optical axis (250) of a camera near a door of vehicle 102 (e.g., camera 105, 125). In another example, computer 104 can select a cropping operation to determine the person having the least-obstructed shoulders, which can guide facial recognition processes to determine whether the person is enrolled as an authorized operator of vehicle 102.

Process 400 begins at block 405, at which computer 104 determines a component to be actuated. A component to be actuated may include an external door unlocking mechanism determined in response, for example, to detecting one or more persons approaching the vehicle. In another example, a component to be actuated may include an ignition component of the vehicle determined in response to a seat sensor registering a non-negligible weight being applied to the driver's position of a vehicle, etc.

Process 400 may continue at block 410, which can include a camera of the vehicle, such as a camera mounted near a door of vehicle 102 (e.g., on a frame member of the vehicle, a dashboard of vehicle 102, etc.), capturing an image of a scene within a camera field-of-view, e.g., 205, 305. A scene may include a portion of an environment external to vehicle 102 (e.g., within camera field-of-view 205), an interior portion of vehicle 102, (e.g., within camera field-of-view 305), etc.

Process 400 may continue at block 415, at which computer 104 can select a cropping operation based on the component to be actuated determined from block 405. For example, responsive to a component to be actuated corresponding to a vehicle door lock, computer 104 may select to apply a cropping operation that directs facial recognition processing toward the area of a scene (e.g., 205) that includes the largest facial features visible within the scene, an area that includes facial features at a central portion of the scene, an area that includes the facial features and least-obstructed shoulders within the scene, an area that includes facial features that intersect an optical axis (e.g., optical axis 250) of a camera (e.g., camera 105, 125). In another example, responsive to a component to be actuated corresponding to a vehicle ignition system, computer