FIG. 4A are to be described using the case where n=3 and m=2 as a comparative example. This comparative example corresponds to operations excluding fourth to sixth exposure pulses E4 to E6 from FIG. 4A so that packets P4 to P6 are not created. For example, if the exposure pulse duration is the same as that in the comparative example, the distance measurement range can be increased 2.5 times in the examples of operations in FIG. 4A. As illustrated in FIG. 4B, if exposure pulse duration Tq is approximately a half of pulse duration Tp in the comparative example, distance measurement variations can be reduced by approximately a half with the same distance measurement range.

Next, operations of the distance measurement device from light exposure to generation of distance images are to be described.

FIG. 5 is a flowchart illustrating an example of operation of the distance measurement device according to Embodiment 1. When a distance image is to be generated, first, distance measurement device determines pixels for a distance image as first pixels that are readout targets, and determines pixels not for a distance image as second pixels that are not readout targets (S100). In the example in FIG. 2, the first pixels are infrared-light pixels 13. The second pixels are visible-light pixels 12.

Next, distance measurement device 5 generates packet 1 to packet n for each pixel by performing processing n times in loop 1 (S101 to S106). A pixel here is a pixel for a distance image, and in the example in FIG. 3, two infrared-light pixels 13, signal charges from which are mixed and added, correspond to one pixel for the distance image. Control variable i in FIG. 5 is an integer from 1 through n. n is an integer of at least 4, and is 6 in the example in FIG. 3.

In loop 1, distance measurement device 5 performs exposure to pulsed light according to an i-th exposure pulse (S102), and reads out a signal charge generated in visible-light pixel 12 by the exposure to vertical transfer portion 14 (S103). Accordingly, packet i is created at vertical transfer portion 14 according to the i-th exposure pulse. Furthermore, distance measurement device 5 transfers each packet at vertical transfer portion 14 by one packet in the vertical direction (S104).

Packets P1 to Pn of n types are generated for each pixel for a distance image by the processing in loop 1.

Solid-state imaging device 10 outputs packet signals of n types that indicate signal charge amounts of packets P1 to Pn to signal processing circuit 40 (S106). A packet signal may be a digital signal or an analog signal.

Signal processing circuit 40 calculates distance values, based on packet signals of n types output from solid-state imaging device 10, in loop 2 (S107 to S114). Loop 2 is processing repeatedly performed for each pixel for the distance image.

In loop 2, signal processing circuit 40 distinguishes, from among the n packet signals, the first packet signal indicating signal charge S0 and corresponding to a portion of reflected light that includes a leading end thereof, and the second packet signal indicating signal charge S1 and corresponding to a portion of the reflected light other than the portion that includes the leading end (S108).

Here, signal charge S0 and signal charge S1 are to be described with reference to FIG. 6A to FIG. 6C. RA, RB, and RC in FIG. 6A to FIG. 6C show examples of timings of reflected light that is reflected off an object.

In the example in FIG. 6A, exposure to a portion that includes the leading end of reflected light RA is performed at first exposure pulse E1, and generates signal charge S0. Signal charge S0 is held in packet P1. Exposure to a portion of reflected light RA other than the portion that includes the leading end is performed at second exposure pulse E2, and generates signal charge S1. Signal charge S1 is held in packet P2. All the signal charges generated by reflected light RA correspond to S0+S1.

In the example in FIG. 6B, exposure to a portion of reflected light RB that includes the leading end thereof is performed at third exposure pulse E3, and generates signal charge S0. Signal charge S0 is held in packet P3. Exposure to a portion of reflected light RB other than the portion that includes the leading end is performed at fourth exposure pulse E4, and generates signal charge S1. Signal charge S1 is held in packet P4. All the signal charges generated by reflected light RB correspond to S0+S1.

In the example in FIG. 6C, exposure to a portion of reflected light RC that includes the leading end thereof is performed at fifth exposure pulse E5, and generates signal charge S0. Signal charge S0 is held in packet P5. Exposure to a portion of reflected light RC other than the portion that includes the leading end is performed at sixth exposure pulse E6, and generates signal charge S1. Signal charge S1 is held in packet P6. All the signal charges generated by reflected light RB correspond to S0+S1.

Out of packet signals of n types, a packet signal corresponding to a packet that holds signal charge S0 is referred to as a first packet signal. Out of the packet signals of the n types, a packet signal corresponding to a packet that holds signal charge S1 is referred to as a second packet signal.

Next, signal processing circuit 40 identifies a group to which the distinguished first packet signal belongs (S109), and selects a first background light packet signal from the identified group (S110). The first background light packet signal does not include a signal charge generated by reflected light out of the n packet signals, and refers to a packet signal that includes a signal charge generated by background light. In FIG. 6A, packets P3 to P6 do not include signal charges generated by reflected light and include signal charges generated by background light. In FIG. 6A, the first packet signal corresponds to packet P1 and is included in first group G1, and thus a packet signal corresponding to packet P3 is selected as the first background light packet signal. In FIG. 6B, a packet signal corresponding to packet P2 is selected as the first background light packet signal. Note that in FIG. 6B, a packet signal corresponding to packet P1 may be selected as the first background light packet signal. In FIG. 6C, a packet signal corresponding to packet P4 is selected as the first background light packet signal.

Furthermore, signal processing circuit 40 identifies a group to which the distinguished second packet signal belongs (S111), and selects a second background light packet signal from the identified group (S112). The second background light packet signal does not include a signal charge generated by reflected light out of the n packet signals, and refers to a packet signal that includes a signal charge generated by background light. In FIG. 6A, packets P3 to P6 do not include reflected light, but include background light. In FIG. 6A, the second packet signal corresponds to packet P2 and is included in first group G1, and thus a packet signal corresponding to packet P3 is selected as the second background light packet signal. In FIG. 6B, a packet signal corresponding to packet P5 is selected as the second background light packet signal. Note that in FIG. 6B, a packet signal corresponding to packet P6 may be selected as the second background light packet signal. In FIG. 6C, a packet signal corresponding to packet P4 is selected as the second background light packet signal.

Furthermore, signal processing circuit 40 subtracts the first background light packet signal from the first packet signal indicating signal charge S0, subtracts the second background light packet signal from the second packet signal indicating signal charge S1, and calculates a distance value from the resultant first packet signal and the resultant second packet signal from which the background light signals have been subtracted (S113).

Distance value z is calculated by Expression 1, for example.

z={c×(i−1)×Tp+c×((S1−BG1)/(S0−BG0+S1−BG1))×Tp}/2 (Expression 1)

The first packet signal and the first background light packet signal belong to the same group, and thus include approximately the same amount of dark current components. Thus, by subtracting the first background light packet signal from the first packet signal, not only the background light components can be cancelled out, but also the dark current components can be cancelled out.

Similarly, the second packet signal and the second background light packet signal belong to the same group, and thus include approximately the same amount of dark current components. Thus, by subtracting the second background light packet signal from the second packet signal, not only the background light components can be cancelled out, but also the dark current components can be cancelled out.

In this manner, according to an example of the operation illustrated in FIG. 5, candidates to be selected for the first background light packet signal and the second background light packet signal are each limited within a group, and thus a dark current component having a magnitude that differs depending on a group can be appropriately cancelled out when calculating a distance value.

As described above, a distance measurement device according to Embodiment 1 includes: light source 20 that emits pulsed light; solid-state imaging device 10; and control circuit 30. Solid-state imaging device 10 includes: pixels that generate signal charges through photoelectric conversion; vertical transfer portions 14 at which packets that hold and transfer the signal charges are created; and vertical transfer electrodes V1 to V10 of plural types that cover vertical transfer portions 14 and create the packets. The pixels include first pixels (for example, infrared-light pixels 13) that are readout targets and included in a predetermined frame, and second pixels (for example, visible-light pixels 12) that are not readout targets and not included in the predetermined frame. Control circuit controls, for the first pixels, exposure operations of n types performed at different timings relative to the pulsed light, and reads out signal charges of n types to n packets at each of vertical transfer portions 14 out of the packets where n is an integer of at least 4, the signal charges of the n types being generated through the exposure operations of the n types. Within a region in which vertical transfer portions 14 are disposed, a first region in which the n packets are created is adjacent to a pixel region in which m pixels are consecutively disposed in a vertical direction where m is an integer of at least 3, and the m pixels include at least one of the first pixels and at least one of the second pixels.

According to this, a distance measurement range can be increased and distance measurement variations can be decreased. Specifically, exposure to reflected light is performed in at least four exposure operations at different timings, and thus a distance measurement range can be increased. Variations in distance measurement can be decreased by shortening exposure pulse durations in the at least four exposure operations.

Here, n may be at least 6, and m may be at least 4.

According to this, for example, in comparison to the case where n=3 and m=2 as a comparative example, if the exposure pulse duration is the same, the distance measurement range can be increased at least 2.5 times, and if the exposure pulse duration is reduced by a half, the distance measurement variations can be reduced by a half or less.

Here, control circuit 30 may generate a first exposure pulse to an n-th exposure pulse that are each combined with the pulsed light. Time differences between the pulsed light and the first exposure pulse to the n-th exposure pulse may increase in order from the first exposure pulse to the n-th exposure pulse, and in the exposure operations of the n types, the control circuit may generate the first exposure pulse to the n-th exposure pulse in the order from the first exposure pulse to the n-th exposure pulse.

According to this, the order in which n packets are arranged at a vertical transfer portion is the same as the descending or ascending order of time differences from the pulsed light. Stated differently, the n packets are arranged in the order of adjacent distance ranges. Thus, signal charges generated by reflected light can be held in two packets adjacent to each other in the arrangement of the n packets.

Here, distance measurement device 5 may further include: signal processing circuit 40 that calculates a distance value, based on n packet signals corresponding to the n packets output by solid-state imaging device 10. Signal processing circuit 40 may distinguish a first packet signal and a second packet signal from among the n packet signals, the first packet signal corresponding to a portion of reflected light that includes a leading end of the reflected light, the second packet corresponding to a portion of the reflected light other than the portion that includes the leading end; select, from among the n packet signals, a first background light packet signal and a second background light packet signal each indicating a background light component; subtract the first background light packet signal from the first packet signal; subtract the second background light packet signal from the second packet signal; and calculate the distance value from a resultant first packet signal and a resultant second packet signal, the resultant first packet signal resulting from subtracting the first background light packet signal from the first packet signal, the resultant second packet signal resulting from subtracting the second background light packet signal from the second packet signal.

According to this, influence of background light and dark currents on distance values can be reduced.

Here, the n packet signals may be grouped into a first group and a second group where n is an even number, the first group including a packet signal corresponding to a packet that has passed through a predetermined readout electrode out of the n packets, the second group including a packet signal corresponding to a packet that has not passed through the predetermined readout electrode out of the n packets, and signal processing circuit 40: may select the first background light packet signal from a group to which the first packet signal belongs, out of the first group and the second group; and may select the second background light packet signal from a group to which the second packet signal belongs, out of the first group and the second group.

According to this, influence of background light on distance values can be reduced, and influence of dark currents on distance values can be further effectively reduced. This is because the first packet signal and the first background light packet belong to the same group, and thus include an approximately same amount of dark current components. Furthermore, the second packet signal and the second background light packet belong to the same group, and thus include an approximately same amount of dark current components.

Here, the first pixels may be infrared-light pixels that photoelectrically convert infrared light, and the second pixels (visible-light pixels 12) may be visible-light pixels that photoelectrically convert visible light.

According to this, brightness images can be generated using visible light, in addition to distance images.

Here, the pixels may be included in first pixel rows and second pixel rows that are alternately disposed, the first pixel rows may each include at least two infrared-light pixels 13 that are one-dimensionally disposed, out of infrared-light pixels 13, and the second pixel rows may each include at least two visible-light pixels 12 that are one-dimensionally disposed, out of visible-light pixels 12.

According to this, a distance measurement range can be increased and distance measurement variations can be decreased.

Here, out of vertical transfer portions 14, vertical transfer portions 14 in predetermined columns may each mix signal charges from two infrared-light pixels that are adjacent to each other in a row direction and included in a predetermined row out of the first pixel rows, in a readout operation on the first pixels, and vertical transfer portions 14 in columns other than the predetermined columns may each mix signal charges from two infrared-light pixels that are adjacent to each other in the row direction and included in a row other than the predetermined row out of the first pixel rows, in the readout operation on the first pixels. The predetermined columns may be odd-numbered columns or even-numbered columns, and the predetermined row may be an odd-numbered first pixel row or an even-numbered first pixel row in a group of the first pixel rows.

According to this, sensitivity can be improved along with mixture of signal charges from pixels.

A distance measurement method according to Embodiment 1 is a distance measurement method for use in a distance measurement device that includes light source 20 that emits pulsed light and solid-state imaging device 10, the distance measurement method including: determining first pixels that are readout targets and second pixels that are not readout targets; controlling, for the first pixels, exposure operations of n types performed at different timings relative to the pulsed light where n is an integer of at least 4; and reading out signal charges of n types to n packets at vertical transfer portion 14 in the solid-state imaging device, the signal charges of the n types being generated through the exposure operations of the n types. Within a region in which vertical transfer portion 14 is disposed, a first region in which the n packets are created is adjacent to a pixel region in which m pixels are consecutively disposed in a vertical direction where m is an integer of at least 3, and the m pixels include at least one of the first pixels and at least one of the second pixels.

According to this, a distance measurement range can be increased, and distance measurement variations can be decreased. Specifically, exposure to reflected light is performed in at least four exposure operations at different timings, and thus distance calculation for three or more distance ranges can be performed or whether an object is present in four or more distance ranges can be determined, so that a distance measurement range can be increased. Variations in distance measurement can be decreased by shortening exposure pulse durations in the at least four exposure operations.

Here, the distance measurement method may further include: distinguishing a first packet signal and a second packet signal from among the n packet signals, the first packet signal corresponding to a portion of reflected light that includes a leading end of the reflected light, the second packet corresponding to a portion of the reflected light other than the portion that includes the leading end; selecting, from among the n packet signals, a first background light packet signal and a second background light packet signal each indicating a background light component; subtracting the first background light packet signal from the first packet signal; subtracting the second background light packet signal from the second packet signal; and calculating the distance value from a resultant first packet signal and a resultant second packet signal, the resultant first packet signal resulting from subtracting the first background light packet signal from the first packet signal, the resultant second packet signal resulting from subtracting the second background light packet signal from the second packet signal.

According to this, influence of background light and dark currents on distance values can be reduced.

Embodiment 2

In Embodiment 1, (S0+S1) that are all the signal charges generated by reflected light are held in two adjacent packets in the arrangement of the n packets at vertical transfer portion 14. In contrast, Embodiment 2 is to describe an example of a configuration in which (S0+S1) that are all the signal charges generated by reflected light are held in two packets not adjacent to each other in the arrangement of the n packets at vertical transfer portion 14.

A configuration of distance measurement device 5 in the present embodiment may be the same as those in FIG. 1 to FIG. 3. Note that the main difference is the order of exposure operations of n types performed by control circuit 30. The following gives a description focusing on different points while avoiding redundant description of the same points.

FIG. 7A is a time chart illustrating an example of operation of distance measurement device 5 according to Embodiment 2. The order in which first exposure pulse E1 to sixth exposure pulse E6 are output in the drawing is different from that in FIG. 4A. In FIG. 7A, the exposure operations of the six types are executed in the order of (L0, E1), (L0, E3), (L0, E5), (L0, E2), (L0, E4), and (L0, E6). As a result, the exposure operations of (L0, E1), (L0, E3), and (L0, E5) are executed in this order, and packets P1, P2, and P3 in a first group are generated. The exposure operations of (L0, E2), (L0, E4), and (L0, E6) are executed in this order, and packets P4, P5, and P6 in a second group are generated. FIG. 7B is similar to FIG. 7A except that the pulse duration is different.

FIG. 8 is a flowchart illustrating an example of operation of the distance measurement device according to Embodiment 2. The drawing is different from FIG. 5 in that loop 1a is executed instead of loop 1.

Loop 1a (S101a to S105a) are different from loop 1 (S101 to S105) in FIG. 5 in that control variable i takes the value in the order of 1, 3, 5, 2, 4, and 6. Control variable i corresponds to an i-th exposure pulse.

Next, a first packet signal, a second packet signal, a first background light packet signal, and a second background light packet signal in Embodiment 2 are to be described with reference to FIG. 9A to FIG. 9C. RA, RB, and RC in FIG. 9A to FIG. 9C show examples of timings of reflected light that is reflected off an object, similarly to FIG. 6A to FIG. 6C.

In the example in FIG. 9A, exposure to a portion of reflected light RA that includes the leading end thereof is performed at first exposure pulse E1, and generates signal charge S0. Signal charge S0 is held in packet P1. Exposure to a portion of reflected light RA other than the portion that includes the leading end is performed at second exposure pulse E2, and generates signal charge S1. Signal charge S1 is held in packet P4.

(S0+S1) that are all the signal charges generated by reflected light RA are held in packet P1 and packet P4, respectively. The first packet signal corresponds to packet P1. The second packet signal corresponds to packet P4.

The first background light packet signal is selected from a group the same as the group that includes the first packet signal, and a packet signal corresponding to packet P2 is selected. Note that the first background light packet signal may be a packet signal corresponding to packet P3. The second background light packet signal is selected from a group the same as the group that includes the second packet signal, and a packet signal corresponding to packet P5 is selected. Note that the second background light packet signal may be a packet signal corresponding to packet P6.

In this manner, in FIG. 9A, (S0+S1) that are all the signal charges generated by reflected light are held in packets P1 and P4, respectively, which are not adjacent to each other in the arrangement of the n packets at vertical transfer portion 14. Accordingly, two candidates for the first background light packet signal can be included. Hence, the first background light packet signal can be more appropriately selected. Similarly, two candidates for the second background light packet signal can be included. In this regard, FIG. 6A shows one candidate for each of the first background light packet signal and the second background light packet signal. In FIG. 9A, the background light components and the dark current components can be cancelled out more appropriately than in FIG. 6A.

In the example in FIG. 9B, exposure to a portion of reflected light RB that includes the leading end thereof is performed at third exposure pulse E3, and generates signal charge S0. Signal charge S0 is held in packet P2. Exposure to a portion of reflected light RB other than the portion that includes the leading end is performed at fourth exposure pulse E4, and generates signal charge S1. Signal charge S1 is held in packet P5.

(S0+S1) that are all the signal charges generated by reflected light RB are held in packet P2 and packet P5, respectively. The first packet signal corresponds to packet P2. The second packet signal corresponds to packet P5.

The first background light packet signal is selected from a group the same as the group that includes the first packet signal, and a packet signal corresponding to packet P3 is selected. Note that the first background light packet signal may be a packet signal corresponding to packet P1. The second background light packet signal is selected from a group the same as the group that includes the second packet signal, and a packet signal corresponding to packet P6 is selected. Note that the second background light packet signal may be a packet signal corresponding to packet P4.

In this manner, in FIG. 9B, (S0+S1) that are all the signal charges generated by reflected light are held in packets P2 and P5, respectively, which are not adjacent to each other in the arrangement of the n packets at vertical transfer portion 14. Accordingly, two candidates for the first background light packet signal can be included. Hence, the first background light packet signal can be more appropriately selected. Similarly, two candidates for the second background light packet signal can be included. This point is the same as FIG. 6B.

In the example in FIG. 9C, exposure to a portion of reflected light RC that includes the leading end thereof is performed at fifth exposure pulse E5, and generates signal charge S0. Signal charge S0 is held in packet P3. Exposure to a portion of reflected light RC other than the portion that includes the leading end is performed at sixth exposure pulse E6, and generates signal charge S1. Signal charge S1 is held in packet P6.

(S0+S1) that are all the signal charges generated by reflected light RC are held in packet P3 and packet P6, respectively. The first packet signal corresponds to packet P3. The second packet signal corresponds to packet P6.

The first background light packet signal is selected from a group the same as the group that includes the first packet signal, and a packet signal corresponding to packet P2 is selected. Note that the first background light packet signal may be a packet signal corresponding to packet P1. The second background light packet signal is selected from a group the same as the group that includes the second packet signal, and a packet signal corresponding to packet P5 is selected. Note that the second background light packet signal may be a packet signal corresponding to packet P4.