For example, in the example shown in FIG. 1, the monitored person D has fallen. In response thereto, the monitoring device 1 can activate the alarm device 2.

Note that in the present embodiment, the monitored person D is a resident of a welfare facility for the elderly, and the area B is a room of the welfare facility for the elderly where the monitored person D resides. That is, the area B in the present embodiment is the room where the monitored person D is present. Note that the present disclosure is not limited to this configuration, and the area B does not need to be a room. For example, if the monitored person D is present in a hallway, the area B is this hallway. Also, if the monitored person D is present in a relatively large space (such as, e.g., an outdoor plaza), the area B may be the entire space or may be only the vicinity of the monitored person D in this space. The area B is preferably within a range where another person (predetermined attribute person E) can visually check the state of the monitored person D.

Also, the predetermined attribute person E is a person who is different from the monitored person D and has a predetermined attribute. In the present embodiment, the predetermined attribute is being an attendant who assists the monitored person D. That is, the predetermined attribute person E is an attendant (staff) in the welfare facility for the elderly.

Note however that the present disclosure is not limited to this configuration. “A predetermined attribute” may be an attribute other than the attribute of “being an attendant who assists the monitored person D”. For example, “a predetermined attribute” may be “being a relative of the monitored person D or an attendant who assists the monitored person D”, or may be being a specific person (for example “a specific X or Y”).

Note that in the present embodiment, the predetermined attribute person E is also a user of the monitoring system A.

The alarm device 2 of the present embodiment is an information terminal (such as a smartphone or tablet) held by the attendant. Also, in the example shown in FIG. 1, the predetermined attribute person E, who is an attendant, is located in a room (e.g., anteroom) other than the room where the monitored person D is located.

The monitoring system A may include one monitoring device 1 or a plurality of monitoring devices 1. That is, the monitoring system A may be a system for monitoring one monitored person D or a system for monitoring a plurality of monitored persons D.

A plurality of monitoring devices 1 may be installed in the welfare facility for the elderly. For example, a monitoring device 1 may be installed in each room of the welfare facility for the elderly, or may be installed in a region (such as, e.g., hallway) other than the rooms.

A single predetermined attribute person E or a plurality of predetermined attribute persons E may be present in the welfare facility for the elderly. The monitoring system A may include one alarm device 2 or a plurality of alarm devices 2. A plurality of predetermined attribute persons E may each hold one alarm device 2. The alarm device 2 may be a personal computer, a display, a lamp, a speaker, or the like installed in a room (e.g., anteroom) for the predetermined attribute person E. The alarm device 2 may be held by a person (e.g., a relative of the monitored person D) who is not the predetermined attribute person E, or may be a personal computer, a display, a lamp, a speaker, or the like installed in the home of a person (e.g., a relative of the monitored person D) who is not the predetermined attribute person E.

In the example shown in FIG. 2, a first person E1 and a second person E2 are shown. The first person E1 and the second person E2 are both predetermined attribute persons E.

The first person E1 is located in a room (e.g., anteroom) different from the area B, which is a room where the monitored person D is located. The second person E2 is located in the area B, which is a room where the monitored person D is located. Also, in the example shown in FIG. 2, the monitored person D has fallen as in the example shown in FIG. 1. In this case, in the monitoring system A of the present embodiment, the monitoring device 1 does not essentially activate the alarm device 2.

Hereinafter, the monitoring system A will be described in detail.

State Detection Unit

As shown in FIG. 3, the monitoring device 1 includes an image capturing unit 3 and a control unit 4. The image capturing unit 3 captures an image of the inside of the area B (see FIG. 1). Note that the image capture range of the image capturing unit 3 may be the entirety of the area B or may be part of the area B. Also, the image capture range of the image capturing unit 3 may include the outside of the area B.

In the present embodiment, the image capturing unit 3 is constituted by an infrared camera. Note however that the present disclosure is not limited to this configuration, and the image capturing unit 3 may be any device other than an infrared camera as long as it can capture an image of the inside of the area B. The image capturing unit 3 is preferably a ToF (Time of Flight) camera. A captured image obtained by the image capturing unit 3 is preferably a depth image, which is an image including depth information. Note that the captured image obtained by the image capturing unit 3 may be a moving image or a still image.

As shown in FIG. 3, the control unit 4 includes a state detection unit 41. The image capturing unit 3 transmits an obtained captured image to the state detection unit 41.

Also, as shown in FIG. 3, the monitoring system A includes a sound detection device 5 for detecting sound. The sound detection device 5 may be installed in the area B. The sound detection device 5 may be a known microphone. The sound detection device 5 transmits information indicating detected sound to the state detection unit 41.

The state detection unit 41 detects the state of the monitored person D based on the captured image received from the image capturing unit 3 and the information indicating the sound received from the sound detection device 5. That is, the monitoring system A includes the state detection unit 41 that detects the state of the monitored person D.

In the present embodiment, the state detection unit 41 generates a skeleton model that is information indicating the skeleton of the monitored person D, based on the received captured image. The skeletal model is generated, by analyzing the captured image to three-dimensionally recognize the positions of joints of the monitored person D and connecting the joints with straight lines.

Furthermore, the state detection unit 41 is configured to detect the body position (e.g., standing, lying, or sitting) of the monitored person D based on the captured image and the generated skeleton model, and to detect whether or not the monitored person D is in a predetermined state and whether or not the monitored person D is in an abnormal state different from the predetermined state, based on the captured image, the skeleton model, and the information indicating sound.

Although not particularly limited, the predetermined state may be a dangerous state (e.g., a fallen state) of the monitored person D. Also, although not particularly limited, the abnormal state may be a state in which the monitored person D is massively bleeding or a state in which the monitored person D is being assaulted by the predetermined attribute person E. Note that techniques for detecting a dangerous state such as a fallen state or an abnormal state based on a captured image or a skeleton model are known and will not be described here.

Also, the state detection unit 41 may be configured to determine whether or not the monitored person D is screaming based on information indicating sound, and to detect a dangerous state or an abnormal state of the monitored person D based on the result of this determination.

Note that there may be a case where a person other than the monitored person D is present in the image capture range of the image capturing unit 3. Therefore, the state detection unit 41 is preferably capable of recognizing a person in a captured image. By recognizing a person in the captured image, it is possible to avoid the situation where the state detection unit 41 mistakenly detects the state of a person other than the monitored person D. This can avoid the situation where, for example, when a person (e.g., predetermined attribute person E) other than the monitored person D is in a predetermined state (e.g., a fallen state), the state detection unit 41 mistakenly detects that the monitored person D is in a predetermined state.

Note that the method for recognizing a person in a captured image by the state detection unit 41 is not particularly limited, and a machine-learned neural network may be used to analyze the captured image and recognize the person.

Person's State Detection Unit

As shown in FIG. 3, the control unit 4 includes a person's state detection unit 42. The image capturing unit 3 transmits an obtained captured image to the person's state detection unit 42. Also, the sound detection device 5 transmits information indicating detected sound to the person's state detection unit 42.

The person's state detection unit 42 detects the state of the predetermined attribute person E (see FIG. 2) present in the area B, based on the captured image received from the image capturing unit 3 and the information indicating sound received from the sound detection device 5. That is, the monitoring system A further includes the person's state detection unit 42 that detects the state of the predetermined attribute person E.

In the present embodiment, the person's state detection unit 42 generates a skeleton model that is information indicating the skeleton of the predetermined attribute person E, based on the received captured image. The skeleton model is generated, by analyzing the captured image to three-dimensionally recognize the positions of joints of the predetermined attribute person E and connecting the joints with straight lines.

Furthermore, the person's state detection unit 42 is configured to detect the body position (e.g., standing, lying, or sitting) of the predetermined attribute person E based on the captured image and the generated skeleton model, and to detect whether or not the predetermined attribute person E is in an abnormal state based on the captured image, the skeleton model, and the information indicating sound.

Although not particularly limited, the abnormal state may be a state where the predetermined attribute person E is assaulting the monitored person D. For example, if the predetermined attribute person E continues to move violently for a predetermined period of time or if the predetermined attribute person E makes a predetermined number of movements to wave his or her hands down, the predetermined attribute person E may be detected as being in an abnormal state (state of assaulting the monitored person D).

Also, the person's state detection unit 42 may be configured to determine whether or not the predetermined attribute person E is shouting in anger based on information indicating sound, and to detect that the predetermined attribute person E is in an abnormal state based on the result of this determination.

Note that the person's state detection unit 42 is preferably capable of recognizing a person in a captured image. By recognizing a person in the captured image, it is possible to avoid the situation where the person's state detection unit 42 mistakenly detects the state of a person other than the predetermined attribute person E. This can avoid the situation where, for example, when a person (e.g., monitored person D) other than the predetermined attribute person E is in an abnormal state, the person's state detection unit 42 mistakenly detects that the predetermined attribute person E is in an abnormal state.

Note that the method for recognizing a person in a captured image by the person's state detection unit 42 is not particularly limited, and a machine-learned neural network may be used to analyze the captured image and recognize the person.

Determination Unit

As shown in FIG. 3, the control unit 4 includes a specific object detection unit 43 (corresponding to a “detection unit” according to the present disclosure). The image capturing unit 3 transmits an obtained captured image to the specific object detection unit 43. The specific object detection unit 43 performs image analysis based on the captured image received from the image capturing unit 3 to detect a specific object. A specific object refers to an object worn by the predetermined attribute person E who is an attendant in the welfare facility for the elderly. That is, the monitoring system A includes the specific object detection unit 43 that detects a specific object worn by the attendant.

Note that the specific object is not particularly limited and may be an attendant's uniform or badge, or the like, for example.

If the predetermined attribute person E is present in the area B, the specific object will appear in the captured image and will be detected by the specific object detection unit 43. If the predetermined attribute person E is not present in the area B, the specific object does not appear in the captured image and thus is not detected by the specific object detection unit 43.

Also, as shown in FIGS. 1 to 3, the monitoring system A includes an entry detection device 6 (corresponding to a “biometric information acquisition unit” according to the present disclosure) and an exit detection device 7. The entry detection device 6 and the exit detection device 7 are both installed in the vicinity of a door 8 provided in a doorway of the area B.

The entry detection device 6 is configured to acquire biometric information of a person entering the area B. That is, the monitoring system A includes the entry detection device 6 that acquires biometric information of a person entering the area B. For example, a configuration is possible in which the door 8 is normally closed, and in response to the entry detection device 6 acquiring biometric information, the door 8 is temporarily opened. Note that the biometric information acquired by the entry detection device 6 is not particularly limited and may be face information or fingerprint information, for example.

The exit detection device 7 is configured to acquire biometric information of a person exiting the area B. For example, a configuration is possible in which the door 8 is normally closed, and in response to the exit detection device 7 acquiring biometric information, the door 8 is temporarily opened. Note that the biometric information acquired by the exit detection device 7 is not particularly limited and may be face information or fingerprint information, for example.

As shown in FIG. 3, the control unit 4 includes a determination unit 44. The result of the detection by the specific object detection unit 43 is transmitted to the determination unit 44. Also, the pieces of biometric information acquired by the entry detection device 6 and the exit detection device 7 are transmitted to the determination unit 44.

The determination unit 44 determines whether or not the predetermined attribute person E is present in the area B, based on the result of the detection by the specific object detection unit 43 and the pieces of biometric information acquired by the entry detection device 6 and the exit detection device 7.

That is, the monitoring system A includes the determination unit 44 that determines whether or not the predetermined attribute person E, who is different from the monitored person D and has a predetermined attribute, is present in the area B where the monitored person D is present. Also, the determination unit 44 determines whether or not the predetermined attribute person E is present in the area B based on the result of the detection by the specific object detection unit 43. The determination unit 44 also determines whether or not the predetermined attribute person E is present in the area B based on the biometric information acquired by the entry detection device 6.

The following will describe in detail the determination performed based on the biometric information acquired by the entry detection device 6 and the exit detection device 7. The determination unit 44 has stored therein biometric information (e.g., face information or fingerprint information) of all attendants in this welfare facility for the elderly. The determination unit 44 determines whether or not the person who has entered the area B is the predetermined attribute person E, by comparing the biometric information stored in the determination unit 44 with the biometric information acquired by the entry detection device 6. The determination unit 44 also determines whether or not the person who has exited the area B is the predetermined attribute person E, by comparing the biometric information stored in the determination unit 44 with the biometric information acquired by the exit detection device 7.

Also, the determination unit 44 determines whether the history of the biometric information acquired by the entry detection device 6 and the exit detection device 7 has a first pattern, a second pattern, or a third pattern. The first pattern is a pattern in which the predetermined attribute person E has entered the area B and then the predetermined attribute person E has not exited the area B. The second pattern is a pattern in which the predetermined attribute person E has entered the area B and then the predetermined attribute person E has exited the area B. The third pattern is a pattern in which the predetermined attribute person E has not entered the area B.

If a specific object is detected by the specific object detection unit 43, the determination unit 44 determines that the predetermined attribute person E is present in the area B regardless of the biometric information acquired by the entry detection device 6 and the exit detection device 7.

Also, if the history of the biometric information acquired by the entry detection device 6 and the exit detection device 7 has the above-described first pattern, the determination unit 44 determines that the predetermined attribute person E is present in the area B regardless of the result of the detection by the specific object detection unit 43.

Also, if no specific object is detected by the specific object detection unit 43 and the history of the biometric information acquired by the entry detection device 6 and the exit detection device 7 has the above-described second or third pattern, the determination unit 44 determines that the predetermined attribute person E is not present in the area B.

Note that the present disclosure is not limited to this configuration. A configuration is also possible in which, only if a specific object is detected by the specific object detection unit 43 and the history of the biometric information acquired by the entry detection device 6 and the exit detection device 7 has the first pattern, the determination unit 44 determines that the predetermined attribute person E is present in the area B.

Signal Transmission Unit

As shown in FIG. 3, the control unit 4 includes a signal transmission unit 45 (corresponding to an “alarm control unit” or a “transmission unit” according to the present disclosure). The results of the detection by the state detection unit 41 and the person's state detection unit 42 are transmitted to the signal transmission unit 45. Also, the result of the determination by the determination unit 44 is transmitted to the signal transmission unit 45.

The signal transmission unit 45 is configured to be able to transmit a predetermined alarm signal (corresponding to “state information” according to the present disclosure) to the alarm device 2 in response to a predetermined state being detected by the state detection unit 41. The alarm signal is information indicating a result of detection by the state detection unit 41.

The alarm device 2 is activated upon receiving the alarm signal. More specifically, the alarm device 2 issues an alarm by image display and a sound that calls the predetermined attribute person E. In other words, the alarm device 2 is a display device capable of issuing an alarm through image display and is a call device capable of calling the predetermined attribute person E.

Note that the image display may be display of a captured image captured by the image capturing unit 3. Also, content of the image display and content of the calling sound may inform the name of the monitored person D, the location of the room (such as the room number), the current state, and the like, for example. The alarm device 2 may also be capable of issuing an alarm through vibration.

That is, the monitoring system A includes the signal transmission unit 45 capable of activating the alarm device 2 in response to a predetermined state being detected by the state detection unit 41. Also, the monitoring device 1 includes the signal transmission unit 45 capable of transmitting an alarm signal that is information indicating the result of detection by the state detection unit 41, to the outside in response to a predetermined state being detected by the state detection unit 41.

The signal transmission unit 45 is configured to decide whether or not to transmit the above-described alarm signal to the alarm device 2 in accordance with an alarm decision flow shown in FIG. 4, based on results of the detection by the state detection unit 41 and the person's state detection unit 42, and a result of the determination by the determination unit 44.

Alarm Decision Flow

Upon start of the alarm decision flow shown in FIG. 4, processing in step S01 is first executed. In step S01, it is determined whether or not a predetermined state has been detected by the state detection unit 41. If it is determined that the predetermined state has not been detected by the state detection unit 41 (“No” in step S01 in FIG. 4), the alarm decision flow is temporarily ended. If it is determined that the predetermined state has been detected by the state detection unit 41 (“Yes” in step S01 in FIG. 4), the processing moves to step S02.

In step S02, it is determined whether or not the determination unit 44 has determined that the predetermined attribute person E is present in the area B. If it is determined that the predetermined attribute person E is present in the area B (“Yes” in step S02 in FIG. 4), the processing moves to step S03. Also, if it is determined that the predetermined attribute person E is not present in the area B (“No” in step S02 in FIG. 4), the processing moves to step S04.

In step S03, it is determined whether or not a predetermined first condition is satisfied. The first condition of the present embodiment is the condition that an abnormal state has been detected by the person's state detection unit 42. Note however that the present disclosure is not limited to this configuration. The first condition may be, for example, the condition that an abnormal state different from the predetermined state has been detected by the state detection unit 41 or the condition that the predetermined state continues for a predetermined period of time (e.g., for five minutes).

If the first condition is satisfied (“Yes” in step S03 in FIG. 4), the processing moves to step S05.

In step S05, the signal transmission unit 45 transmits the above- described alarm signal to the alarm device 2. That is, in this case, the signal transmission unit 45 activates the alarm device 2. Then, this alarm decision flow is temporarily ended.

In this way, in a case where the predetermined state is detected by the state detection unit 41 and it is determined by the determination unit 44 that the predetermined attribute person E is present in the area B, if the predetermined first condition is satisfied, the signal transmission unit 45 activates the alarm device 2.

If the first condition is not satisfied (“No” in step S03 in FIG. 4), the alarm decision flow is temporarily ended. Accordingly, in this case, the signal transmission unit 45 does not transmit the above-described alarm signal to the alarm device 2. That is, in this case, the signal transmission unit 45 does not activate the alarm device 2.

In step S04, it is determined whether or not a predetermined second condition is satisfied. The second condition of the present embodiment is the condition that the current time is included in preset hours of the day. The hours of the day may be time set by being input in advance according to the will of the monitored person D or an agreement on the monitoring of the monitored person D (e.g., an agreement on the monitoring time), for example, or may be time for care of the monitored person D, such as time for which the predetermined attribute person E wipes the body of the monitored person D or drain time.

Note however that the present disclosure is not limited to this configuration. The second condition may be, for example, the condition that the time at which the predetermined state is detected by the state detection unit 41 is included in preset hours of the day.

If the second condition is satisfied (“Yes” in step S04 in FIG. 4), the alarm decision flow is temporarily ended. Accordingly, in this case, the signal transmission unit 45 does not transmit the above-described alarm signal to the alarm device 2. That is, in this case, the signal transmission unit 45 does not activate the alarm device 2.

In this way, in a case where the predetermined state is detected by the state detection unit 41 and it is determined by the determination unit 44 that the predetermined attribute person E is not present in the area B, if the predetermined second condition is satisfied, the signal transmission unit 45 does not activate the alarm device 2.

If the second condition is not satisfied (“No” in step S04 in FIG. 4), the processing moves to step S05. In this case, the signal transmission unit 45 transmits the above-described alarm signal to the alarm device 2. That is, in this case, the signal transmission unit 45 activates the alarm device 2. Then, this alarm decision flow is temporarily ended.

As is clear from the description above, in a case where the predetermined state is detected by the state detection unit 41 (“Yes” in step S01 in FIG. 4) and it is determined by the determination unit 44 that the predetermined attribute person E is not present in the area B (“No” in step S02 in FIG. 4), the signal transmission unit 45 transmits an alarm signal except if the second condition is satisfied. That is, if the predetermined state is detected by the state detection unit 41 and it is determined by the determination unit 44 that the predetermined attribute person E is not present in the area B, the signal transmission unit 45 activates the alarm device 2. Also, if the predetermined state is detected by the state detection unit 41 and it is determined by the determination unit 44 that the predetermined attribute person E is not present in the area B, the signal transmission unit 45 transmits an alarm signal to the outside.

With this configuration, as shown in FIG. 1, if the monitored person D is in a predetermined state (e.g., state where the monitored person D has fallen) and the predetermined attribute person E is not present in the area B where the monitored person D is present, the monitoring device 1 transmits an alarm signal to the alarm device 2. As a result, the monitoring device 1 activates the alarm device 2.

Also, as is clear from the description above, if the predetermined state is detected by the state detection unit 41 (“Yes” in step S01 in FIG. 4) and it is determined by the determination unit