104 to a surface adjacent to the smart TV 102 such as a furniture item on which the smart TV 102 is supported, or to a portion of a wall proximate to where the smart TV 102 is supported. Additional examples are contemplated.

As described above, the camera 104 in certain examples is a pan-tilt-zoom (PTZ) camera. The gimbal 142 allows for movement of the image sensor 150. For example, the gimbal 142 can be used to pan the image sensor 150 left to right, and to tilt the image sensor 150 up and down to adjust the field of view of the video stream captured by the image sensor 150.

The electric motor 144 is connected to the gimbal 142. As will be described in more detail below, the electric motor 144 can be used to move the gimbal 142 to pan left to right, and to tilt up and down based commands received from the controller 148.

In some examples, the electric motor 144 is connected to the lens 146 to control an optical zoom of the camera 104. For example, the lens 146 is mounted in front of the image sensor 150, and the electric motor 144 can move the lens 146 relative to the image sensor 150 to adjust the focal length of the lens 146 to zoom-in and zoom-out based the commands received from the controller 148. Alternatively, the controller 148 can perform digital zoom on the video stream captured by the image sensor 150 such as by enlarging pixels within an area of interest.

The controller 148 is an example of a computing device that can control operation of the camera 104. For example, the controller 148 can instruct the electric motor 144 to move the gimbal 142 such as to pan the image sensor 150 left and right, and to tilt the image sensor 150 up and down. Additionally, the controller 148 can instruct the electric motor 144 to move the lens 146 relative to the image sensor 150 to adjust optical zoom. In further examples, the controller 148 can provide digital zoom on the video stream captured by the image sensor 150.

The communications interface 152 allows the camera 104 to connect to the smart TV 102 and/or to the communications network 124. The communications interface 152 can include both wired interfaces (e.g., USB port) and wireless interfaces (e.g., Bluetooth connection).

FIG. 5 schematically illustrates examples of operating modes 500 under which the camera 104 can be configured to operate. In certain examples, the camera 104 receives instructions from the camera control module 400 installed on the telehealth device 200 to move the gimbal 142 using the electric motor 144 to adjust the field of view of the camera 104.

In the example shown in FIG. 5, the camera 104 can operate under a manual mode 502 in which the remote clinician uses one or more controls provided by the telehealth device 200 to manually control the field of view of the camera 104. Alternatively, the camera 104 can operate under a tracking mode 504 in which the camera 104 automatically adjusts the field of view to track an object of interest identified by the remote clinician on the telehealth device 200.

FIG. 6 illustrates an example of a user interface 600 displayed on the display device 210 of the telehealth device 200. As shown in FIG. 6, the user interface 600 displays a video stream of a patient P and a nurse N captured by the camera 104 inside the patient environment PE. The user interface 600 further displays a message 602 such as “Who do you want to follow?” and displays one or more options for the remote clinician to select. In the example shown in FIG. 6, the remote clinician can select an icon 604 to perform the manual mode 502.

FIG. 7 schematically illustrates an example of a method 700 of conducting a telehealth consultation on the telehealth device 200 under the manual mode 502. The method 700 can be performed by the camera control module 400 installed on the telehealth device 200. The method 700 can be performed in response to the remote clinician selecting the icon 604 displayed on the user interface 600 shown in FIG. 6.

As shown in FIG. 7, the method 700 includes an operation 702 of receiving one or more inputs from the remote clinician. The one or more inputs can be received in a plurality of different ways at least some of which are described herein.

As an illustrative example, the remote clinician can enter inputs by using a tool 610 displayed on the user interface 600. In this example, the remote clinician can select a left or right side of a horizontal axis H using a mouse cursor to enter an input to pan the gimbal 142 of the camera 104 left or right, and can select an upper or lower portion of a vertical axis V using the mouse cursor to enter an input to tilt the gimbal 142 of the camera 104 up or down.

As a further illustrative example, the remote clinician can click on the video stream displayed in the user interface 600, and while holding the click, can move the mouse to enter inputs to pan and/or tilt the gimbal 142. As a further illustrative example, the remote clinician can select one or more keyboard arrow keys to enter inputs to pan and/or tilt the gimbal 142. In yet further examples, the display device 210 of the telehealth device 200 includes a touchscreen allowing the remote clinician to use their fingers or a stylus to enter inputs to pan and/or tilt the gimbal 142. In yet further examples, the camera 212 of the telehealth device 200 can recognize gestures from the remote clinician to enter inputs to pan and/or tilt the gimbal 142. Additional examples for the remote clinician to enter inputs to pan and/or tilt the gimbal 142 are possible.

The remote clinician can also scroll a wheel in clockwise or counterclockwise directions on the mouse to enter inputs to zoom in or out. In another example, the remote clinician while clicking on the video stream displayed in the user interface 600, can move the mouse up or down to enter inputs to zoom in or out. As a further illustrative example, the remote clinician can use one or more keys on a keyboard to enter inputs to zoom in or out. In examples where the display device 210 includes a touchscreen, the remote clinician can move their fingers together or apart on the touchscreen to enter inputs to zoom in or out. In further examples, the camera 212 can recognize gestures from the remote clinician to enter inputs to zoom in or out. Additional examples for the remote clinician to enter inputs to zoom in or out are possible.

Next, the method 700 further includes an operation 704 of converting the one or more inputs into commands for panning, tilting, and/or zooming the camera 104 in the patient environment PE. For example, operation 704 can include generating commands for the controller 148 to instruct the electric motor 144 to move the gimbal 142 of the camera 104 to pan the image sensor 150 left or right, and to tilt the image sensor 150 up or down. Additionally, operation 704 can include generating commands for the controller 148 to instruct the electric motor 144 to move the lens 146 relative to the image sensor 150 to zoom in or out. In further examples, operation 704 can include generating commands to provide a digital zoom on the video stream displayed in the user interface 600 such as by enlarging pixels within an area of interest.

Next, the method 700 includes an operation 706 of sending the commands to the camera 104 located in the patient environment PE. In some examples, the camera 104 receives the commands directly from the telehealth device 200 via the connection through the communications network 124. In alternative examples, the smart TV 102 receives the commands from the telehealth device 200 via the communications network 124, and the camera 104 thereafter receives the commands from a connection to the smart TV 102.

Referring back to FIG. 6, the remote clinician can also identify one or more objects of interest in the video stream displayed in the user interface 600 to perform the tracking mode 504 on the camera 104 located in the patient environment PE. For example, the remote clinician can select an icon 606 to automatically track the field of view of the camera 104 the movement of the patient P. Alternatively, the remote clinician can select an icon 608 to automatically track the field of view of the camera 104 with the movement of the nurse N who is present inside the patient environment PE. In further examples, the remote clinician can select an icon to automatically track an object such as a medical device including, for example, a spot monitor, hospital bed, and the like. An illustrative example of performing the tracking mode 504 based on the selection of the icons 606, 608 will next be described with reference to FIG. 8.

FIG. 8 schematically illustrates an example of a method 800 of conducting a telehealth consultation on the telehealth device 200 under the tracking mode 504. The method 800 can be performed by the camera control module 400 installed on the telehealth device 200.

The method 800 includes an operation 802 of receiving an instruction to follow an object in the video stream captured by the camera 104 in the patient environment PE. For example, operation 802 can include receiving a selection of the icon 606 displayed on the user interface 600 to follow the patient P. As a further example, operation 802 can include receiving a selection of the icon 608 displayed on the user interface 600 to follow the nurse N.

Additional icons can be displayed in the user interface 600 to track additional persons including doctors, medical specialists, and family members, and to track additional types of objects such as medical devices including a spot monitor, a hospital bed, and the like.

In some examples, operation 802 can include using the speaker and microphone unit 214 to receive a verbal input from the remote clinician that can be processed by the computing device 202 of the telehealth device 200 for recognizing an object such as person or piece of medical equipment. Further examples of receiving the instructions in operation 802 are possible.

Next, the method 800 includes an operation 804 of identifying the object in the video stream captured by the camera 104 in the patient environment PE. As an illustrative example, a person such as the patient P or the nurse N can be identified by facial recognition. For example, an image of the patient P's face is captured and stored in the EMR 302 when the patient P is admitted to the healthcare facility, and the telehealth device 200 can retrieve the image from the EMR 302 via the communications network 124. Thereafter, the telehealth device 200 uses the image to identify the patient P in the video stream captured by the camera 104.

As another example, the telehealth device 200 can recognize a signal from a tag worn by the patient P to identify the patient in the video stream captured by the camera 104. The signal can be sensed by a sensor on the smart TV 102 or the camera 104, and the signal can be communicated to the telehealth device 200 via the communications network 124. In examples where the object is a medical device, the telehealth device 200 can identify the object by recognizing its shape (e.g., the shape of a hospital bed or the shape of a spot monitor), or by recognizing a signal from a tag attached to the medical device. Additional examples of identifying the object in operation 804 are possible.

Next, the method 800 includes an operation 806 of generating commands for the camera 104 to track the movement of the object identified in operation 804, followed by an operation 808 of sending the commands to the camera 104 via the communications network 124. The commands are received by the camera 104 either directly from the telehealth device 200 via the communications network 124, or alternatively, the camera 104 receives the commands from the smart TV 102, which receives the commands via the communications network 124.

In some examples, the commands are generated and sent by the telehealth device 200 each time the telehealth device 200 detects movement of the object. In alternative examples, operation 806 can include generating a command for the camera 104 to track the movement of the object identified in operation 804, and thereafter the camera 104 tracks the movement of the object such that telehealth device 200 does not generate commands each time the object moves. Instead, the camera 104 recognizes the movements of the object, and the controller 148 of the camera 104 instructs the electric motor 144 to move the gimbal 142 to pan left and right and/or to tilt up and down based on the movements of the object detected by the camera 104. Also, the controller 148 can instruct the electric motor 144 to move the lens 146 to zoom in when the object moves away from the camera, or to zoom out when the object moves toward the camera.

In some examples, the camera 104 can use facial recognition to identify a person and track their movements. In further examples, the camera 104 can use motion sensing to follow a moving object such as a patient or a caregiver in the patient environment PE.

FIG. 9 schematically illustrates an example of a method 900 of conducting a telehealth consultation using the telehealth device 200. Advantages of the method 900 can include reducing barriers to connecting with patients via video communications by reducing the steps, time, and costs often associated with telehealth consultations.

As shown in FIG. 9, the method 900 includes an operation 902 of initiating the telehealth consultation. As an example, the telehealth consultation can be initiated following acceptance of a telehealth consultation request on the telehealth device 200 by the remote clinician. The telehealth consultation request can be generated by the patient P using the smart TV 102 or the mobile device 106, and can be routed to the telehealth device 200 for acceptance by the remote clinician via the communications network 124. In certain examples, the app 114 displayed on the smart TV 102 (see FIG. 1) can be selected by the patient using the remote control 108 to request or initiate the telehealth consultation with the remote clinician.

Operation 902 can include security measures to ensure compliance with the Health Insurance Portability and Accountability Act of 1996 (HIPAA), which is a federal law that requires the creation of national standards to protect sensitive patient health information from being disclosed without the patient's consent or knowledge. For example, telehealth consultation request can be generated after the camera 104 captures an image of the patient and facial recognition is performed on the captured image by the controller 128 of the smart TV 102 or the computing device 202 of the telehealth device 200 to confirm the identity of the patient. When the identity of the patient is not confirmed, the telehealth consultation request is not generated, or the telehealth consultation is otherwise not initiated such that the method 900 terminates. In further examples, the identity of the patient can be confirmed by other means such as by using at least one of an admissions, transfer, discharge (ATD) data feed, and bed or patient location determination by a real-time locating system (RTLS), and any combination thereof. In further examples, the telehealth consultation request and any information or data exchanged during the telehealth consultation can be encrypted to provide further data security.

When the smart TV 102 is being used by the patient P for consuming media content, the method 900 can include an operation 904 of adjusting playback of the media content on the smart TV 102 to prevent the patient P from being distracted. For example, operation 904 can include pausing the playback of the media content during the telehealth consultation. In some examples, the playback of the media content is adjusted by lowering or muting the volume of the media content, and/or by decreasing or minimizing the display of the media content without pausing the media content to reduce distraction during the telehealth consultation. In further examples, operation 904 can prevent the patient P from adjusting the playback of the media content (e.g., prevent the patient from increasing the volume) and/or from selecting new media content for playback on the smart TV 102 during the telehealth consultation. Additional examples of adjusting the playback of the media content on the smart TV 102 are possible.

Next, the method 900 includes an operation 906 of allocating a portion of the display 126 of the smart TV 102 for displaying a video stream of the remote clinician captured by the camera 212 and speaker and microphone unit 214 of the telehealth device 200. As an example, operation 906 can include allocating the portion 118 shown in FIG. 1 for displaying the video stream of the remote clinician on the smart TV 102 during the telehealth consultation. In some examples, operation 906 displays the video stream of the remote clinician while also displaying media content from one or more applications included in the application list 110. In some further examples, operation 906 displays the video stream of the remote clinician while also displaying the messaging chat with the remote clinician (see portion 120 in FIG. 1).

Next, the method 900 includes an operation of adjusting a field of view of the camera 104 based on commands received during the telehealth consultation. In some examples, the field of view of the camera 104 is manually adjusted, in accordance with the method 700. In alternative examples, the field of view of the camera 104 is automatically adjusted to track the movement of an object such as the patient, in accordance with the method 800.

In some further examples, the method 900 can include sharing views of one or more applications being displayed on the smart TV 102 and the telehealth device 200. For example, when the patient is viewing an educational video from the app 112 on the smart TV 102, a view of the educational video can also be displayed on the telehealth device 200. In such examples, the remote clinician can view what the patient is watching on the smart TV 102. This allows the remote clinician to provide feedback such as when the patient has a question based on the educational video that is being output on the smart TV 102 in the patient environment PE.

FIG. 10 schematically illustrates another example of a method 1000 of conducting a telehealth consultation. The method 1000 when performed by the telehealth device 200 provides an automatic smart data display for viewing by the remote clinician during the telehealth consultation with the patient. For example, the method 1000 enables the telehealth device 200 to acquire relevant patient data and display the data during the telehealth consultation allowing for faster clinical analysis. This can reduce the time to provide a medical intervention, which can improve clinical outcomes for critical health conditions such as stroke, sepsis, and hypoxia.

The method 1000 includes an operation 1002 of receiving a telehealth consultation request. In accordance with the examples described above, the telehealth consultation request can be generated by the patient using the smart TV 102 or the mobile device 106, and can be routed to the telehealth device 200 via the communications network 124. In certain examples, the app 114 displayed on the smart TV 102 (see FIG. 1) can be selected by the patient using the remote control 108 to generate and send the telehealth consultation request. In further examples, the telehealth consultation request can be generated by a caregiver who is located in the patient environment PE such as the nurse N or a family member of the patient P (e.g., see FIG. 6).

FIG. 11 schematically illustrates an example of a telehealth consultation request 1100. The telehealth consultation request 1100 includes a request type 1102, and in some instances, can also include a description 1104 that briefly describes patient symptoms and/or a reason for the telehealth consultation. The request type 1102 can be selected from a list of predefined categories displayed on the smart TV 102 when the app 114 is opened such as stroke, sepsis, hypoxia, heart failure, and other categories. In further examples, the request type 1102 can be selected from a list of predefined categories displayed on the mobile device 106.

In further examples, the patient P or the caregiver in the patient environment PE (e.g., the nurse N shown in FIG. 6), can speak one or more words to describe the request type, and the speaker and microphone unit 130 of the smart TV 102 recognizes the speech, and matches a request type based on the one or more spoken words to generate the telehealth consultation request