100 can include a microbeamformer located within the probe 10 and in communication with the system beamformer 22. The microbeamformer may perform preliminary beamforming and/or signal processing that can reduce the number of communication channels required to transmit the receive signals to the computing device 28.

Control of ultrasound system parameters such as scanning mode (e.g., B-mode, M-mode), probe selection, beam steering and focusing, and signal and image processing is done under control of a system controller 26 which is coupled to various modules of the system 100. The system controller 26 may be formed by application specific integrated circuits (ASICs) or microprocessor circuitry and software data storage devices such as RAMs, ROMs, or disk drives. In the case of the probe 10, some of this control information may be provided to the electronic circuitry 14 from the computing device 28 over the cable 16, conditioning the electronic circuitry 14 for operation of the array as required for the particular scanning procedure. The user inputs these operating parameters by means of a user interface device 20.

In some embodiments, the image processor 24 is configured to generate images of different modes to be further analyzed or output to the display 30. For example, in some embodiments, the image processor can be configured to compile a B-mode image, such as a live B-mode image, of an anatomy of the patient. In other embodiments, the image processor 24 is configured to generate or compile an M-mode image. An M-mode image can be described as an image showing temporal changes in the imaged anatomy along a single scan line.

It will be understood that the computing device 28 may comprise hardware circuitry, such as a computer processor, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), capacitors, resistors, and/or other electronic devices, software, or a combination of hardware and software. In some embodiments, the computing device 28 is a single computing device. In other embodiments, the computing device 28 comprises separate computer devices in communication with one another.

The computing device 28 further includes a guidance system 25, which is used to generate and output guidance instructions for a user to move the ultrasound probe 10 to a desired or selected pose. The guidance system 25 may be configured to receive various inputs from the system, including inputs from the interface device 20, ultrasound imaging data from the ultrasound probe 10, the system beamformer 22, and/or the signal and image processor 24. In some embodiments, the guidance system 25 is configured to receive an input from the interface device 20 corresponding to a desired or selected view of anatomy. The desired or selected view of the anatomy comprises, or is associated with, a desired or selected pose of the ultrasound probe 10. The guidance system 25 may determine, based on ultrasound imaging data and/or position data from a positioning system or sensor (e.g., medical positioning system (MPS), optical image sensor, accelerometer, gyroscope), a current pose of the ultrasound probe 10. For example, in some embodiments, the guidance system 25 includes an optical imaging sensor, such as a camera, and the position data comprises images of the ultrasound probe positioned relative to the patient. In this embodiment, the guidance system is configured to determine, by image processing the images, the pose of the ultrasound probe relative to the anatomy of the patient.

In one embodiment, the guidance system 25 determines the pose of the ultrasound probe 10 in terms of one or more physical dimensions relative to the anatomy. For example, the guidance system 25 may determine the pose of the probe in terms of x-y-z coordinates, rocking angle, fanning angle, rotation angle, etc., relative to the anatomy. In some embodiments, the guidance system 25 employs an anatomical model to determine the current pose of the ultrasound probe 10. In some embodiments, the guidance system 25 is configured to determine the current pose using various image processing techniques, including artificial intelligence (A.I.), machine learning, deep learning, and/or neural network architectures. For example, in some embodiments, a convolution neural network (CNN) is used. Based on the determined current pose of the ultrasound probe 10 and the selected or desired pose, the guidance system 25 computes a movement to align the current pose of the ultrasound probe 10 with the desired pose, and outputs a graphical representation of the movement to the display 30.

FIG. 2 is a schematic diagram of a processor circuit 150, according to embodiments of the present disclosure. The processor circuit 150 may be implemented in the computing device 28, the signal and image processor 24, the controller 26, and/or the probe 10 of FIG. 1. As shown, the processor circuit 150 may include a processor 160, a memory 164, and a communication module 168. These elements may be in direct or indirect communication with each other, for example via one or more buses.

The processor 160 may include a central processing unit (CPU), a digital signal processor (DSP), an ASIC, a controller, an FPGA, another hardware device, a firmware device, or any combination thereof configured to perform the operations described herein. The processor 160 may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

The memory 164 may include a cache memory (e.g., a cache memory of the processor 160), random access memory (RAM), magnetoresistive RAM (MRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), flash memory, solid state memory device, hard disk drives, other forms of volatile and non-volatile memory, or a combination of different types of memory. In an embodiment, the memory 164 includes a non-transitory computer-readable medium. The memory 164 may store instructions 166. The instructions 166 may include instructions that, when executed by the processor 160, cause the processor 160 to perform the operations described herein with reference to the computing device 28 and/or the probe 10 (FIG. 1). Instructions 166 may also be referred to as code. The terms “instructions” and “code” should be interpreted broadly to include any type of computer-readable statement(s). For example, the terms “instructions” and “code” may refer to one or more programs, routines, sub-routines, functions, procedures, etc. “Instructions” and “code” may include a single computer-readable statement or many computer-readable statements.

The communication module 168 can include any electronic circuitry and/or logic circuitry to facilitate direct or indirect communication of data between the computing device 28, the probe 10, and/or the display 30. In that regard, the communication module 168 can be an input/output (I/O) device. In some instances, the communication module 168 facilitates direct or indirect communication between various elements of the processor circuit 150 and/or the processing system 106 (FIG. 1A).

A sonographer may desire to obtain an ultrasound image representative of a particular view or imaging plane of an organ (e.g., apical view of the heart) in order to evaluate the condition of the organ and/or make measurements of the organ. However, the movements involved to position and orient the probe may be complex and may include adjustments in several degrees of freedom in three-dimensional space. Thus, it can be challenging for inexperienced sonographers to achieve the desired view. A guidance scheme or procedure may be used to position the ultrasound probe at a desired position and orientation (i.e., pose). Accordingly, the present disclosure describes a guidance interface that displays the movements associated with the desired view in an intuitive manner that involves fewer mental translations for the sonographer.

FIG. 3 is a flow diagram illustrating a method 200 for providing image-based guidance to achieve a desired view using an ultrasound probe and a display. It will be understood that the method 200 may be performed using an ultrasound imaging device and/or an ultrasound imaging system, such as the system 100 shown in FIG. 1. For example, one or more steps of the method 200 may be performed using the guidance system 25 of the system 100 as described above, which may include one or more features of the processor circuit 150 described with respect to FIG. 2.

In step 210, a processor circuit of a guidance system receives ultrasound imaging data representative of a field of view obtained by an ultrasound imaging transducer. The ultrasound imaging data may comprise raw analog ultrasound signals, digital ultrasound signals or data, filtered data, beamformed data, or any other suitable type of data. In some embodiments, the ultrasound imaging data may include B-mode data obtained by an imaging probe or imaging device. In some embodiments, the ultrasound imaging data may include Doppler data such as power Doppler or color Doppler, M-mode data, or any other suitable type of imaging data.

In step 220 the processor circuit receives an input indicating a desired pose of the ultrasound transducer, where the desired pose represents the position and/or orientation of the ultrasound transducer associated with a desired view, such as the apical view of the heart. As stated above, the desired pose may be represented by positional information that includes values associated with one or more physical dimensions or geometric parameters, such as x-y-z coordinates, angles (e.g., fanning, rotation, rocking), spherical coordinates, cylindrical coordinates, etc. The desired pose and/or positional information may be stored in a memory of the processor circuit and recalled or retrieved from the memory in response to receiving the input.

The input may be received by the processor circuit from a user interface device, such as the user interface of the system 100 shown in FIG. 1. In some embodiments, the user interface device comprises a touch-screen display that shows a graphical display or interface including a list or selection of different echocardiography imaging windows. In some embodiments, the user interface device comprises a touch-less, augmented reality display (e.g., glasses or screen), such as a HOLOLENS display manufactured by Microsoft, Inc. FIG. 4A illustrates a graphical user interface 222 used for receiving an input indicating a desired window. The graphical interface includes a diagrammatic view 224 of a patient's anatomy and a plurality of window indicators 226 overlaid at corresponding locations of the anatomy 224 in the interface. The user may select an input by tapping on a window indicator 226 on a touch screen display, using a mouse, trackball, keyboard input, voice command, or any other suitable type of input. In some embodiments, once the window indicator 226 is selected, a set of views—typically defined by the scanning protocol—is displayed in the form of view indicators. For example, if an apical window is selected, the user may be provided an option to switch between A4C, A2C, A3C, or A5C views by selecting the associated view indicator. In some embodiments, if no view selection is provided by the user, the system automatically selects a commonly used or predefined view that is obtainable in a previously selected acoustic window. In some embodiments, once the user selects the window indicator 226, an example of a standard view is shown to the user. In some embodiments, the example standard view comprises a previously-acquired image obtained at the selected window and/or view. In some embodiments, the example standard view is an illustration or cartoon of the standard view. The example standard view may be saved to a memory in communication with the processor. In some embodiments, the indicators 226 comprise view indicators. In some embodiments, the indicators 226 comprise a combination of window indicators and view indicators.

In some embodiments, the input selecting the window and/or view is automatically created using an image processing algorithm. For example, the processor circuit may determine, by image processing of the ultrasound imaging signals, a pose of the ultrasound transducer relative to the anatomy of the patient and automatically selects a window or view based on the determined pose. For example, the processor circuit may generate the input based on a view which the ultrasound transducer is closest to achieving. In another example, the processor circuit may determine—using classification algorithm—a type of an acoustic window, and select such window based on the prediction.

In some embodiments, once the desired pose or view is received or determined by the processor circuit, the processor circuit outputs a user instruction to place the ultrasound transducer at a particular position and/or orientation on the patient's anatomy. FIG. 4B shows an exemplary graphical user interface 232 that shows a diagrammatic view of a patient's anatomy and an ultrasound probe or ultrasound transducer positioned with the desired pose relative to the anatomy 224 of the patient. The graphical user interface includes in indicator 234 of the ultrasound transducer as well as a textual instruction corresponding to the indicator. In some embodiments, the graphical user interface 232 includes only of the indicator 234 of the ultrasound transducer or the textual instruction. In other embodiments, the user instruction includes an audible instruction, a haptic instruction, or any other suitable type of instruction.

In some embodiments, the processor circuit receives the input indicating the desired pose or imaging plane before receiving the ultrasound imaging data. For example, the sonographer may begin the imaging procedure by selecting the desired window and/or view (e.g., apical window and A4C view), and then position the transducer to obtain the ultrasound imaging data. In some embodiments, the processor circuit is configured to receive a continuous stream of ultrasound imaging data, and may receive the input indicating the desired view simultaneously with the ultrasound imaging data.

In step 230, the processor circuit determines, based on the ultrasound imaging data received in step 210 and the input received in step 220, a movement to align the current pose of the ultrasound transducer with the desired pose. In other words, step 230 includes determining a movement to achieve the view or pose associated with the input received in step 220. In an exemplary embodiment, step 230 includes determining the current pose of the ultrasound transducer by image processing the ultrasound imaging data. In some embodiments, the processor circuit uses artificial intelligence (A.I.), machine learning, and/or deep learning architectures to determine the current pose of the ultrasound transducer. In one embodiment, a convolutional neural network (CNN) architecture is trained to predict a relative pose of the ultrasound transducer in respect to a predefined reference plane. Such algorithms are described in, for example, U.S. Provisional Patent Application No. 62/641,540, titled “ULTRASOUND IMAGING PLANE GUIDANCE FOR NEURAL NETWORKS AND ASSOCIATED DEVICES, SYSTEMS, AND METHODS,” filed Mar. 12, 2018, U.S. Provisional Patent Application No. 62/641,508, titled “ULTRASOUND IMAGING PLANE ALIGNMENT USING NEURAL NETWORKS AND ASSOCIATED DEVICES, SYSTEMS, AND METHODS,” filed Mar. 12, 2018, U.S. Provisional Patent Application No. 62/641,493, titled “ULTRASOUND IMAGING DATASET ACQUISITION FOR NEURAL NETWORK TRAINING AND ASSOCIATED DEVICES, SYSTEMS, AND METHODS,” filed Mar. 12, 2018, U.S. Provisional Patent Application No. 62/700,960, titled “ULTRASOUND IMAGING BY DEEP LEARNING AND ASSOCIATED DEVICES, SYSTEMS, AND METHODS,” filed Jul. 20, 2018, U.S. Patent Application No. 62/906,368, titled “AUTOMATIC CLOSED-LOOP ULTRASOUND PLANE STEERING FOR TARGET LOCALIZATION IN ULTRASOUND IMAGING AND ASSOCIATED DEVICES, SYSTEMS, AND METHODS,” filed Sep. 26, 2019, and U.S. Patent Application No. 62/746,042 titled “DEEP LEARNING-BASED ULTRASOUND IMAGING GUIDANCE AND ASSOCIATED DEVICES, SYSTEMS, AND METHODS,” filed Oct. 16, 2018, each of which is hereby incorporated by reference in its entirety.

In that regard, FIG. 5 is a diagrammatic view of an ultrasound image 242 being processed according to an algorithm to determine the current pose of the ultrasound transducer. The image 242 includes outlines 244 of anatomical features, such as organs, chambers of the heart, etc., that may be used to identify the current pose or view of the ultrasound transducer in combination with anatomical features, such as mitral valve, aortic valve, septum, etc. The processor circuit may employ one or more of the architectures described above, including A.I., machine learning, deep learning, and/or neural networks. In some embodiments, the identified anatomical features may be compared to a model of anatomical features to determine the current pose or view. In some embodiments, a state machine comprising a plurality of exemplary image frames associated with a plurality of different poses or views is used to determine the current pose or view of the ultrasound transducer with respect to the patient's anatomical features.

With the current pose or view of the ultrasound transducer determined by the processor circuit, the processor circuit determines a movement of the ultrasound transducer to achieve the desired pose or view. The movement may involve or include a number of physical adjustments in one or more degrees of freedom. The movement may be determined by comparing the current pose to the desired pose. In that regard, the processor may determine a plurality of dimensional values associated with the current pose, such as x-y-z coordinates, rotational angle, fanning angle, rocking angle, etc. These dimensional values can then be compared to the dimensional values associated with the desired pose, such as x-y-z coordinates, rotational angle, fanning angle, rocking angle, etc. The movement can then be determined based on this comparison by, for example, a subtraction of the dimensional values of the current pose from the dimensional values of the desired prose. Accordingly, in some embodiments, the movement is determined based on a direct comparison of the current pose to the desired pose. In other embodiments, the movement is computed by determining the current pose of the ultrasound transducer in a reference coordinate system, and comparing the current pose in the reference coordinate system with the desired pose in the reference coordinate system.

In step 240, the processor circuit generates a graphical representation of the movement determined in step 230. The graphical indicator may include a first indicator of the current pose of the ultrasound transducer and a second indicator of the desired pose of the ultrasound transducer positioned an oriented relative to the first indicator to indicate the movement. One or both of the indicators may indicate the movement such that one or more adjustments in one or more degrees of freedom are shown. In some embodiments, multiple adjustments in multiple degrees of freedom are shown illustrated by the first and/or second indicator.

FIG. 6 is a diagrammatic view of a plurality of ultrasound transducer movements and the graphical indicators associated with the movements. The shapes of the graphical indicators may approximate or represent the ultrasound transducer array of the ultrasound transducer. The movements include sliding, rocking, sweeping, fanning, compression, and rotation. Sliding involves a lateral translation of the transducer in which the angular orientation of the ultrasound transducer remains substantially unchanged. The sliding movement is shown as a lateral translation that includes a first indicator 310a of the ultrasound transducer laterally spaced from a second indicator 310b representative of the desired lateral position. Rocking involves a lateral tilt of the ultrasound transducer along a lateral axis (i.e. the longer dimension) of the ultrasound transducer. The rocking movement is shown using a gradient in which the darker side of an indicator 312 represents the desired direction of rocking or tilting to achieve the desired pose or view. Sweeping involves a forward or backward translation of the ultrasound transducer. The sweeping movement is shown by a first indicator 314a and a second indicator 314b spaced from the first indicator 314a in a sweeping direction (e.g., forward/backward). Fanning involves a forward or backward tilt of the ultrasound transducer along a transverse axis that is perpendicular to the lateral axis. The fanning movement is shown similar to the representation for rocking in that the indicator 316 includes a gradient in which the darker side of the indicator 316 represents the desired direction of fanning or tilting to achieve the desired pose or view. Compression involves pressing the ultrasound transducer into the patient along a vertical axis of the ultrasound probe. Accordingly, compression involves applying increased force to the ultrasound probe in a downward direction along the vertical axis. By contrast, decompression involves reducing the pressure or force applied to the patient by the ultrasound probe. The compression motion is illustrated by an outline displayed around an indicator 318 of the ultrasound transducer. The decompression motion is illustrated by a dashed outline around the indicator 321. In some aspects, the boldness, width, color, or other visual aspect of the outlines may correspond to the amount of compression/decompression needed to achieve the desired view. The rotation movement involves rotating the ultrasound transducer about the vertical axis of the ultrasound probe to a particular angular orientation. The rotation movement may be shown by an indicator 320, such as the second indicator, being oriented or rotated relative to the other indicator to the angular orientation associated with the desired pose. In some embodiments, the indicator 320 of representative of rotation includes an arrow. However, in other embodiments, the indicator