Hereinafter, although a case where the insert is a catheter with a puncture needle will be described as an example, the ultrasound diagnostic apparatus of the present invention can also be applied to a case where an insert other than the catheter with a puncture needle is inserted into the blood vessel. Here, the insert extends linearly, and can puncture a body surface and a vascular wall of the subject.

Hereinafter, a blood vessel satisfying a condition set in advance among blood vessels present inside the subject is referred to as a “target blood vessel”, and the target blood vessel includes a blood vessel into which the insert puncturing the subject is inserted.

First Embodiment

As shown in FIG. 1, an ultrasound diagnostic apparatus (hereinafter, referred to as an ultrasound diagnostic apparatus 1) according to the first embodiment of the present invention has an ultrasound probe 21 that comprises a transducer array 2, and a processor 22 that is connected to the ultrasound probe 21. Each of a transmission circuit 3 and a reception circuit 4 is connected to the transducer array 2. The transmission circuit 3 and the reception circuit 4 configure a transmission and reception circuit 5 and are included in the ultrasound probe 21 in the configuration shown in FIG. 1. An image generation unit 6 is connected to the reception circuit 4, a display controller 7 is connected to the image generation unit 6, and a display device 8 is connected to the display controller 7.

A scene determination unit 12, a blood vessel information acquisition unit 16, and a region-of-interest detection unit 18 are connected to the image generation unit 6, and a scene determination unit 12 is connected to the region-of-interest detection unit 18. A blood vessel information display unit 17 is connected to the blood vessel information acquisition unit 16, and the display controller 7 is connected to the blood vessel information display unit 17. An apparatus controller 13 is connected to each of the transmission and reception circuit 5, the image generation unit 6, the display controller 7, the scene determination unit 12, the blood vessel information acquisition unit 16, the blood vessel information display unit 17, and the region-of-interest detection unit 18, and the region-of-interest detection unit 18, an input device 14, and a storage unit 15 are connected to the apparatus controller 13. The apparatus controller 13 and the storage unit 15 are connected in a state in which information can be transferred therebetween.

In the configuration of FIG. 1, the image generation unit 6, the display controller 7, the scene determination unit 12, the apparatus controller 13, the blood vessel information acquisition unit 16, the blood vessel information display unit 17, and the region-of-interest detection unit 18 are provided in the processor 22. The transmission and reception circuit 5 (that is, the transmission circuit 3 and the reception circuit 4) of the ultrasound probe 21 and the image generation unit 6 of the processor 22 cooperate with each other to configure an image acquisition unit 11 that acquires an ultrasound image.

The transducer array 2 has a plurality of transducers arranged in a one-dimensional or two-dimensional manner. A plurality of transducers may be arranged linearly like a linear ultrasound probe or may be arranged in a curved manner like a convex or sector ultrasound probe. Each of a plurality of transducers transmits an ultrasonic wave in response to a drive signal supplied from the transmission circuit 3, receives an ultrasound echo generated inside the subject, and outputs an electric signal based on the ultrasound echo. Each transducer is configured by forming electrodes at both ends of a piezoelectric body consisting of, for example, piezoelectric ceramic represented by lead zirconate titanate (PZT), a polymer piezoelectric element represented by poly vinylidene di fluoride (PVDF), piezoelectric single crystal represented by lead magnesium niobate-lead titanate (PMN-PT), or the like.

The transmission circuit 3 causes the transducer array 2 to transmit an ultrasound beam toward the subject. Specifically, the transmission circuit 3 includes, for example, a plurality of pulse generators, and adjusts a delay amount of each drive signal to a plurality of transducers of the transducer array 2 based on a transmission delay pattern selected in response to a control signal from the apparatus controller 13 and supplies the drive signals to a plurality of transducers. Each drive signal is a pulsed or continuous-wave voltage signal, and in a case where the drive signal is applied to the electrodes of each transducer of the transducer array 2, the piezoelectric body expands and contracts. As a result, a pulsed or continuous-wave ultrasonic wave is generated from each transducer, and an ultrasound beam is formed from a combined wave of the ultrasonic waves.

The transmitted ultrasound beam is reflected by, for example, each part (for example, an organ and a blood vessel) inside the subject and appliance disposed inside the subject. With this, an ultrasound echo is generated and propagates inside the subject toward the transducer array 2, and is finally received by a plurality of transducers of the transducer array 2. In this case, each transducer expands and contracts with the reception of the ultrasound echo to generate an electric signal, and outputs the electric signal to the reception circuit 4.

The reception circuit 4 executes processing on a signal (strictly, an analog electric signal) output from the transducer array 2 in response to a control signal from the apparatus controller 13 to generate a sound ray signal. For example, as shown in FIG. 2, the reception circuit 4 has a configuration in which an amplification unit 23, an analog-digital (AD) conversion unit 24, and a beam former 25 are connected in series.

The amplification unit 23 amplifies a signal output from each of a plurality of transducers of the transducer array 2 and transmits the amplified signal to the AD conversion unit 24. The AD conversion unit 24 converts the amplified signal into digital reception data and transmits each piece of converted reception data to the beam former 25. The beam former 25 executes reception focus processing of giving a delay to each piece of reception data converted by the AD conversion unit 24 conforming to a sound speed or a distribution of a sound speed set based on a reception delay pattern selected in response to a control signal from the apparatus controller 13 and performing addition. With the reception focus processing, each piece of reception data converted by the AD conversion unit 24 is subjected to phasing addition, and a sound ray signal in which a focus of the ultrasound echo is narrowed is acquired.

The image generation unit 6 generates an ultrasound image based on the sound ray signal generated by the reception circuit 4, and as shown in FIG. 3, has a configuration in which a signal processing unit 26, a digital scan converter (DSC) 27, and an image processing unit 28 are sequentially connected in series.

The signal processing unit 26 performs correction of attenuation on the sound ray signal generated by the reception circuit 4 due to a distance depending on a depth of a reflection position of the ultrasonic wave, and then, executes envelope detection processing, thereby generating a B mode image signal indicating an ultrasound image.

The DSC 27 converts (raster-converts) the B mode image signal generated by the signal processing unit 26 into an image signal conforming to a normal television signal scanning system.

The image processing unit 28 executes various kinds of necessary image processing, such as gradation processing, on the B mode image signal input from the DSC 27, and then, outputs the B mode image signal to the display controller 7 and the image analysis unit 9. The B mode image signal subjected to the image processing by the image processing unit 28 corresponds to an ultrasound image.

The apparatus controller 13 performs control such that the transmission and reception circuit 5 and the image generation unit 6 (in other words, the image acquisition unit 11) continuously acquire ultrasound images at a given frame rate multiple times in an acquisition period of the ultrasound image.

The display controller 7 executes predetermined processing on the ultrasound image generated by the image generation unit 6 (in other words, the ultrasound image acquired by the image acquisition unit 11) and displays the ultrasound image on the display device 8 under the control of the apparatus controller 13. As shown in FIG. 4, the ultrasound image (hereinafter, referred to as an ultrasound image U) displayed on the display device 8 is developed in a depth direction and a width direction. Here, the width direction of the ultrasound image U is a direction in which a plurality of scanning lines configuring the ultrasound image U are arranged. The depth direction of the ultrasound image U is a direction in which the scanning lines extend. Each portion in the ultrasound image U is displayed at a position depending on a distance (depth) from a body surface of the subject with which the ultrasound probe 21 is brought into contact, in the depth direction.

As the ultrasound probe 21 is moved in a state of being brought into contact with the subject, a part of a tomographic plane of which is observed is changed depending on the ultrasound image U, and a direction in bringing the ultrasound probe 21 into contact with the subject is changed. Thereby, it is possible to switch an observation direction of the blood vessel inside the subject and the insert. For example, in a case where the ultrasound probe 21 is brought into contact with the subject in an orientation in which a direction (that is, a scanning direction) in which a plurality of transducers are arranged in the transducer array 2 follows an extension direction of the blood vessel and the insert, that is, in a case where a major axis method (paralleling method) is employed, the longitudinal sections of the blood vessel and the insert are observed in the ultrasound image U. Here, the longitudinal section of each of the blood vessel and the insert means a cut section along the extension direction of each of the blood vessel and the insert.

On the other hand, in a case where the ultrasound probe 21 is brought into contact with the subject in an orientation in which the arrangement direction (scanning direction) of the transducers in the transducer array 2 crosses the extension direction of the blood vessel and the insert, that is, in a case where a minor axis method (crossover method) is employed, as shown in FIG. 4, the transverse sections of the blood vessel B and the insert C are observed in the ultrasound image U. Here, the transverse section of each of the blood vessel and the insert means a cut section perpendicular to the extension direction of each of the blood vessel and the insert.

The display device 8 displays the ultrasound image U and the like under the control of the display controller 7, and includes, for example, a display, such as a liquid crystal display (LCD) or an organic electroluminescence display (organic EL display).

As shown in FIG. 4 and the like, blood vessel information J is displayed on the display device 8 along with the ultrasound image U. The blood vessel information J is described in, for example, a dialog box type display object as shown in FIG. 4, and may be displayed on the ultrasound image U in a superimposed manner or may be embedded and displayed as a part of the ultrasound image U.

The scene determination unit 12 determines a scene in a case where the ultrasound image U is acquired, by analyzing the ultrasound image U generated by the image generation unit 6 (in other words, the ultrasound image acquired by the image acquisition unit 11) with a known image processing technique. Examples of the scene that can be determined by the scene determination unit 12 include, for example, a search scene where a target blood vessel Bx is searched, an insertion scene where a catheter with a puncture needle as the insert C punctures and is inserted toward the target blood vessel Bx, and a placement scene where a distal end part of the insert C is placed inside the target blood vessel Bx.

The search scene is a scene at a stage before the insert C is inserted, and in this case, the operator is in a situation of moving the ultrasound probe 21 on the body surface of the subject to search for the target blood vessel Bx in the ultrasound image U.

The insertion scene is a scene until the distal end of the insert C reaches the target blood vessel Bx after puncturing the body surface of the subject, and in this case, the operator confirms the ultrasound image U while inserting the insert C and switches the orientation of the ultrasound probe 21 to change the observation direction of the blood vessel B and the insert C as needed. Along with this, the cross sections of the blood vessel B and the insert C rendered in the ultrasound image U are switched between the longitudinal sections (the cross sections in the major axis method) and the transverse sections (the cross sections in the minor axis method).

The placement scene is a scene at a stage where a distal end of the catheter with a puncture needle as the insert C, that is, a distal end of the puncture needle breaks through a vascular wall of the target blood vessel Bx to enter the target blood vessel Bx along with the distal end part of the catheter, then, the puncture needle is removed, and the distal end part of the catheter is placed inside the target blood vessel Bx. In this case, the distal end part of the insert C is only the distal end part of the catheter since the puncture needle is removed from the inside of the catheter. Thus, the transverse section (for example, see FIG. 4) of the distal end part of the insert C that previously appears as a circular shape with the presence of the puncture needle is separated in a lip shape and is configured with a pair of fragment images (for example, see FIG. 8).

In determining the scene, the scene determination unit 12 may analyze a video of ultrasound images U acquired by the image generation unit 6 at a given frame rate (in other words, ultrasound images U of a plurality of frames continuously acquired by the image acquisition unit 11) or may analyze a static image of an ultrasound image U (that is, an ultrasound image U of one frame). That is, in the first embodiment, the scene determination unit 12 can analyze the ultrasound images U as a video to determine the scene, and can also analyze the ultrasound image U as a static image to determine the scene.

The scene determination by the scene determination unit 12 will be described in detail in the following paragraph.

The blood vessel information acquisition unit 16 detects the target blood vessel Bx in the ultrasound image U through analysis and acquires blood vessel information including at least one of the diameter or the depth of the detected target blood vessel Bx.

The blood vessel information acquisition unit 16 analyzes the ultrasound image U generated by the image generation unit 6 (in other words, the ultrasound image U acquired by the image acquisition unit 11) conforming a known algorithm, thereby detecting the blood vessel B in the ultrasound image U. For example, the blood vessel information acquisition unit 16 can store typical pattern data of a blood vessel region as a template in advance, can calculate similarity to the pattern data while searching the ultrasound image U with the templates, and can regard that the blood vessel B is present at a location where the similarity is equal to or greater than a reference value and is the maximum.

In the calculation of the similarity, in addition to simple template matching, for example, a machine learning method described in Csurka et al.: Visual Categorization with Bags of Keypoints, Proc. of ECCV Workshop on Statistical Learning in Computer Vision, pp. 59-74 (2004) or a general image recognition method using deep learning described in Krizhevsk et al.: ImageNet Classification with Deep Convolutional Neural Networks, Advances in Neural Information Processing Systems 25, pp. 1106-1114 (2012) can be used.

The blood vessel information acquisition unit 16 measures the diameter and the depth of the blood vessel B by applying known image analysis processing to the detected blood vessel B in the ultrasound image U. Here, the diameter of the blood vessel B means the width (in FIG. 4, a length indicated by a symbol d1) of the blood vessel B in the depth direction of the ultrasound image U. The depth of the blood vessel B means the shortest distance (in FIG. 4, a distance indicated by a symbol d2) between a position in the ultrasound image U corresponding to the body surface of the subject and the blood vessel B in the depth direction.

In the first embodiment, although the blood vessel information acquisition unit 16 measures both of the diameter and the depth of the blood vessel B, the blood vessel information acquisition unit 16 may measure only one of the diameter and the depth. A feature quantity (for example, an area or the like) other than the diameter and the depth of the blood vessel B may be further measured.

The blood vessel information acquisition unit 16 specifies the blood vessel B having the measured values of the diameter and the depth within a predetermined range among the blood vessels B the diameter and the depth of which are measured, sets the specified blood vessel B as the target blood vessel Bx, and acquires the blood vessel information J indicating the measured values of the diameter and the depth of the target blood vessel Bx.

In the first embodiment, the blood vessel information acquisition unit 16 sets the blood vessel B the depth of which is comparatively small and the diameter of which is a size enough to insert the insert C, among the detected blood vessels B as the target blood vessel Bx and acquires the blood vessel information J of the target blood vessel Bx. Here, the target blood vessel Bx will be supplemented. The blood vessel B into which the insert C puncturing the subject is actually inserted, among the blood vessels B rendered in the ultrasound image U corresponds to the target blood vessel Bx. In this case, the size of the insert C may be decided in advance, and the blood vessel B having the diameter depending on the size may be selected. On the contrary, the intended blood vessel B may be decided, and the size of the insert C may be set depending on the diameter of the blood vessel B.

The blood vessel information J may indicate at least one of the diameter or the depth of the target blood vessel Bx or may indicate any one of the diameter or the depth.

The blood vessel information display unit 17 displays the blood vessel information J acquired by the blood vessel information acquisition unit 16 within a display range of the ultrasound image U in the display device 8 under the control of the display controller 7. The display range of the ultrasound image U is a range in which the ultrasound image U is actually displayed on an image display screen of the display device 8, and is, for example, a range inside a display frame of the ultrasound image U or a range in which a display window is rendered.

The region-of-interest detection unit 18 detects a region of interest in the ultrasound image U by analyzing the ultrasound image U generated by the image generation unit 6 (in other words, the ultrasound image U acquired by the image acquisition unit 11). The region of interest is a region to be noticed other than the target blood vessel Bx in the ultrasound image U and is different for each scene described above.

While a detection procedure of the region of interest by the region-of-interest detection unit 18 will be described below in detail, as an algorithm in detecting the region of interest, a known algorithm, such as template matching, a machine learning method, or a general image recognition method using deep learning, can be used.

The apparatus controller 13 performs control of each unit of the ultrasound diagnostic apparatus 1 based on a program stored in advance in the storage unit 15 or the like, information input from the operator through the input device 14, and the like.

In a case where the region of interest is detected in the ultrasound image U by the region-of-interest detection unit 18, the apparatus controller 13 decides a display region of the blood vessel information J within the display range of the ultrasound image U based on a position of the region of interest. Then, the apparatus controller 13 performs control such that the blood vessel information display unit 17 displays the blood vessel information J in the decided display region.

In detail, the apparatus controller 13 extracts a display candidate region avoiding the target blood vessel Bx and the region of interest within the display range of the ultrasound image U. Here, the display candidate region avoiding the target blood vessel Bx and the region of interest means a region where the blood vessel information J can be displayed not to overlap (not to cover) the target blood vessel Bx and the region of interest, and hereinafter, is simply referred to as a “display candidate region”.

Then, the apparatus controller 13 decides the display region from the extracted display candidate region, in a case where only one display candidate region is extracted, decides the display candidate region as the display region, and in a case where a plurality of display candidate regions are extracted, decides a region closest to the target blood vessel Bx as the display region.

On the other hand, in a case where the display candidate region is not extracted within the display range of the ultrasound image U, the apparatus controller 13 decides the display region such that at least a part of the blood vessel information J overlaps the region of interest. More specifically, for example, in a case where the region-of-interest detection unit 18 detects a plurality of regions of interest in the ultrasound image U, the apparatus controller 13 sets priority to each of a plurality of regions of interest. In this case, the apparatus controller 13 sets the priority of each of a plurality of regions of interest in the display range of the ultrasound image U based on the position of each of a plurality of regions of interest. For example, the priority of each region of interest is set to increase in an order of lower, lower right, lower left, right, left, upper right, upper left, and upper.

A rule in setting the priority of each region of interest based on the position of each region of interest is stored in the storage unit 15, and the apparatus controller 13 reads out the above-described rule from the storage unit 15 in setting the priority of each region of interest.

Then, in a case where the display candidate region is not extracted within the display range of the ultrasound image U, the apparatus controller 13 decides the display region such that at least a part of the blood vessel information J overlaps a region of interest with lower priority (for example, a region of interest with the lowest priority).

The input device 14 is provided for the operator to perform an input operation, and can be configured with, for example, a keyboard, a mouse, a track ball, a touch pad, and a touch panel.

The priority set to each region of interest in a case where the region-of-interest detection unit 18 detects a plurality of regions of interest in the ultrasound image U may be input from the operator through the input device 14. In this case, since the apparatus controller 13 sets the priority of each region of interest based on input contents to the input device 14, it is possible to reflect user's intention or the like to freely set the priority of each region of interest.

The storage unit 15 stores a control program of the ultrasound diagnostic apparatus 1 and various kinds of information, and a recording medium, such as a flash memory, a hard disk drive (HDD), a solid state drive (SSD), a flexible disc (FD), a magneto-optical disc (MO disc), a magnetic tape (MT), a random access memory (RAM), a compact disc (CD), a digital versatile disc (DVD), a secure digital card (SD card), and a universal serial bus memory (USB memory), a server computer, or the like can be used.

As described above, the storage unit 15 stores the rule in setting the priority to each region of interest in a case where the region-of-interest detection unit 18 detects a plurality of regions of interest in the ultrasound image U.

The storage unit 15 includes a cine memory (not shown), and the cine memory has a capacity for accumulating an ultrasound image (B mode image signal) for one frame or ultrasound images for several continuous frames. The apparatus controller 13 reads out the ultrasound image of one frame stored in the cine memory and transfers the ultrasound image to the display controller 7 in a freeze mode. With this, the ultrasound image U of one frame (that is, a static image) is displayed on the display device 8.

The processor 22 in which the image generation unit 6, the display controller 7, the scene determination unit 12, the blood vessel information acquisition unit 16, the apparatus controller 13, the blood vessel information display unit 17, and the region-of-interest detection unit 18 described above are provided is configured with, for example, a central processing unit (CPU) and a control program causing the CPU to execute various kinds of processing. The present invention is not limited thereto, and the processor 22 may be configured using a field programmable gate array (FPGA), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a graphics processing unit (GPU), or other integrated circuits (ICs), or may be configured by combining such ICs.

The image generation unit 6, the display controller 7, the scene determination unit 12, the apparatus controller 13, the blood vessel information acquisition unit 16, the blood vessel information display unit 17, and the region-of-interest detection unit 18 provided in the processor 22 may be configured to be partially or wholly integrated into one CPU or the like.

The processor 22 may be mounted in, for example, a stationary type apparatus or may be mounted in a portable type apparatus, such as a notebook type personal computer (PC), a smartphone, or a tablet terminal.

Next, the operation of the ultrasound diagnostic apparatus 1 according to the first embodiment will be described in detail referring to a flowchart shown in FIG. 5.

In a display flow of an ultrasound image by the ultrasound diagnostic apparatus 1, Step S1 is executed, and in Step S1, the ultrasound image U is generated. Specifically, first, the ultrasound probe 21 is brought into contact with the body surface of the subject, an ultrasound beam is transmitted from each of a plurality of transducers of the transducer array 2 into the subject in response to the drive signals from the transmission circuit 3, and a reception signal is output from each transducer that receives the ultrasound echo, to the reception circuit 4 subject. Next, the reception signal received by the reception circuit 4 is amplified by the amplification unit 23, is AD-converted by the AD conversion unit 24, and then, is subjected to phasing addition by the beam former 25. As a result, a sound ray signal is generated. The sound ray signal is subjected to the envelope detection processing by the signal processing unit 26 in the image generation unit 6 to become a B mode image signal, and the B mode image signal is output to the display controller 7 through the DSC 27 and the image processing unit 28. With this, an ultrasound image U is generated (in other words, the ultrasound image U is acquired).

In next Step S2, the scene determination unit 12 determines the scene where the ultrasound image U is generated (acquired), by analyzing the generated (acquired) ultrasound image U. In the scene determination by the scene determination unit 12, there are a mode that is performed based on the ultrasound image U of one frame (that is, a static image) and a mode that is performed based on the continuously acquired ultrasound images U of a plurality of frames (that is, a video), and one of such modes is selected. The selection of the mode may be performed, for example, by the operator through the input device 14 or may be automatically performed based on an elapsed time from when the scene determination unit 12 starts the acquisition of the ultrasound image U.

Hereinafter, a flow of the scene determination in each mode will be described.

Scene Determination Based on Static Image

In a case of determining the scene based on the static image, the scene determination unit 12 performs the determination along a flow shown in FIG. 6. First, the scene determination unit 12 executes Step S11, analyzes the ultrasound image U as the static image, and determines whether or not only the blood vessel B is detected in the ultrasound image U or whether or not the blood vessel B and the catheter with a puncture needle as the insert C are detected. In detecting the blood vessel B and the insert C in the ultrasound image U in Step S11, a known algorithm, such as template matching, a machine learning method, or a general image recognition method using deep learning, can be used.

In a case where only the blood vessel B is detected but the insert C is not detected in the ultrasound image U, Step S12, the scene determination unit 12 determines that the scene where the ultrasound image U as the static image is acquired is the search scene.

In a case where both of the blood vessel B and the insert C are detected in the ultrasound image U, the scene determination unit 12 executes Step S13 and determines whether or not the detected insert C is the distal end of the puncture needle. During the observation in the minor axis method (crossover method), the distal end of the puncture needle is rendered as a circular (in detail, a circular dot-shaped) image in the ultrasound image U. In light of this, the scene determination unit 12 determines whether or not the detected insert C is the distal end of the puncture needle, for example, using a method, such as image matching.

The determination about whether or not the detected insert C is the distal end of the puncture needle can also be performed using a method other than the above-described method. Specifically, there is a case where a part (non-distal end part) other than the distal end of the catheter with a puncture needle is rendered as an isolated dot-shaped image in the ultrasound image U. In this case, determination can be performed whether or not the image represents the distal end of the puncture needle based on whether or not acoustic shadow (shadow) appears behind the dot. That is, in a case where an image of a shaft part (non-distal end part) of the needle is rendered in the ultrasound image U, a sound wave is strongly reflected, and accordingly, acoustic shadow appears behind the image. In contrast, in a case where the image of the distal end of the needle is rendered in the ultrasound image U, the reflection of the sound wave is slightly weak, and acoustic shadow behind the image does not appear or is weakened. This is because a reflection area of the sound wave at the needle distal end is smaller than a reflection area in the shaft part for the reason that a needle tip is obliquely cut, the needle is thinned toward the distal end of the needle, or the like.

Incidentally, for the reason that the puncture needle with the needle distal end subjected to echogenic machining is deficient in practicability due to hardness of insertion or the like, in many cases, the shaft part slightly in front of the needle distal end is subjected to echogenic machining. Even in a case where such a puncture needle is used, with the determination based on acoustic shadow described above, the needle distal end and the non-distal end can be distinguished.

In a case where determination is made that the detected insert C is the distal end of the puncture needle, Step S14, the scene determination unit 12 determines that the scene where the ultrasound image U as the static image is acquired is the insertion scene.

In a case where determination is made that the detected insert C is not the distal end of the puncture needle, the scene determination unit 12 executes Step S15 and determines whether or not the detected insert C is present inside the blood vessel B. In more detail, in Step S15, the scene determination unit 15 determines whether or not the distal end part of the catheter in a state in which the puncture needle is removed is present inside the blood vessel B. During the observation in the minor axis method (crossover method), as shown in FIG. 8, the distal end part of the catheter in a state in which the puncture needle is removed is separated in a lip shape and is rendered as a pair of fragment images in the ultrasound image U. In light of this, the scene determination unit 12 determines whether or not the detected insert C inside the blood vessel B is the distal end part of the catheter, for example, using a method, such as image matching.

In a case where determination is made that the detected insert C is the distal end part of the catheter that is present inside the blood vessel B in a state in which the puncture needle is removed, Step S16, the scene determination unit 12 determines that the scene where the ultrasound image U as the static image is acquired is the placement scene.

On the other hand, for example, in a case where the puncture needle is not yet removed from the catheter, or the like, the scene determination unit 12 determines that the distal end part of the catheter in a state in which the puncture needle is removed is not inside the blood vessel B. In this case, Step S15 is repeated.

With the procedure described above, based on the static image of the ultrasound image U of one frame, it is possible to appropriately determine the scene when the ultrasound image U is acquired.

Scene Determination Based on Video

In determining the scene based on a video, the scene determination unit 12 performs the determination along a flow shown in FIG. 7. First, the scene determination unit 12 executes Step S21, analyzes the continuously acquired ultrasound images U of a plurality of frames, and determines whether or not appearance and disappearance of the blood vessel B are repeated in the ultrasound images U of a plurality of frames. Specifically, in Step S21, as shown in FIG. 9, the scene determination unit 12 determines whether or not the ultrasound image U in which the blood vessel B is detected and the ultrasound image U in which the blood vessel B is not detected are alternately switched within a short time in the ultrasound images U of a plurality of frames. In detecting the blood vessel B, the insert C, a lesion portion E, and the like in the ultrasound image U, a known algorithm, such as template matching, a machine learning method, and a general image recognition method using deep learning, can be used.

The repetition of appearance and disappearance of the blood vessel B in the ultrasound images U of a plurality of frames means that the operator is in a situation of moving the ultrasound probe 21 to search for the blood vessel B (that is, the target blood vessel Bx) into which the insert C is to be inserted. Accordingly, in a case where determination is made that appearance and disappearance of the blood vessel B are repeated in the ultrasound images U of a plurality of frames, in Step S22, the scene determination unit 12 determines that the scene where the ultrasound images U of a plurality of frames are acquired is the search scene.

On the other hand, in a case where the blood vessel B is detected in all the ultrasound images U of a plurality of frames, the scene determination unit 12 executes Step S23, analyzes the continuously acquired ultrasound images U of a plurality of frames, and determines whether or not the position of the blood vessel B is changed in the ultrasound images U of a plurality of frames.

The change of the position of the blood vessel B in the ultrasound images U of a plurality of frames means that the positions of the blood vessel B and other tissues rendered in the ultrasound images U are unstable, and the operator is in a situation of moving the ultrasound probe 21 to search for the blood vessel B (target blood vessel Bx) into which the insert C is to be inserted. Accordingly, in a case where determination is made that the position of the blood vessel B is changed in the ultrasound images U of a plurality of frames, the scene determination unit 12 executes Step S22 and determines that the scene where the ultrasound images U of a plurality of frames are acquired is the search scene.

In a case where the position of the blood vessel B is stable (not changed) in the ultrasound images U of a plurality of frames, the scene determination unit 12 executes Step S24, analyzes the continuously acquired ultrasound images U of a plurality of frames, and determines whether or not a position of the distal end of the insert C is changed to approach the target blood vessel Bx in the ultrasound images U of a plurality of frames. Here, the target blood vessel Bx is the blood vessel B that is positioned on a path of the insert C and into which the insert C is to be inserted later, among the blood vessels B in the ultrasound image U. Upon specifying the target blood vessel Bx, the scene determination unit 12 analyzes the ultrasound images U of a plurality of frames to detect a trajectory of the distal end of the insert C and estimates the path of the insert C based on the detected trajectory.

In a case where determination is made that the position of the distal end of the insert C gradually approaches the target blood vessel Bx in the ultrasound images U of a plurality of frames as shown in FIG. 10, in Step S25, the scene determination unit 12 determines that the scene where the ultrasound images U of a plurality of frames are acquired is the insertion scene.

On the other hand, there is a case where the operator changes the orientation of the ultrasound probe 21 to confirm that the insert C is appropriately inserted along an inner wall of the blood vessel B during an insertion operation of the insert C (that is, alternately employs the minor axis method and the major axis method) and switches the observation direction of the blood vessel B and the insert C as shown in FIG. 11. In this case, it may not be possible to clearly determine whether or not the distal end position of the insert C approaches the target blood vessel Bx in the ultrasound images U of a plurality of frames. In this case, the scene determination unit 12 executes Step S26, analyzes the continuously acquired ultrasound images U of a plurality of frames, and determines whether or not the observation direction of the blood vessel B and the insert C is switched a predetermined number of times or more in the ultrasound images U of a plurality of frames.

In a case where determination is made that the observation direction of the blood vessel B and the insert C is switched a predetermined number of times or more in the ultrasound images U of a plurality of frames, the scene determination unit 12 executes Step S25 and determines that the scene where the ultrasound images U of a plurality of frames are acquired is the insertion scene.

At a stage where the operation of the insert C is stable, and for example, the insert C is inserted into the target blood vessel Bx, the number of times the observation direction of the blood vessel B and the insert C is switched is less than the predetermined number of times. In this case, the scene determination unit 12 executes next Step S27, analyzes the continuously acquired ultrasound images U of a plurality of frames, and determines whether or not appearance and disappearance of the distal end part of the catheter as the insert C are repeated in the ultrasound images U of a plurality of frames.

The repetition of the appearance and disappearance of the distal end part of the catheter in the ultrasound images U of a plurality of frames means that the distal end part of the catheter is already placed inside the target blood vessel Bx as shown in FIG. 12 and the operator is in a situation of moving sweeping) the ultrasound probe 21 to confirm the distal end part of the catheter. Accordingly, in a case where determination is made that appearance and disappearance of the distal end part of the catheter are repeated in the ultrasound images U of a plurality of frames, in Step S28, the scene determination unit 12 determines that the scene where the ultrasound images U of a plurality of frames are acquired is the placement scene.

On the other hand, in a case where the distal end part of the catheter present inside the blood vessel B is detected in all the ultrasound images U of a plurality of frames, the scene determination unit 12 executes Step S29, analyzes the continuously acquired ultrasound images U of a plurality of frames, and determines whether or not appearance and disappearance of the lesion portion E (for example, phlebitis) inside the subject are repeated in the ultrasound images U of a plurality of frames.

The repetition of appearance and disappearance of the lesion portion E in the ultrasound images U of a plurality of frames as shown in FIG. 13 means that the operator moves the ultrasound probe 21 to confirm the lesion portion E, such as phlebitis, caused by the placement of the distal end part of the catheter inside the blood vessel B. Accordingly, in a case where determination is made that appearance and disappearance of the lesion portion E are repeated in the ultrasound images U of a plurality of frames, the scene determination unit 12 executes Step S28 and determines that the scene where the ultrasound images U of a plurality of frames are acquired is the placement scene.

On the other hand, in a case where determination is made that appearance and disappearance of the lesion portion E are not repeated in the ultrasound images U of a plurality of frames, the process returns to Step S27, and the steps after S27 are repeated.

With the procedure described above, based on the continuously acquired ultrasound images U of a plurality of frames (that is, a video), it is possible to appropriately determine the scene where the ultrasound images U are acquired.

Returning to the description on the flowchart of FIG. 5, Step S3 is executed after the execution of the scene determination. In Step S3, the blood vessel information acquisition unit 16 analyzes the ultrasound image U generated (acquired) in Step S1, detects the target blood vessel Bx in the ultrasound image U, and acquires the blood vessel information J of the detected target blood vessel Bx.

In subsequent Step S4, the region-of-interest detection unit 18 analyzes the ultrasound image U like the blood vessel information acquisition unit 16 and detects the region of interest in the ultrasound image U. In this case, the region-of-interest detection unit 18 detects the region of interest corresponding to the scene determined by the scene determination unit 12 in Step S2. Specifically, in a case where the scene determined by the scene determination unit 12 is the search scene, the region-of-interest detection unit 18 detects the lesion portion E inside the subject or the blood vessel B other than the target blood vessel Bx as the region of interest.

In more detail, in a case where the scene is the search scene, and as shown in FIG. 14, the blood vessels B of various sizes are densely rendered in the ultrasound image U, the region-of-interest detection unit 18 detects the lesion portion E (for example, a blood clot or edema) in the ultrasound image U as the region of interest. In this case, the blood vessel B the depth of which is comparatively small and the diameter of which has a size enough to insert the insert C, among a plurality of blood vessels B in the ultrasound image U corresponds to the target blood vessel Bx.

In a case where the scene is the search scene, and as shown FIG. 15, the number of blood vessels B is small but various peripheral tissues F (for example, nerves, organs, and bones) are present around the blood vessel B in the ultrasound image U, the region-of-interest detection unit 18 detects the blood vessel B other than the target blood vessel Bx in the ultrasound image U as the region of interest. Even in this case, similarly to the ultrasound image U shown in FIG. 14, the blood vessel B the depth of which is comparatively small and the diameter of which has a size enough to insert the insert C corresponds to the target blood vessel Bx.

In a case where the scene determined by the scene determination unit 12 is the insertion scene, the region-of-interest detection unit 18 detects at least one of the distal end of the insert C or a tissue positioned near the distal end of the insert C inside the subject as the region of interest.

In more detail, in a case where the scene is the insertion scene, and as shown in FIG. 16, the distal end (needle tip) of the catheter with a puncture needle as the insert C in the ultrasound image U is positioned near the target blood vessel Bx, the region-of-interest detection unit 18 detects the needle tip of the puncture needle in the ultrasound image U and the peripheral tissue F near the needle tip as the region of interest. The peripheral tissue F near the needle tip includes the blood vessel other than the target blood vessel Bx positioned near the needle tip.

In a case where the scene is the insertion scene, and as shown in FIG. 17, the distal end of the catheter with a puncture needle (that is, the distal end of the puncture needle) as the insert C in the ultrasound image U is inserted into the target blood vessel Bx, the region-of-interest detection unit 18 detects the needle tip of the puncture needle in the ultrasound image U as the region of interest.

In a case where the scene determined by the scene determination unit 12 is the placement scene, the region-of-interest detection unit 18 detects the distal end part of the insert C or the lesion portion E inside the subject as the region of interest.

In more detail, in a case where the scene is the placement scene, and as shown in FIG. 18, the distal end part of the catheter in a state in which the puncture needle is removed, as the insert C in the ultrasound image U is present inside the target blood vessel Bx, the region-of-interest detection unit 18 detects the distal end part of the catheter in the ultrasound image U as the region of interest.

In a case where the scene is the placement scene, and as shown in FIG. 19, phlebitis as the lesion portion E is rendered near the target blood vessel Bx in which the distal end part of the catheter is placed, in the ultrasound image U, the region-of-interest detection unit 18 detects phlebitis in the ultrasound image U as the region of interest.

After the detection of the region of interest is performed in the above-described manner, in Step S5, the apparatus controller 13 decides the display region of the blood vessel information J within a display range H of the ultrasound image U based on the position of the region of interest. Step S5 progresses following a flow shown in FIG. 20. Specifically, first, the apparatus controller 13 executes Step S31 and determines whether or not the region of interest in the ultrasound image U is detected by the region-of-interest detection unit 18 in previous Step S4.

In a case where determination is made that the region of interest is not detected, the apparatus controller 13 executes Step S32, and in Step S32, selects a region near the target blood vessel Bx while avoiding the target blood vessel Bx in the display range H of the ultrasound image U and decides the selected region as the display region.

On the other hand, in a case where determination is made that the region of interest is detected, in Step S33, the apparatus controller 13 determines whether or not there are a plurality of regions of interest detected by the region-of-interest detection unit 18. In a case where determination is made that only one region of interest is detected, the apparatus controller 13 executes Step S34 and extracts a display candidate region avoiding the target blood vessel Bx and the region of interest within the display range H of the ultrasound image U.

Here, a specific example of the display candidate region will be described. In the ultrasound image U that is acquired on the search scene and in which the blood vessels B of various sizes are densely rendered as shown in FIG. 14, a region where the blood vessel B the depth of which is comparatively great is rendered or a region where the blood vessel B the diameter of which is sufficiently smaller than the target blood vessel Bx is rendered is extracted as the display candidate region (in FIG. 14, a region indicated by a dotted line).

In the ultrasound image U that is acquired on the search scene and in which the number of blood vessels B is small but the peripheral tissues F is present around the blood vessels B as shown in FIG. 15, a region where the peripheral tissue F is rendered is extracted as the display candidate region (in FIG. 15, a region indicated by a dotted line).

In the ultrasound image U that is acquired on the insertion scene and in which the distal end of the catheter with a puncture needle as the insert C is positioned near the target blood vessel Bx as shown in FIG. 16, a region where the blood vessel B far from the distal end of the puncture needle is rendered is extracted as the display candidate region (in FIG. 16, a region indicated by a dotted line). In a case where the above-described ultrasound image U is an ultrasound image acquired in the major axis method, a region where the shaft part (not shown) of the puncture needle is rendered is extracted as the display candidate region.

In the ultrasound image U that is acquired on the insertion scene and in which the distal end of the catheter with a puncture needle as the insert C is inserted into the target blood vessel Bx as shown in FIG. 17, a region where the blood vessel B in which the puncture needle does not enter or the peripheral tissue F is rendered is extracted as the display candidate region (in FIG. 17, a region indicated by a dotted line). In a case where the above-described ultrasound image U is an ultrasound image acquired in the major axis method, a region where the shaft part (not shown) of the puncture needle is rendered is extracted as the display candidate region.

In the ultrasound image U that is acquired on the placement scene and in which the distal end part of the catheter in a state in which the puncture needle is removed is present inside the target blood vessel Bx as shown in FIGS. 18 and 19, a region where the blood vessel B in which the catheter does not enter is rendered is extracted as the display candidate region (in FIGS. 18 and 19, a region indicated by a dotted line). In a case where the above-described ultrasound image U is an ultrasound image acquired in the major axis method, a region where the shaft part of the catheter is rendered is extracted as the display candidate region.

Then, the apparatus controller 13 executes Step S35 after the execution of Step S34 and decides the display region of the blood vessel information J from the display candidate region extracted in Step S34. In this case, in a case where one display candidate region is extracted, the display candidate region may be decided as the display region of the blood vessel information J, and in a case where a plurality of display candidate regions are extracted, the display candidate region closest to the target blood vessel Bx among the display candidate regions may be decided as the display region of the blood vessel information J. Note that a method of deciding the display region from the display candidate region is not particularly limited, and for example, the display region may be decided from the display candidate region by any method.

On the other hand, in a case where determination is made in Step S33 that there are a plurality of regions of interest, the apparatus controller 13 executes Step S36 and determines whether or not the display candidate region is (can be extracted) within the display range H of the ultrasound image U. In a case where determination is made that there is the display candidate region, the apparatus controller 13 executes Step S37 and decides the display region of the blood vessel information J from the display candidate region.

In contrast, in a case where determination is made that the display candidate region is not (cannot be extracted) within the display range H of the ultrasound image U, the apparatus controller 13 executes Step S38. In Step S38, the apparatus controller 13 refers to the rule for setting the priority stored in the storage unit 15 and sets the priority to each of a plurality of detected regions of interest in compliance with the rule. In this case, the apparatus controller 13 sets the priority to each region of interest based on the position of each of a plurality of regions of interest in the display range H of the ultrasound image U.

After the setting of the priority, the apparatus controller 13 executes Step S39, and in Step S39, decides the display region such that at least a part of the blood vessel information J overlaps the region of interest with lower priority (for example, the region of interest with the lowest priority). With this, even though the display candidate region cannot be extracted within the display range of the ultrasound image U since a plurality of regions of interest are detected, it is possible to appropriately decide the display region of the blood vessel information J while considering the priority of each region of interest.

Returning to the description on the flowchart of FIG. 5, Step S6 is executed after the display region of the blood vessel information J is decided, and in Step S6, the ultrasound image U generated (acquired) in Step S1 is displayed on the display device 8 under the control of the display controller 7. In Step S6, the apparatus controller 13 performs control such that the blood vessel information display unit 17 displays the blood vessel information J in the display region decided in Step S5. With this, the blood vessel information J is displayed in the display region decided based on the position of the region of interest within the display range H of the ultrasound image U.

The process progresses to Step S7 after the execution of Step S6, and in Step S7, determination is made whether or not to end the operation of the ultrasound diagnostic apparatus 1. For example, in a case where the operator inputs an instruction for the guidance on ending the operation of the ultrasound diagnostic apparatus 1 through the input device 14 or the like, determination is made to end the operation of the ultrasound diagnostic apparatus 1, and in a case where the instruction to end the operation of the ultrasound diagnostic apparatus 1 is not input, determination is made not to end the operation of the ultrasound diagnostic apparatus 1. In a case where determination is made not to end the operation of the ultrasound diagnostic apparatus 1, the process returns to Step S1, the ultrasound image U is newly generated, and then, the steps after Step S2 are repeated.

On the other hand, in a case where determination is made to end the operation of the ultrasound diagnostic apparatus 1, the operation of the ultrasound diagnostic apparatus 1 ends.

As described above, with the ultrasound diagnostic apparatus 1 according to the first embodiment, the target blood vessel Bx (for example, the blood vessel B into which the insert C is to be inserted) in the ultrasound image U is detected, and the blood vessel information J indicating the diameter and the depth of the detected target blood vessel Bx is displayed in the display region decided based on the position of the region of interest within the display range H of the ultrasound image U.

In more detail, the blood vessel information J is displayed to overlap the target blood vessel Bx and the region of interest. With this, it is possible to confirm the target blood vessel Bx in the ultrasound image U and the blood vessel information J in real time, and simultaneously, it is possible to confirm the region of interest (for example, another blood vessel B present near the target blood vessel Bx) together.

That is, with the use of the ultrasound diagnostic apparatus 1 according to the first embodiment, it is possible to display the blood vessel information J at an appropriate position within the display range H of the ultrasound image U. In this case, for example, usability (convenience) for the operator is improved compared to a case where the blood vessel information J is constantly displayed in the same region within the display range H.

The region of interest can be changed depending on the scene in a case where the ultrasound image U is acquired. Thus, in the first embodiment, in light of this, the display region of the blood vessel information J is decided for each scene. With this, the display region of the blood vessel information J is appropriately decided corresponding to the scene at this point of time.

In the first embodiment, in a case where a plurality of regions of interest are detected in the ultrasound image U, and the blood vessel information J cannot be displayed while avoiding all of a plurality of regions of interest, the priority is set to each region of interest, and the display region is decided such that at least a part of the blood vessel information J overlaps the region of interest with lower priority. With this, it is possible to display the blood vessel information J in the most appropriate display region.

In the above-described case, the transmission circuit