15 is phased and added and the focus of the ultrasound echo is narrowed is acquired.

As illustrated in FIG. 3, the image generation unit 21 of the apparatus main body 20 has a configuration in which a signal processing unit 41, a digital scan converter (DSC) 42, and an image processing unit 43 are sequentially connected in series.

The signal processing unit 41 generates an ultrasound image signal (B-mode image signal), which is tomographic image information regarding tissues inside the subject, by performing, on the sound ray signal sent from the transmission and reception circuit 12 of the ultrasound probe 10, correction of the attenuation due to the distance according to the depth of the reflection position of the ultrasonic wave and then performing envelope detection processing.

The DSC 42 converts (raster conversion) the ultrasound image signal generated by the signal processing unit 41 into an image signal according to a normal television signal scanning method.

The image processing unit 43 performs various kinds of necessary image processing such as gradation processing on the ultrasound image signal input from the DSC 42, and then outputs the signal representing the ultrasound image to the display controller 22 and the image memory 24. The signal representing the ultrasound image generated by the image generation unit 21 in this manner is simply referred to as an ultrasound image.

The image memory 24 is a memory that stores the ultrasound image generated by the image generation unit 21 under the control of the main body controller 28. For example, the image memory 24 can hold the ultrasound images of a plurality of frames, which are generated by the image generation unit 21, corresponding to the diagnosis for the breast of the subject.

Here, as the image memory 24, recording media such as a flash memory, a hard disc drive (HDD), a solid state drive (SSD), a flexible disc (FD), a magneto-optical disc (MO disc), a magnetic tape (MT), a random access memory (RAM), a compact disc (CD), a digital versatile disc (DVD), a secure digital card (SD card), and a universal serial bus memory (USB memory), or the like can be used.

The determination procedure memory 32 is a memory that stores a predetermined determination procedure for determining which one of a plurality of standardized medical examination categories does the patient fall into on the basis of the benignancy or malignancy of the lesion part. For example, as illustrated in FIG. 4, the determination procedure for determining the medical examination category of a mass formed in the mammary glands is stored in the determination procedure memory 32. The determination procedure is shown in the guideline created by the Japanese Association of Breast and Thyroid Sonology (JABTS), and has a plurality of branches in which the route is divided according to each determination item.

As the determination procedure memory 32, for example, a flash memory, an HDD, an SSD, an FD, an MO disc, an MT, a RAM, a CD, a DVD, an SD card, and a USB memory can be used.

JABTS defines medical examination categories 1 to 5 as follows.

- Medical examination category 1: no abnormal findings
- Medical examination category 2: there are findings but detailed examination is not required
- Medical examination category 3: benign but malignancy cannot be ruled out
- Medical examination category 4: suspicious of malignancy
- Medical examination category 5: malignant

For example, in a first branch B1, three determination items of a cystic pattern (non-echo), a mixed pattern (having a solid part and a liquid part), and a solid pattern are set regarding ultrasound echoes from the mass inside in the ultrasound image.

Then, in a case where the cystic pattern is selected on the basis of the characteristics of the ultrasound echoes, it is determined that the mass belongs to the medical examination category 2, and in a case where the mixed pattern is selected, it is determined that the mass belongs to the medical examination categories 3 and 4, but it is determined that the tumor belongs to the medical examination category 2 in a case where the maximum diameter of the tumor is 5 mm or less.

In the branch B1, in a case where the solid pattern is selected, in a case of corresponding to “the maximum diameter is 2 cm or less, the aspect ratio is sufficiently small, and the circumference is clear and smooth” or the like according to the characteristics of the ultrasound echo, it is determined that the tumor belongs to the medical examination category 2, and in a case of not corresponding thereto, the determination proceeds to a next branch B2.

In the branch B2, a border portion hyperechoic image and the rupture of the mammary gland border line are set as the determination items, in a case of corresponding to at least one of the border portion hyperechoic image or the rupture of the mammary gland border line, it is determined that the mass belongs to medical examination categories 4 and 5. In a case of corresponding to neither the border portion hyperechoic image nor the rupture of the mammary gland border line, the medical examination category is determined according to whether or not there are a plurality of fine and punctate hyperechoic portions, and according to the maximum diameter of the tumor and the aspect ratio of the mass. Furthermore, the medical examination categories may be adjusted depending on the shape of the mass.

Here, for example, the diameter of the tumor is expressed as L1×L2×L3 [mm] by measuring a maximum length L1 of a tumor N and a maximum width L3 on a line orthogonal to the maximum length L1 in a cross section including the maximum dimension of the tumor N as illustrated in FIG. 5, and further measuring a maximum diameter L2 in a cross section orthogonal to the cross section including the maximum dimension of the tumor N as illustrated in FIG. 6. These maximum length L1, maximum diameter L2, and maximum width L3 are measured including a border hyperechoic portion H.

The aspect ratio of the mass can be expressed as a value obtained by measuring a horizontal diameter W of a mass M in a direction parallel to the body surface (skin) of the subject and a vertical diameter D in a direction orthogonal to the horizontal diameter W without including the border hyperechoic portion, for example, as illustrated in FIG. 7, and dividing the vertical diameter D by the horizontal diameter W.

Further, for example, the shape of the mass is classified into a circular shape or oval shape (without narrow part, without angulated part) as illustrated in FIG. 8, a polygonal shape (without narrow part, with angulated part) as illustrated in FIG. 9, a lobulated shape (with narrow part, without angulated part) as illustrated in FIG. 10, and an irregular shape (with narrow part, with angulated part) as illustrated in FIG. 11, according to whether or not the shape including the border hyperechoic portion has a “narrow part” and an “angulated part”.

The identification unit 25 identifies the characteristics of the lesion part for the determination items for the route selection in the plurality of branches of the determination procedure as illustrated in FIG. 4 by reading out the determination procedure from the determination procedure memory 32 and analyzing the plurality of ultrasound images that are generated by the image generation unit 21 and are stored in the image memory 24. The “cystic pattern”, the “mixed pattern”, the “solid pattern”, the “border portion hyperechoic image”, the “rupture of the mammary gland border line”, the “fine and punctate hyperechoic portions”, the “maximum diameter of the tumor”, the “aspect ratio of the mass”, the “shape of the mass”, the “narrow part”, the “angulated part”, and the like which are exemplified in the determination procedure described above correspond to the characteristics of the lesion part to be identified by the identification unit 25.

The identification of the characteristics of the lesion part in the identification unit 25 can be executed by using at least one of template matching, an image analysis technique using feature amounts such as Adaptive Boosting (Adaboost), support vector machines (SVM), or scale-invariant feature transform (SIFT), or a determination model trained using a machine learning technique such as deep learning. The determination model is a trained model that is trained using learning ultrasound images of the breast including a lesion region, for example.

By the identification unit 25, the border of the lesion part is determined, and the size such as the maximum diameter and the aspect ratio of the lesion part is measured on the basis of the border.

The plurality of ultrasound images to be analyzed by the identification unit 25 may form a video in which the same lesion part of the subject is imaged, and each of the plurality of ultrasound images is an image in which the entire or a part of the lesion part is imaged.

The identification unit 25 may analyze the plurality of ultrasound images forming the video in which the lesion part is imaged as it is, or can analyze an image thinned out of the plurality of ultrasound images forming the video, an image interpolated with respect to the plurality of ultrasound images forming the video, or an image synthesized from the plurality of ultrasound images forming the video.

The identification result of the characteristics of the lesion part by the identification unit 25 is sent from the identification unit 25 to the extraction unit 26 and the determination unit 27.

The extraction unit 26 receives the identification result of the characteristics of the lesion part from the identification unit 25, and extracts, as the basis image, at least one ultrasound image contributing to the identification of the characteristics in the plurality of branches of the determination procedure by the identification unit 25, from the plurality of ultrasound images that are generated by the image generation unit 21 and stored in the image memory 24. That is, in the plurality of branches of the determination procedure illustrated in FIG. 4, in a case of determining whether or not the characteristics of the lesion part correspond to the determination item, the ultrasound image that best represents the determination, that is, the ultrasound image that is a decisive factor for the determination is extracted.

In this case, for example, the likelihood of determining each characteristic corresponding to the branch of the determination procedure is scored and recorded for the plurality of ultrasound images, the characteristic with the highest score or the characteristic with the highest mean, median, or mode of the scores of the plurality of ultrasound images is selected as a branch route, and the ultrasound image with the highest score of the characteristic corresponding to the selected route is extracted as the basis image.

The number of extracted ultrasound images is not limited to one, and the extraction unit 26 can extract a plurality of ultrasound images as the basis image.

The determination unit 27 receives the identification result of the characteristics of the lesion part from the identification unit 25, performs the route selection of the determination procedure by applying the characteristics of the lesion part identified by the identification unit 25 to the determination item corresponding to each branch in the plurality of branches of the determination procedure stored in the determination procedure memory 32, and thereby determines the medical examination category of the lesion part.

The display controller 22 performs predetermined processing on the ultrasound image sent from the image generation unit 21, and displays the ultrasound image on the monitor 23, under the control of the main body controller 28.

Further, the display controller 22 displays the determination procedure stored in the determination procedure memory 32 on the monitor 23, and displays the route in the determination procedure in a case of determining the medical examination category by the determination unit 27 and the basis image extracted by the extraction unit 26 on the monitor 23 by superimposing the route and the basis image on the determination procedure.

The monitor 23 is for displaying the ultrasound image, the determination procedure, and the like under the control of the display controller 22, and includes a display device such as a liquid crystal display (LCD), or an organic electroluminescence (EL) display.

The main body controller 28 controls each unit of the apparatus main body 20 and the transmission and reception circuit 12 of the ultrasound probe 10 on the basis of a control program and the like stored in advance.

Although not illustrated, a main body-side storage unit is connected to the main body controller 28. The main body-side storage unit stores a control program and the. As the main body-side storage unit, for example, a flash memory, a RAM, an SD card, an SSD, and the like can be used.

The input device 29 is for a user to perform an input operation, and is configured by, for example, a device such as a keyboard, a mouse, a trackball, a touchpad, and a touch sensor superimposed on the monitor 23.

The processor 30 having the image generation unit 21, the display controller 22, the identification unit 25, the extraction unit 26, the determination unit 27, and the main body controller 28 is configured by a central processing unit (CPU) and a control program for causing the CPU to execute various kinds of processing, but the processor 30 may be configured by using a field programmable gate array (FPGA), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a graphics processing unit (GPU), or other integrated circuits (IC) or may be configured by a combination thereof.

Further, the image generation unit 21, the display controller 22, the identification unit 25, the extraction unit 26, the determination unit 27, and the main body controller 28 of the processor 30 can also be configured by being integrated partially or entirely into one CPU or the like.

Next, the operation of the ultrasound diagnostic apparatus according to the first embodiment will be described with reference to the flowchart illustrated in FIG. 12.

First, in Step S1, by the identification unit 25, the determination procedure is read out from the determination procedure memory 32, the plurality of ultrasound images in which the same lesion part of the subject is imaged and which are stored in the image memory 24 are analyzed, and the characteristics of the lesion part for the determination item set in each branch of the determination procedure are identified.

For example, it is determined whether the lesion part corresponds to one of three determination items set in the first branch B1 of the determination procedure illustrated in FIG. 4, that is, the cystic pattern, the mixed pattern, or the solid pattern.

Next, in Step S2, by the extraction unit 26, the determination procedure is read out from the determination procedure memory 32, and at least one ultrasound image contributing to the identification of the characteristics of the lesion part in the plurality of branches of the determination procedure performed in Step S1 is extracted as the basis image. in a case of determining whether or not the characteristics of the lesion part correspond to the determination item, the extraction unit 26 extracts the ultrasound image that best represents the determination as the basis image from the plurality of ultrasound images which are analyzed by the identification unit 25 in Step S1.

For example, the likelihood of determining each characteristic corresponding to the branch of the determination procedure is scored for the plurality of ultrasound images, the characteristic with the highest score or the characteristic with the highest mean, median, or mode of the scores of the plurality of ultrasound images is selected as a branch route, and the ultrasound image with the highest score of the characteristic corresponding to the selected route is extracted as the basis image.

In a case where there are a plurality of ultrasound images most contributing to the identification by the identification unit 25, the basis image is not limited to one ultrasound image, and a plurality of ultrasound images are extracted as the basis image.

In a case where the basis image is extracted, the processing proceeds to Step S3, and it is determined whether or not the identification of the characteristics of the lesion part in Step S1 and the extraction of the basis image in Step S2 are ended for all of the plurality of branches of the determination procedure.

In a case where there remains a branch for which the identification of the characteristics of the lesion part and the extraction of the basis image are not performed, the processing returns to Step S1, and for the next branch in the determination procedure, the identification of the characteristics of the lesion part is performed by the identification unit 25, and the extraction of the basis image is performed by the extraction unit 26 in Step S2.

In this manner, Steps S1 to S3 are repeated until the identification of the characteristics of the lesion part and the extraction of the basis image are ended for all the branches in Step S3.

Then, in a case where it is determined in Step S3 that the identification of the characteristics of the lesion part and the extraction of the basis image are ended for all the branches, the processing proceeds to Step S4, and by the determination unit 27, the determination procedure is read out from the determination procedure memory 32, and the medical examination category of the lesion part is determined according to the determination procedure. In this case, the determination unit 27 can perform the route selection of the determination procedure by applying the characteristics of the lesion part identified by the identification unit 25 to the determination item corresponding to each branch in the plurality of branches of the determination procedure, and thereby determine the medical examination category of the lesion part.

In a case where the medical examination category of the lesion part is determined in this manner, the processing proceeds to Step S5, and as illustrated in FIG. 13, by the display controller 22, the determination procedure used in the determination of the medical examination category is displayed on the monitor 23, and further, the route in a case where the medical examination category is determined by the determination unit 27 and the basis image extracted by the extraction unit 26 are superimposed and displayed on the determination procedure.

The border of the lesion part determined by the identification unit 25 can be superimposed and displayed on the basis image.

In FIG. 13, the route in a case where the medical examination category is determined by the determination unit 27 is displayed by being emphasized by a thick line, so that it can be seen that the lesion part of the subject is determined to belong to the medical examination categories 4 and 5.

The route of the determination can be displayed by being emphasized by a line with a specific color without being limited to the thick line.

In the example illustrated in FIG. 13, in the first branch B1, it is determined that the lesion part corresponds to the solid pattern among three determination items of the cystic pattern, the mixed pattern, and the solid pattern, and the ultrasound image that best represents the determination is extracted by the extraction unit 26, and is displayed as a basis image U1 near the branch B1 in the determination procedure.

Similarly, in the branch B2, it is determined that the lesion part corresponds to at least one of the border portion hyperechoic image or the rupture of the mammary gland border line, and the ultrasound image that best represents the determination is extracted by the extraction unit 26, and is displayed as a basis image U2 near the branch B2 in the determination procedure.

In this manner, the characteristics of the lesion part for the determination item in the plurality of branches of the determination procedure are identified by the identification unit 25 on the basis of the plurality of ultrasound images in which the same lesion part of the subject is imaged, at least one ultrasound image contributing to the identification of the characteristics by the identification unit 25 is extracted as the basis image from the plurality of ultrasound images by the extraction unit 26, and the route in the determination procedure in a case of determining the medical examination category and the basis image extracted by the extraction unit 26 are displayed on the monitor 23 by the display controller 22. Therefore, it is possible to determine the medical examination category of the lesion part and accurately understand the basis of the determination from the plurality of ultrasound images regardless of the skill level of the user.

The extraction unit 26 can form an enclosing line to surround a portion contributing to the identification by the identification unit 25, in the basis image extracted from the plurality of ultrasound images on the basis of the identification result of the characteristics of the lesion part received from the identification unit 25, and an enclosing line E can be superimposed and displayed on the basis image U1 by the display controller 22 as illustrated in FIG. 14. Instead of the enclosing line E, a border line of the portion contributing to the identification may be superimposed and displayed on the basis image U1.

Further, the extraction unit 26 may form a heat map in which a contribution rate of the portion contributing to the identification by the identification unit 25 is represented by color shading or color difference, on the basis of the basis image U1. As illustrated in FIG. 15, a heat map U1H can be displayed side by side with the basis image U1 on the monitor 23 by the display controller 22.

Instead of displaying the heat map U1H side by side with the basis image U1, the basis image U1 and the heat map U1H may be alternately displayed on the basis of the user's input operation via the input device 29, as illustrated in FIG. 16.

In this manner, by superimposing and displaying the enclosing line E on the basis image U1 or displaying the heat map U1H, the user can intuitively understand the portion contributing to the identification by the identification unit 25.

In each branch of the determination procedure, in a case where a plurality of basis images contributing to the identification by the identification unit 25 are extracted by the extraction unit 26, for example, as illustrated in FIG. 17, a plurality of corresponding basis images U1a and U1b can be displayed side by side near the branch.

Alternatively, a plurality of basis images extracted by the extraction unit 26 may be sequentially displayed on the monitor 23 on the basis of the user's input operation via the input device 29. In this case, for example, as illustrated in FIG. 18, a page number m/n indicating which of n basis images extracted by the extraction unit 26 is a basis image U1m that is currently being displayed, that is, indicating what number of the basis image U1m, can be displayed in an overlaid manner on the basis image U1m.

In a case where the plurality of basis images are sequentially displayed in this manner, even in a case where the monitor 23 does not have a large display space, the plurality of basis images can be displayed, and the page number is displayed in an overlaid manner so that the user can easily understand the plurality of basis images.

Further, the plurality of basis images may be sequentially and automatically displayed at certain time intervals without depending on the user's input operation.

As in the determination procedure illustrated in FIG. 4, the size of the lesion part is set as the determination item in a plurality of branches. Therefore, as illustrated in FIG. 19, the size of the lesion part measured by the identification unit 25, for example, the maximum diameter of the tumor and the aspect ratio of the mass can be displayed on the monitor 23 by being superimposed on the determination procedure.

It is configured such that the region of the tumor and the region of the mass are displayed on the image and the displayed regions can be modified by the user via the input device 29, and the identification unit 25 may correct the measurement value by measuring again the maximum diameter of the tumor and the aspect ratio of the mass on the basis of the modified regions. For example, in a case where the contour lines of the displayed regions are modified by the user, the measurement values of the maximum diameter of the tumor and the aspect ratio of the mass are corrected by the identification unit 25 on the basis of the modified region, and the medical examination category is determined again by the determination unit 27 on the basis of the corrected measurement values. The corrected maximum diameter of the tumor and the corrected aspect ratio of the mass are displayed on the monitor 23 by being superimposed on the determination procedure.

Further, the identification unit 25 can calculate a ratio of the region where the identified characteristics of the lesion part appear, to the total region of the lesion part for each set determination item of the determination procedure. For example, the identification unit 25 creates a three-dimensional image on the basis of the plurality of ultrasound images in which the same lesion part of the subject is imaged and which are analysis targets, and calculates the ratio of the region recognized as “rough” in the border portion of the lesion part to the entire region of the lesion part, for example. The calculated ratio can be displayed on the monitor 23 by being superimposed on the determination procedure as illustrated in FIG. 20.

Instead of creating the three-dimensional image, for example, the lengths of the rough portion and the other portions may be calculated for each frame, such as 2 cm of the rough portion and 8 cm of the other portions in the first frame, and 1.5 cm of the rough portion and 7 cm of the other portions in the second frame, and the ratio of the sum of the lengths of the rough portions in all the frames and the sum of the lengths of the other portions in all the frames may be calculated and displayed on the monitor 23.

As described above, the shape of the mass is classified according to whether the shape including the border hyperechoic portion has a “narrow part” and an “angulated part”, and for example, the irregular shape as illustrated in FIG. 11 has both the “narrow part” and the “angulated part”, but it is desirable to check the number of “narrow parts” and “angulated parts” in order to recognize that the mass has an irregular shape. Thus, the number of each of “narrow parts” and “angulated parts” of the lesion part can be calculated from the plurality of ultrasound images by the identification unit 25, and can be displayed on the monitor 23 by being superimposed on the determination procedure as illustrated in FIG. 21.

In this manner, by displaying the size of the lesion part, the ratio of the region where the identified characteristics of the lesion part appear to the entire region of the lesion part, and the number of “narrow parts” and “angulated parts” on the monitor 23 on the basis of the analysis by the identification unit 25, the user can understand in more detail the basis for the route selection in the plurality of branches of the determination procedure.

In the first embodiment described above, as illustrated in FIG. 12, the characteristics of the lesion part are identified for the plurality of ultrasound images by the identification unit