FIG. 5, for convenience, a region having large plaque burden is shown with the same hatching.

In the present embodiment, the description is given assuming that the object indicating the magnitude of plaque burden is the gradation bar 6, but the object is not limited to the gradation bar 6. For example, the diagnostic imaging device 2 may display a line graph indicating the magnitude of plaque burden on the upper side of the longitudinal tomographic image 5. As described above, the gradation bar 6 is an example of the object, and any object may be used as long as the object appropriately indicates the magnitude of plaque burden.

For example, the diagnostic imaging device 2 refers to a predetermined threshold (for example, 50%) set by default, and sets a display color for a region having many plaques. Specifically, the diagnostic imaging device 2 sets the display color for a region having plaque burden equal to or greater than the threshold to a warm color (orange, for example), and sets the display color for a region having plaque burden less than the threshold to a default color (white, for example). The diagnostic imaging device 2 then changes the display color (for example, increases the lightness) so that the larger the plaque burden, the more the region having plaque burden equal to or greater than the threshold is emphasized, and performs gradation display.

As a method of gradation display, for example, it is assumed that lightness is changed, but hue, saturation, and the like may be changed, and a specific display method is not particularly limited.

In a case where the gradation bar 6 is displayed as described above, it is preferable that the diagnostic imaging device 2 hides a region in which the reliability of the calculation result of the plaque burden in the calculation model 51 is relatively low (hereinafter, referred to as a “low reliability region”). In FIG. 5, the low reliability region is illustrated in black. For the calculation of the plaque burden in each frame using the calculation model 51, the diagnostic imaging device 2 acquires the reliability of the calculated value (for example, a probability value of 0 to 1) from the calculation model 51 together with the plaque burden in each frame. In a case where the reliability at each position (frame) of the blood vessel is equal to or less than a predetermined value, that is, in a case where the reliability is relatively low, the diagnostic imaging device 2 leaves a region on the corresponding gradation bar 6 blank as the low reliability region. As a result, the certainty of the calculation result of the plaque burden can be presented to the user.

As described above, the diagnostic imaging device 2 displays the gradation bar 6 together with the longitudinal tomographic image 5, and quantitatively presents the state of the blood vessel at each position. As illustrated in FIG. 5, the diagnostic imaging device 2 can display a maximum value, a minimum value, and an average value of the plaque burden, which helps enable the user (for example, a medical worker) to grasp the state of the blood vessel and determine a treatment area to be treated with a treatment device such as a stent.

FIG. 6 is an explanatory diagram illustrating another display example of the plaque burden. In FIG. 5, the display example has been described in which the warm color is displayed for the region having the plaque burden equal to or greater than the threshold, but gradation display may be performed also for the region having the plaque burden less than the threshold, that is, the region having few plaques.

For example, the diagnostic imaging device 2 sets the display color for the region having the plaque burden equal to or greater than the threshold to a warm color, and sets the display color for the region having the plaque burden less than the threshold to a cold color. In FIG. 6, the region having the plaque burden equal to or greater than the threshold and the region having the plaque burden less than the threshold are shown with different hatching. Similarly to the region having the plaque burden equal to or greater than the threshold, the diagnostic imaging device 2 changes the display color (for example, increases the lightness) as the plaque burden is smaller, and performs gradation display. As described above, the diagnostic imaging device 2 may present a region that has a large number of plaques and needs to be treated, and may also present a region that has a small number of plaques and has a relatively low need for treatment.

In the example of FIG. 6, the number of regions for gradation display is two, but it is possible that a threshold is further provided and gradation display is performed in three or more regions.

In addition, in a case where the gradation display is performed, the user may set a threshold as a display reference. Specifically, the diagnostic imaging device 2 receives, from the user, a setting input for changing the threshold from a default value to another value (for example, a threshold based on user preference), determines the region having the plaque burden equal to or greater than the threshold and the region having the plaque burden less than the threshold according to the set threshold, and changes the display color, which helps enable the user to visualize a region of interest by himself/herself, resulting in supporting the user more suitably.

FIG. 7 is an explanatory diagram related to determination processing of a treatment area of a blood vessel. In the present embodiment, a treatment area to be treated with a predetermined treatment device is further presented to the user based on the calculation result of the plaque burden.

Specifically, the diagnostic imaging device 2 determines positions of both ends of a stent indwelled in the blood vessel. In the case of vascular treatment such as PCI, it can be preferable to indwell the stent so as to cover the entire lesion, however in practice, it can be difficult to cover the entire lesion because plaques are continuously present. For example, the stent can be indwelled so that both ends of the stent are located in a normal site where no plaque is present; however, when it is difficult to do so, the stent is indwelled so that both ends of the stent are located at a site where the plaque burden is a predetermined value (for example, 50%) or less. If that is also difficult, the stent can be indwelled so that both ends of the stent are located at a site with the least plaque burden.

At present, the plaque burden of all frames is not calculated as described above, and a stenosed site is visually identified on the image, and the plaque burden of several frames is measured as necessary. In view of this, in the present embodiment, an area where both ends of the stent should be located is determined based on the plaque burden calculated by the calculation model 51.

In the present embodiment, the stent is exemplified as a treatment device, but the treatment device is not limited to the stent. For example, the diagnostic imaging device 2 may determine both end positions of an expansion area to be expanded with a balloon used for expanding a blood vessel. Alternatively, for example, the diagnostic imaging device 2 may determine both end positions of a cut area to be cut by a rotablator used for scraping a hard lesion (for example, calcified tissue). As described above, the diagnostic imaging device 2 only needs to be able to determine both end positions of a treatment area to be treated with a treatment device inserted into a blood vessel, and the treatment device is not limited to the stent.

First, the diagnostic imaging device 2 divides the longitudinal tomographic image 5 (longitudinal cross section) of the blood vessel into a first region in which the plaque burden is equal to or greater than a predetermined threshold and a second region in which the plaque burden is less than the threshold. The threshold can be, for example, the same value as the threshold used as the reference for the gradation display described above, but may be a different value. The diagnostic imaging device 2 divides, based on the plaque burden in each frame calculated by the calculation model 51, the longitudinal tomographic image 5 into first regions L1, L2, L3 (L: Lesion) in which the plaque burden is equal to or greater than the threshold and second regions H1, H2, H3 (H: Healthy) in which the plaque burden is less than the threshold.

In the present embodiment, the diagnostic imaging device 2 identifies the first regions LX (X=1, 2, 3 . . . ) including the position (frame) where the plaque burden takes the maximum value as a lesion region LX in which the stent is to be indwelled. The diagnostic imaging device 2 then determines two second regions HY and HZ (Y, Z=1, 2, 3 . . . ) located on both sides of the identified lesion region LX as regions where both ends of the stent should be located.

FIG. 7 illustrates the longitudinal tomographic image 5 in which one first region L1 and two second regions HP1 and HD1 (P: Proximal, D: Distal) are present. In a case where the plaque burden takes the maximum value in the first region L1, the diagnostic imaging device 2 determines the second regions HP1 and HD1 sandwiching the first region L1 between the second regions HP1 and HD1 as the regions where both ends of the stent should be located.

For example, in a case where there is a low reliability region in which the reliability of the calculation result of the plaque burden is relatively low, it is preferable that the diagnostic imaging device 2 groups a plurality of the first regions or the second regions extending over the low reliability region as the same region. For example, in FIG. 7, two low reliability regions are present in the first region L1. In this case, the diagnostic imaging device 2 determines whether or not the length of each low reliability region in the axial direction of the blood vessel is equal to or less than a predetermined length (for example, 1 mm). In a case where the length of the low reliability region is equal to or less than the predetermined length, the diagnostic imaging device 2 treats the plurality of first regions extending over the low reliability region as the same region and divides the regions. In the example of FIG. 7, since the length of both of the two low reliability regions is equal to or less than the predetermined length, the diagnostic imaging device 2 divides the three regions extending over the low reliability region into the same first region L1.

Similarly for the second region, in a case where there is a plurality of second regions extending over the low reliability region of which the length is equal to or less than the predetermined length, the diagnostic imaging device 2 divides the plurality of second regions as the same region. As described above, in a case where the length of the low reliability region in the axial direction is equal to or less than the predetermined length, the diagnostic imaging device 2 divides the plurality of first regions or the plurality of second regions extending over the low reliability region as the same region. As a result, the indwelling position of the stent can be suitably determined while reliability can be ensured.

In addition, the user may change the predetermined length. For example, the diagnostic imaging device 2 receives a setting input for changing a default value of the predetermined length to a value (for example, lengths in predetermined increments of 0.6 mm, 0.8 mm, 1.2 mm, . . . ). The diagnostic imaging device 2 divides each region by determining whether the length is equal to or less than a set predetermined length, which helps enable the user to determine the indwelling position of the stent more suitably.

Similarly to the gradation display, in the diagnostic imaging device 2, the user may change a threshold serving as a reference for the region division. In this case, the diagnostic imaging device 2 may be able to set different thresholds for the proximal side (Proximal) and the distal side (Distal) of the blood vessel. For example, in a case where 50% is set as the threshold on the proximal side and 60% is set as the threshold on the distal side, the diagnostic imaging device 2 performs division with a position where the plaque burden is equal to or greater than 50% as viewed from the left side of FIG. 7 used as the start point of the first region and a position where the plaque burden is less than 60% used as the end point of the first region, which helps enable the user to finely adjust the first region and the second region by himself/herself.

FIG. 8 is an explanatory diagram illustrating a display example of a lesion length. As described above, the diagnostic imaging device 2 divides the longitudinal tomographic image 5 into the first regions L1, L2, L3 . . . and the second regions H1, H2, H3 . . . , and identifies a lesion region L1. In a case where the lesion region L1 is identified, the diagnostic imaging device 2 measures the length of the lesion region L1 in the axial direction of the blood vessel, that is, the lesion length, and displays the length in association with the lesion region L1 in the longitudinal tomographic image 5. Specifically, as illustrated in FIG. 8, the diagnostic imaging device 2 displays a portion corresponding to the lesion region L1 on the gradation bar 6 with an arrow, and displays the lesion length. Thereby, the length of a lesion site can be visualized.

FIG. 9 is an explanatory diagram illustrating an example where a stent is superimposed. In the present embodiment, the diagnostic imaging device 2 further presents, to the user, a treatment area to be treated with the treatment device, that is, an area in which the stent is to be indwelled. Specifically, the diagnostic imaging device 2 superimposes and displays a stent region 7 (second object) representing the size of the stent (treatment device) on the longitudinal tomographic image 5.

The stent region 7 can be an object simulating the stent indwelled in the blood vessel, and can be, for example, a rectangular object representing an area in which the stent is indwelled. It is only required that at least the length of the stent region 7 in the axial direction is equal to the actual length of the stent, and the length of the stent region 7 in the radial direction (length in the longitudinal direction in FIG. 8) is not required to be equal to the actual length of the stent.

Further, in the present embodiment, since the treatment device is a stent, the shape of the object is a rectangular shape; however, for example, in a case where the treatment device is a balloon, the shape of the object may be a sandbag shape (i.e., a generally rectangular shape having an increased diameter in a central portion of the object). As described above, the diagnostic imaging device 2 only needs to be able to display an object having a shape corresponding to the treatment device.

For example, the diagnostic imaging device 2 displays the stent region 7 having a length equal to or longer than the lesion length measured above such that both ends of the stent region 7 are located in the second regions HP1 and HD1 determined as the regions where both ends should be located. For example, the diagnostic imaging device 2 stores, in the auxiliary storage unit 27, information on stent products in predetermined length increments (for example, 9 mm, 12 mm, 15 mm, . . . ) that can be used for a procedure. The diagnostic imaging device 2 selects a stent that exceeds the lesion length and has the minimum length from among the stent products. The diagnostic imaging device 2 then superimposes and displays the stent region 7 having the length of the selected stent such that both ends of the selected stent are located in the second regions HP1 and HD1. The selection of the stent, the determination of the superimposing position of the stent region 7, and the like may be performed by manual operation by the user.

For example, the diagnostic imaging device 2 superimposes and displays the stent region 7 such that the center position (midpoint) of the lesion region L1 in the axial direction of the blood vessel coincides with the center position of the stent region 7. In this case, as illustrated in FIG. 9, downward arrows are displayed at positions corresponding to both ends of the stent, and bidirectional arrows are displayed at portions exceeding the first region L1. The diagnostic imaging device 2 moves the stent region 7 in accordance with an operation input (for example, mouse operation) by the user, and allows the user to determine the optimum indwelling position of the stent.

In this case, as illustrated in FIG. 9, the diagnostic imaging device 2 preferably displays transverse tomographic images at positions corresponding to both ends of the stent region 7, which helps enable the user to search for the optimum indwelling position of the stent while moving the stent region 7.

Although the stent region 7 is superimposed and displayed on the longitudinal tomographic image 5 in the above description, the diagnostic imaging device 2 may display the stent region 7 at a location other than the longitudinal tomographic image 5 (for example, on the gradation bar 6, inside the gradation bar 6, or the like). As described above, the diagnostic imaging device 2 only needs to be able to display the object simulating the treatment device, and the display position is not limited on the longitudinal tomographic image 5.

FIG. 10 is a flowchart depicting a procedure of generation processing of the calculation model 51. Processing details for a case where the calculation model 51 is generated by machine learning will be described with reference to FIG. 10.

The control unit 11 of the server 1 acquires training data (second training data) for generating the calculation model 51 (S11). The training data is data in which a correct value of plaque burden is correlated with a transverse tomographic image of a blood vessel for training.

The control unit 11 generates, based on the training data, the calculation model 51 for calculating plaque burden in a case where the transverse tomographic image of the blood vessel is input (S12). Specifically, as described above, the control unit 11 generates a neural network such as a CNN as the calculation model 51. The control unit 11 inputs the blood vessel tomographic image for training to the calculation model 51 to calculate the plaque burden, and compares the plaque burden with the correct value. The control unit 11 optimizes parameters such as weights between neurons so that the plaque burden calculated is approximate to the correct value, and generates the calculation model 51. The control unit 11 ends the series of processing.

FIG. 11 is a flowchart depicting a procedure of plaque burden calculation processing. Processing details executed by the diagnostic imaging device 2 will be described with reference to FIG. 11.

The control unit 21 of the diagnostic imaging device 2 acquires transverse tomographic images of a plurality of frames obtained by imaging a blood vessel of a patient (S31). The control unit 21 inputs the transverse tomographic image of each frame to the calculation model 51 and calculates the plaque burden in each frame (S32). Specifically, the control unit 21 acquires, from the calculation model 51, the plaque burden in each frame and the reliability of the calculated value.

The control unit 21 identifies a blood vessel position (frame) at which the plaque burden takes the maximum value (S33). The control unit 21 also identifies a low reliability region in which the reliability of the plaque burden is equal to or less than a predetermined value in the axial direction of the blood vessel (S34). Further, the control unit 21 receives a setting input of a threshold (S35). The control unit 21 may skip S35 and set a default threshold.

The control unit 21 generates (reconstructs) a longitudinal tomographic image of the blood vessel on the basis of the transverse tomographic images of the plurality of frames acquired in S31 (S36). The control unit 21 displays the generated longitudinal tomographic image and an object (first object) indicating the magnitude of the plaque burden at each position of the longitudinal tomographic image along the axial direction of the blood vessel (S37). Specifically, as described above, the control unit 21 displays a gradation bar in which the display color is gradually changed according to the magnitude of the plaque burden. In this case, the control unit 21 displays the gradation bar in which the low reliability region is hidden.

The control unit 21 divides, based on the plaque burden at each position (frame) of the blood vessel, the longitudinal tomographic image (longitudinal cross section) of the blood vessel into a first region having the plaque burden equal to or greater than the threshold and a second region having the plaque burden less than the threshold (S38). Specifically, the control unit 21 divides the longitudinal tomographic image into a plurality of regions according to whether or not the plaque burden is equal to or greater than the threshold, and in a case where the length of the low reliability region is equal to or less than a predetermined length, the control unit 21 divides the plurality of first regions or second regions extending over the low reliability region as the same region.

The control unit 21 identifies a first region including the maximum value of the plaque burden as a lesion region (S39). The control unit 21 then determines a region in which both ends of a treatment area to be treated with the treatment device are to be located according to the identified lesion region (S40). Specifically, the control unit 21 determines second regions located on both sides of the lesion region as regions where both ends of the stent indwelled in the blood vessel should be located.

The control unit 21 measures the length in the axial direction of the lesion region identified in S39, that is, the lesion length, and displays the lesion length in correlation with the lesion region in the longitudinal tomographic image (S41). The control unit 21 also displays an object (second object) representing the size of the treatment device in correlation with the longitudinal tomographic image (S42). For example, as described above, the control unit 21 superimposes and displays the stent region representing the length of the stent in the axial direction of the blood vessel on the longitudinal tomographic image such that both ends are located in the region determined in S40. The control unit 21 ends the series of processing.

As described above, according to the first embodiment, the plaque burden at each position of the blood vessel is calculated based on the transverse tomographic images of the plurality of frames obtained by imaging the blood vessel of the patient, and the object such as the gradation bar representing the magnitude of the calculated plaque burden is displayed in correlation with the longitudinal tomographic image, which makes it possible to suitably support diagnostic imaging related to vascular treatment such as determination of the indwelling position of the stent.

In addition, according to the first embodiment, the certainty of the calculation result of the plaque burden can be presented to the user by hiding the low reliability region in which the reliability of the calculation result of the plaque burden is relatively low.

Further, according to the first embodiment, it is possible to more suitably support diagnostic imaging by presenting the length of the lesion region (lesion length), the indwelling area of the stent (stent region), and the like.

Further, according to the first embodiment, the longitudinal cross section (longitudinal tomographic image) of the blood vessel is divided into a plurality of regions according to the plaque burden, and regions where both ends of a treatment area of the blood vessel by the treatment device such as the stent are to be located are determined, which makes it possible to more suitably support diagnostic imaging related to vascular treatment such as determination of the indwelling position of the stent.

According to the first embodiment, in a case where the length of the low reliability region is equal to or less than the predetermined length, the plurality of regions (first region or second region) extending over the low reliability region is regarded as the same region and divided. As a result, the indwelling position of the stent and the like can be suitably determined while reliability can be ensured.

First Modification

In the first embodiment, the first region where the plaque burden takes the maximum value is identified as the lesion region. However, the lesion region is not necessary to be a region where the plaque burden takes the maximum value.

FIG. 12 is an explanatory diagram related to processing for identifying a lesion region according to the first modification. In the first embodiment, the lesion region is identified first, and then the region where both ends of the stent should be located is determined; however, the region where both ends of the stent should be located may be determined first, and then the lesion region may be identified.

Specifically, after dividing the longitudinal tomographic image 5 into the first regions L1, L2, L3 . . . and the second regions H1, H2, H3 . . . , the diagnostic imaging device 2 searches for two second regions Hy and Hz sandwiching a first region Lx between the two second regions Hy and Hz. In a case where the two second regions Hy and Hz sandwiching the first region Lx are searched, the diagnostic imaging device 2 identifies, as the lesion region Lx, the first region Lx sandwiched between the two second regions Hy and Hz. The diagnostic imaging device 2 then measures the length of the lesion region Lx (lesion length) and superimposes and displays the stent region 7.

As described above, the diagnostic imaging device 2 may determine the region in which the stent is to be indwelled from the positional relationship between the first region and the second region. The determination method described in the first embodiment is an example, and various methods are assumed.

Second Embodiment

In the first embodiment, the configuration in which the calculation model 51 directly calculates plaque burden has been described. In the present embodiment, description is provided of a configuration in which a plaque region is identified from a blood vessel tomographic image (transverse tomographic image) using another machine learning model to indirectly calculate plaque burden. The same reference numerals are given to the same contents as those of the first embodiment, and the description of the same reference numerals of the first embodiment will be omitted.

FIG. 13 is a block diagram illustrating a configuration example of the diagnostic imaging device 2 according to the second embodiment. The auxiliary storage unit 27 of the diagnostic imaging device 2 according to the present embodiment stores an identification model 52. As with the calculation model 51 according to the first embodiment, the identification model 52 is a machine learning model generated by learning predetermined training data, and is a learned model that identifies a plaque region in a blood vessel tomographic image using the blood vessel tomographic image as an input. The identification model 52 is assumed to be used as a program module constituting a part of artificial intelligence software.

FIG. 14 is an explanatory diagram related to calculation processing of plaque burden according to the second embodiment. FIG. 14 illustrates a state in which an image region (referred to as a “plaque region” in the following description) corresponding to a plaque and a tunica media is identified from a transverse tomographic image of a blood vessel using the identification model 52, and plaque burden is calculated from an area of the identified plaque region. An outline of the present embodiment will be described with reference to FIG. 14.

The identification model 52 is a machine learning model that has learned predetermined training data, and can be, for example, a neural network generated by deep learning. In the present embodiment, the description will be given on the assumption that the identification model 52 is a semantic segmentation model which is a type of CNN.

The semantic segmentation model is a neural network that identifies an object in an image in units of pixels, and includes a convolutional layer (encoder) that convolves an input image and a deconvolutional layer (decoder) that maps a convoluted feature amount to the original image size. In the deconvolutional layer, which object exists at which position in the image is identified based on the feature amount extracted in the convolutional layer, and data obtained by binarizing which object each pixel corresponds to is generated.

The server 1 generates the identification model 52 using training data (first training data) in which data indicating a correct plaque region is correlated with a tomographic image for training. The server