The discrepancy report generation component may transmit generated discrepancy reports and/or alerts to one or more users for further evaluation. For those images that are not associated with significant discrepancies (e.g., the data across modalities is substantially congruent), the system may store an indication of such determinations and/or may also transmit a report reflecting the substantially congruent condition and parameter data across modalities to a user.

The disclosed systems and techniques provide a more efficient and effective means of determining discrepancies between modalities that may be associated with an image. By facilitating a more efficient determination of such discrepancies, the examples described herein may provide faster and more efficient means of identifying potential misdiagnoses and/or missed diagnoses, which may in turn reduce health risk and medical costs. For example, in the disclosed system, discrepancies between conditions represented on an image and those described by a user as being represented in the image are identified using the disclosed systems and techniques, while those images without such discrepancies are removed from further processing, thereby greatly reducing the quantity of images that must be reevaluated by a human user and/or further processed by a discrepancy analysis system. By removing from the processing pipeline those images without significant discrepancies sooner, the number of processor cycles needed to process (e.g., substantially congruent) images, memory resources to store images and image data, and network resources required to exchange images and image data may be reduced. Furthermore, by providing images for further analysis that are more likely to be associated with errors, resources (human and otherwise) are not wasted processing images that are less likely to be associated with errors. Thus, the disclosed systems and techniques improve the user experience while minimizing inefficacies, increasing available network bandwidth (and thereby reducing latency), increasing available memory for other processes and operations, and improving overall system performance.

The techniques and systems described herein may be implemented in a number of ways. Example implementations and additional details are provided below with reference to the following figures.

FIG. 1 is a block diagram of an illustrative environment 100 that may be a portion of, or otherwise associated with, a system incorporating a discrepancy determination framework or any other type of data processing and/or storage system. The environment 100 may include a discrepancy determination system 102 and a machine-learned model training system 142. The environment 100 may also include a source data store 104 that may be one or more data storage systems of any type. The systems 102 and 142, as well as the data store 104, may each represent one or more physical and/or logical resources, such as one or more server computers, virtual machines, software resources, databases, notebooks clusters, datasets, etc., or any combination thereof.

The discrepancy determination system 102 may be configured to receive source data from the source data store 104. Source data may be one or more images and associated data of one or more other modalities. For example, the discrepancy determination system 102 may receive a source data record 106 that may include or otherwise indicate an image 108, text 110, and one or more codes 112. The image 108 may be an image of any type captured by an imaging system (e.g., X-ray, scan, etc.) and/or data representative thereof. In a non-limiting example that will be used throughout the description of FIG. 1, the image 108 may be a digital representation of an X-ray. The text 110 may be textual data associated with the image 108. For example, the text 110 may be text entered and/or selected by a user for association with the image 108. For instance, where the image 108 is an X-ray, the text 110 may be a description of conditions and/or parameters identified in the X-ray by a radiologist. The code(s) 112 may be code data associated with the image 108. For example, the code(s) 112 may be codes selected by a user as representing one or more conditions and/or parameters for association with the image 108. For instance, where the image 108 is an X-ray, the code(s) 112 may be a description of conditions and/or parameters identified in the X-ray by a radiologist.

The source data record 106 may be transmitted to, acquired by, accessed, or otherwise received at a prediction determination component 114 of the discrepancy determination system 102. The prediction determination component 114 may include one or more components configured to determine prediction data for individual modalities that may be associated with an image. Prediction data may be any value or range of values determined for a particular condition or parameter. For example, for a particular condition (e.g., medical condition), a prediction value may be a likelihood of that condition being represented in the associated image. In another example, for a particular condition (e.g., medical condition), a prediction value may be an indication that the condition is present in the associated image based on a likelihood. For instance, a prediction generation component may determine that the likelihood of the condition being present in the subject of the image is over a threshold likelihood of 80%, and therefore may set the prediction value for the particular condition to an indication that the condition is present (e.g., “yes,” “present,” “likely,” “high risk,” etc.). In another example, for a particular parameter (e.g., age of the subject of the image), a prediction value may be an estimated age of the subject.

In examples, the prediction determination component 114 may include an image prediction generation component 116. The image prediction component 116 may be configured to process image modality data, such as the image 108. The image prediction generation component 116 may include an image processing model 118 that may be a machine-learned model trained to identify conditions and/or parameters in an image. In examples, the image processing model 118 may be configured to determine prediction values for conditions and parameters of a set of conditions and parameters. The image prediction component 116 may provide an image (e.g., image 108) as input to the image processing model 118 and execute the image processing model 118 to generate output that includes predicted values for conditions and parameters of the set of conditions and parameters. The image prediction component 116 may use this output to generate image prediction data 128. In examples, the image prediction data 128 may include the (e.g., unmodified) output of the image processing model 118. Alternatively or additionally, the image prediction data 128 may include data resulting from processing of the output of the image processing model 118 by the image prediction component 116 (e.g., predicted values determined by the image prediction component 116 based on the output of the image processing model 118).

In examples, the prediction determination component 114 may also, or instead, include a text prediction generation component 120. The text prediction generation component 120 may be configured to process text modality data, such as the text 110. The text prediction generation component 120 may include a text processing model 122 that may be a machine-learned model trained to identify conditions and/or parameters in textual data. In examples, the text processing model 122 may be configured to determine prediction values for conditions and parameters of a set of conditions and parameters (e.g., the same set of conditions and parameters for which the image processing model 118 is configured). The text prediction generation component 120 may provide textual data (e.g., text 110) as input to the text processing model 122 and execute the text processing model 122 to generate output that includes predicted values for conditions and parameters of the set of conditions and parameters. The text prediction generation component 120 may use this output to generate text prediction data 130. In examples, the text prediction data 130 may include the (e.g., unmodified) output of the text processing model 122. Alternatively or additionally, the text prediction data 130 may include data resulting from processing of the output of the text processing model 122 by the text prediction generation component 120 (e.g., predicted values determined by text prediction generation component 120 based on the output of the text processing model 122).

The prediction determination component 114 may also, or instead, include a code prediction generation component 124. The code prediction generation component 124 may be configured to process code modality data, such as the code(s) 112. The code prediction generation component 124 may include a code processing model 126 that may be a machine-learned model trained to identify conditions and/or parameters in code data. In examples, the code processing model 126 may be configured to determine prediction values for conditions and parameters of a set of conditions and parameters (e.g., the same set of conditions and parameters for which the image processing model 118 and/or the text processing model 122 is configured). The code prediction generation component 124 may provide code data (e.g., code(s) 112) as input to the code processing model 126 and execute the code processing model 126 to generate output that includes predicted values for conditions and parameters of the set of conditions and parameters. The code prediction generation component 124 may use this output to generate code prediction data 132. In examples, the code prediction data 132 may include the (e.g., unmodified) output of the code processing model 126. Alternatively or additionally, the code prediction data 132 may include data resulting from processing of the output of the code processing model 126 by the code prediction generation component 124 (e.g., predicted values determined by code prediction generation component 124 based on the output of the code processing model 126).

The output for the various modalities of the various prediction generation components of the prediction determination component 114 may be provided to a discrepancy determination component 134. The discrepancy determination component 134 may be configured to determine discrepancies for values of individual conditions and/or parameters determined for individual modalities. The discrepancy determination component 134 may be further configured to determine, for identified discrepancies, whether such discrepancies are significant. For example, as described in more detail herein, the discrepancy determination component 134 may determine whether detected discrepancies meet or exceed a discrepancy threshold for the associated condition or parameter. The discrepancy determination component 134 may generate discrepancy data 136 that may represent those discrepancies determined to be significant. The discrepancy data 136 may further include any or all of the data associated with the source data record 106, the image prediction data 128, the text prediction data 130, and/or the code prediction data 132.

The discrepancy data 136 may be provided to a discrepancy report generation component 138 that may be configured to generate a discrepancy report 140. The discrepancy report 140 may present, or facilitate presentation of, the discrepancy data 136 to a user and/or any related data. In examples, the discrepancy report generation component 138 may generate, or cause the generation of, a graphical user interface presenting the discrepancy report 140 to a user. For example, the discrepancy report generation component 138 may instruct a computing device 160 to generate one or more discrepancy interfaces 158 that may present the discrepancy report 140 and/or any associated data to a user.

As described above, the prediction determination component 114 may include prediction generation components (e.g., components 116, 120, 124) that are configured to execute one or more ML models (e.g., models 118, 122, 126) to perform one or more of the prediction value determinations functions for various conditions and/or parameters that may be associated with an image. In examples, an ML model training system 142 may be configured with a model training component 144 that trains models to generate, as output, predicted values for various conditions and/or parameters using data of particular modalities as input. For example, the model training component 144 may train an image processing model to generate, as output, predicted values for various conditions and/or parameters using image data as input to generate a trained image processing model 152. The ML model training system 142 may provide the trained image processing model 152 to the image prediction generation component 116 for use as the image processing model 118. Similarly, the model training component 144 may train a text processing model to generate, as output, predicted values for various conditions and/or parameters using textual data as input to generate a trained text processing model 154. The ML model training system 142 may provide the trained text processing model 154 to the text prediction generation component 120 for use as the text processing model 122. Likewise, the model training component 144 may train a code processing model to generate, as output, predicted values for various conditions and/or parameters using code data as input to generate a trained code processing model 156. The ML model training system 142 may provide the trained code processing model 152 to the text prediction generation component 120 for use as the code processing model 126.

The training data used by the model training component 144 may include modality-specific training data. For example, to train the trained image processing model 152, the model training component 144 may use image training data 146 that may include images and associated, identified conditions and/or parameters. To train the trained text processing model 154, the model training component 144 may use image training data 148 that may include textual data and associated, identified conditions and/or parameters. To train the trained code processing model 156, the model training component 144 may use code training data 150 that may include code data and associated, identified conditions and/or parameters.

The model training component 144 may train a model with one or more layers. For example, any or all of the trained image processing model 152, the trained text processing model 154, and the trained code processing model 156, may include one or more neural network layers trained to process input data (e.g., images, text, codes, etc.) and/or one or more language model (LM) layers trained to process, as input, output from one or more neural network layers. Any combination and number of neural networks and machine-learned models may be used to implement the prediction generation processes described herein.

The systems and methods described herein may be faster, more efficient, and more robust than conventional systems and methods for determining discrepancies in medical image analysis and other image analysis, as they may facilitate the efficient determination of images that require additional manual analysis while removing from further manual consideration those images with which no discrepancies have been identified. That is, the systems and methods described herein provide a technological improvement over existing discrepancy determination processes that involve manually evaluating individual images and associated textual and other data that may result in less thorough analysis for discrepancies and therefore more frequent missing of such discrepancies, increasing the speed and efficiency of such operations and overall outcomes, especially in a medical context. Moreover, the systems and methods described herein provide a significant increase in the relevance of recommended additional image analysis by using trained and executed machine-learned models that increase the accuracy of discrepancy determinations, while greatly reducing the likelihood of presenting images with no discrepancies to a user for additional manual analysis.

By efficiently identifying discrepancies between images and data of other modalities associated with such images, processing requirements for determining discrepancies are reduced because the frequency of performing discrepancy determination-related operations is reduced. For example, because more discrepancies are accurately identified sooner, various users may not be required to repeatedly access images and related data to analyze such images and related data for different types of discrepancies. The discrepancy determination techniques described herein further increase the efficiency of the disclosed systems and methods because more relevant discrepancy data is presented while less relevant or irrelevant discrepancy data is eliminated before presentation on an interface, thereby reducing the processing and memory requirements of the present systems and methods even further over conventional techniques. Accordingly, fewer storage resources and fewer processing resources may be required to determine and further analyze discrepancies using these systems and methods and fewer network resources are needed to transmit and receive associated data.

FIGS. 2 and 3 are flow diagrams of illustrative processes illustrated as a collection of blocks in a logical flow graph, which represent a sequence of operations that can be implemented in hardware, software, or a combination thereof. In the context of software, the blocks represent computer-executable instructions stored on one or more computer-readable storage media that, when executed by one or more processors, perform the recited operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures, and the like that perform particular functions or implement particular abstract data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described blocks can be combined in any order and/or in parallel to implement the processes.

FIG. 2 is a flow diagram of an illustrative process 200 to determine discrepancies between conditions and/or parameters associated with an image determined using two or more modalities. The process 200 may be described at times with reference to the environment 100 and may be performed by the discrepancy determination system 102, but the process 200 may also, or instead, be performed in other similar and/or different environments by other similar and/or different systems.

At 202, source data from, or associated with, two or more modalities may be determined. This operation may include receiving or otherwise acquiring such source data from a data store, a source data generation system or component, an electronic communication, etc. The source data may be associated with an image and may include data associated with two or more modalities. For example, the source data may include two or more of an image, image data, image metadata, textual data, code data, diagnostic data, lab data, and any other type of data that may be processed as described herein to determine predicted conditions and/or parameters that may be associated with an image.

At 204, the source data may be used as input to one or more machine-learned models that may be executed to generate predicted conditions and/or parameters as output. For example, the system may include machine-learned models trained to process data associated with particular modalities to generate a set of conditions and/or parameters. A first such machine-learned model may be trained to process an image as input to generate predicted values for the set of conditions and/or parameters, while a second such machine-learned model may be trained to process textual data as input to generate predicted values for the set of conditions and/or parameters. Alternatively or additionally, models may be used that have been trained to process code data, diagnostic data, lab data, etc. as input to generate predicted values for the set of conditions and/or parameters.

At 206, the resulting predicted condition and/or parameter values may be compared to determine whether any discrepancies between such values are present. For example, the system may compare a first value determined for a particular condition by a first modality component (e.g., executing an image processing machine-learned model using an image as input to generate the first value as output) to a second value determined for the same particular condition by a second modality component (e.g., executing a text processing machine-learned model using textual data associated with the image as input to generate the second value as output, executing a code prediction generation component using code data associated with the image as input to generate the second value as output, executing an image metadata prediction generation component using image metadata for the image (e.g., DICOM data) as input to generate the second value as output, etc.). If such a comparison determines a difference (e.g., of quantity or quality) between the first value and the second value, the system may determine that a discrepancy for that condition is present in the prediction data for the two modalities associated with the image. In examples, as described in more detail herein, the system may be configured to determine whether such discrepancies are significant based on various criteria.

At 208, the system may determine whether any such determined discrepancies are relevant. In examples, a relevance may be based on one or more factors that may be used to determine whether the determined discrepancy should be indicated (e.g., to a user) as a discrepancy. For example, a user may set one or more parameters at a discrepancy determination system that may indicate conditions and/or parameters of interest. The system may determine whether a condition or parameter associated with a determined discrepancy is indicated as one of these conditions and/or parameters of interest and, if so, include the determined discrepancy in a discrepancy alert and/or report (e.g., that may be transmitted or otherwise provided to the user). Other factors may also, or instead, be used to determine the relevance of a determined discrepancy between predicted values.

At 210, the system may generate a discrepancy report and/or alert indicating determined discrepancies (e.g., to a user and/or for storage at a data store). For example, the system may transmit an individual alert (e.g., as an email, present as an interface element, etc.) for each individual discrepancy determined. Alternatively or additionally, the system may transmit a discrepancy report (e.g., as an email, present as an interface element, etc.) that includes or indicates one or more determined discrepancies, for example, associated with a batch of images and associated data processed by a discrepancy determination system.

FIG. 3 is a flow diagram of an illustrative process 300 to determine discrepancies between predicted values determined using image and textual modalities. The process 300 may be described at times with reference to the environment 100 and may be performed by the discrepancy determination system 102, but the process 300 may also, or instead, be performed in other similar and/or different environments by other similar and/or different systems.

At 302, image source data may be determined (e.g., received, retrieved, acquired, etc.). Image source data may include, but is not limited to, an image of any type and/or image data representing an image of any type. For example, the image source data may include an X-ray, an MRI image, a CT scan image, a CAT scan image, an ultrasound image, a histology image, an OCT image, a fundus image, a histology image, a pathology image, a histopathology image, and/or any other form of digital and/or analog image.

At 304, text source data associated with the image source data determined at 302 may be determined (e.g., received, retrieved, acquired, etc.). Text source data may include, but is not limited to, descriptive text provided by a user that has visually analyzed an image or image data associated with the image source data of 302. For example, the text source data may be textual data input by a radiologist analyzing an X-ray or other type of medical image, an engineer analyzing an image of a mechanical component generated using a computer vision system, a technician or doctor analyzing an image of an implanted medical device, etc. Such textual data may be a textual description of one or more conditions and/or parameters identified by the analyzing user.

At 306, the image source data of 302 may be used as input to an image processing model that may be executed to generate one or more prediction values as output. The image processing model may include one or more machine-learned models and/or neural networks trained to determine values for a set of conditions and/or parameters based on image input. For example, the image processing model may be configured to determine predicted values for a predetermined set of medical conditions, demographic parameters, and image parameters based on an image or image data. The image processing model may be executed by an image processing component that may generate predicted values based on the output of the image processing model, which may include simply providing the output of the image processing model as predicted values and/or modifying and/or further processing the output of the image processing model to generate the predicted values (e.g., as described herein).

At 308, the text source data of 304 may be used as input to a text processing model that may be executed to generate one or more prediction values as output. The text processing model may include one or more machine-learned models (e.g., one or more natural language processing (NLP) models) and/or neural networks trained to determine values for a set of conditions and/or parameters based on textual data input. For example, the text processing model may be configured to determine predicted values for a predetermined set of medical conditions, demographic parameters, and image parameters based on textual data. The text processing model may be executed by a text processing component that may generate predicted values based on the output of the text processing model, which may include simply providing the output of the text processing model as predicted values and/or modifying and/or further processing the output of the text processing model to generate the predicted values (e.g., as described herein).

At 310, the predicted values generated at 306 may be compared to corresponding predicted values generated at 308 to determine discrepancies. For example, a predicted value generated for a particular condition or parameter at 306 may be compared to a predicted value generated for the same particular condition or parameter at 308 to determine if there is a difference between such values. This discrepancy determination operation may be performed for the predicted values generated for the individual conditions and/or parameters in the set of conditions and/or parameters (e.g., the predetermined set of medical conditions, demographic parameters, and image parameters). In examples, differences of any type may be considered discrepancies.

At 312, the system may determine whether a determined discrepancy is sufficiently significant. As described herein, this sufficiency of significance may be based on a variety of factors. In examples, any difference between two predicted values of a same condition or parameter may be considered sufficiently significant. Alternatively or additionally, a difference between two predicted values of a same condition or parameter that is equal to or greater than a threshold value may be considered sufficiently significant. Other criteria for determining significance may be utilized in addition to or instead of absolute difference and threshold amounts of difference.

If, at 312, a determined discrepancy for a particular condition or parameter is determined to be insufficiently significant, at 320, data may be stored indicating the discrepancy determination, the significance determination, and/or other related data. In examples, the source data of 302 and 304 may be supplemented or augmented with the discrepancy determination results associated with such source data. This augmentation data may indicate that the values for the particular condition or parameter are sufficiently congruent. Alternatively or additional, if a determined discrepancy is determined to be insufficiently significant, no action may be taken at 320.

If, at 312, a determined discrepancy is determined to be sufficiently significant, at 314 the system may determine the relevance of the determined discrepancy. As described herein, the relevance of a determined discrepancy may be based on one or more factors that may be used to determine whether the determined discrepancy should be indicated (e.g., to a user) as a discrepancy. As described herein, one or more parameters and/or conditions in a set may be configured at a discrepancy determination system as relevant and/or irrelevant (e.g., as configured by a user, according to attributes of the parameter or condition, etc.). The system may determine whether a condition or parameter associated with the determined discrepancy is indicated as a relevant condition or parameter at 314.

At 316, if the determined discrepancy for a particular condition or parameter is determined to be not relevant, data may be stored indicating the discrepancy determination, the relevance determination, and/or other related data. In examples, the source data of 302 and 304 may be supplemented or augmented with the discrepancy determination results associated with such source data. This augmentation data may indicate that the determined discrepancy was not relevant. Alternatively or additionally, if a determined discrepancy is determined not to be relevant, no action may be taken at 320.

If, at 316, the determined discrepancy for a particular condition or parameter is determined to be relevant, at 318 the discrepancy determination system may generate a discrepancy report and/or alert indicating the determined discrepancy (e.g., to a user and/or for storage at a data store). For example, the system may transmit an individual alert (e.g., as an email, present as an interface element, etc.) for the determined discrepancy. Alternatively or additionally, the system may ass the determined discrepancy to a discrepancy report alert (e.g., as an email, present as an interface element, etc.) that may include or indicate one or more determined discrepancies, for example, as a summary of the results of processing a batch of images and associated data by a discrepancy determination system.

FIG. 4A illustrates an example 401 of prediction data and an exemplary interface on which such data may be presented. In various examples, the processing and generation of data illustrated in the figure may be performed, at least in part, by a discrepancy determination system, such as the discrepancy determination system 102 of FIG. 1. For example, image prediction data 410 may correspond to image prediction data 128 of FIG. 1, and may be generated using an image prediction generation component, such as the image prediction generation component 116 of FIG. 1, that may include one or more ML models and/or neural networks, such as the image processing model 118 of FIG. 1. The image prediction data 410 may include output data generated by an image processing model executed using an image as input.

In examples, the image prediction generation component may be configured to determine predicted values for a set of conditions and/or parameters using an image. For example, the image prediction generation component may be configured to determine condition data 420 that includes predicted values for conditions 421-425, demographic data 430 that includes predicted values for demographic parameters 431-433, and image metadata 440 that includes predicted values for image parameters 441-443. In a demonstrative medical example, the conditions of the condition data 420 may include any one or more medical conditions, such as atelectasis, consolidation, edema, emphysema, fibrosis, effusion, pleural thickening, cardiomegaly, mass, hernia, lung opacity, and enlarged cardiomedia. In this example, the parameters of the demographic data 430 may include any one or more demographic parameters that may be associated with a patient or human subject, such as sex, race, age, gender, behavioral attribute(s), environmental attribute(s), social determinant(s), etc. Continuing with this medical example, the parameters of the image data 440 may include any one or more image parameters that may be associated with a scan or image of a patient or human subject, such as patient position, imaging device or system (e.g., camera) data (e.g., position, focal length, model, make, etc.), image data and/or metadata (e.g., resolution, time/date of capture, location, etc.), etc. Such image parameters may include or be represented as DICOM tags, DICOM header data, and/or other DICOM data.

The predicted values and associated data of the image prediction data 410 may be presented to a user in a variety of forms. For example, the predicted values may be presented as interface elements on an interface, as shown here for the demographic data 430 (e.g., “43” for demographic parameter 431, “X” for demographic parameter 432, and “Y” for demographic parameter 433) and the image metadata 440 (e.g., “Frontal” for image parameter 441, “I” for image parameter 442, and “J” for image parameter 443). The predicted values for the various conditions of the condition data 420 are also shown here as risk values 428.

One or more graphical representations of predicted values may also be presented to a user as one or more interface elements. For example, the predicted condition values of the condition data 420 may be represented as graphical elements 426 that indicate the predicted value along a spectrum or scale of possible values.

In examples, the predicted values illustrated as the image condition data 410 may also, or instead, be stored for use in discrepancy determination operations. For instance, no interfaces may be generated to present predicted values generated for the individual modalities used in discrepancy determination operations, but rather such data may be stored for performing the discrepancy determination operations.

FIG. 4B illustrates an example 402 of prediction data and an exemplary interface on which such data may be presented. In various examples, the processing and generation of data illustrated in the figure may be performed, at least in part, by a discrepancy determination system, such as the discrepancy determination system 102 of FIG. 1. For example, text prediction data 450 may correspond to text prediction data 130 of FIG. 1, and may be generated using a text prediction generation component, such as the text prediction generation component 120 of FIG. 1, that may include one or more ML models and/or neural networks, such as the text processing model 122 of FIG. 1. The text prediction data 450 may include output data generated by a text processing model executed using a textual data associated with an image as input. In this example, the textual data may be associated with the image processed to generate the exemplary prediction values described in regard to FIG. 4A.

In examples, the text prediction generation component may be configured to determine predicted values for a set of conditions and/or parameters using textual data associated with an image, such as the image processed to generate the predicted values described in regard to FIG. 4A. For example, the text prediction generation component may be configured to determine condition data 460 that includes predicted values for the conditions 421-425, predicted values for the demographic data 470 that includes the demographic parameters 431-433, and predicted values for the image metadata 480 that includes image parameters 441-443. In this example, the conditions of the condition data 460 may be the same conditions as those of the condition data 420 described in regard to FIG. 4A, the demographic parameters of the demographic data 470 may be the same parameters as those of demographic data 430 described in regard to FIG. 4A, and the image parameters of the image metadata 480 may be the same parameters as those of image metadata 440 described in regard to FIG. 4A. For example, the text prediction generation component may be configured to determine predicted DICOM data and/or parameter values for comparison to the image metadata 440 that may accompany an image. Continuing the demonstrative medical example initiated above, the conditions of the condition data 460 may include any one or more of the medical conditions described in regard to FIG. 4A, the demographic data 470 may include any one or more of the demographic parameters described in regard to FIG. 4A, and the image metadata 480 may include any one or more image parameters described in regard to FIG. 4A.

The predicted values and associated data of the text prediction data 450 may be presented to a user in a variety of forms, as with the example of FIG. 4A, including predicted values presented as interface elements on an interface, as shown here for the demographic data 470 (e.g., “13” for demographic parameter 431, “X” for demographic parameter 432, and “Y” for demographic parameter 433) and the image metadata 480 (e.g., “Side” for image parameter 441, “Q” for image parameter 442, and “J” for image parameter 443). The predicted values for the various conditions of the condition data 460 are also shown here as risk values 468.

As noted, one or more graphical representations of predicted values may also be presented to a user as one or more interface elements. For example, the predicted condition values of the condition data 460 may be represented as graphical elements 466 that indicate the predicted value along a spectrum or scale of possible values.

Here again, the predicted values illustrated as the text condition data 450 may also, or instead, be stored for use in discrepancy determination operations. For instance, no interfaces may be generated to present predicted values generated for the individual modalities used in discrepancy determination operations, but rather such data may be stored for performing the discrepancy determination operations.

FIG. 5 illustrates an example 501 of discrepancy data and an exemplary interface on which such data may be presented. In various examples, the determination of the discrepancy data illustrated in the figure may be performed, at least in part, by a discrepancy determination system, such as the discrepancy determination system 102 of FIG. 1. For example, discrepancy data 510 may correspond to discrepancy data 136 of FIG. 1, and may be generated using a discrepancy determination component, such as the discrepancy determination component 134 of FIG. 1. The text prediction data 450 may input data generated by an image prediction generation component and/or a text prediction generation component as described herein (e.g., in regard to FIGS. 4A and 4B above).

In examples, the discrepancy determination system component may be configured to determine discrepancies between predicted values generated for a set of conditions and/or parameters using modality data associated with an image, such as the image and data processed to generate the predicted values described in regard to FIGS. 4A and 4B. For example, the discrepancy determination system may be configured to determine condition discrepancy data 510 that includes comparisons and discrepancies between predicted values for the conditions 421-425, demographic discrepancy data 530 that includes comparisons and discrepancies between predicted values for the demographic parameters 431-433, and image discrepancy data 540 that includes comparisons and discrepancies between predicted values for the image parameters 441-443. In this examples, the comparisons performed to determine discrepancies may be comparisons of the predicted values of the condition data 420 described in regard to FIG. 4A and the predicted values of the condition data 460 described in regard to FIG. 4B, comparisons of the predicted values of the demographic data 430 described in regard to FIG. 4A and the predicted values of the demographic data 470 described in regard to FIG. 4B, and comparisons of the predicted values of the image metadata 440 described in regard to FIG. 4A and the predicted values of the image metadata 480 described in regard to FIG. 4B.

The discrepancy data 510 may be presented to a user in a variety of forms, similar to the examples of FIGS. 4A and 4B, including predicted values for the compared modalities presented as interface elements on an interface, as shown here for the demographic discrepancy data 530 (e.g., “43/13” for demographic parameter 431, “X/X” for demographic parameter 432, and “Y/Y” for demographic parameter 433) and the image discrepancy data 540 (e.g., “Frontal/Side” for image parameter 441, “I/Q” for image parameter 442, and “J/J” for image parameter 443). The compared predicted values for the various conditions 421-425 are also shown here as compared risk values 528.

One or more graphical representations of compared predicted values may also be presented to a user as one or more interface elements. For example, the compared predicted condition values for the conditions 421-425 may be represented as graphical elements 526 that indicate the predicted value for each modality along a spectrum or scale of possible values. This may facilitate easily representing those comparisons that are associated with a (e.g., significant) discrepancy.

Indications of various determined discrepancies may be included in the discrepancy data 510 and/or indicated on an interface presenting such data. For example, the condition data discrepancy alerts 527 interface element may include a textual indication that may describe a determined condition discrepancy (e.g., with regard to condition 421 in this example). A graphical representation 529 of this discrepancy may also, or instead, be included and/or indicated in the condition discrepancy data 520.

Similarly, the demographic discrepancy data 530 may include a textual indication, such as a demographic data discrepancy alerts 537 interface element, that may describe a determined demographic data discrepancy (e.g., with regard to demographic parameter 431 in this example). A graphical representation 539 of this discrepancy may also, or instead, be included and/or indicated in the demographic discrepancy data 530.

Likewise, the image discrepancy data 540 may include textual indications, such as those represented in an image data discrepancy alerts 547 interface element, that may describe determined image data discrepancies (e.g., with regard to image parameters 441 and 442 in this example). Graphical representations 546 and 549 of these discrepancies may also, or instead, be included and/or indicated in the image discrepancy data 540.

As with the data of FIGS. 4A and 4B, Here again, the data illustrated as discrepancy data 510 may also, or instead, be stored for other uses and future reference. For instance, no interfaces may be generated to present discrepancy determination data, but rather such data may be stored for generating the discrepancy determination alerts and operations and/or for operations.

FIG. 6 is a system and network diagram that shows an illustrative operating environment that includes a discrepancy determination system 102, a source data store 104, and a machine-learned model training system 142 that can be configured to implement aspects of the functionality described herein. The systems 102, 104, and 142 can each execute and/or implement various types of computing and network services, such as data storage and data processing, and/or utilize various computing resources of various types of systems on a permanent or an as-needed basis. Among other types of functionality, the computing resources utilized and/or implemented by the systems 102, 104, and 142, or by a larger system of which one or more of these systems may be a part, can be utilized to implement the various discrepancy determination operations and model training operations described herein. One or more of the systems 102, 104, and 142 may be part of a larger system that provides additional computing resources that include, without limitation, data storage resources, data processing resources, such as virtual machine (VM) instances, networking resources, data communication resources, network services, and other types of resources.

Each type of computing resource utilized and/or implemented at the systems 102, 104, and 142, or by a larger system of which one or more of these systems may be a part, can be general-purpose or can be available in a number of specific configurations. For example, data processing resources can be implemented as physical computers or VM instances in a number of different configurations. The VM instances can be configured to execute applications, including web servers, application servers, media servers, database servers, some or all of the discrepancy determination operations and/or model training operations described above, and/or other types of programs. Data storage resources can include file storage devices, block storage devices, and the like. The systems 102, 104, and 142, or a larger system of which one or more of these systems may be a part, can also be configured to perform other types of operations and/or utilize and/or implement other computing resources not mentioned specifically herein.

The computing resources utilized and/or implement by systems 102, 104, and 142, or a larger system of which one or more of these systems may be a part, may be enabled in one implementation by one or more data centers 600(1), 600(2), 600(3), . . . , 600(N) that may be configured in (partially or wholly) and/or communicative connected to the environment 100. The data centers are facilities utilized to house and operate computer systems and associated components. The data centers typically include redundant and backup power, communications, cooling, and security systems. The data centers can also be located in geographically disparate locations. One illustrative configuration for a data center that can be utilized to implement the technologies disclosed herein will be described below with regard to FIG. 7.

The users of the system can access the computing resources, such as systems 102, 104, and 142, and/or any of the computing resources in the environment 100, provided by the system over a network 602, which can be a wide area communication network (“WAN”), such as the Internet, an intranet, an Internet service provider (“ISP”) network, or a combination of such networks. For example, and without limitation, a computing device (e.g., the discrepancy determination system 102) operated by a user of the system can be utilized to access the system by way of the network 602. It should be appreciated that a local-area network (“LAN”), the Internet, or any other networking topology known in the art that connects the data centers to remote users and other users can be utilized. It should also be appreciated that combinations of such networks can also be utilized.

FIG. 7 is a computing system diagram that illustrates one configuration for a data center 600(N) that can be utilized to implement the systems 102, 104, and/or 142, as described above in FIGS. 1-5 and/or any other discrepancy determination system and/or model training system disclosed herein. The example data center 600(N) shown in FIG. 7 includes several server computers 700A-700E (collectively 700) for providing the computing resources 702A-702E (collectively 702), respectively.

The server computers 700 can be standard tower, rack-mount, or blade server computers configured appropriately for providing the various computing resources described herein (illustrated in FIG. 7 as the computing resources 702A-702E). As mentioned above, the computing resources 702 may be utilized and/or configured at one or more of systems 102, 104, and 142, or a larger system of which these systems may be a part, and can include, without limitation, analytics applications, data storage resources, data processing resources such as VM instances or hardware computing systems, database resources, networking resources, model execution resources, model training resources, and others. Some of the servers 700 can also be configured to execute access services 704A-704E (collectively 704) capable of instantiating, providing and/or managing the computing resources 702, some of which are described in detail herein.

The data center 600(N) shown in FIG. 7 also includes a server computer 700F that can execute some or all of the software components described above. For example, and without limitation, the server computer 700F can be configured to execute one or more of the discrepancy determination system 102, the source data store 104, the machine-learned model training system 142, and/or one or more components associated therewith. The server computer 700F can also be configured to execute other components and/or to store data for providing some or all of the functionality described herein. In this regard, it should be appreciated that components or different instances of one or more of the systems 102, 104, and 142 can execute on many other physical or virtual servers in the data centers 600 in various configurations.

In the example data center 600(N) shown in FIG. 7, an appropriate LAN 706 is also utilized to interconnect the server computers 700A-700F. The LAN 706 is also connected to the network 602 illustrated in FIG. 6. It should be appreciated that the configuration of the network topology described herein has been greatly simplified and that many more computing systems, software components, networks, and networking devices can be utilized to interconnect the various computing systems disclosed herein and to provide the functionality described above.

Appropriate load balancing devices or other types of network infrastructure components can also be utilized for balancing a load between each of the data centers 600(1)-(N), between each of the server computers 700A-700F in each data center 600, and, potentially, between computing resources 702 in each of the data centers 600. It should be appreciated that the configuration of the data center 600 described with reference to FIG. 7 is merely illustrative and that other implementations can be utilized.

FIG. 8 is a system services diagram that shows aspects of several services that can be provided by and utilized within the discrepancy determination system 102 and/or a larger system of which this system may be a part, which is configured to implement the various technologies disclosed herein. In particular, and as discussed above, this system, or a larger system of which this system may be a part, can provide a variety of services to users and other users including, but not limited to, discrepancy determination services performed by a prediction determination component 114, a discrepancy determination component 134, discrepancy repost generation component 138, and/or one or more computing instances performing one or more functions thereof; a storage service 800A; an on-demand computing service 800B; a serverless compute service 800C; a cryptography service 800D; an authentication service 800E; a policy management service 800F; and a deployment service 800G. The system, or a larger system of which the system is a part, can also provide other types of computing services, some of which are described below.

It is also noted that not all configurations described include the services shown in FIG. 8 and that additional services can be provided in addition to, or as an alternative to, the services explicitly described herein. Each of the systems and services shown in FIG. 8 can also expose web service interfaces that enable a caller to submit appropriately configured API calls to the various services through web service requests. The various web services can also expose GUIs, command line interfaces (“CLIs”), and/or other types of interfaces for accessing the functionality that they provide. In addition, each of the services can include service interfaces that enable the services to access each other. Additional details regarding some of the services shown in FIG. 8 will now be provided.

The storage service 800A can be a network-based storage service that stores data obtained from users of the system and/or from computing resources in the system, or a larger system of which the system is a part. The data stored by the storage service 800A can be obtained from computing devices of users. The data stored by the storage service 800A may also be activity data logged to the storage system 800A that may be functioning as a logging system or service.

The on-demand computing service 800B can be a collection of computing resources configured to instantiate VM instances and to provide other types of computing resources on demand. For example, a user of the system, or a larger system of which the system is a part, can interact with the on-demand computing service 800B (via appropriately configured and authenticated API calls, for example) to provision and operate VM instances that are instantiated on physical computing devices hosted and operated by the system, or a larger system of which the system is a part. The VM instances can be used for various purposes, such as to operate as servers supporting the network services described herein, a web site, to operate business applications or, generally, to serve as computing resources for the user.

Other applications for the VM instances can be to support database applications, electronic commerce applications, business applications and/or other applications. Although the on-demand computing service 800B is shown in FIG. 8, any other computer system or computer system service can be utilized in the system, or a larger system of which the system is a part, to implement the functionality disclosed herein, such as a computer system or computer system service that does not employ virtualization and instead provisions computing resources on dedicated or shared computers/servers and/or other physical devices.

The serverless compute service 800C is a network service that allows users to execute code (which might be referred to herein as a “function”) without provisioning or managing server computers in the system, or a larger system of which the system is a part. Rather, the serverless compute service 800C can automatically run code in response to the occurrence of events. The code that is executed can be stored by the storage service 800A or in another network accessible location.

In this regard, it is to be appreciated that the term “serverless compute service” as used herein is not intended to infer that servers are not utilized to execute the program code, but rather that the serverless compute service 800C enables code to be executed without requiring a user to provision or manage server computers. The serverless compute service 800C executes program code only when needed, and only utilizes the resources necessary to execute the code. In some configurations, the user or entity requesting execution of the code might be charged only for the amount of time required for each execution of their program code.

The system, or a larger system of which the system is a part, can also include a cryptography service 800D. The cryptography service 800D can utilize storage services of the system, or a larger system of which the system is a part, such as the storage service 800A, to store encryption keys in encrypted form, whereby the keys can be usable to decrypt user keys accessible only to particular devices of the cryptography service 800D. The cryptography service 800D can also provide other types of functionality not specifically mentioned herein.

The system, or a larger system of which the system is a part, in various configurations, also includes an authentication service 800E and a policy management service 800F. The authentication service 800E, in one example, is a computer system (i.e., collection of computing resources 800B) configured to perform operations involved in authentication of users or customers. For instance, one of the services shown in FIG. 8 can provide information from a user or customer to the authentication service 800E to receive information in return that indicates whether or not the requests submitted by the user or the customer are authentic.

The policy management service 800F, in one example, is a network service configured to manage policies on behalf of users or customers of the system, or a larger system of which the system is a part. The policy management service 800F can include an interface (e.g., API or GUI) that enables customers to submit requests related to the management of policy, such as a security policy. Such requests can, for instance, be requests to add, delete, change, or otherwise modify policy for a customer, service, or system, or for other administrative actions, such as providing an inventory of existing policies and the like.

The system, or a larger system of which the system is a part, can additionally maintain other network services based, at least in part, on the needs of its customers. For instance, the system, or a larger system of which the system is a part, can maintain a deployment service