Distances were calculated with the standard Euclidean metric, i.e., the square form of pairwise Euclidean distance, so that closer neighbors contribute more.

In experimentation, as depicted in FIGS. 6-8, feature variance method resulted in better clusters among other mass calculation techniques because it's a measure that quantifies the amount of variation or dispersion within a set of data points which helps to form natural clusters.

With mass and distance in hand, we summed the torque that every point's neighbors exerted on it. Points with larger mass and stronger pull thus shifted more in the direction of the net torque, while distant points had a weaker effect. The system iteratively updated all points using a small learning rate; the iterations stopped when the average displacement fell below a tolerance or when a preset maximum was reached.

Once torque-adjusted positions were stable, we re-applied a conventional clustering routine (e.g., k-means). The elbow criterion and silhouette scores guided the choice of the final number of clusters. Compared with the previous “straight-forward” pipeline, the torque-preprocessing produced clusters that were both more tightly packed internally and more clearly separated from one another, leading to a noticeable improvement in downstream grouping quality.

Conclusion: Our experiments on high-dimensional natural language utterances demonstrate that applying K-Means to torque-adjusted embeddings delivers consistently higher cluster quality than traditional K-Means or agglomerative methods. By letting data points “pull” one another based on locally estimated mass and mutual distances, the torque pre-processing discovers natural groupings without the need for manual tuning of cluster counts or distance metrics. To our knowledge, this is the first use of a torque-based framework for clustering textual embeddings, and it provides a robust, hyper-parameter-free pathway for uncovering hidden, semantically meaningful patterns in unstructured natural language text data.

End of Scientific or Technical Experimentation Description.

Currently, there exists a problem with using existing clustering techniques with financial utterances due to the nuanced nature of financial utterances which cause either too many clusters which overlap each other by more than a desirable degree or too few clusters that leave gaps of uncovered subject-matter between the identified clusters. Overlapping clusters create ambiguity in generating actionable outputs from the clustering and too many clusters require further processes—either human oversight or additional code to identify significant clusters. Both of these problems lead to difficulty in automatic triggering of actions in machines and data processing systems in response to financial utterances. A different problem occurs when too few clusters leave gaps of unaddressed subject-matter from the financial utterances—a system may fail to respond to a requested or required actions according to the utterances.

The present disclosure addresses these deficiencies by providing a computer implemented method, a computer usable program product, and a data processing system for mass-based clustering of utterances in financial contexts, improve the present state of the art technology by processing the financial utterances in a financial application-specific manner to generate an optimal number of clusters with optimal coherency and overlap. Optimality of cluster numbers is tunable for specific financial applications, and is generally intended to mean a number that does not exceed a set optimum threshold number. Similarly, optimal coherence means the coherence of any generated cluster not being below a set coherence threshold value; and optimal overlap means the overlap between any two generated clusters not exceeding a set overlap threshold value The improvements made by the illustrative embodiments to the existing clustering technology result in enabling automatic action identification in a manner that is suitable for triggering automatic operations in one or more other data processing systems in the financial data processing environment, as shown by a non-limiting example in FIG. 4.

FIG. 1 depicts a block diagram of a computing environment in which an illustrative embodiment may be implemented. The computing environment, labeled and referred to as system 10, includes user device 100, other device 120, and server 170 connected to a network. System 10 is configured to load and execute at least some of the computer code involved in performing the inventive methods, such as server instructions 158, which include the server-side computer program instructions of an embodiment, user device instructions 108, which include the user device-side computer program instructions of an embodiment, and other device instructions 126, which include the other device-side computer program instructions of an embodiment, as may be needed to perform the dynamic adaptation of identity related user interface.

User device 100 is a device used by a user that can be used as part of processes described herein. User device 100 can include one or more aspects described elsewhere herein such as in reference to computing environment 900 of FIG. 9. In many examples, user device 100 is a personal computing device, such as a smart phone, tablet, laptop computer, or desktop computer. But device 100 need not be so limited and may instead encompass other devices used by a user as part of processes described herein. In the illustrated example, user device 100 can include one or more user device processors 102, one or more user device interfaces 104, and user device memory 106, among other components.

The one or more user device processors 102 are one or more components of user device 100 that execute instructions, such as instructions that obtain data, process the data, and provide output based on the processing. The one or more user device processors 102 can include one or more aspects described below in relation to the one or more processors 912 of FIG. 9.

The one or more user device interfaces 104 are one or more components of user device 100 that facilitate receiving input from and providing output to something external to user device 100. The one or more user device interfaces 104 can include one or more aspects described below in relation to the one or more interfaces 918 of FIG. 9.

User device memory 106 is a collection of one or more components of user device 100 configured to store instructions and data for later retrieval and use. User device memory 106 can include one or more aspects described below in relation to memory 914 of FIG. 9. As illustrated, user device memory 106 stores user device instructions 108 and other suitable user device code.

User device instructions 108 are a set of instructions that, when executed by one or more of the one or more user device processors 102, cause the one or more user device processors 102 to perform an operation described herein. In examples, instructions 108 can be those of a mobile application (e.g., that may be obtained from a mobile application store, such as the APPLE APP STORE or the GOOGLE PLAY STORE). The mobile application can provide a user interface for receiving user input from a user and acting in response thereto. User interface 104 can further provide output to the user. In some examples, client instructions 108 are instructions that cause a web browser of user device 100 to render a web page associated with a process described herein. The web page may present information to the user and be configured to receive input from the user and take actions in response thereto.

Other device 120 may be a self-service kiosk, a computer terminal, a dedicated transactions terminal—such as an electronic payment terminal, an automated customer interaction machine or device—such as an Automated Teller Machine (ATM), and the like.

In the illustrated example, other device 120 includes one or more other device processors 122, other device memory 124, and other device interface 130.

The one or more other device processors 122 are one or more components of other device 120 that execute instructions, such as instructions that obtain data, process the data, and provide output based on the processing. The one or more other device processors 122 can include one or more aspects described below in relation to the one or more processors 912 of FIG. 9.

Other device memory 124 is a collection of one or more components of other device 120 configured to store instructions and data for later retrieval and use. Other device memory 120 can include one or more aspects described below in relation to memory 914 of FIG. 9. Other device memory 120 can store other device instructions 126.

Other device instructions 126 are instructions that, when executed by the one or more processors 122, cause the one or more processors 122 to perform one or more operations described elsewhere herein.

The one or more other device interfaces 130 are one or more components of other device 120 that facilitate receiving input from and providing output to something external to other device 120. The one or more other device interfaces 130 can include one or more aspects described below in relation to the one or more interfaces 918 of FIG. 9.

Server 150 is a server device that functions as part of one or more processes described herein. In the illustrated example, server 150 includes one or more server processors 152, one or more server interfaces 154, and server memory 150, among other components.

The one or more server processors 152 are one or more components of server 150 that execute instructions, such as instructions that obtain data, process the data, and provide output based on the processing. The one or more server processors 152 can include one or more aspects described below in relation to the one or more processors 912 of FIG. 9.

The one or more server interfaces 154 are one or more components of server 150 that facilitate receiving input from and providing output to something external to server 150. The one or more server interfaces 154 can include one or more aspects described below in relation to the one or more interfaces 918 of FIG. 9.

Server memory 150 is a collection of one or more components of server 150 configured to store instructions and data for later retrieval and use. Server memory 150 can include one or more aspects described below in relation to memory 914 of FIG. 9. Server memory 150 can store server instructions 158.

Server instructions 158 are instructions that, when executed by the one or more processors 152, cause the one or more processors 152 to perform one or more operations described elsewhere herein.

Network 190 is a set of devices that facilitate communication from a sender to a destination, such as by implementing communication protocols. Example networks 190 include local area networks, wide area networks, intranets, and the Internet.

FIG. 2 depicts an existing clustering process that can be improved in accordance with an illustrative embodiment. In an existing implementation of process 200, utterance/data pull operation 202 initiates the process by gathering relevant utterance data. Text Preprocessing 204 processes the gathered utterance data to prepare it for embedding.

Embeddings generation operation 206 generates embeddings from the pre-processed text (e.g., using Sentence Transformers). Clustering operation 208 clusters the embeddings using specified clustering algorithms, e.g., using K-Means or Agglomerative models. Evaluation operation 210 evaluates the clusters based on silhouette coefficient and threshold criteria, e.g., using a silhouette coefficient and/or a threshold. Cluster Labeling operation 212 labels the evaluated clusters. Cluster visualization operation 214 visualizes the labeled clusters, e.g., using T-SNE plots. Cluster/Label review operation 216 reviews the visualized clusters and labels, e.g., through human evaluation. The process ends thereafter. This process would suffer from the deficiencies identified herein, including producing too many or too few clusters with less than optimal coherency and overlap for actioning in a financial application environment.

FIG. 3 depicts an improved clustering process using torque principles in accordance with an illustrative embodiment. Process 300 can be implemented in server 150 as server instructions 158 in FIG. 1. Utterance/Data pull operation 302 initiates the process by gathering relevant financial utterances data. Financial utterances may be obtained as textual data, natural language (NL) speech, video or graphical input of a gesture indicative of an intended action, or some combination of these and other forms of utterances or inputs during an interaction with a financial application. a gesture may include verbal clues or may be a nonverbal gesture without any associated verbal clues. A text pre-processing operation, such as a modified form of operation 204 in FIG. 2, processes the gathered financial utterances data to prepare it for embedding. For example, the pre-processing operation may preprocess speech into text by applying a speech-to-text transformation to speech data, or by applying a gesture recognition model to decode a video or graphical gesture into a textual description of an intended utterance relative to a financial application or device. These examples of pre-processing are not intended to be limiting. From this disclosure, those of ordinary skill in the art will be able to conceive many other preprocessing and the same are contemplated within the scope of the illustrative embodiments.

Embeddings generation operation 304 generates embeddings from the gathered Natural language utterances, the preprocessed textual data, or some combination thereof. Dimensionality reduction operation 306 reduces the dimensionality of the embeddings in a manner described earlier. Mass assignment operation 308 assigns mass to data points (i.e., to the reduced dimension embedding) using the Torque Principle described earlier. In other words, operation 308 assigns a value representing a mass or mass-like property to the reduced data points such that the data point behaves like, or emulates, a physical object having a mass in a simulated physical dimensioned space in which the mass-bearing data points (MD(s)) behave and move relative to one another according to the physics of forces acting on physical objects consistent with principles governing torque. Clustering operation 310 applies a suitable clustering technique, such as K-Means clustering, while adjusting the positions of the data points. The position of a data point “A” relative to other data points “B”, “C”, . . . “N” is adjusted by computing the forces exerted on one mass-bearing data point A by other mass-bearing data points B, D, . . . F, situated in a neighborhood of the data point A at distances computed according to difference between each data point vector pair AB, AD, . . . AF. The neighborhood can be defined either by the closes “n” data points to data point A regardless of their distance from A, or all data points within a threshold distance “d” from A regardless of the number of data points falling within distance d from A. a neighborhood can also be defined by selecting only a subset of data points within distance d from A using a suitable selection criterion.

The resulting position-adjusted data points are referred to as torque-adjusted data points. Scoring operation 312 calculates one or more score for a generated cluster, such as by using the Adjusted Rand Index (ARI) and/or silhouette scores for the clusters. A score of a cluster is indicative of the cluster's cohesiveness, overlap with another cluster, or some combination thereof. The mass assignment, torque-based clustering and scoring repeats iteratively until an exit condition is reached. One exit condition may be that the cluster scores for a determined number of clusters indicates optimum coherence, optimum overlap, or both. Another exit condition may be that a determined number of iterations have been executed. Another exit condition may be that an improvement in a score from one iteration to the next is below an improvement threshold indicating that further iterations are not going to be productive in improving the scores. An exit condition can be a combination of one or more conditions described herein or other similarly purposed conditions.

Label generation operation 314 generates one or more labels corresponding to the generated and scored clusters, such as by using one or more large language models (LLMs) trained to convert a cluster of data points into an actionable label. The process ends thereafter. Some non-limiting examples of actionable labels include code that is configured to trigger a specific action at another data processing system in a financial data processing environment, code that is configured to cause a specific financial application to perform a specific function in a financial data processing environment, code that is configured to generate another code to implement a requested feature corresponding to the action at a data processing system in a financial data processing environment, code that is configured to trigger a notification in a financial data processing environment, or some combination of these and other manifestations of actionable labels that are readily conceivable from this disclosure by those of ordinary skill in the art.

FIG. 4 depicts a financial utterance clustering application in accordance with an illustrative embodiment. Financial utterance clustering application 402 can be implemented as server code 158 in FIG. 1, to perform all or a part of process 300 of FIG. 3.

Application 402 receives utterance data component 404 as input. Input 404 includes financial utterances as described herein. Preprocessing component 406 preprocesses the utterance data of input 404. Embedding generation component 408 generates embeddings from the preprocessed data. Dimensionality reduction component 410 reduces the dimensionality of the generated embeddings. Mass assignment 412 assigns, reassigns, and/or adjusts a mass of the reduced data point embeddings vectors according to one or more different mass computation methods as described with respect to FIG. 5.

Torque adjusted clustering component 414, operating iteratively in conjunction with mass assignment component 412 towards an exit condition in a manner described herein, clusters the mass-assigned data points using torque principles. Cluster scoring component 416 scores the generated clusters. Label generation component 418 generates actionable labels (also interchangeably referred to herein as labeled actions) corresponding to the scored clusters.

Application 402 outputs one or more actionable labels 420, which cause or trigger further actions, such as automated machine operations and/or functions, in a data processing system in the data processing environment providing financial services. Some non-limiting examples of the systems where the contemplated further actions may trigger automated operations or functions include, feature development system 422 that develops code of a feature based on an instruction according to an actionable label; user management and support system 424 that performs management and support functions for a user according to an actionable label; workflow management and change system 426 that creates a workflow or causes a change in a managed workflow based on an actionable label; human supervision system 428 that receives a notification for human attention to an actionable label; or some combination of these and other systems or components of a financial services data processing environment.

FIG. 5 depicts various modes of mass assignment and torque adjusted clustering in accordance with an illustrative embodiment. Mass assignment component 502 is an example implementation of mass assignment component 412 in FIG. 4. Torque adjusted clustering component 512 is an example implementation of torque adjusted clustering component 414 in FIG. 4.

Mass assignment component 502 includes several subcomponents, each subcomponent computing and adjusting the mass values using a different strategy. The same or different strategies for mass determination can be used during an iterative torque-based clustering described herein. Subcomponent 504 assigns uniform mass to all data points in a given set. Subcomponent 506 assigns mass based on a local density of a neighborhood of a data point. As computed by subcomponent 506, mass is proportional to local density (, e.g., as determined using k-nearest neighbors (KNN) algorithm). To assign mass using local density, subcomponent 506 implemented the following steps:

- Calculate square form of pairwise distances for each data point
- Sort the distances in an ascending order
- Find KNN distances for each data point where k=a predetermined value, e.g., 5
- local_density (for each data point)=1/(mean KNN distances+negligible value)
- Finally normalize local density by dividing sum of local densities

Subcomponent 508 assigns mass based on the sum of features of data points. Mass of a data point is proportional to the sum of that data point's absolute feature values. In one embodiment, all feature values are selected in the summation. In another embodiment, a subset of features is selected according to a suitable selection criterion and only the selected subset of features are included in the summation. The summation is performed as follows—

feature_sum=sum(abs(feature_values over a given axis)

Subcomponent 510 assigns mass based on the variance of features of data points. Mass of a data point is proportional to the variance of the data point's features. In one embodiment, all feature values are selected in the variance determination. In another embodiment, a subset of features is selected according to a suitable selection criterion and only the selected subset of features are included in the variance determination. The variance determination is performed as follows—

feature_variance=var(feature_values over a given axis)/sum(var(feature_values over a given axis)

Torque adjusted clustering component 512 includes several sub-components. Torque Calculation 514 calculates torque for data points. Data Point Shifting 516 shifts data points based on calculated torque. Cluster Generation 518 generates clusters from the shifted data points. The process ends thereafter.

FIG. 6 depicts the comparative results of one experiment. Table 600 shows a distinct improvement in clustering with the inventive torque adjusted method over an existing K-Means method of clustering on vectors of 768 features. The ARI index, Silhouette score, and the number of clusters all show improvement with torque based clustering, especially using density method, feature sum method, and feature variance method of mass assignments.

FIG. 7 depicts the comparative results of another experiment. Table 700 shows a distinct improvement in clustering with the inventive torque adjusted method over an existing K-Means method of clustering on UMAP reduced feature vectors. The ARI index, Silhouette score, and the number of clusters all show improvement with torque based clustering, especially using density method, feature sum method, and feature variance method of mass assignments.

FIG. 8 depicts the comparative results of another experiment. Table 800 shows a distinct improvement in clustering with the inventive torque adjusted method over an existing K-Means method of clustering on vectors of 768 features with torque adjusted using a k=5 neighbors. The ARI index, Silhouette score, and the number of clusters all show improvement with torque based clustering, especially using density method, feature sum method, and feature variance method of mass assignments.

FIG. 9 depicts a data processing environment 20 in which aspects of the present disclosure may be implemented. Computing environment 900 is a set of one or more virtual or physical computers 910 that individually or in cooperation achieve tasks, such as implementing one or more aspects described herein. Computers 910 have components that cooperate to cause output based on input. Example computers 910 include desktops, servers, mobile devices (e.g., smart phones and laptops), wearables, virtual reality devices, augmented reality devices, expanded reality devices, spatial computing devices, virtualized devices, other computers, or combinations thereof. In particular example implementations, computing environment 900 includes at least one physical computer.

Computing environment 900 may specifically be used to implement one or more aspects described herein. In some examples, one or more of computers 910 may be implemented as a user device, such as mobile device and others of computers 910 may be used to implement aspects of a machine learning framework useable to train and deploy models exposed to the mobile device or provide other functionality, such as through exposed application programming interfaces.

Computing environment 900 can be arranged in any of a variety of ways. Computers 910 can be local to or remote from other computers 910 of environment 900. Computing environment 900 can include computers 910 arranged according to client-server models, peer-to-peer models, edge computing models, other models, or combinations thereof.

In many examples, computers 910 are communicatively coupled with devices internal or external to computing environment 900 via network 902. Network 902 is a set of devices that facilitate communication from a sender to a destination, such as by implementing communication protocols. Example networks 902 include local area networks, wide area networks, intranets, or the Internet.

In some implementations, computers 910 can be general-purpose computing devices (e.g., consumer computing devices). In some instances, via hardware or software configuration, computers 910 can be special purpose computing devices, such as servers able to practically handle large amounts of client traffic, machine learning devices able to practically train machine learning models, data stores able to practically store and respond to requests for large amounts of data, other special purposes computers, or combinations thereof. The relative differences in capabilities of different kinds of computing devices can result in certain devices specializing in certain tasks. For instance, a machine learning model may be trained on a powerful computing device and then stored on a relatively lower powered device for use.

Many example computers 910 include one or more processors 912, memory 914, and one or more interfaces 918. Such components can be virtual, physical, or combinations thereof.

The one or more processors 912 are components that execute instructions, such as instructions that obtain data, process the data, and provide output based on the processing. The one or more processors 912 often obtain instructions and data stored in memory 914. The one or more processors 912 can take any of a variety of forms, such as central processing units, graphics processing units, coprocessors, tensor processing units, artificial intelligence accelerators, microcontrollers, microprocessors, application-specific integrated circuits, field programmable gate arrays, other processors, or combinations thereof. In example implementations, the one or more processors 912 include at least one physical processor implemented as an electrical circuit. Example providers of processors 912 include INTEL, AMD, QUALCOMM, TEXAS INSTRUMENTS, and APPLE.

Memory 914 is a collection of components configured to store instructions 916 and data for later retrieval and use. Instructions 916 can, when executed by the one or more processors 912, cause execution of one or more operations that implement aspects described herein. In many examples, memory 914 is a non-transitory computer readable medium, such as random-access memory, read only memory, cache memory, registers, portable memory (e.g., enclosed drives or optical disks), mass storage devices, hard drives, solid state drives, other kinds of memory, or combinations thereof. In certain circumstances, transitory memory 914 can store information encoded in transient signals.

The one or more interfaces 918 are components that facilitate receiving input from and providing output to something external to computer 910, such as visual output components (e.g., displays or lights), audio output components (e.g., speakers), haptic output components (e.g., vibratory components), visual input components (e.g., cameras), auditory input components (e.g., microphones), haptic input components (e.g., touch or vibration sensitive components), motion input components (e.g., mice, gesture controllers, finger trackers, eye trackers, or movement sensors), buttons (e.g., keyboards or mouse buttons), position sensors (e.g., terrestrial or satellite-based position sensors such as those using the Global Positioning System), other input components, or combinations thereof (e.g., a touch sensitive display). The one or more interfaces 918 can include components for sending or receiving data from other computing environments or electronic devices, such as one or more wired connections (e.g., Universal Serial Bus connections, THUNDERBOLT connections, ETHERNET connections, serial ports, or parallel ports) or wireless connections (e.g., via components configured to communicate via radiofrequency signals, such as according to WI-Fl, cellular, BLUETOOTH, ZIGBEE, or other protocols). One or more of the one or more interfaces 918 can facilitate connection of computing environment 900 to network 990.

Computers 910 can include any of a variety of other components to facilitate performance of operations described herein. Example components include one or more power units (e.g., batteries, capacitors, power harvesters, or power supplies) that provide operational power, one or more busses to provide intra-device communication, one or more cases or housings to encase one or more components, other components, or combinations thereof.

A person of skill in the art, having benefit of this disclosure, may recognize various ways for implementing technology described herein, such as by using any of a variety of programming languages (e.g., a C-family programming language, PYTHON, JAVA, RUST, HASKELL, other languages, or combinations thereof), libraries or packages (e.g., that provide functions for obtaining, processing, and presenting data, such as may be obtained using a package manager like PIP or CONDA), compilers, and interpreters to implement aspects described herein. Example libraries include NLTK (Natural Language Toolkit) by Team NLTK (providing natural language functionality), PYTORCH by META (providing machine learning functionality), NUMPY by the NUMPY Developers (providing mathematical functions), and BOOST by the Boost Community (providing various data structures and functions) among others. Operating systems (e.g., WINDOWS, LINUX, MACOS, IOS, and ANDROID) may provide their own libraries or application programming interfaces useful for implementing aspects described herein, including user interfaces and interacting with hardware or software components. Web applications can also be used, such as those implemented using JAVASCRIPT or another language. A person of skill in the art, with the benefit of the disclosure herein, can use programming tools to assist in the creation of software or hardware to achieve techniques described herein, such as intelligent code completion tools (e.g., INTELLISENSE) and artificial intelligence tools (e.g., GITHUB COPILOT by MICROSOFT or CODE LLAMA by META).

In some examples, large language models can be used to understand natural language, generate natural language, or perform other tasks. Examples of such large language models include CHATGPT by OPENAI, a LLAMA model by META, a CLAUDE model by ANTHROPIC, others, or combinations thereof. Such models can be fine-tuned on relevant data using any of a variety of techniques to improve the accuracy and usefulness of the answers. The models can be run locally on server or client devices or accessed via an application programming interface. Some of those models or services provided by entities responsible for the models may include other features, such as speech-to-text features, text-to-speech, image analysis, research features, and other features, which may also be used as applicable.

FIG. 10 illustrates an example machine learning framework 1000 that techniques described herein may benefit from or improve on. A machine learning framework 1000 is a collection of software and data that implements artificial intelligence trained to provide output, such as predictive data, based on input. Examples of artificial intelligence that can be implemented with machine learning way include neural networks (including recurrent neural networks), language models (including so-called “large language models”), generative models, natural language processing models, adversarial networks, decision trees, Markov models, support vector machines, genetic algorithms, others, or combinations thereof. A person of skill in the art having the benefit of this disclosure will understand that these artificial intelligence implementations need not be equivalent to each other and may instead select from among them based on the context in which they will be used. Machine learning frameworks 1000 or components thereof are often built or refined from existing frameworks, such as TENSORFLOW by GOOGLE, INC. or PYTORCH by the PYTORCH community.

The machine learning framework 1000 can include one or more models 1002 that are the structured representation of learning and an interface 1004 that supports use of the model 1002.

The model 1002 can take any of a variety of forms. In many examples, the model 1002 includes representations of nodes (e.g., neural network nodes, decision tree nodes, Markov model nodes, other nodes, or combinations thereof) and connections between nodes (e.g., weighted or unweighted unidirectional or bidirectional connections). In certain implementations, the model 1002 can include a representation of memory (e.g., providing long short-term memory functionality). Where the set includes more than one model 1002, the models 1002 can be linked, cooperate, or compete to provide output.

The interface 1004 can include software procedures (e.g., defined in a library) that facilitate the use of the model 1002, such as by providing a way to establish and interact with the model 1002. For instance, the software procedures can include software for receiving input, preparing input for use (e.g., by performing vector embedding, such as using Word2Vec, BERT, or another technique), processing the input with the model 1002, providing output, training the model 1002, performing inference with the model 1002, fine tuning the model 1002, other procedures, or combinations thereof.

In an example implementation, interface 1004 can be used to facilitate a training method 1010 that can include operation 1012. Operation 1012 includes establishing a model 1002, such as initializing a model 1002. The establishing can include setting up the model 1002 for further use (e.g., by training or fine tuning). The model 1002 can be initialized with values. In examples, the model 1002 can be pretrained. Operation 1014 can follow operation 1012. Operation 1014 includes obtaining training data. In many examples, the training data includes pairs of input and desired output given the input. In supervised or semi-supervised training, the data can be prelabeled, such as by human or automated labelers. In unsupervised learning the training data can be unlabeled. The training data can include validation data used to validate the trained model 1002. Operation 1016 can follow operation 1014. Operation 1016 includes providing a portion of the training data to the model 1002. This can include providing the training data in a format usable by the model 1002. The framework 1000 (e.g., via the interface 1004) can cause the model 1002 to produce an output based on the input. Operation 1018 can follow operation 1016. Operation 1018 includes comparing the expected output with the actual output. In an example, this can include applying a loss function to determine the difference between expected and actual. This value can be used to determine how training is progressing. Operation 1020 can follow operation 1018. Operation 1020 includes updating the model 1002 based on the result of the comparison. This can take any of a variety of forms depending on the nature of the model 1002. Where the model 1002 includes weights, the weights can be modified to increase the likelihood that the model 1002 will produce correct output given an input. Depending on the model 1002, backpropagation or other techniques can be used to update the model 1002. Operation 1022 can follow operation 1020. Operation 1022 includes determining whether a stopping criterion has been reached, such as based on the output of the loss function (e.g., actual value or change in value over time). In addition, or instead, whether the stopping criterion has been reached can be determined based on a number of training epochs that have occurred or an amount of training data that has been used. In some examples, satisfaction of the stopping criterion can include If the stopping criterion has not been satisfied, the flow of the method can return to operation 1014. If the stopping criterion has been satisfied, the flow can move to operation 1022. Operation 1022 includes deploying the trained model 1002 for use in production, such as providing the trained model 1002 with real-world input data and produce output data used in a real-world process. The model 1002 can be stored in memory 914 of at least one computer 910, or distributed across memories of two or more such computers 910 for production of output data (e.g., predictive data).

In addition to the depicted components of system 10 in FIG. 1, system 10 can also include many other components including, but not limited to—a public cloud, a private cloud, a gateway, a cloud orchestration module, a computer (which may further include specialized processing circuitry, cache, a communication fabric, a volatile memory, a persistent storage including an operating system, and peripheral devices), Internet of Things (IoT) sensors, a remote database, and a container set.

A device contemplated herein may take the form of a desktop computer, laptop computer, tablet computer, smart phone, smart watch or other wearable computer, mainframe computer, quantum computer or any other form of computer or mobile device now known or to be developed in the future that is capable of running a program, accessing a network or querying a database. As is well understood in the art of computer technology, and depending upon the technology, performance of a computer-implemented method may be distributed among multiple computers and/or between multiple locations. Where the detailed discussion in this disclosure is focused on a single computer, such focus is only to keep the presentation as simple as possible. A computer or device contemplated herein may be located in or participate in a cloud, even though it is not shown in a cloud. On the other hand, a computer or device is not required to participate in a cloud except to any extent as may be affirmatively indicated.

A processor device described herein may be of any type now known or to be developed in the future, may be distributed over multiple packages or coordinated integrated circuit chips, and may implement multiple processor threads and/or multiple processor cores. Cache memory may be located in the processor chip package(s) and is typically used for data or code that should be available for rapid access by the threads or cores running on a processor. Cache memories are typically organized into multiple levels depending upon relative proximity to the processing circuitry. Alternatively, some, or all, of the cache for the processor set may be located “off chip.” In some computing environments, a processor may be designed for working with qubits and performing quantum computing.

Computer readable program instructions are typically loaded onto a computing device to cause a series of operational steps to be performed by a processor set of the computing device and thereby effect a computer-implemented method, such that the instructions thus executed will instantiate the methods specified in flowcharts and/or narrative descriptions of computer-implemented methods included in this document (collectively referred to as “the inventive methods”). These computer readable program instructions are stored in various types of computer readable storage media, such as cache and the other storage media discussed below. The program instructions, and associated data, are accessed by the processor set to control and direct performance of the inventive methods. At least some of the instructions for performing the inventive methods may be stored in a persistent storage.

Measured service: cloud systems automatically control and optimize resource use by leveraging a metering capability at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, reported, and invoiced, providing transparency for both the provider and consumer of the utilized service.

For the sake of clarity of the description, and without implying any limitation thereto, the illustrative embodiments are described using some example configurations. From this disclosure, those of ordinary skill in the art will be able to conceive many alterations, adaptations, and modifications of a described configuration for achieving a described purpose, and the same are contemplated within the scope of the illustrative embodiments.

Furthermore, simplified diagrams of the data processing environments are used in the figures and the illustrative embodiments. In an actual computing environment, additional structures or components that are not shown or described herein, or structures or components different from those shown but for a similar function as described herein may be present without departing the scope of the illustrative embodiments.

Furthermore, the illustrative embodiments are described with respect to specific actual or hypothetical components only as examples. Any specific manifestations of these and other similar artifacts are not intended to be limiting to the embodiments. Any suitable manifestation of these and other similar artifacts can be selected within the scope of the illustrative embodiments.

The examples in this disclosure are used only for the clarity of the description and are not limiting to the illustrative embodiments. Any advantages listed herein are only examples and are not intended to be limiting to the illustrative embodiments. Additional or different advantages may be realized by specific illustrative embodiments. Furthermore, a particular illustrative embodiment may have some, all, or none of the advantages listed above.

Furthermore, the illustrative embodiments may be implemented with respect to any type of data, data source, or access to a data source over a data network. Any type of data storage device may provide the data to an embodiment, either locally at a data processing system or over a data network, within the scope of the illustrative embodiments. Where an embodiment is described using a mobile device, any type of data storage device suitable for use with the mobile device may provide the data to such embodiment, either locally at the mobile device or over a data network, within the scope of the illustrative embodiments.

The illustrative embodiments are described using specific code, computer readable storage media, high-level features, designs, architectures, protocols, layouts, schematics, and tools only as examples and are not limiting to the illustrative embodiments. Furthermore, the illustrative embodiments are described in some instances using particular software, tools, and data processing environments only as an example for the clarity of the description. The illustrative embodiments may be used in conjunction with other comparable or similarly purposed structures, systems, applications, or architectures. For example, other comparable mobile devices, structures, systems, applications, or architectures therefor, may be used in conjunction with such embodiment within the scope of the illustrative embodiments. An illustrative embodiment may be implemented in hardware, software, or a combination thereof.