Cloud components 310 may include model 302, which may be a machine learning model, artificial intelligence model, etc. (which may be referred collectively as “models” herein). Model 302 may take inputs 304 and provide outputs 306. The inputs may include multiple datasets, such as a training dataset and a test dataset. Each of the plurality of datasets (e.g., inputs 304) may include data subsets related to user data, predicted forecasts and/or errors, and/or actual forecasts and/or errors. In some embodiments, outputs 306 may be fed back to model 302 as input to train model 302 (e.g., alone or in conjunction with user indications of the accuracy of outputs 306, labels associated with the inputs, or with other reference feedback information). For example, the system may receive a first labeled feature input, wherein the first labeled feature input is labeled with a known prediction for the first labeled feature input. The system may then train the first machine learning model to classify the first labeled feature input with the known prediction (e.g., training Preliminary Model 112 to output embeddings corresponding to text tokens).

In a variety of embodiments, model 302 may update its configurations (e.g., weights, biases, or other parameters) based on the assessment of its prediction (e.g., outputs 306) and reference feedback information (e.g., user indication of accuracy, reference labels, or other information). In a variety of embodiments, where model 302 is a neural network, connection weights may be adjusted to reconcile differences between the neural network's prediction and reference feedback. In a further use case, one or more neurons (or nodes) of the neural network may require that their respective errors are sent backward through the neural network to facilitate the update process (e.g., backpropagation of error). Updates to the connection weights may, for example, be reflective of the magnitude of error propagated backward after a forward pass has been completed. In this way, for example, the model 302 may be trained to generate better predictions.

In some embodiments, model 302 may include an artificial neural network. In such embodiments, model 302 may include an input layer and one or more hidden layers. Each neural unit of model 302 may be connected with many other neural units of model 302. Such connections can be enforcing or inhibitory in their effect on the activation state of connected neural units. In some embodiments, each individual neural unit may have a summation function that combines the values of all of its inputs. In some embodiments, each connection (or the neural unit itself) may have a threshold function such that the signal must surpass it before it propagates to other neural units. Model 302 may be self-learning and trained, rather than explicitly programmed, and can perform significantly better in certain areas of problem solving, as compared to traditional computer programs. During training, an output layer of model 302 may correspond to a classification of model 302, and an input known to correspond to that classification may be input into an input layer of model 302 during training. During testing, an input without a known classification may be input into the input layer, and a determined classification may be output. (e.g., training Preliminary Model 112 to produce accurate embeddings).

In some embodiments, model 302 may include multiple layers (e.g., where a signal path traverses from front layers to back layers). In some embodiments, back propagation techniques may be utilized by model 302 where forward stimulation is used to reset weights on the “front” neural units. In some embodiments, stimulation and inhibition for model 302 may be more free-flowing, with connections interacting in a more chaotic and complex fashion. During testing, an output layer of model 302 may indicate whether or not a given input corresponds to a classification of model 302.

In some embodiments, the model (e.g., model 302) may automatically perform actions based on outputs 306. In some embodiments, the model (e.g., model 302) may not perform any actions.

System 300 also includes API layer 350. API layer 350 may allow the system to generate summaries across different devices. In some embodiments, API layer 350 may be implemented on mobile device 322 or user terminal 324. Alternatively or additionally, API layer 350 may reside on one or more of cloud components 310. API layer 350 (which may be A REST or Web services API layer) may provide a decoupled interface to data and/or functionality of one or more applications. API layer 350 may provide a common, language-agnostic way of interacting with an application. Web services APIs offer a well-defined contract, called WSDL, that describes the services in terms of its operations and the data types used to exchange information. REST APIs do not typically have this contract; instead, they are documented with client libraries for most common languages, including Ruby, Java, PHP, and JavaScript. SOAP Web services have traditionally been adopted in the enterprise for publishing internal services, as well as for exchanging information with partners in B2B transactions.

API layer 350 may use various architectural arrangements. For example, system 300 may be partially based on API layer 350, such that there is strong adoption of SOAP and RESTful Web services, using resources like Service Repository and Developer Portal, but with low governance, standardization, and separation of concerns. Alternatively, system 300 may be fully based on API layer 350, such that separation of concerns between layers like API layer 350, services, and applications are in place.

In some embodiments, the system architecture may use a microservice approach. Such systems may use two types of layers: Front-End Layer and Back-End Layer where microservices reside. In this kind of architecture, the role of the API layer 350 may provide integration between Front-End and Back-End. In such cases, API layer 350 may use RESTful APIs (exposition to front-end or even communication between microservices). API layer 350 may use AMQP (e.g., Kafka, RabbitMQ, etc.). API layer 350 may use incipient usage of new communications protocols such as gRPC, Thrift, etc.

In some embodiments, the system architecture may use an open API approach. In such cases, API layer 350 may use commercial or open-source API Platforms and their modules. API layer 350 may use a developer portal. API layer 350 may use strong security constraints applying WAF and DDOS protection, and API layer 350 may use RESTful APIs as standard for external integration.

FIG. 4 shows a flowchart of the steps involved in providing search query responses using adjacent keywords and search filters, in accordance with one or more embodiments. For example, the system may use process 400 (e.g., as implemented on one or more system components described above) in order to generate adjacent keywords based on embeddings of search queries, compute similarity and popularity metrics for each adjacent keyword, generate filters based on adjacent keywords, and use filters to provide responses to search queries.

At step 402, process 400 (e.g., using one or more components described above) retrieves a training dataset for the generation of a first semantic standardization model. The training dataset may comprise a set of concepts and a set of descriptors. For example, the training dataset may correspond to a body of text. The body of text may contain the set of descriptors in discussions of one or more subjects. The body of text may be written by different authors over time and may consist of articles, technical documentations, instruction sets, and other communications. Thus, the descriptors may be heterogenous in meaning, and more than one descriptor may be used to serve the same purpose. Descriptors can map onto concepts either explicitly mentioned in the body of text or indirectly referred to. More than one descriptor may describe the same concept. The system may tokenize the body of text by transforming it into a text token format. Each word, sentence, paragraph, or other semantic element may be represented as a text token, which may be translated to a real-valued embedding. These tokens can be words, sentences, or even individual characters, depending on the level of granularity required. The system may identify sentence boundaries, such as periods, question marks, or exclamation marks to select sentence-level tokens.

Each sentence is further divided into individual words. This step involves removing punctuation marks, splitting words based on spaces, and handling contractions or hyphenated words. The system may convert all words to lowercase to ensure consistency and avoid duplication of words due to case differences. Depending on the requirements, special characters like numbers, symbols, or emojis may be treated as separate tokens or removed altogether. Stop words (e.g., “the,” “is,” “and”) that do not carry significant meaning are often removed to reduce noise and improve the efficiency of subsequent text analysis tasks. The system may additionally reduce words to their base or root form (e.g., “running” to “run”), while lemmatization aims to convert words to their dictionary form (e.g., “better” to “good”). These techniques help to consolidate similar words and reduce vocabulary size. A training dataset based on high-quality text tokens is instrumental in reducing bloat in the set of descriptors for easier, faster, and more accurate consolidation onto concepts.

At step 404, process 400 (e.g., using one or more components described above) trains a preliminary model based on the training dataset, wherein the preliminary model correlates the set of descriptors with a set of embeddings. For example, the preliminary model is an embedding model using a deep neural network algorithm trained to produce vector representations of input text tokens. The system may train the preliminary model in an unsupervised fashion to embed text tokens into real-valued vectors utilizing advanced techniques such as word2vec or GloVe. These algorithms aim to capture the semantic and syntactic relationships between words by learning distributed representations, also known as word embeddings. The system may provide the training dataset for the preliminary model to predict embeddings as a part of the training and evaluation process. The model then iterates over the text, considering the context in which each word appears. It learns to predict the probability of a word occurring given its neighboring words (skip-gram model) or predicts the surrounding words given a target word (continuous bag-of-words model). This prediction task forces the model to learn meaningful representations of words based on their contextual usage.

During training, the model adjusts its internal parameters to minimize the difference between predicted and actual word occurrences. This optimization process is typically performed using stochastic gradient descent or other optimization algorithms. The system may use a loss function to enforce the integrity of predictions by the preliminary model. As the model iterates over the text, it gradually learns to encode words into dense, low-dimensional vectors, where similar words are represented by vectors that are closer together in the vector space.

The resulting word embeddings capture various linguistic properties, such as semantic similarity and syntactic regularities. For example, words with similar meanings or that often appear in similar contexts will have similar vector representations. Additionally, vector arithmetic operations can be performed on the embeddings, allowing for intriguing analogies like “king−man+woman=queen.”

Once the preliminary model is trained, the embedded vectors can be used as features in downstream tasks such as text classification, information retrieval, or sentiment analysis. These real-valued representations enable the model to leverage the semantic relationships between words, improving the performance of subsequent machine learning algorithms.

At step 406, process 400 (e.g., using one or more components described above) trains the first semantic standardization model based on the set of embeddings. The first semantic standardization model may be trained to find patterns and relations between embeddings. For example, the first semantic standardization model is a language processing model trained to perform pattern mining and sequence mining based on a set of embeddings. The training process leverages a supervised or semi-supervised approach where the model learns to recognize and predict patterns and sequences within these embeddings. This is achieved by employing techniques such as RNNs or transformers, which are adept at handling sequential data and capturing dependencies over long ranges. During training, the model optimizes a loss function designed to measure its performance in identifying correct patterns and sequences, using backpropagation and gradient descent to update its weights. The output layer of the model is structured to produce a relational representation, which involves generating tuples that indicate correspondences and relations between the embeddings. The training process includes fine-tuning on a validation set to ensure generalization and robustness of the model. Additionally, techniques like attention mechanisms are employed to focus on relevant parts of the input embeddings, enhancing the model's ability to discern intricate patterns and sequences.

At step 408, process 400 (e.g., using one or more components described above) generates a relational semantic structure comprising a relation from each concept in the set of concepts to one or more descriptors in the set of descriptors using the first semantic standardization model. For example, the relational semantic structure is a relational data structure comprising a set of vertices and connections, where each vertex in the relational semantic structure corresponds to a descriptor in the set of descriptors, and each connection in the relational semantic structure corresponds to a pattern of proximity between a first descriptor in the set of descriptors and a second descriptor in the set of descriptors. The system may generate the relational semantic structure based on the output of Semantic Standardization Model 116. However, the system may make additional modifications to further fine-tune the relational semantic structure.

For example, the system may determine a set of pairwise distances from embeddings for each descriptor to each other descriptor in the set of descriptors. Each distance is based on a mathematical comparison of a first embedding to a second embedding. The distances in the set of pairwise distances may be indicative of the estimated similarities between a first embedding and a second embedding. For example, the system may calculate a Euclidean distance between each embedding to each other embedding in Embeddings 132. Based on the set of pairwise distances, the system may determine a distance threshold, for example, a percentile of the set of pairwise distances. The distance threshold may indicate the sufficient similarity for the system to group descriptors as corresponding to the same concept. Based on the distance threshold and the set of pairwise distances, the system selects a set of clusters from the set of embeddings. Each cluster in the set of clusters comprises embeddings with a mutual distance less than the distance threshold. The system may thus identify clusters of descriptors that can be combined into a single vertex in the relational semantic structure. The system generates the relational semantic structure to include the set of clusters. Doing so reduces the number of redundant vertices and makes for a more accurate and powerful relational semantic structure for the task of aggregating descriptors of common meaning.

In some embodiments, the system may generate the relational semantic structure a set of prototype concepts generated based on a distance-based clustering algorithm applied to the set of embeddings. The set of prototype concepts comprises real-value vectors generated from weighted averages of embeddings from the set of embeddings. In one example, the system may identify clusters of embeddings in Embeddings