12 (FIG. 1) in responding to queries 10 provided by the users 106.

At 202, the LLM pipeline 200 includes obtaining a custom dataset. The LLM pipeline includes a storage layer for the custom dataset. The custom dataset is a curated dataset for an industry or domain of the user. The dataset may include documents with various forms of data (e.g., PDFs, images, text, tables, graphs, etc.). Obtaining the custom dataset includes the identification and collection of relevant documents encompassing a wide range of topics for the industry. In some implementations, the custom dataset is provided by a user.

In some implementations, the custom dataset is automatically obtained by sourcing data from various high-quality repositories such as government agencies, scientific knowledge databases, and proprietary data, if needed, and gathering a diverse and curated dataset pertinent to the industry domain. The data sources selected for sourcing the data contain high-quality, authoritative information on the topic of interest. For example, in agriculture, this includes agricultural and environmental government agencies, scientific knowledge repositories, and agronomist exams databases. In some implementations, web scraping tools (e.g., SCRAPY and BEAUTIFULSOUP) are used to automatically gather the required data for the custom dataset of the identified sources. In some implementations, the user 106 is in a specific location or geographic region and the data obtained is related to the specific location or geographic region. For example, in the case of agricultural data, guidelines and procedures that are geography-specific, i.e., with a shared location among documents, are obtained for the custom dataset.

At 204, the LLM pipeline 200 includes extracting data by extracting text from the custom dataset. In some implementations, data extraction components use a combination of different machine learning based approaches to transform the information in the documents in the custom dataset into a standard format usable by LLMs. For example, the if the documents included in the custom dataset include visual based documents (e.g., PDFs, images, tables, graphs, etc.) the data extraction components extract the information from the documents and coverts the information into text usable by the LLMs.

At 206, the LLM pipeline 200 includes training a LLM model using the custom dataset. In some implementations, the training is used to finetune a LLM model to answer specific questions. In some implementations, the training is used to specialize a LLM model for a specific industry or domain. The training 206 includes generating training data and evaluating the training data.

At 208, the LLM pipeline 200 includes generating questions and answers from the text extracted from the documents and using the questions and answers pairs as training data to train the LLM model. In some implementations, a question and answer generation component use the text extracted from the documents to generate question and answers pairs to create a training dataset to train the LLM model to answer questions based on the dataset. The training dataset generates contextually grounded and high-quality questions that accurately reflect the content of the extracted text. Controlling the structural composition of both inputs and outputs during the training of the LLM model results in outputs that are more precise and exhibit enhanced coherence and contextual relevance.

In some implementations, context-specific prompts are created for the questions that contribute to a heightened level of semantic relevance in the resultant text providing by the LLM model in providing answers to the questions. The content and structure of available documents is augmented by explicitly adding supporting tags from the text. Prompts are created for the questions that extract a list of locations and agronomic topics mentioned in each section of the document (e.g., if that section refers to crops, cattle, or diseases), and task the LLM to answering them based on the data extracted. The additional information used by the LLM to answer the questions, including locations and mentioned topics, grounds the generation process, enhancing the relevance of the questions and guiding the model to cover a wide range of topics and challenges.

In some implementations, the question and answer generation component combines the supporting context and section content, prompting the LLM to generate a set of questions based on the context and section content. The prompt includes a system preamble that guides the LLM into formulating assessment questions related to industrial topics based on the document content. In some implementations, a user portion provides few-shot examples of the types of expected questions, as well as the content and supporting context used for generation. For example, the LLM generates a set of 5 to 15 questions per section of the document. In some implementations, Retrieval-Augmented Generation (RAG) is used by the LLM to create answers in response to the set of questions the LLM generated.

At 210, the LLM pipeline 200 includes evaluating the question and answer pairs and the responses provided by the LLM using the question and answer pairs. The generated pairs of questions and answers are subsequently evaluated and filtered based on quality using a combination of proposed metrics.

In some implementations, the proposed metrics include relevance of the informativeness of a generated question and answer pair from a perspective of the user 106 ensuring that the generated content is relevant and accurate to the target audience, considering all provided information. For example, LLMs (e.g., GPT-4) are used to rate the question on a scale of 1 to 5, with 5 being a question that would be asked by the user and 1 a question that would not, given the context.

In some implementations, the proposed metrics include global relevance to measure the information generated by a question and answer pair from a perspective of a user 106 without considering any context ensuring the generated content is relevant and accurate to a target audience. For example, a LLM (e.g., GPT-4) rates the question and answer pair generated during the training process on a scale of 1 to 5 whether the question would be asked by the user and 1 a question that would not.

In some implementations, the proposed metrics include coverage to gauge whether the generated answers can be directly extracted from the provided context and how well the model avoids hallucinating. For example, LLMs are used to rate the answerability of each question and answer pair on a scale of 1 to 5. A higher score indicates that the answer can be more reliably extracted from the context, ensuring the model's output is grounded in the available information.

In some implementations, the proposed metrices include overlap to assess the semantic similarity between the generated question and the source text/section. For example, Kullback-Leibler (KL) divergence is used as a measure. KL divergence is an asymmetric measure that quantifies the difference between two probability distributions, in this case, the word probability distributions of the source text and the concatenated questions. By leveraging the smoothed counts of words in the texts, allows for a more nuanced comparison of their semantic content. A lower KL divergence value indicates a higher degree of semantic overlap between the questions and the text, which is desired for effective question and answer generation.

In some implementations, the proposed metrics include diversity to evaluate the variety of questions generated by the system. For example, Word Mover's Distance (WMD) is used as a measure for measuring the semantic similarity between two documents or text samples. WMD leverages word embeddings, such as Word2Vec or GloVe, which represent words as high-dimensional vectors capturing their semantic meaning. The embeddings enable the computation of distances between words in a continuous semantic space, allowing for a more nuanced comparison of the meanings of two text samples. A smaller WMD value indicates a higher degree of overlap in meaning between the question and the text, which is desired for effective question and answer generation. A similarity matrix is computed between all generated questions. By calculating the average of the off-diagonal elements in this matrix, a measure of diversity is obtained. A smaller average value indicates greater similarity among questions and, therefore, less diversity.

In some implementations, the proposed metrics include assessing the level of details in both the generated questions and answers by counting the number of tokens (words) in each, providing insight into the depth and specificity of the content generated by the question and answer system.

In some implementations, the proposed metrics include assessing the fluency and coherence of the question and answer pairs. For example, LLMs, such as GPT-4, assess fluency and coherence by preparing a prompt that instructs the LLMs to rate a given question on a scale of 1 to 5 and provide an explanation, the model can generate insightful scores for each question. After appending the generated question to the prompt, submitting it to GPT-4, and parsing the response to extract the fluency score and explanation, the results can be stored for further analysis. Using GPT-4's language understanding capabilities to assess the quality of generated questions, aids in refining question-generation models and selecting the best questions for specific applications.

In some implementations, the proposed metrics include evaluation a coherence of the answers provided to the questions by comparing coherence between ground truths and predictions given the context. The metric provides a score between one to five, where one means that the answer lacks coherence and five means the answer has perfect coherency.

In some implementations, the proposed metrics include evaluation a relevance of the answers provided to the questions by measuring how well the answer addresses the main aspects of the question based on the context. The metric rates from 1 to 5, where 5 means the answer has perfect relevance.

In some implementations, the proposed metrics include evaluating how grounded an answer is to the question. The metric defines weather the answer follows logically from the information contained in the context or not and provides and integer score to whether the answer is grounded.

In some implementations, the proposed metrics include evaluating a succinctness of the answers using a scoring sheet describing what succinct and verbose answers might contain. For example, the scoring sheet is provided as a prompt to a LLM (e.g., GPT-4), the ground truth answer, and the LLM answer, and asked for a grade on a scale from 1 to 5.

In some implementations, the proposed metrics include evaluating a correctness of the answers using a scoring sheet describing what a complete, partially correct, or incorrect answer contains. For example, the scoring sheet is provided as a prompt to a LLM (e.g., GPT-4), the ground truth answer, and the LLM answer, and asked for a grade of correct, incorrect, partially correct.

By employing a combination of the proposed metrics, the evaluation process can effectively evaluate and refine the question and answer generation process, ensuring that the generated content is informative, relevant, diverse, and grounded in the source material leading to a more useful and effective question and answer generation system for the target audience.

At 212, the LLM pipeline 200 includes using the custom dataset to answer questions. The custom dataset is provided as a knowledge resource used by the plugins 104 and/or the LLMs (e.g., the trained LLMs from 206 and/or the LLM 12 in the copilot 102) in the copilot 102 to answer queries 10 from the user 106. In some implementations, Retrieval-Augmented Generation (RAG) is used to query the index to identify the information to provide for use with the LLMs to answer the queries.

At 214, the LLM pipeline 200 includes creating an index for the extracted data from the documents at 204. The index includes embeddings for portions of the extracted data that supports similarity search of vectors. In some implementations, the portions of smaller chunks of the extracted data. For example, a PDF document with 300 pages is divided into 50 page chunks and stored in the index with embeddings for each 50 page chunk of the PDF document. In some implementations, the embeddings are computed from text chunks extracted from the PDF documents in the custom dataset, using sentence transformers.

At 216, the LLM pipeline 200 includes querying the index to retrieve information from the index and provide it the information to an LLM (e.g., the trained LLMs from 206 and/or the LLM 12 in the copilot 102) to answer queries (e.g., the queries 10 provided by the user 106). Given an input question (e.g., the queries 10 or a portion of the queries 10), an embedding is computed for the input question and used to perform a similarity search with score to return relevant chunks from data from the index (e.g., a top number of similar portions of data in the index are returned that are related to the question). The retrieved information (e.g., the chunks of data obtained from the index) provides context to the LLM (e.g., the trained LLMs from 206 and/or the LLM 12 in the copilot 102) in a custom prompt that allows the LLM to generate context specific answers to the queries 10.

By using the LLM pipeline 200 to retrieve information from a custom dataset specific to an industry of the user 106 for responding to the queries 10, the LLMs may provide responses that are grounded to the queries 10 in a specific context of the industry of the user 106.

FIG. 3 illustrates an example architecture 300 for use with the LLM pipeline 200 (FIG. 2). In some implementations, the architecture 300 is used to implement the LLM pipeline 200 and streamline the process of data acquisition and analysis. The architecture 300 includes an orchestrator 302 that defines when each component of the architecture 300 is executed in the LLM pipeline 200. In some implementations, a default order is implemented. For example, the default order is a data extraction component 304, a parser component 306, an LLM metadata component 308, a translation component 314, an embeddings component 310, and index creation component 312. In some implementations, the order is implemented in response to information provided by the user 106.

The data extraction component 304 extracts data from various document formats. In some implementations, the data extraction component 304 is a machine learning library for extracting, parsing, and analyzing bibliographic information from raw documents such as academic papers, journals, and technical reports. In some implementations, the data extraction component 304 is Grobid (GeneRation Of Bibliographic Data), an open-source machine learning library for extracting, parsing, and analyzing bibliographic information from raw documents. Developed primarily for processing PDF files, Grobid leverages a combination of techniques such as CRF (Conditional Random Fields), LSTM (Long Short-Term Memory) neural networks, and other probabilistic models to recognize and structure various components of scholarly documents, including metadata (e.g., titles, authors, affiliations), citation details, and references. By transforming unstructured data into structured XML or TEI (Text Encoding Initiative) formats, Grobid facilitates information retrieval, digital library management, and academic research.

The parser component 306 processes and converts articles from XML or TEI formats into structured data formats, such as, JSON, JSONL, and Markdown, which are understandable and usable by AI models. In some implementations, the parser component 306 is a Grobid parser that parses the XML, extracts metadata, handles content chunking, and transforms tables into CSV format. The Grobid parser also provides a command-line interface for batch processing, leveraging multiprocessing for efficiency, and supports custom chunking strategies and metadata inclusion, facilitating the use of the extracted data for further analysis. In some implementations, two different chunking strategies are implemented: one strategy that uses a text splitter recommended for generic text and is parameterized by a list of characters and chunks the text based on a specific number of tokens. The other strategy chunks the document by the sections provided by Grobid.

The LLM metadata component 308 extracts and organizes metadata from PDF documents. In some implementations, the LLM metadata component 308 leverages OpenAI GPT-4 to extract and organize metadata from PDF documents, and machine learning tables (MLTables) for handling structured data. The LLM metadata component 308 processes an input MLTable of PDF files, extracts content, queries the OpenAI model for metadata on specified topics, and aggregates the results into a structured output. Configuration is managed through command-line arguments, and the script culminates in saving processed data as a JSON Lines file in an MLTable format in the designated output directory.

The translator component 314 interacts with a translation service API to translate text segments from various languages to English. The translator component 314 has methods to detect the language of the input text and to translate a given text segment. In some implementations, the method used by the translator component 314 loads data from an input MLTable, iterates over partitions, detects languages, translates text contents, and logs the status codes of translation requests.

The embeddings component 310 generates embeddings of text documents that capture the semantic meaning of texts. In some implementations, the embeddings component 310 creates embeddings of text documents using the HUGGINGFACEEMBEDDINGS class from the LANGCHAIN library. The embeddings component 310 takes an input MLTable containing text data, applies a pre-trained model to create embeddings for each text entry, and saves the resulting embeddings to a specified output MLTable. The script uses ARGPARSE for command-line interface, allowing users to specify the input MLTable, the embedding model (with a default provided), the device for computation (CPU or GPU), and the output MLTable path. The embeddings component 310 may also leverage external services such as OpenAI embedding models.

The index creation component 312 provides filtering mechanisms throughout the retrieval process that quickly and accurately retrieve the most relevant embeddings for a given query. In some implementations, the index creation component 312 leverages search services to provide the filtering mechanisms throughout the retrieval process. One example search service is AZURE AI SEARCH. The index creation component 312 enables the users 106 to tailor search queries more precisely by including or excluding content from the index based on predefined criteria, either before the search is conducted or as the results are processed. For example, the users 106 can refine searches to focus on documents from certain locations (e.g., the state of Washington or Brazil), or the users 106 can narrow down the search to categories, such as, crop diseases, cattle, or specific types of farming equipment. Another dimension of filtering is by the data source. For example, the users 106 may choose to retrieve content solely from a specific company's tractor manuals. To further enhance the specificity of the search results, the filters may be combined, allowing for complex queries, such as, identifying all documents related to cattle from Washington state that were published after the year 2020. The filtering capabilities are made possible by defining “filterable” fields or columns within the search index. Such fields are populated with metadata extracted from the documents during the indexing pipeline, ensuring that users 106 can find the most relevant information efficiently using search service.

In some implementations, the LLM pipeline 200 includes two paths finetuning (e.g., 206 in FIG. 2) and RAG (e.g., 212 in FIG. 2). Metadata flows between the components (the orchestrator 302, the data extraction component 304, the parser component 306, the LLM metadata component 308, the translation component 314, the embeddings component 310, and the index creation component 312) in the architecture 300 ensuring a coherent and integrated LLM pipeline 200. The metadata helps handle the complex data ingestion, extraction, and question and answer generating processes of the LLM pipeline 200.

In some implementations, the metadata includes pipeline-related metadata with information relating to the component. For example, the pipeline-related metadata includes component name and version that identify the specific components and versions used within the LLM pipeline 200. The component name and version are used for version control, debugging, and system updates. The pipeline-related metadata also includes input arguments that provide specific instructions or parameters that guide the functioning of the components in the LLM pipeline 200, enabling tailored operations based on the data or task.

In some implementations, the metadata includes file-related metadata. The file-related metadata is extracted from the parser component 306 (e.g., a PDF parser). For example, the file-related metadata includes a title of the document, a date of creation, author information, a directory indicating a location where the document is stored, images (visual data or diagrams) included in the document, and user-defined metadata with information known or defined by a user that might ground the LLM. The file-related metadata may be used to filter searches. For example, the user-defined metadata includes assigning a geographic polygon or set of coordinates to a document to filter searches. The file-related metadata may also be used to track when a particular portion of the document has been processed or changed by the LLM pipeline 200.

In some implementations, the metadata includes LLM-generated metadata. One example of LLM-generated metadata is GPT (Generative Pretrained Transformer)-generated metadata. For example, the LLM-generated metadata includes topic and categories with classifications of the content within the document. Examples include specific areas of interest, such as, crop diseases, cattle, or farming equipment. The LLM-generated metadata helps categorize and sort the data for easier retrieval and analysis by the LLM pipeline 220. In some implementations, the areas of interests are defined by the user 106.

FIG. 4 illustrates example questions generated during the question and answer generation portion 208 (FIG. 2) of the LLM pipeline 200 (FIG. 2). The question and answer generation portion of the LLM pipeline 200 combines the supporting context and section content of the data extracted from the custom dataset, prompting an LLM to generate a set of questions based on the context and section content. The prompt includes a system preamble that guides the LLM into formulating assessment questions related to industrial topics based on the document content (e.g., the list of locations and agronomic topics mentioned in the document). For example, the LLM generates the questions 402, 404, 406, 408, 410, 412, 414, 416 in response to the prompt from the data extracted from the custom dataset.

FIG. 5 illustrates an example evaluation (e.g., the evaluation portion 210) of questions generated during the question and answer generation portion 208 (FIG. 2) of the LLM pipeline 200 (FIG. 2). The illustrated evaluation uses the relevance metric to evaluate the informativeness of a generated question and answer pair from a perspective of the user 106 ensuring that the generated content is relevant and accurate to the target audience, considering all provided information. For example, a LLM (e.g., GPT-4) is used to rate the question on a scale of 1 to 5, with 5 being a question that would be asked by the user and 1 a question that would not, given the context. The relevance score 502 is provided with an explanation 504 of the relevance score 502.

FIG. 6 illustrates an example evaluation (e.g., the evaluation portion 210) of answers generated during the question and answer generation portion 208 (FIG. 2) of the LLM pipeline 200 (FIG. 2). The illustrated evaluation uses the succinctness metric evaluating a succinctness of the answers. For example, a scoring sheet describing what succinct and verbose answers might contain is provided as a prompt to a LLM (e.g., GPT-4), the ground truth answer, and the LLM answer, and asked for a grade on a scale from 1 to 5. The LLM provides the scores 602, 606 along with an explanation 604, 608 for the scores 602, 606.

FIG. 7 illustrates an example (e.g., the evaluation portion 210) of answers generated during the question and answer generation portion 208 (FIG. 2) of the LLM pipeline 200 (FIG. 2). The illustrated evaluation uses a correctness metric that evaluates a correctness of the answers. For example, a scoring sheet describing what a complete, partially correct, or incorrect answer contains is provided as a prompt to a LLM (e.g., GPT-4), the ground truth answer, and the LLM answer, and asked for a grade of correct, incorrect, partially correct. In the illustrated example, the LLM provides a score 702 of partially correct with an explanation 704 with reasons supporting why the answer is partially correct.

FIG. 8 illustrates an example output 800 provided by the copilot 102 (FIG. 1) in response to a query 10 provided by the user 106 (FIG. 1). The query 10 references a field (Lowery Field) of the user 106. The LLM 12 (FIG. 1) evaluates the descriptions 14 (FIG. 1) provided by the tool parser 26 (FIG. 1) of the available tools 16 for use with the copilot 102 and determines a match occurs between a description of a tool 16 with a curated dataset for the Lowery Field. The LLM 12 calls a plugin 104 (FIG. 1) with the tool 16 having the information for the Lowery Field in response to the query 10. The LLM 12 uses the information provided by the plugin 104 in response to the tool 16 performing a function (e.g., retrieving information for Lowery Field) and uses the information provided by the plugin 104 to generate a response 802 to the query 10 using the data available for Lowery Field. By using a curated dataset available for use with the plugin 104, the LLM 12 provides the user 106 with a response 802 grounded with relevant factors for Lowery Field.

FIG. 9, illustrates an example method 900 for responding to queries using a copilot 102 (FIG. 1). The actions of the method 900 are discussed below in reference to FIGS. 1-8.

At 902, the method 900 includes receiving, at an LLM, a query and a description of a plurality of tools available for use with the LLM. For example, a user 106 accesses the copilot 102 using a device and the provides a query 10 to the copilot 102. The copilot 102 uses one or more LLMs 12 to understand the queries 10, summarize, predict, and generate content in response to the queries 10.

In some implementations, the copilot 102 uses one or more plugins 104 to enhance the capabilities of the LLM 12 and provide responses to the queries 10 for a specific context of the user 106. Each plugin 104 performs a function or set of functions. Each plugin 104 has a plurality of tools 16 that perform the functions of the plugin 104. Each tool 16 of the plurality of tools performs a function that provides the LLM 12 additional capabilities in providing a response to the query 10. The plugins 104 are dynamically added for use with the copilot 102. In some implementations, a set of the plugins 104 are automatically selected for use with the copilot 102 for a specific industry or domain of the user 106. In some implementations, the plugins 104 are selected by the user 106 for use with the copilot 102. By allowing the dynamic selection or creation of the plugins 104, the user 106 may create a custom copilot 102 to help the LLM 12 provide responses to the queries 10 of the user 106 in a specific context of the user 106.

Descriptions are provided to the LLM 12 that describe the functions provided by the plurality of tools. In some implementations, the tool parser 26 sends a request 18 to a rest API 20 of the plugin 104 and receives from the rest API 20 a set of tools 16 available for the one or more plugins 104 available for use with the copilot 102 and a description of the functions performed by the set of tools 16. The tool parser 26 provides the description 14 of the functions available for the set of tools 16 to the LLM 12. In some implementations, the plurality of tools are included in a plugin 104 remote from the copilot 102 hosting the LLM 12 and the copilot 102 accesses the plugin 104 via a network. In some implementations, the plurality of tools are included in a plurality of plugins 104 remote from the copilot 102 hosting the LLM 12 and the copilot 102 accesses the plurality of plugins 104 via a network. In some implementations, the plugins 104 use a standard format and are created by the users 106 for use with the copilot 102.

In some implementations, the tool 16 is a specialized model trained using a LLM pipeline 200 to answer specific questions by retrieving information from a custom dataset by searching an index of embeddings of the custom dataset and provides the information to the LLM 12 to use in responding to the query 10. The LLM pipeline 200 includes creating a dataset to train the specialized model by generating questions and answers from a custom dataset; using the questions and answers to train the specialized model; and using metrics to evaluate a quality of the questions and answers and a quality of the specialized model.

At 904, the method 900 includes determining, by the LLM, to use a tool in response to the description of the tool matching a portion of the query. The LLM 12 determines to use a tool 16 in response to the description 14 of the tool 16 matching a portion of the query 10.

At 906, the method 900 includes providing a call to the tool with the query. The LLM 12 provides a call 30 to the tool 16 with the query 10. In some implementations, the call 30 includes a subquery generated by the LLM 12 to aid the LLM 12 in responding to the query 10.

At 908, the method 900 includes receiving information from the tool in response to the call. In some implementations, the LLM 12 sends the query 10 in a call 30 to the tool 16 as input to the tool 16. The tool 16 uses the query 10 as input, performs a function on the query 10, and provides a response 22 with the result of the function performed by the tool 16 to the rest API 20. The rest API 20 forwards the response 22 to the copilot 102 for the LLM 12 to use in responding to the query 10. In some implementations, the LLM 12 generates a sub query from the query 10 and sends a sub query in a call 30 as input to the tool 16. The tool 16 performs a function on the sub query and provides a response 22 with the result of the function performed by the tool 16 to the rest API 20.

At 910, the method 900 includes using, by the LLM, the information in providing a response to the query. The rest API 20 forwards the response 22 to the copilot 102 for the LLM 12 to use in responding to the query 10. The LLM 12 uses the information from each tool 16 to provide a response to the query 10. In some implementations, the LLM 12 uses a custom dataset for a context of a user 106 that provided the query 10 in generating a tailored response to the context of the user 106.

In some implementations, the LLM 12 determines to use a set of tools in response to the description of the set of tools matching a portion of the query 10 and the LLM 12 provides a call 30 to each tool 16 of the set of tools with the query 10. The LLM 12 receives the information from each tool 16 in response to each call 30 and uses the information from each tool 16 to provide a response to the query 10. In some implementations, the LLM 12 provides the calls to each tool in parallel. In some implementations, the LLM 12 provides the calls to each tool in a chain.

At 912, the method 900 includes outputting the response to the query. The copilot 102 outputs the response to the query 10. For example, the copilot 102 presents the response on a user interface of a device of the user 106.

The method 900 use the tools 16 to extend the capability of the LLM 12 in the copilot 102 to provide fast responses to the queries 10 provided by the user 106. The LLM 12 in the copilot 102 also uses the tools 16 to provide tailored responses to a context of the user 106 in responding to the queries 10 provided by the user 106.

FIG. 10 illustrates components that may be included within a computer system 1000. One or more computer systems 1000 may be used to implement the various methods, devices, components, and/or systems described herein.

The computer system 1000 includes a processor 1001. The processor 1001 may be a general-purpose single or multi-chip microprocessor (e.g., an Advanced RISC (Reduced Instruction Set Computer) Machine (ARM)), a special purpose microprocessor (e.g., a digital signal processor (DSP)), a graphics processing unit (GPU), a microcontroller, a programmable gate array, etc. The processor 1001 may be referred to as a central processing unit (CPU). Although just a single processor 1001 is shown in the computer system 1000 of FIG. 10, in an alternative configuration, a combination of processors (e.g., an ARM and DSP) could be used.

The computer system 1000 also includes memory 1003 in electronic communication with the processor 1001. The memory 1003 may be any electronic component capable of storing electronic information. For example, the memory 1003 may be embodied as random access memory (RAM), read-only memory (ROM), magnetic disk storage mediums, optical storage mediums, flash memory devices in RAM, on-board memory included with the processor, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM) memory, registers, and so forth, including combinations thereof.

Instructions 1005 and data 1007 may be stored in the memory 1003. The instructions 1005 may be executable by the processor 1001 to implement some or all of the functionality disclosed herein. Executing the instructions 1005 may involve the use of the data 1007 that is stored in the memory 1003. Any of the various examples of modules and components described herein may be implemented, partially or wholly, as instructions 1005 stored in memory 1003 and executed by the processor 1001. Any of the various examples of data described herein may be among the data 1007 that is stored in memory 1003 and used during execution of the instructions 1005 by the processor 1001.

A computer system 1000 may also include one or more communication interfaces 1009 for communicating with other electronic devices. The communication interface(s) 1009 may be based on wired communication technology, wireless communication technology, or both. Some examples of communication interfaces 1009 include a Universal Serial Bus (USB), an Ethernet adapter, a wireless adapter that operates in accordance with an Institute of Electrical and Electronics Engineers (IEEE) 802.11 wireless communication protocol, a Bluetooth® wireless communication adapter, and an infrared (IR) communication port.

A computer system 1000 may also include one or more input devices 1011 and one or more output devices 1013. Some examples of input devices 1011 include a keyboard, mouse, microphone, remote control device, button, joystick, trackball, touchpad, and lightpen. Some examples of output devices 1013 include a speaker and a printer. One specific type of output device that is typically included in a computer system 1000 is a display device 1015. Display devices 1015 used with embodiments disclosed herein may utilize any suitable image projection technology, such as liquid crystal display (LCD), light-emitting diode (LED), gas plasma, electroluminescence, or the like. A display controller 1017 may also be provided, for converting data 1007 stored in the memory 1003 into text, graphics, and/or moving images (as appropriate) shown on the display device 1015.

The various components of the computer system 1000 may be coupled together by one or more buses, which may include a power bus, a control signal bus, a status signal bus, a data bus, etc. For the sake of clarity, the various buses are illustrated in