106, which can then communicate it to the relevant users, such as user 104. This communication may include, for example, notifying the user 104 of the reasons for the failure and workable solutions. The error analysis engine 106 can also send reports to a database 114 for later analysis and tracking, such as by development operations teams, infrastructure teams, information technology teams, etc.

In a further example embodiment, the fix itself can be automatically initiated by the error analysis engine 106 creating an open merge request with a code repository service where the code is published, such as GitHub.

In some example embodiment, the LLM is trained generally to generate text based on a natural language query and contextual information passed to it. As described above, the LLM may be implemented as a generative pre-trained transformer (GPT) model or a bidirectional encoder, but in example embodiments the LLM, regardless of implementation, is trained to generate text regardless of the purpose of the text generation. While contextual information may be passed to the LLM to aid in the generation of text that is relevant to the specific purpose being sought (or environment in which it is being sought), the model itself is trained using training data across many different purposes and environments.

In other example embodiments, however, while the LLM itself is still trained in a general manner using training data across many different purposes and environments, a specific instance of the LLM is fine-tuned to operate more effectively specifically for the purpose and environment of generating code in a declarative language format. This re-trains the LLM to be a special-purpose LLM for declarative language format code generation.

The fine tuning may include obtaining an LLM trained generally (here denoted as a “general LLM”, and then rerunning a machine learning algorithm that was used to train the general LLM using purpose and environment/specific data. In this case, the training data includes data models, such as CAP data models. The machine learning model then retrains the general LLM to produce a special-purpose LLM.

The retraining may include the following operations. First, the training data is cleaned and preprocessed to remove any irrelevant or sensitive information. It is then split into smaller chunks or sequences that the model can process effectively. The model is fine-tuned using a process called unsupervised learning. The primary objective during training is to predict the next word in a sequence given the previous words. This is done using a technique called “masked language modeling” or “cloze task.”

The objective of masked language modeling is to predict missing words in a sentence or sequence of words. During the training process, some words in the input text are randomly masked, and the model then predicts the masked words based on the context provided by the surrounding words. To accomplish masked language modeling, various operations are performed: masking, context-based prediction, loss calculation, and finally backpropagation and updating. For masking, in the input text or sequence of words, a certain percentage of words is randomly selected and replaced with a special token. In context-based prediction, the LLM then processes the masked input and uses the context from the surrounding words to predict the masked word. The model considers the relationships between words, the grammar of the language, and the overall semantics to make an informed guess about the masked word.

In loss calculation, the model's prediction for the masked word is compared to the actual word (the original word that was masked) using a loss function, such as cross-entropy loss. The loss function quantifies the difference between the predicted probability distribution over words and the true distribution (one-hot encoded vector representing the actual word). Finally, for backpropagation and Update, the loss is backpropagated through the model, and the model's parameters (weights and biases) are updated to minimize the loss and improve the model's ability to predict masked words accurately.

As far as cloze task, this involves the model being presented with a passage of text with certain words or phrases removed, and the model is challenged to fill in the missing words based on the context provided.

Returning now to the retraining process, the model is exposed to the preprocessed data and learns from it iteratively over multiple training iterations. During each iteration, the model adjusts its internal parameters (weights and biases) to minimize the prediction errors and improve its performance. The training process involves optimization algorithms like Stochastic Gradient Descent (SGD) or its variants, which fine-tune the model's parameters to make it better at predicting the next word in a sequence.

Hyperparameter Tuning can then be performed where various hyperparameters (e.g., learning rate, batch size, number of layers) are tuned to find the best configuration that results in optimal performance and generalization.

FIG. 4 is a flow diagram illustrating a method 400, in accordance with an example embodiment. At operation 402, an identification of an error that occurred during operation of the CI/CD pipeline on first software code created by a first user and an event log containing events occurring during the operation are received from a continuous integration/continuous deployment (CI/CD) pipeline. At operation 404, an error analysis engine is used on the identification of the error and the event log to identify a root cause of the error.

At operation 406, a model determination component is used to determine whether to utilize an internal artificial intelligence (AI) tool or an external AI tool, the internal AI tool operated by an entity operating the CI/CD pipeline.

At operation 408, in response to a determination that the external AI tool should be used, the root cause of the error is passed to the external AI tool, the external AI tool being a Generative Artificial Intelligence (GAI) model that generates one or more solutions to the root cause of the error. At operation 410 the one or more solutions are caused to be displayed to the first user in a user interface.

In view of the disclosure above, various examples are set forth below. It should be noted that one or more features of an example, taken in isolation or combination, should be considered within the disclosure of this application.

Example 1. A system comprising:

- at least one hardware processor; and
- a computer-readable medium storing instructions that, when executed by the at least one hardware processor, cause the at least one hardware processor to perform operations comprising:
- receiving, from a continuous integration/continuous deployment (CI/CD) pipeline, an identification of an error that occurred during operation of the CI/CD pipeline on first software code created by a first user and an event log containing events occurring during the operation;
- using an error analysis engine on the identification of the error and the event log to identify a root cause of the error;
- using a model determination component to determine whether to utilize an internal artificial intelligence (AI) tool or an external AI tool, the internal AI tool operated by an entity operating the CI/CD pipeline, the external AI tool operated by an entity other than the entity operating the CI/CD pipeline;
- in response to a determination that the external A I tool should be used, passing the root cause of the error to the external AI tool, the external AI being a Generative Artificial Intelligence (GAI) model that generates one or more solutions to the root cause of the error; and
- causing the one or more solutions to be displayed to the first user in a user interface.

Example 2 The system of Example 1, wherein the passing further includes contextual information regarding the error to the GAI model.

Example 3. The system of Example 2, wherein the contextual information includes an identification of the language in which the first software code was generated.

Example 4. The system of any of Examples 1-3, wherein the error analysis engine is a first machine learning model trained by a first machine learning algorithm.

Example 5. The system of Example 4, wherein the first machine learning model is trained by extracting features from sample labeled event logs and passing the features and labels from the sample labeled event logs to the first machine learning algorithm.

Example 6. The system of Example 5, wherein the first machine learning model is retrained based on input by the user in response to the causing of the display.

Example 7. The system of any of Examples 1-6, wherein the model determination component is a second machine learning model trained by a second machine learning algorithm.

Example 8. A method comprising:

- receiving, from a continuous integration/continuous deployment (CI/CD) pipeline, an identification of an error that occurred during operation of the CI/CD pipeline on first software code created by a first user and an event log containing events occurring during the operation;
- using an error analysis engine on the identification of the error and the event log to identify a root cause of the error;
- using a model determination component to determine whether to utilize an internal artificial intelligence (AI) tool or an external AI tool, the internal AI tool operated by an entity operating the CI/CD pipeline, the external AI tool operated by an entity other than the entity operating the CI/CD pipeline;
- in response to a determination that the external AI tool should be used, passing the root cause of the error to the external AI tool, the external AI being a Generative Artificial Intelligence (GAI) model that generates one or more solutions to the root cause of the error; and
- causing the one or more solutions to be displayed to the first user in a user interface.

Example 9 The method of Example 8, wherein the passing further includes contextual information regarding the error to the GAI model.

Example 10. The method of Example 9, wherein the contextual information includes an identification of the language in which the first software code was generated.

Example 11. The method of any of Examples 8-10, wherein the error analysis engine is a first machine learning model trained by a first machine learning algorithm.

Example 12. The method of Example 11, wherein the first machine learning model is trained by extracting features from sample labeled event logs and passing the features and labels from the sample labeled event logs to the first machine learning algorithm.

Example 13. The method of Example 12, wherein the first machine learning model is retrained based on input by the user in response to the causing of the display.

Example 14. The method of any of Examples 8-13, wherein the model determination component is a second machine learning model trained by a second machine learning algorithm.

Example 15. A non-transitory machine-readable medium storing instructions which, when executed by one or more processors, cause the one or more processors to perform operations comprising:

- receiving, from a continuous integration/continuous deployment (CI/CD) pipeline, an identification of an error that occurred during operation of the CI/CD pipeline on first software code created by a first user and an event log containing events occurring during the operation;
- using an error analysis engine on the identification of the error and the event log to identify a root cause of the error;
- using a model determination component to determine whether to utilize an internal artificial intelligence (AI) tool or an external AI tool, the internal AI tool operated by an entity operating the CI/CD pipeline, the external AI tool operated by an entity other than the entity operating the CI/CD pipeline;
- in response to a determination that the external AI tool should be used, passing the root cause of the error to the external AI tool, the external AI being a Generative Artificial Intelligence (GAI) model that generates one or more solutions to the root cause of the error; and
- causing the one or more solutions to be displayed to the first user in a user interface.

Example 16 The non-transitory machine-readable medium of Example 15, wherein the passing further includes contextual information regarding the error to the GAI model.

Example 17. The non-transitory machine-readable medium of Example 17, wherein the contextual information includes an identification of the language in which the first software code was generated.

Example 18. The non-transitory machine-readable medium of any of Examples 15-17, wherein the error analysis engine is a first machine learning model trained by a first machine learning algorithm.

Example 19. The non-transitory machine-readable medium of Example 18, wherein the first machine learning model is trained by extracting features from sample labeled event logs and passing the features and labels from the sample labeled event logs to the first machine learning algorithm.

Example 20. The non-transitory machine-readable medium of Example 19, wherein the first machine learning model is retrained based on input by the user in response to the causing of the display.

FIG. 5 is a block diagram 500 illustrating a software architecture 502, which can be installed on any one or more of the devices described above. FIG. 5 is merely a non-limiting example of a software architecture, and it will be appreciated that many other architectures can be implemented to facilitate the functionality described herein. In various embodiments, the software architecture 502 is implemented by hardware such as a machine 600 of FIG. 6 that includes processors 610, memory 630, and input/output (I/O) components 650. In this example architecture, the software architecture 502 can be conceptualized as a stack of layers where each layer may provide a particular functionality. For example, the software architecture 502 includes layers such as an operating system 504, libraries 506, frameworks 508, and applications 510. Operationally, the applications 510 invoke API calls 512 through the software stack and receive messages 514 in response to the API calls 512, consistent with some embodiments.

In various implementations, the operating system 504 manages hardware resources and provides common services. The operating system 504 includes, for example, a kernel 520, services 522, and drivers 524. The kernel 520 acts as an abstraction layer between the hardware and the other software layers, consistent with some embodiments. For example, the kernel 520 provides memory management, processor management (e.g., scheduling), component management, networking, and security settings, among other functionalities. The services 522 can provide other common services for the other software layers. The drivers 524 are responsible for controlling or interfacing with the underlying hardware, according to some embodiments. For instance, the drivers 524 can include display drivers, camera drivers, BLUETOOTH® or BLUETOOTH® Low-Energy drivers, flash memory drivers, serial communication drivers (e.g., Universal Serial Bus (USB) drivers), Wi-Fi® drivers, audio drivers, power management drivers, and so forth.

In some embodiments, the libraries 506 provide a low-level common infrastructure utilized by the applications 510. The libraries 506 can include system libraries 530 (e.g., C standard library) that can provide functions such as memory allocation functions, string manipulation functions, mathematic functions, and the like. In addition, the libraries 506 can include API libraries 532 such as media libraries (e.g., libraries to support presentation and manipulation of various media formats such as Moving Picture Experts Group-4 (MPEG4), Advanced Video Coding (H.264 or AVC), Moving Picture Experts Group Layer-3 (MP3), Advanced Audio Coding (AAC), Adaptive Multi-Rate (AMR) audio codec, Joint Photographic Experts Group (JPEG or JPG), or Portable Network Graphics (PNG)), graphics libraries (e.g., an OpenGL framework used to render in two dimensions (2D) and three dimensions (3D) in a graphic context on a display), database libraries (e.g., SQLite to provide various relational database functions), web libraries (e.g., WebKit to provide web browsing functionality), and the like. The libraries 506 can also include a wide variety of other libraries 534 to provide many other APIs to the applications 510.

The frameworks 508 provide a high-level common infrastructure that can be utilized by the applications 510, according to some embodiments. For example, the frameworks 508 provide various GUI functions, high-level resource management, high-level location services, and so forth. The frameworks 508 can provide a broad spectrum of other A PIs that can be utilized by the applications 510, some of which may be specific to a particular operating system 504 or platform.

In an example embodiment, the applications 510 include a home application 550, a contacts application 552, a browser application 554, a book reader application 556, a location application 558, a media application 560, a messaging application 562, a game application 564, and a broad assortment of other applications, such as a third-party application 566. According to some embodiments, the applications 510 are programs that execute functions defined in the programs. Various programming languages can be employed to create one or more of the applications 510, structured in a variety of manners, such as object-oriented programming languages (e.g., Objective-C, Java, or C++) or procedural programming languages (e.g., C or assembly language). In a specific example, the third-party application 566 (e.g., an application developed using the ANDROID™ or IOS™ software development kit (SDK) by an entity other than the vendor of the particular platform) may be mobile software running on a mobile operating system such as IOS™, ANDROID™, WINDOWS® Phone, or another mobile operating system. In this example, the third-party application 566 can invoke the API calls 512 provided by the operating system 504 to facilitate functionality described herein.

FIG. 6 illustrates a diagrammatic representation of a machine 600 in the form of a computer system within which a set of instructions may be executed for causing the machine 600 to perform any one or more of the methodologies discussed herein, according to an example embodiment. Specifically, FIG. 6 shows a diagrammatic representation of the machine 600 in the example form of a computer system, within which instructions 616 (e.g., software, a program, an application, an applet, an app, or other executable code) for causing the machine 600 to perform any one or more of the methodologies discussed herein may be executed. For example, the instructions 616 may cause the machine 600 to execute the method 400 of FIG. 4. Additionally, or alternatively, the instructions 616 may implement FIGS. 1-4 and so forth. The instructions 616 transform the general, non-programmed machine 600 into a particular machine 600 programmed to carry out the described and illustrated functions in the manner described. In alternative embodiments, the machine 600 operates as a standalone device or may be coupled (e.g., networked) to other machines. In a networked deployment, the machine 600 may operate in the capacity of a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine 600 may comprise, but not be limited to, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), a personal digital assistant (PDA), an entertainment media system, a cellular telephone, a smart phone, a mobile device, a wearable device (e.g., a smart watch), a smart home device (e.g., a smart appliance), other smart devices, a web appliance, a network router, a network switch, a network bridge, or any machine capable of executing the instructions 616, sequentially or otherwise, that specify actions to be taken by the machine 600. Further, while only a single machine 600 is illustrated, the term “machine” shall also be taken to include a collection of machines 600 that individually or jointly execute the instructions 616 to perform any one or more of the methodologies discussed herein.

The machine 600 may include processors 610, memory 630, and I/O components 650, which may be configured to communicate with each other such as via a bus 602. In an example embodiment, the processors 610 (e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a radio-frequency integrated circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, a processor 612 and a processor 614 that may execute the instructions 616. The term “processor” is intended to include multi-core processors that may comprise two or more independent processors (sometimes referred to as “cores”) that may execute instructions