400 may structure a field to provide guidance to a user to include required information in a prompt. For instance, a field may include initial text which may define a structure of the prompt and sub-fields where a user may enter information to complete the prompt. In cases where information is extracted from a workflow document, the sub-fields may include initial text, which may be edited by a user.

A field 430 for the first prompt 332 includes an introduction of an input data format followed by a sub-field 432 for defining the input data format. The field 430 further includes an introduction to the second format followed by a sub-field 434 for defining the second format. The field 430 also includes an introduction to transform rules followed by a sub-field 436 for defining the transform rules.

A field 440 for the second prompt 342 may include initial text with a picklist 442 to select a number of steps. The field 440 may then include an introduction to each step followed by one or more sub-fields for defining the respective step. In some implementations, the sub-fields may include a purpose of the step 444, an input schema definition 446, an operation 448, and a rule 450. For example, as illustrated for the first step, the purpose of the step 444 is “Call account API to get user context.” The input schema definition 446 defines a JSON schema. The operation 448 defines a mapping of the input schema to an API data contract. The rule 450 defines a relationship between a property of the JSON schema and the API data contract. As another example, for the second step, the purpose of the step 444 is “Execute local code to generate a file with xml format.” The input schema definition 446 defines the same input schema as in the first step. The operation 448 defines data processing. The rule 450 defines business logic for processing the input data.

In some implementations, the user interface 400 provides a template for defining the steps. For instance, the third step includes the introduction to the input schema, but the sub-fields include descriptions of the sub-fields rather than initial text. Accordingly, the user interface 400 provides structure for generating a prompt for performing a step of a workflow.

The user interface 400 in FIG. 4B includes fields 410 for generating prompts to perform the business logic of individual steps. For example, the field 460 may define a prompt to implement the first step by posting JSON data to a REST API endpoint using Java. In some implementations, the LLM 140 may generate the Java code based on a more general prompt that defines the platform specific requirements such as data model, API type, and location. As another example, the field 470 may include a prompt to execute a local executable file on the input schema.

FIG. 5 is a schematic diagram of an example of an apparatus 500 (e.g., a computing device) for implementing a workflow using the LLM 140. The apparatus 500 may be implemented as one or more computing devices in the system 120.

In an example, the apparatus 500 includes at least one processor 502 and a memory 504 configured to execute or store instructions or other parameters related to providing an operating system 506, which can execute one or more applications or processes, such as, but not limited to, the workflow automation application 130. For example, processors 502 and memory 504 may be separate components communicatively coupled by a bus (e.g., on a motherboard or other portion of a computing device, on an integrated circuit, such as a system on a chip (SoC), etc.), components integrated within one another (e.g., a processor 502 can include the memory 504 as an on-board component), and/or the like. Memory 504 may store instructions, parameters, data structures, etc. for use/execution by processor 502 to perform functions described herein. In some implementations, the memory 504 includes the database 552 for use by the workflow automation application 130. In some implementations, the apparatus 500 includes the LLM 140, for example, as another application executing on the processors 502. Alternatively, the LLM 140 may be executed on a different device that may be accessed via an API 550.

In an example, the workflow automation application 130 includes the user interface 132, the document analyzer 134, the prompt verifier 136, the saved context 138, and the local executables 139 discussed above with respect to FIG. 1.

The workflow automation application 130 may further include a data parser or transformer 330 configured to send a first prompt to a large language model (LLM) to transform a first input data source in a first format to a second format. The workflow automation application 130 may further include a workflow orchestration component 340 configured to send a second prompt to the LLM to define multiple steps of a workflow starting on the data source in the second format. The workflow automation application 130 may further include a business logic component 350 configured to send a third prompt to the LLM to define execution of business logic for each step of the workflow. The workflow automation application 130 may be configured to receive, from the LLM, output data indicating that each step of the workflow has been executed.

In some implementations, the apparatus 500 is implemented as a distributed processing system, for example, with multiple processors 502 and memories 504 distributed across physical systems such as servers, virtual machines, or datacenters 122. For example, one or more of the components of the workflow automation application 130 may be implemented as services executing at different datacenters 122. The services may communicate via an API.

FIG. 6 is a flow diagram of an example of a method 600 for executing a workflow using a large language model. For example, the method 600 can be performed by the workflow automation application 130, the apparatus 500 and/or one or more components thereof to automate workflows using the LLM 140.

At block 610, the method 600 may optionally include receiving a workflow document and a prompt to extract the first format, the second format, and the multiple steps. For example, in an aspect, apparatus 500, processor 502, memory 504, and/or workflow automation application 130 may be configured to or may comprise means for receiving a workflow document and a prompt to extract the first format, the second format, and the multiple steps. For example, the workflow automation application 130 may receive the workflow document 114 from the user device 110. The workflow automation application 130 may add a prompt for the LLM 140 to extract the first format (e.g., of input data source 210), the second format (e.g., a transformed format 334), and the multiple steps.

Referring to an alternative or additional aspect, at block 620, the method 600 may further include providing a user interface with fields to enter the first prompt, the second prompt, and the third prompt. For example, in an aspect, apparatus 500, processor 502, memory 504, workflow automation application 130, and/or user interface 132 may be configured to or may comprise means for providing a user interface 400 with fields 410 to enter the first prompt 332, the second prompt 342, and the third prompt 352. In some implementations, the user interface 400 may prepopulate the fields 410 based on information extracted from the workflow document 114. For example, the user interface may insert the first format into the sub-field 432 and insert the second format into the sub-field 434.

Referring to an alternative or additional aspect, at block 630, the method 600 may further include verifying whether text entered into each of the fields includes required content for each prompt. For example, in an aspect, apparatus 500, processor 502, memory 504, workflow automation application 130, and/or prompt verifier 136 may be configured to or may comprise means for verifying whether text entered into each of the fields 410 includes required content for each prompt. For example, the prompt verifier 136 may ensure that each sub-field includes content that matches a rule for the sub-field.

At block 640, the method 600 includes receiving a first prompt at the LLM to transform a first input data source in a first format to a second format. For example, in an aspect, apparatus 500, processor 502, memory 504, and/or the LLM 140 may be configured to or may comprise means for receiving the first prompt 332 at the LLM 140 to transform a first input data source 210 in a first format to a second format. For example, the LLM 140 may receive the first prompt 332 from the workflow automation application 130 and/or the data parser or transformer 330. For example, the data parser or transformer 330 may output the first prompt 332 based on the field 430. The workflow automation application 130 may call the API 550 for the LLM 140 with the first prompt 332. In some implementations, the first prompt 332 defines an input data format (e.g., in sub-field 432), an output data format (e.g., in sub-field 434), and a transformation rule (e.g., in sub-field 436).

At block 650, the method 600 may optionally include outputting a transformed data set in the output data format to a local memory. For example, in an aspect, apparatus 500, processor 502, memory 504, and/or the LLM 140 may be configured to or may comprise means for outputting a transformed data set in the output data format to a local memory (e.g., memory 504 and/or saved context 138). For example, the LLM 140 may output the transformed data set in the output data format to the workflow automation application 130 for storage and local operations. For instance, the workflow automation application 130 may select data from the transformed data set when generating the second prompt 342 and/or the third prompt 352.

At block 660, the method 600 includes receiving a second prompt at the LLM to define multiple steps of a workflow starting on the data source in the second format. For example, in an aspect, apparatus 500, processor 502, memory 504, and/or the LLM 140 may be configured to or may comprise means for receiving the second prompt 342 at the LLM to define multiple steps of a workflow starting on the data source in the second format. For example, the LLM 140 may receive the second prompt 342 from the workflow automation application 130 and/or the workflow orchestration component 340. For example, the workflow orchestration component 340 may output the second prompt 342 based on the field 440. The workflow automation application 130 may call the API 550 for the LLM 140 with the second prompt 342. In some implementations, the second prompt 342 identifies, for a first step, a property of the input data source in the second format (e.g., input schema definition 446) and an operation on the property (e.g., operation 448). In some implementations, the second prompt 342 identifies a rule for performing the operation (e.g., rule 450).

At block 670, the method 600 includes receiving a third prompt at the LLM to define execution of business logic for each step of the workflow. For example, in an aspect, apparatus 500, processor 502, memory 504, and/or the LLM 140 may be configured to or may comprise means for receiving the third prompt 352 at the LLM 140 to define execution of business logic for each step of the workflow. For example, the LLM 140 may receive the third prompt 352 from the workflow automation application 130 and/or the business logic component 350. For example, the business logic component 350 may output the third prompt 352 based on the field 460 or 470 corresponding to a respective step of the workflow. The workflow automation application 130 may call the API 550 for the LLM 140 with the third prompt 352. In some implementations, the third prompt 352 calls for execution of code via one or more of an API 252, local code 254, or an external executable 256. The LLM 140 may determine how to execute the code on the respective execution platform 320.

Referring to an alternative or additional aspect, at block 680, the method 600 may further include nesting the third prompt within a corresponding step of the second prompt. For example, in an aspect, apparatus 500, processor 502, memory 504, the workflow automation application 130, and/or the LLM 140 may be configured to or may comprise means for nesting the third prompt 352 within a corresponding step of the second prompt 342. In some implementations, the workflow automation application 130 may nest the third prompt 352 prior to sending the second prompt 342 to the LLM 140. In other implementations, when orchestrating the workflow according to the second prompt 342, the LLM 140 may locate the third prompt 352 and nest the third prompt 352 at the corresponding step of the workflow.

At block 690, the method 600 includes executing, by the LLM, each step of the workflow. For example, in an aspect, apparatus 500, processor 502, memory 504, and/or the LLM 140 may be configured to or may comprise means for executing, by the LLM 140, each step of the workflow. For example, the LLM 140 may evaluate each of the first prompt 332, the second prompt 342, and the third prompt 352. When executing the second prompt 342, the LLM 140 may store the output of a previous step as saved context 138 and/or return the output to the workflow automation application 130 for inclusion in a subsequent prompt (e.g., a third prompt 352). In some implementations, executing a step of a workflow may include executing code as defined by the third prompt 352. The LLM 140 may output the output data 360 to the workflow automation application 130 after executing each step of the workflow.

FIG. 7 illustrates an example of a device 700 including additional optional component details as those shown in FIG. 5. In one aspect, device