314 constructs abstract syntax trees (ASTs) from the token-based diff output and produces a syntax-based diff output that shows the added, deleted, or modified AST nodes in PR 200. The AST nodes can represent the syntactic elements of software code 102, such as statements, expressions, variables, types, or functions.

Semantic-based diff algorithm 316 analyzes the semantic meaning and the context of the AST nodes and produces a semantic-based diff output that shows the added, deleted, or modified semantic units. The semantic units can represent the semantic elements of the code, such as control flow, data flow, variable scope, type inference, exception handling, or function calls. Logical change extraction algorithm 318 groups the semantic units into logical changes, which are coherent and meaningful units of code modification that affect the functionality or behavior of the code. A logical change can consist of one or more semantic units that are related by a common purpose, intention, or functionality. A logical change can also be annotated with metadata, such as the author, date, message, or rationale of the change. The set of these is set of logical changes 302.

FIG. 4 illustrates further detail for noise detector 400. Noise detector 400 is able to identify and remove code change noise 402 within logical changes 302 of files 209, and which may include formatting, renaming, refactoring, comments, and/or whitespace changes that do not affect the logic or functionality of software code 102. Noise detector 400 performs several tasks in support of this functionality, such as a reformatting detection task 410, a renaming detection task 412, a refactoring detection task 414, a comment detection task 416, and a whitespace detection task 418. Noise detector 400 may use multiple AI models, each trained specifically for one of the identified tasks, although some examples may be able to perform multiple tasks with a single AI model.

Reformatting detection task 410 detects formatting changes from PR 200, such as indentation, alignment, spacing, or punctuation that do not affect the syntax or the semantics of the code. The formatting detection may use a code formatter or a code style checker to identify and remove the formatting changes. Renaming detection task 412 detects renaming changes in files 209 of PR 200, such as file or variable renames, that do not affect the functionality or the behavior of software code 102. The renaming detector can use a code similarity or a code clone detection technique to identify and remove the renaming changes.

Refactoring detection task 414 detects refactoring changes from PR 200, such as method extraction, method in-lining, method moving, or variable extraction, that do not affect the functionality or the behavior of software code 102. The refactoring detection may use a code analysis or a code transformation technique to identify and remove the refactoring changes. Comment detection task 416 detects comment changes from files 209 of PR 200, such as adding, deleting, or modifying comments that do not affect the functionality or the behavior of software code 102. The comment detection may use natural language processing or a semantic analysis technique to identify and remove the comment changes. Whitespace detection task 418 detects whitespace changes from files 209 of PR 200, such as adding, deleting, or modifying blank lines, tabs, or spaces that do not affect the functionality or the behavior of software code 102. The whitespace detection may use a regular expression or a string comparison technique to identify and remove the whitespace changes.

FIG. 5 illustrates further detail for risk assessor 500. Risk assessor 500 receives set of logical changes 302 in files 209 of software code 102, with code change noise 402 having been removed. A bug data risk assessor 510 of risk assessor 500, which may be an AI model, queries bug history 110 to compare logical changes 302 with the linked PRs of bug history 110 in order to determine bug data risk score 512. A bug data risk reduction action 514 (one or more of them) is identified based on bug data risk score 512, and test cases related to bug data risk may be added to test cases 124.

A code feature risk assessor 520 of risk assessor 500, which may be an AI model, queries PR database 112 to compare logical changes 302 with the linked PRs in order to determine code feature risk score 522. A code feature risk reduction action 524 (one or more of them) is identified based on code feature risk score 522, and test cases related to code feature risk may be added to test cases 124. A test coverage risk assessor 530 of risk assessor 500, which may be an AI model, queries test database 114 to compare the planned test coverage of logical changes 302 with the test coverage of the main branch of code repository 106 to assign a test coverage risk score 532. A test coverage risk reduction action 534 (one or more of them) is identified based on test coverage risk score 532, and test cases identified as missing or needed may be added to test cases 124.

A static code risk assessor 540 of risk assessor 500, which may be an AI model, uses one or more static code analysis tools to assign a static code risk score 542. A static code risk reduction action 544 (one or more of them) is identified based on static code risk score 542, and test cases related to static code risk may be added to test cases 124. A PR comment risk assessor 550 of risk assessor 500, which may be an AI model, assigns a PR comment risk score 552. A PR comment risk reduction action 554 (one or more of them) is identified based on PR comment risk score 552, and test cases related to PR comment risk may be added to test cases 124.

Risk score components 502, which include bug data risk score 512, code feature risk score 522, test coverage risk score 532, static code risk score 542, and PR comment risk score 552 may each comprise a numerical value that reflects the probability of introducing a bug. Risk reduction actions 504, which include bug data risk reduction action 514, code feature risk reduction action 524, test coverage risk reduction action 534, static code risk reduction action 544, and PR comment risk reduction action 554 are each designed to reduce a respective one of bug data risk score 512, code feature risk score 522, test coverage risk score 532, static code risk score 542, and PR comment risk score 552.

Score aggregator 560 aggregates risk score components 502 first by each file of files 209, and then for PR 200 into PR aggregate risk score 506. A file aggregate risk score 562a and a file aggregate risk score 562b are shown to represent two files of files 209.

FIG. 6 illustrates further detail for report generator 600 and risk reduction report 602. In some examples, risk reduction report 602 has a PR title 610, a date 612, and a description 614 of PR 200, in addition to PR aggregate risk score 506, file aggregate risk scores 562a and 562b, risk reduction actions 504, and test cases 124. Report generator 600 may include a generative AI model that generates natural language explanations 620 for the factors affecting PR aggregate risk score 506, file aggregate risk scores 562a and 562b, risk reduction actions 504, and/or test cases 124.

Report generator 600 produces risk reduction report 602 to presentation suggestions and recommendations (e.g., risk reduction actions 504), and test cases 124 to the developers and reviewers of PR 200 (e.g., set of PR reviewers 134) in a clear, concise, and actionable manner. Natural language generation, such as template-based or neural-based, may be used to generate natural language sentences or paragraphs that explain the suggestions, recommendations, and identify the various test cases. Some examples may also use visualization presentations, such as charts or graphs, to provide a visualization of PR aggregate risk score 506, file aggregate risk scores 562a and 562b, and risk reduction actions 504. Risk reduction actions 504 and risk score components 502 may be identified according to each logical change of set of logical changes 302. Links (such as hyperlinks) may be provided as references to the sources of data, such as bug history 110, PR database 112, test database 114, code repository 106, and tools (e.g., the static code analysis tool), to provide support for risk reduction actions 504 and test cases 124.

FIG. 7 illustrates an exemplary training arrangement 700 for components of architecture 100. In some examples, each of logical change identifier 300, noise detector 400, risk assessor 500, risk ranker 120, and training test case selector 122 comprise one or more AI models (or ML models). A trainer 702 has training data 704 that includes labeled files 706 and feedback 150. Training is described in relation to FIG. 8.

FIG. 8 shows a flowchart 800 illustrating exemplary operations that may be performed by architecture 100. In some examples, operations described for flowchart 800 are performed by computing device 1000 of FIG. 10. Flowchart 800 commences with training various AI (ML) powered components of architecture 100 in operation 802. This may include any of training logical change identifier 300 to differentiate between logical changes and non-logical changes in software code (e.g., set of logical changes 302 and non-logical changes 322 in software code 102), training noise detector 400 to identify code change noise (e.g., code change noise 402), training risk assessor 500 to determine risk score components for logical changes in software code (e.g., risk score components 502 for the logical changes in software code 102), training risk ranker 120 to rank risk reduction actions (e.g., risk reduction actions 504), and training test case selector 122 to select test cases (e.g., test cases 124).

Operations 804-808 loop until decision operation 810 determines that set of logical changes 302 are sufficiently finalized to proceed with risk analysis. Operation 804 identifies set of logical changes 302 within software code 102 of PR 200. This may include performing any or all of line-based diff algorithm 310, token-based diff algorithm 312, syntax-based diff algorithm 314, semantic-based diff algorithm 316, and logical change extraction algorithm 318.

Operation 806 identifies code change noise 402 within set of logical changes 302, and operation 808 removes code change noise 402 from set of logical changes 302. In some examples, identifying code change noise 402 comprises performing a noise detection task such as reformatting detection, renaming detection, refactoring detection, comment detection, and whitespace detection. After noise removal, flowchart 800 may return to operation 804, and this cycle may continue until code change noise 402 is sufficiently small that set of logical changes 302 is finalized for risk analysis.

Moving on, operations 812-838 may be looped for risk analysis and reduction until PR 200 is approved in decision operation 836. Operation 812 may be performed using operations 814-818 to determine PR aggregate risk score 506 for set of logical changes 302. Operation 814 determines risk score components such as bug data risk score 512, code feature risk score 522, test coverage risk score 532, static code risk score 542, and PR comment risk score 552 for each of files 209 of PR 200 having a logical change. Risk score components 502 are aggregated for each file into file aggregate risk scores, such as file aggregate risk scores 562a and 562b, in operation 816. Then, these file aggregate risk scores are aggregated into PR aggregate risk score 506 in operation 818, and operation 820 selects test cases 124 for software code 102 of PR 200 using bug history 110.

Risk reduction report 602 is generated in operation 822 using operations 824-828. Operation 824 generates natural language explanations 620 for the factors affecting PR aggregate risk score 506, risk reduction actions 504, and recommended test cases (e.g., test cases 124). Operation 826 ranks risk reduction actions 504, within risk reduction report 602, according to priority, and operation 828 indicates PR aggregate risk score 506, file aggregate risk scores 562a and 562b (color-coded according to risk, in some examples), and test cases 124 for software code 102 of PR 200, within risk reduction report 602.

Set of PR reviewers 134 is curated based on at least identifying set of logical changes 302 and non-logical changes 322 in software code 102 of PR 200, in operation 830. In some examples, this includes selecting, for each file of PR 200 having a logical change, subset 136 of set of PR reviewers 134, based on at least file aggregate risk score 562a or 562b and identification of file-specific reviewers 209a. In some examples, curating set of PR reviewers 134 comprises removing, from set of PR reviewers 134, removed PR reviewer 146, who is associated with only non-logical changes 322 and/or is not associated with set of logical changes 302. This may include moving removed PR reviewer 146 to set of optional PR reviewers 144.

Operation 832 transmits risk reduction report 602 across computer network 1030 to at least a portion of (newly-curated) set of PR reviewers 134, and feedback 150 relating to risk reduction report 602 is received from at least a portion of set of PR reviewers 134 in operation 834. If decision operation 836 determines that PR 200 is approved, flowchart 800 moves on to operation 840. Otherwise, operation 838 performs the risk reduction by altering software code 102 of PR 200 in accordance with risk reduction actions 504 identified within risk reduction report 602. In some examples, this may be performed automatically or semi-automatically, using ML code completion tools. Flowchart 800 then returns to operation 812.

When PR 200 is approved, software code 102 of PR 200 is merged into software code project 104 in operation 840, based on at least altering software code 102 of PR 200 in accordance with risk reduction actions 504 (in operation 838).

Software application 108 (which is now based on at least software code 102 being merged into software code project 104) is deployed in operation 842. Operation 844 performs further training on AI powered components of architecture 100 using feedback 150. For example, logical change identifier 300 is trained further to differentiate between logical changes and non-logical changes in software code, and risk assessor 500 is further trained to determine risk score components.

FIG. 9 shows a flowchart 900 illustrating exemplary operations that may be performed by architecture 100. In some examples, operations described for flowchart 900 are performed by computing device 1000 of FIG. 10. Flowchart 900 commences with operation 902, which includes identifying, within software code of a PR, a set of logical changes. Operation 904 includes determining a PR aggregate risk score for the set of logical changes. Operation 906 includes generating a risk reduction report indicating the PR aggregate risk score, natural language explanations for factors affecting the PR aggregate risk score, and risk reduction actions. Operation 908 includes transmitting the risk reduction report across a computer network to at least a portion of a set of PR reviewers.

Additional Examples

An example system comprises: a processor; and a computer-readable medium storing instructions that are operative upon execution by the processor to: identify, within software code of a PR, a set of logical changes; determine a PR aggregate risk score for the set of logical changes; generate a risk reduction report indicating the PR aggregate risk score, natural language explanations for factors affecting the PR aggregate risk score, and risk reduction actions; and transmit the risk reduction report across a computer network to at least a portion of a set of PR reviewers.

An example computer-implemented method comprises: identifying, within software code of a PR, a set of logical changes; determining a PR aggregate risk score for the set of logical changes; generating a risk reduction report indicating the PR aggregate risk score, natural language explanations for factors affecting the PR aggregate risk score, and risk reduction actions; and transmitting the risk reduction report across a computer network to at least a portion of a set of PR reviewers.

One or more example computer storage devices have computer-executable instructions stored thereon, which, on execution by a computer, cause the computer to perform operations comprising: identifying, within software code of a PR, a set of logical changes; identifying, within the set of logical changes, code change noise; removing, from the set of logical changes, the code change noise; determining a PR aggregate risk score for the set of logical changes; generating a risk reduction report indicating the PR aggregate risk score, natural language explanations for factors affecting the PR aggregate risk score, and risk reduction actions; transmitting the risk reduction report across a computer network to at least a portion of a set of PR reviewers; altering the software code of the PR in accordance with the risk reduction actions identified within the risk reduction report; and based on at least altering the software code of the PR in accordance with the risk reduction actions, merging the software code of the PR into a software code project.

Alternatively, or in addition to the other examples described herein, examples include any combination of the following:

- altering the software code of the PR in accordance with the risk reduction actions identified within the risk reduction report;
- based on at least altering the software code of the PR in accordance with the risk reduction actions, merging the software code of the PR into a software code project;
- identifying, within the set of logical changes, code change noise;
- removing, from the set of logical changes, the code change noise;
- identifying the code change noise comprises performing a noise detection task selected from the set of tasks consisting of: reformatting detection, renaming detection, refactoring detection, comment detection, and whitespace detection;
- identifying the set of logical changes comprises performing a diff algorithm selected from the set of algorithms consisting of: a line-based diff, a token-based diff, a syntax-based diff, a semantic-based diff, and a logical change extraction;
- determining the PR aggregate risk score comprises determining two or more risk score components from the set of risk score components consisting of: a bug data risk score, a code feature risk score, a test coverage risk score, a static code risk score, and a PR comment risk score;
- aggregating the determined risk score components into the PR aggregate risk score;
- for each file of the PR having a logical change, determining two or more risk score components from the set of risk score components;
- aggregating, for each file, the determined risk score components into a file aggregate risk score;
- aggregating the file aggregate risk scores into the PR aggregate risk score;
- selecting, for each file of the PR having a logical change, a subset of the set of PR reviewers, based on at least the file aggregate risk score and an identification of file-specific reviewers;
- ranking, within the risk reduction report, the risk reduction actions according to priority;
- indicating, within the risk reduction report, test cases for the software code of the PR;
- receiving, from at least a portion of the set of PR reviewers, feedback relating to the risk reduction report;
- further training a logical change identifier to differentiate between logical changes and non-logical changes in software code;
- further training a risk assessor to determine risk score components using the feedback;
- training the logical change identifier to differentiate between logical changes and non-logical changes in software code;
- the logical change identifier comprises an AI model;
- training a noise detector to identify code change noise;