540 for CODE4 is created. Since at least one new node was created, the process repeats for the new (third) level of the dependency graph 500. Since the dependency table 410 does not show any dependencies on CODE4, no child nodes of the node 540 are created. No nodes were created in the next (fourth) level of the dependency graph 500, so the dependency graph building process is complete. By reference to the source files identified by the dependency graph 500 and the test files identified by the test table 440, the test selection module 220 of FIG. 2 is enabled to select the test files corresponding to the source files that are, directly or indirectly, affected by a modification to one source file.

Alternatively, the dependency graph 500 may be built by repeatedly searching the text of the source code files. For example, if a change is made to the file 310 of FIG. 3, the root node 510 is created in the dependency graph 500 for the file 310. Then, each other source file can be searched for the text:

- #include “code1.h”

A child node of the node for the file 310 is created for each source file that includes the file 320. After all child nodes of the root node 510 are created (e.g., the child nodes 520 and 530), the source files are searched again, this time for the text:

- #include “code2.h”

Child nodes of the node 520 are created for each matching source file (e.g., the node 540). This process is repeated until no new nodes are generated. To avoid circular graphs, source files that are already represented by nodes in the dependency graph 500 may not be searched again.

FIG. 6 is a block diagram illustrating a user interface 600 for informing a user of errors detected by executing an intelligent test plan, according to some example embodiments. The user interface 600 may be presented to a developer using the client device (160A or 160B) of FIG. 1 to attempt to commit changes to a source code file. The user interface 600 includes a title 610, an informational message 620, and a button 630.

The title 610 indicates that the user interface 600 reports test results. As indicated by the informational message 620, the changes in the source code file CODE1 caused errors in the test file CODE4.TEST. The developer is invited to revise the source code and repeat the commit attempt. The button 630 enables the user to dismiss the user interface 600.

FIG. 7 is a block diagram illustrating a user interface 700 for informing a user that all tests of an intelligent test plan have passed, according to some example embodiments. The user interface 700 may be presented to a developer using the client device 160A of FIG. 1 to attempt to commit changes to a source code file. The user interface 700 includes a title 710, an informational message 720, and a button 730.

The title 710 indicates that the user interface 700 reports test results. As indicated by the informational message 720, the changes in the source code file CODE1 did not cause any test errors.

FIG. 8 is a flowchart illustrating operations of an example method 800 suitable for intelligent test plan generation and execution. The method 800 includes operations 810, 820, 830, 840, 850, and 860. By way of example and not limitation, the method 800 is described as being performed by the change analyzer 130, the test plan generator 140, the test execution system 150, and the plugin for test execution 170, all of FIG. 1, using the modules 210-260 of FIG. 2 and the database schema 400 of FIG. 4.

The change analyzer 130, in operation 810, determines one or more changed files for a software project. For example, a developer using the client device 160A may have uploaded changed files (e.g., using a commit command of a VCS) to the source code server 120. The change analyzer 130 either requests information from the source code server 120 that identifies the changed files or retrieves the files for the software project and compares them to stored versions to determine which files have changed.

In operation 820, the test selection module 220 of the test plan generator 140 generates a dependency graph that identifies a set of files that depend on the one or more changed files for the software project. In some example embodiments, the generating of the dependency graph comprises determining which files import the file containing the change. This may be accomplished by searching the files for the software project to determine if they contain the name of the changed file. Such files are affected by the changed file. The search process may be repeated to build the dependency graph 500. For example, if the file “example.cpp” was modified, files containing “include example.cpp” or “include example.h” are affected by the change.

In other example embodiments, the dependency table 410 may be used to generate the dependency graph 500 of FIG. 5. For example, if the file CODE1 was modified in version 1.1, a root node is created for CODE1 and the dependency data from the rows 430A and 430B is read to determine that CODE2 and CODE3 depend on CODE1. Child nodes in the dependency graph are created for the files that depend on the changed file. The database search process may be repeated to build the dependency graph 500. For example, after adding a child node for CODE2, the dependency data from the row 430C is read to determine that CODE4 depends on CODE2, so a node for CODE4 is added to the dependency graph 500.

A developer may commit changes to multiple files at once. Accordingly, the dependency graph generated in operation 820 may identify files that are affected by changes in multiple files for the software project. Thus, a single dependency graph may be generated that identifies all files affected by changes in multiple files.

Based on the dependency graph, the test selection module 220 identifies a set of tests to be run (operation 830). For example, the test table 440 may be used to determine the tests to run for each file in the generated dependency graph. As another example, test definition files may be searched to determine if they refer to files in the generated dependency graph (e.g., by importing or including the source code files identified by the dependency graph). Thus, the identifying of the set of tests to be run may comprise identifying a set of tests that refer to the set of files identified in the dependency graph.

The set of tests may further include one or more tests that are run regardless of the files identified by the dependency graph. For example, a global configuration file, a project-specific configuration file, or both, may be accessed by the test selection module 220. Before running the set of tests, the test selection module 220 adds the one or more tests identified by the one or more configuration files to the set of tests.

Based on the set of tests to be run, the test plan generator 140 generates a test plan in a first file format (operation 840). For example, the test plan may be generated in a machine-readable file format that is parseable by the plugin for test execution 170. The first file format may be a common file format that is used by different plugins for a variety of test execution systems. The test plan generator 140 may send the test plan in the first file format to the plugin for test execution 170 via a network (e.g., using an API call).

In operation 850, the plugin for test execution 170 converts the test plan from the first file format to a second file format of a test execution system. Different test execution systems may use different file formats for test plans. Accordingly, different plugins for test execution can receive the test plan in the first file format and generate test plans in different second file formats for the particular test execution system being used. For example, the first file format may be an extended markup language (XML) file format that includes an element for each test; the second file format may be a Python script that, when executed, runs the tests and aggregates the results. In these example embodiments, the running of the set of tests comprises running the set of tests by the test execution system using the converted test plan.

The test execution system 150, in operation 860, runs the set of tests to generate test results. For example, each test may generate a Boolean result that indicates whether the test passed or failed. As another example, each test may generate an integer result wherein a zero indicates success and a non-zero value indicates a particular error code.

In view of the above-described implementations of subject matter this application discloses the following list of examples, wherein one feature of an example in isolation or more than one feature of an example, taken in combination and, optionally, in combination with one or more features of one or more further examples are further examples also falling within the disclosure of this application.

Example 1 is a system comprising: a memory that stores instructions; and one or more processors coupled to the memory and configured to execute the instructions to perform operations comprising: determining one or more changed files for a software project; generating a dependency graph that identifies a set of files that depend on the one or more changed files for the software project; based on the dependency graph, identifying a set of tests to be run; based on the set of tests to be run, generating a test plan in a first file format; converting the test plan from the first file format to a second file format of a test execution system; and running the set of tests by the test execution system based on the converted test plan to generate test results.

In Example 2, the subject matter of Example 1, wherein the operations further comprise: accessing a configuration file that identifies a test; and before running the set of tests, adding the identified test to the set of tests.

In Example 3, the subject matter of Examples 1-2, wherein the generating of the dependency graph comprises determining which source code files for the software project import the one or more changed files.

In Example 4, the subject matter of Examples 1-3, wherein the converting of the test plan from the first file format to the second file format of the test execution system is performed by a plugin for the test execution system.

In Example 5, the subject matter of Examples 1-4, wherein the identifying of the set of tests to be run comprises identifying a set of tests that refer to the set of files identified in the dependency graph.

In Example 6, the subject matter of Examples 1-5, wherein the generating of the dependency graph comprises generating a root node for each of the one or more changed files.

In Example 7, the subject matter of Examples 1-6, wherein the determining of the one or more changed files for the software project comprises receiving a notification from a version control system (VCS).

Example 8 is a non-transitory computer-readable medium that stores instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising: determining one or more changed files for a software project; generating a dependency graph that identifies a set of files that depend on the one or more changed files for the software project; based on the dependency graph, identifying a set of tests to be run; and running the set of tests to generate test results.

In Example 9, the subject matter of Example 8, wherein the operations further comprise: accessing a configuration file that identifies a test; and before running the set of tests, adding the identified test to the set of tests.

In Example 10, the subject matter of Examples 8-9, wherein the generating of the dependency graph comprises determining which source code files for the software project import the one or more changed files.

In Example 11, the subject matter of Examples 8-10, wherein the converting of the test plan from the first file format to the second file format of the test execution system is performed by a plugin for the test execution system.

In Example 12, the subject matter of Examples 8-11, wherein the identifying of the set of tests to be run comprises identifying a set of tests that refer to the set of files identified in the dependency graph.

In Example 13, the subject matter of Examples 8-12 includes wherein the generating of the dependency graph comprises generating a root node for each of the one or more changed files.

In Example 14, the subject matter of Examples 8-13, wherein the determining of the one or more changed files for the software project comprises receiving a notification from a version control system (VCS).

Example 15 is a method comprising: determining, by one or more processors, one or more changed files for a software project; generating a dependency graph that identifies a set of files that depend on the one or more changed files for the software project; based on the dependency graph, identifying a set of tests to be run; and running the set of tests to generate test results.

In Example 16, the subject matter of Example 15 includes accessing a configuration file that identifies a test; and before running the set of tests, adding the identified test to the set of tests.

In Example 17, the subject matter of Examples 15-16, wherein the generating of the dependency graph comprises determining which source code files for the software project import the one or more changed files.

In Example 18, the subject matter of Examples 15-17, wherein the converting of the test plan from the first file format to the second file format of the test execution system is performed by a plugin for the test execution system.

In Example 19, the subject matter of Examples 15-18, wherein the generating of the dependency graph comprises generating a root node for each of the one or more changed files.

In Example 20, the subject matter of Examples 15-19, wherein the determining of the one or more changed files for the software project comprises receiving a notification from a version control system (VCS).

Example 21 is an apparatus comprising means to implement any of Examples 1-20.

FIG. 9 is a block diagram 900 showing one example of a software architecture 902 for a computing device. The software architecture 902 may be used in conjunction with various hardware architectures, for example, as described herein. FIG. 9 is merely a non-limiting example of a software architecture and many other architectures may be implemented to facilitate the functionality described herein. A representative hardware layer 904 is illustrated and can represent, for example, any of the above referenced computing devices. In some examples, the hardware layer 904 may be implemented according to the architecture of the computer system of FIG. 9.

The representative hardware layer 904 comprises one or more processing units 906 having associated executable instructions 908. Executable instructions 908 represent the executable instructions of the software architecture 902, including implementation of the methods, modules, subsystems, and components, and so forth described herein and may also include memory and/or storage modules 910, which also have executable instructions 908. Hardware layer 904 may also comprise other hardware as indicated by other hardware 912 which represents any other hardware of the hardware layer 904, such as the other hardware illustrated as part of the software architecture 902.

In the example architecture of FIG. 9, the software architecture 902 may be conceptualized as a stack of layers where each layer provides particular functionality. For example, the software architecture 902 may include layers such as an operating system 914, libraries 916, frameworks/middleware 918, applications 920, and presentation layer 944. Operationally, the applications 920 and/or other components within the layers may invoke application programming interface (API) calls 924 through the software stack and access a response, returned values, and so forth illustrated as messages 926 in response to the API calls 924. The layers illustrated are representative in nature and not all software architectures have all layers. For example, some mobile or special purpose operating systems may not provide a frameworks/middleware 918 layer, while others may provide such a layer. Other software architectures may include additional or different layers.

The operating system 914 may manage hardware resources and provide common services. The operating system 914 may include, for example, a kernel 928, services 930, and drivers 932. The kernel 928 may act as an abstraction layer between the hardware and the other software layers. For example, the kernel 928 may be responsible for memory management, processor management (e.g., scheduling), component management, networking, security settings, and so on. The services 930 may provide other common services for the other software layers. In some examples, the services 930 include an interrupt service. The interrupt service may detect the receipt of an interrupt and, in response, cause the software architecture 902 to pause its current processing and execute an interrupt service routine (ISR) when an interrupt is accessed.

The drivers 932 may be responsible for controlling or interfacing with the underlying hardware. For instance, the drivers 932 may include display drivers, camera drivers, Bluetooth® drivers, flash memory drivers, serial communication drivers (e.g., Universal Serial Bus (USB) drivers), Wi-Fi® drivers, NFC drivers, audio drivers, power management drivers, and so forth depending on the hardware configuration.

The libraries 916 may provide a common infrastructure that may be utilized by the applications 920 and/or other components and/or layers. The libraries 916 typically provide functionality that allows other software modules to perform tasks in an easier fashion than to interface directly with the underlying operating system 914 functionality (e.g., kernel 928, services 930 and/or drivers 932). The libraries 916 may include system libraries 934 (e.g., C standard library) that may provide functions such as memory allocation functions, string manipulation functions, mathematic functions, and the like. In addition, the libraries 916 may include API libraries 936 such as media libraries (e.g., libraries to support presentation and manipulation of various media format such as MPEG4, H.264, MP3, AAC, AMR, JPG, PNG), graphics libraries (e.g., an OpenGL framework that may be used to render two-dimensional and three-dimensional in a graphic content on a display), database libraries (e.g., SQLite that may provide various relational database functions), web libraries (e.g., WebKit that may provide web browsing functionality), and the like. The libraries 916 may also include a wide variety of other libraries 938 to provide many other APIs to the applications 920 and other software components/modules.

The frameworks/middleware 918 may provide a higher-level common infrastructure that may be utilized by the applications 920 and/or other software components/modules. For example, the frameworks/middleware 918 may provide various graphic user interface (GUI) functions, high-level resource management, high-level location services, and so forth. The frameworks/middleware 918 may provide a broad spectrum of other APIs that may be utilized by the applications 920 and/or other software components/modules, some of which may be specific to a particular operating system or platform.

The applications 920 include built-in applications 940 and/or third-party applications 942. Examples of representative built-in applications 940 may include, but are not limited to, a contacts application, a browser application, a book reader application, a location application, a media application, a messaging application, and/or a game application. Third-party applications 942 may include any of the built-in applications as well as a broad assortment of other applications. In a specific example, the third-party application 942 (e.g., an application developed using the Android™ or iOS™ software development kit (SDK) by an entity other than the vendor of the particular platform) may be mobile software running on a mobile operating system such as iOS™, Android™, Windows® Phone, or other mobile computing device operating systems. In this example, the third-party application 942 may invoke the API calls 924 provided by the mobile operating system such as operating system 914 to facilitate functionality described herein.

The applications 920 may utilize built in operating system functions (e.g., kernel 928, services 930 and/or drivers 932), libraries (e.g., system libraries 934, API libraries 936, and other libraries 938), and frameworks/middleware 918 to create user interfaces to interact with users of the system. Alternatively, or additionally, in some systems, interactions with a user may occur through a presentation layer, such as presentation layer 944. In these systems, the application/module “logic” can be separated from the aspects of the application/module that interact with a user.

Some software architectures utilize virtual machines. In the example of FIG. 9, this is illustrated by virtual machine 948. A virtual machine creates a software environment where applications/modules can execute as if they were executing on a hardware computing device. A virtual machine is hosted by a host operating system (operating system 914) and typically, although not always, has a virtual machine monitor 946, which manages the operation of the virtual machine 948 as well as the interface with the host operating system (i.e., operating system 914). A software architecture executes within the virtual machine 948 such as an operating system 950, libraries 952, frameworks/middleware 954, applications 956 and/or presentation layer 958. These layers of software architecture executing within the virtual machine 948 can be the same as corresponding layers previously described or may be different.

Modules, Components and Logic

A computer system may include logic, components, modules, mechanisms, or any suitable combination thereof. Modules may constitute either software modules (e.g., code embodied (1) on a non-transitory machine-readable medium or (2) in a transmission signal) or hardware-implemented modules. A hardware-implemented module is a tangible unit capable of performing certain operations and may be configured or arranged in a certain manner. One or more computer systems (e.g., a standalone, client, or server computer system) or one or more hardware processors may be configured by software (e.g., an application or application portion) as a hardware-implemented module that operates to perform certain operations as described herein.

A hardware-implemented module may be implemented mechanically or electronically. For example, a hardware-implemented module may comprise dedicated circuitry or logic that is permanently configured (e.g., as a special-purpose processor, such as a field programmable gate array [FPGA] or an application-specific integrated circuit [ASIC]) to perform certain operations. A hardware-implemented module may also comprise programmable logic or circuitry (e.g., as encompassed within a general-purpose processor or another programmable processor) that is temporarily configured by software to perform certain operations. It will be appreciated that the decision to implement a hardware-implemented module mechanically, in dedicated and permanently configured circuitry, or in temporarily configured circuitry (e.g., configured by software) may be driven by cost and time considerations.

Accordingly, the term “hardware-implemented module” should be understood to encompass a tangible entity, be that an entity that is physically constructed, permanently configured (e.g., hardwired), or temporarily or transitorily configured (e.g., programmed) to operate in a certain manner and/or to perform certain operations described herein. Hardware-implemented modules may be temporarily configured (e.g., programmed), and each of the hardware-implemented modules need not be configured or instantiated at any one instance in time. For example, where the hardware-implemented modules comprise a general-purpose processor configured using software, the general-purpose processor may be configured as respective different hardware-implemented modules at different times. Software may accordingly configure a processor, for example, to constitute a particular hardware-implemented module at one instance of time and to constitute a different hardware-implemented module at a different instance of time.

Hardware-implemented modules can provide information to, and receive information from, other hardware-implemented modules. Accordingly, the described hardware-implemented modules may be regarded as being communicatively coupled. Where multiples of such hardware-implemented modules exist contemporaneously, communications may be achieved through signal transmission (e.g., over appropriate circuits and buses that connect the hardware-implemented modules). Multiple hardware-implemented modules are configured or instantiated at different times. Communications between such hardware-implemented modules may be achieved, for example, through the storage and retrieval of information in memory structures to which the multiple hardware-implemented modules have access. For example, one hardware-implemented module may perform an operation, and store the output of that operation in a memory device to which it is communicatively coupled. A further hardware-implemented module may then, at a later time, access the memory device to retrieve and process the stored output. Hardware-implemented modules may also initiate communications with input or output devices, and can operate on a resource (e.g., a collection of information).

The various operations of example methods described herein may be performed, at least partially, by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, such processors may constitute processor-implemented modules that operate to perform one or more operations or functions. The modules referred to herein may comprise processor-implemented modules.

Similarly, the methods described herein may be at least partially processor-implemented. For example, at least some of the operations of a method may be performed by one or more processors or processor-implemented modules. The performance of certain of the operations may be distributed among the one or more processors, not only residing within a single machine, but deployed across a number of machines. The processor or processors may be located in a single location (e.g., within a home environment, an office environment, or a server farm), or the processors may be distributed across a number of locations.

The one or more processors may also operate to support performance of the relevant operations in a “cloud computing” environment or as a “software as a service” (SaaS). For example, at least some of the operations may be performed by a group of computers (as examples of machines including processors), these operations being accessible via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., APIs).

Electronic Apparatus and System

The systems and methods described herein may be implemented using digital electronic circuitry, computer hardware, firmware, software, a computer program product (e.g., a computer program tangibly embodied in an information carrier, e.g., in a machine-readable medium for execution by, or to control the operation of, data processing apparatus, e.g., a programmable processor, a computer, or multiple computers), or any suitable combination thereof.

A computer program can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a standalone program or as a module, subroutine, or other unit suitable for use in a computing environment. A computer program can be deployed to be executed on one computer or on multiple computers at one site or distributed across multiple sites (e.g., cloud computing) and interconnected by a communication network. In cloud computing, the server-side functionality may be distributed across multiple computers connected by a network. Load balancers are used to distribute work between the multiple computers. Thus, a cloud computing environment performing a method is a system comprising the multiple processors of the multiple computers tasked with performing the operations of the method.