CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of International Application No. PCT/CN2021/077413, filed on Feb. 23, 2021, the disclosure of which is hereby incorporated by reference in its entirety.

TECHNICAL FIELD

This application relates to the field of artificial intelligence, and in particular, to a data processing method, an apparatus, a device, and a medium.

BACKGROUND

A neural network compiler considers a neural network (NN) model as a computation graph. First, graph compilation is performed, graph topology is analyzed, and computing nodes are converted into tasks on different computing engines, to determine a computation order and form an actual computation stream. Then, operator compilation is performed to generate computation task code that can be run on an accelerator SoC. In an entire compilation process, many optimization and processing passes in graph optimization and operator optimization are involved. Most important optimization is how to efficiently use a system-on-chip high-speed cache & buffer to reduce data access overheads, and reduce a bandwidth limit of external storage access, thereby improving efficiency of data loading and data pipeline computation.

As shown in FIG. 1, FIG. 1 is a diagram of an architecture in which a high-speed cache & buffer run on an SoC. In the buffer, a memory DDR 101 sends data to a processor Engine 103 for computation, and the Engine 103 buffers intermediate data in a Buffer 102 in a computation process, thereby implementing buffering of the intermediate data. In the cache, a Cache 105 is a part of an Engine 106. The Engine 106 exchanges with a DDR 104, and the Engine 106 determines caching of intermediate data in the Cache 105.

In conventional technologies, a large data volume is computed by an operator in a neural network, and a data throughput of a task usually reaches dozens of MB or even more than one GB. However, capacities of the SoC cache & buffer are limited. Therefore, cache & buffer failure caused by a cache miss or a buffer overflow often occurs.

Therefore, the foregoing problems in the conventional technologies still need to be resolved.

SUMMARY

Embodiments of this application provide a data processing method, an apparatus, a device, and a medium, to reduce an intermediate data volume in processing a neural network model, reduce a frequency of data exchange between cache & buffer and external storage, thereby avoiding a cache & buffer overflow.

To resolve the foregoing technical problems, embodiments of this application provide the following technical solutions.

According to a first aspect, an embodiment of this application provides a data processing method, including: splitting a first tensor into at least two first sub-tensors, where the first tensor is a to-be-processed multi-dimensional tensor; determining a target computation order of the at least two first sub-tensors, where the target computation order is a processing order of the at least two first sub-tensors; and processing the at least two first sub-tensors based on the target computation order.

In this embodiment, the first tensor is split into the at least two first sub-tensors, so that a data volume of a first sub-tensor that needs to be processed each time is reduced, and intermediate data is also reduced accordingly. This avoids a cache & buffer overflow, so that in a subsequent working process, a data reuse rate of a high-speed cache & buffer can be increased, a computation data throughput can be increased, and a frequency of data exchange between the cache & buffer and external storage can be reduced.

Optionally, the splitting a first tensor into at least two first sub-tensors includes: inputting the first tensor into a left-hand matrix; splitting the first tensor into N parts along an axis of the left-hand matrix, where N is a positive integer greater than or equal to 2; inputting the first tensor into a right-hand matrix; splitting the first tensor into M parts along an axis of the right-hand matrix, where M is a positive integer greater than or equal to 2; and performing tensor concatenation aggregation on the N parts of the split first tensor and the M parts of the split first tensor to obtain a split first tensor, where the split first tensor includes the at least two first sub-tensors.

In this embodiment, the split first tensor includes the at least two first sub-tensors. Tensor concatenation aggregation needs to be performed on data slices obtained through splitting along the left-hand matrix and data slices obtained through splitting along the right-hand matrix on two different axes when tensors are output, to obtain a plurality of split first sub-tensors. The plurality of first sub-tensors are concatenated to form the first tensor. In this way, operator splitting is implemented in the foregoing manner.

Optionally, the splitting a first tensor into at least two first sub-tensors includes: splitting the first tensor along an axis of the first tensor; and performing tensor reduction aggregation on the split first tensor to obtain a split first tensor, where the split first tensor includes the at least two first sub-tensors.

In this embodiment, the split first tensor includes the at least two first sub-tensors. Two first tensors that are split along a same axis are aggregated, so that operator splitting is implemented, and an obtained split tensor includes at least two first sub-tensors.

Optionally, before the splitting a first tensor into at least two first sub-tensors, the method further includes: obtaining all splitting manners of the first tensor; obtaining all split aggregation streams of the first tensor in a depth direction of a computation graph based on a mapping relationship, where the mapping relationship is a relationship between slices of the split first tensor, and each different splitting manner corresponds to one split aggregation stream; and determining a splitting manner corresponding to a split aggregation stream with a longest transmission distance of the mapping relationship as a target splitting manner; and the splitting a first tensor into at least two first sub-tensors includes: splitting the first tensor into the at least two first sub-tensors in the target splitting manner.

In this embodiment, the split aggregation stream corresponding to each splitting manner has a different transmission distance in the depth direction of the computation graph, and the splitting manner corresponding to the aggregation stream with the longest transmission distance is determined as the target splitting manner. A splitting method corresponding to the splitting manner can enable an operator of the first tensor to be transmitted farther in the depth direction after splitting, to minimize a processing volume of intermediate data and maximize cache & buffer benefit.

Optionally, before the splitting a first tensor into at least two first sub-tensors, the method further includes: obtaining training tensors, where the training tensors and the first tensor are different tensors; determining a plurality of different computation policies based on the training tensors, where the computation policy includes a splitting quantity of the training tensor and a computation order; and generating a target policy model through training based on the different computation policies, where the target policy model includes feedback time of a same training tensor corresponding to a case in which the different computation policies are executed.

In this embodiment, the target policy model is obtained based on the training tensors in a reinforcement learning manner, and the target policy model includes the feedback time of the same training tensor when the different computation policies are executed. In this way, the feedback time of the same training tensor when the different computation policies are executed can be obtained based on the target policy model, so that an optimal computation policy can be determined based on the target policy model when a new tensor is obtained.

Optionally, after the generating a target policy model through training based on the different computation policies, the method further includes: using, in each of a plurality of rounds of iterations, a target training tensor as fixed input that is input into the target policy model, where the target training tensor and the first tensor are different tensors; and optimizing the target policy model based on results output by the target policy model in the plurality of rounds of iterations.

In this embodiment, through a reinforcement learning method, a policy model can be quickly obtained based on a training set, so that a long process of an evolutionary algorithm is not required, thereby improving computation efficiency.

Optionally, the generating a target policy model through training based on the different computation policies includes: encoding the different computation policies as gene sequences; performing iterative verification on a gene sequence of each individual by using each computation policy as an individual; and obtaining a converged optimal solution in iteration results as a computation policy in the target policy model.

n this embodiment, each computation policy is used as an individual, and is sampled as on-board data. A new individual (a computation policy) generated in each iteration is for on-board verification or is input into a simulator for verification, and a direction of population evolution is adjusted based on a feedback result to eventually converge an optimal solution through continuous iterations.

Optionally, the encoding the different computation policies as gene sequences includes: encoding the computation policy in the optimized target policy model as the gene sequence.

In this embodiment, a policy model obtained through a reinforcement learning algorithm is used as an initial gene sequence of an evolutionary algorithm, so that an initial point of the evolutionary algorithm has already been an ideal policy model. This improves a performance lower limit of the evolutionary algorithm. In other words, performance of a first policy model generated by the evolutionary algorithm in a subsequent working process is not lower than performance of the initial policy model. An initial sample of the iteration method is a policy model computed by the reinforcement algorithm. Therefore, initial point selection should be better, so that the evolutionary algorithm can obtain an optimal solution in a quicker and better manner, and the computation efficiency is greatly improved.

Optionally, after the generating a target policy model through training based on the different computation policies, the method further includes: inputting the target policy model into a simulator; and obtaining a feedback result output after the simulator performs data simulation on the target policy model, where the feedback result indicates performance of the target policy model; or inputting the target policy model into a performance predictor; and obtaining a prediction result output by the performance predictor, where the prediction result is for predicting performance of the target policy model.

In this embodiment, the simulator or the performance predictor may predict feedback time based on a preset method, replace actual on-board verification through prediction of the performance predictor, and feed back performance data through a prediction method, to improve the computation efficiency.

Optionally, the method further includes: adding the target policy model to a policy knowledge library; and the determining a target computation order of the at least two first sub-tensors includes: obtaining the target policy model from the policy knowledge library; and obtaining a target computation policy based on the target policy model, where the target computation policy includes a splitting quantity of the first tensor and the target computation order.

In this embodiment, a policy search algorithm inputs the obtained model into the policy knowledge library, so that in a subsequent working process, whenever a new tensor is input, a corresponding computation policy can be directly queried from the policy knowledge library, to determine a splitting quantity of a current vector. This does not require re-execution of the evolutionary algorithm, and shortens time for determining a splitting quantity of each tensor data, thereby improving the computation efficiency.

According to a second aspect, an embodiment of this application provides a data processing apparatus, including:

- a splitting unit, configured to split a first tensor into at least two first sub-tensors, where the first tensor is a to-be-processed multi-dimensional tensor;
- an execution unit, configured to determine a target computation order of the at least two first sub-tensors split by the splitting unit, where the target computation order is a processing order of the at least two first sub-tensors; and
- a processing unit, configured to process the at least two first sub-tensors based on the target computation order determined by the execution unit.

Optionally, the splitting unit is further configured to:

- input the first tensor into a left-hand matrix;
- split the first tensor into N parts along an axis of the left-hand matrix, where N is a positive integer greater than or equal to 2;
- input the first tensor into a right-hand matrix;
- split the first tensor into M parts along an axis of the right-hand matrix, where M is a positive integer greater than or equal to 2; and
- perform tensor concatenation aggregation on the N parts of the split first tensor and the M parts of the split first tensor to obtain a split first tensor, where the split first tensor includes the at least two first sub-tensors.

Optionally, the splitting unit is further configured to:

- split the first tensor along an axis of the first tensor; and
- perform tensor reduction aggregation on the split first tensor to obtain a split first tensor, where the split first tensor includes the at least two first sub-tensors.

Optionally, the apparatus further includes a determining unit, configured to:

- obtain all splitting manners of the first tensor;
- obtain all split aggregation streams of the first tensor in a depth direction of a computation graph based on a mapping relationship, where the mapping relationship is a relationship between slices of the split first tensor, and each different splitting manner corresponds to one split aggregation stream; and
- determine a splitting manner corresponding to a split aggregation stream with a longest transmission distance of the mapping relationship as a target splitting manner.

The splitting unit is further configured to:

- split the first tensor into the at least two first sub-tensors in the target splitting manner.

Optionally, the apparatus further includes a training unit, configured to:

- obtain training tensors, where the training tensors and the first tensor are different tensors;
- determine a plurality of different computation policies based on the training tensors, where the computation policy includes a splitting quantity of the training tensor and a computation order; and
- generate a target policy model through training based on the different computation policies, where the target policy model includes feedback time of a same training tensor corresponding to a case in which the different computation policies are executed.