In some possible embodiments, as shown in FIG. 4, if the target object is a “cube” which is selected, it may be displayed in a “highlighted” state (the “highlighted” state is indicated by an increase in a grayscale value in FIG. 4) to remind the user that the “cube” has been selected.

At step 202, in response to the hand gesture of the hand of the user changing to a predefined movement confirmation gesture, a target display position is determined.

At step 203, the target object to be displayed at the target display position is controlled.

It is to be understood that the predefined movement confirmation gesture is used to indicate the end of the movement of the target object. Therefore, in response to the hand gesture of the hand of the user changing to a predefined movement confirmation gesture, a target display position is determined. Because the target display position is associated with the hand gesture of the user, a visual effect of displaying the target object pulled by the “bare hand” of the user is achieved. The target object is switched to display at the target display position, achieving the movement control of the target object. This control method may be applied in scene building in a “game scene”, or the like. Thus, the process of displaying the target object from being selected to moved may be achieved through a hand gesture operation without the need for a handle device, or the like, which expands the operation mode in the extended reality scene and enhances the operation intelligence.

In the method of object movement control in embodiments of the present disclosure, in response to a hand gesture of a hand of a user being a predefined selection gesture, a target object corresponding to the hand gesture is determined; in response to the hand gesture of the hand of the user changing to a predefined movement confirmation gesture, a target display position is determined; and the target object to be displayed at the target display position is controlled. In this way, movement control of an object according to the hand gesture operation and “bare hand” control of an object are achieved. This further enhances an operating experience of the user while improving the operation intelligence of object movement control.

When controlling the target object with a gesture as described above, in order to realize different control operation experiences, the ways for movement control may be divided into “quick fetch control”, “drag control”, or the like. The following examples illustrate the process of displaying the target object from the initial display position to the target display position in these two ways.

In an embodiment of the present disclosure, in the case that the movement control is in the way of “quick fetch control”, if the target display position is determined in the aforementioned step 202, a current hand position of the hand of the user may be determined as the target display position in response to the hand gesture of the hand of the user changing to a predefined fetch gesture. The predefined fetch gesture may be any predefined hand gesture that signifies “fetch”. In this embodiment, a “sudden” rapid transformation is performed to the target display position. After the hand gesture of the user is detected to be the predefined selection gesture, once the hand gesture of the user changes to the predefined fetch gesture, the current hand position of the hand of the user is determined as the target display position. Then the target object is displayed at the current hand position of the hand of the user, achieving the effect of “fetch” control of the target object.

For example, if the predefined selection gesture and the predefined fetch gesture are as shown in FIG. 5, the corresponding target object may be highlighted when the predefined selection gesture is detected. Then, after the hand gesture of the user changes to the predefined fetch gesture, the target object “cube” may be displayed on the hand of the user. As such, the visual effect of “fetching” the target object is achieved, which improves user experience of operations.

In an embodiment of the present disclosure, if the movement control is in the way of “drag control”, in response to the hand gesture of the hand of the user being a predefined drag gesture and before the hand gesture of the hand of the user changes to the predefined movement confirmation gesture, a real-time movement displacement of the hand of the user is determined, and the target object is moved according to the real-time movement displacement.

That is, in this embodiment, in order to achieve a “drag” effect, the target object is moved along with the real-time movement of the hand of the user, achieving a visual effect of controlling a distant target object in close proximity. Because a real-time displacement is a vector with direction, the distance and direction of the movement of the target object both correspond to the real-time displacement of the hand of the user, achieving the “drag” effect of associated motions visually.

When moving the target object according to the real-time displacement of the hand of the user, the real-time displacement of the target object may be completely consistent with the real-time displacement of the hand of the user, and the real-time displacement of the target object may also maintain a fixed mapping relationship with the real-time displacement of the hand of the user. For example, if the real-time displacement of the hand of the user is a, the real-time displacement of the target object is 2a or the like, which suggests no limitations here.

Thus, during the determination of the target display position in the aforementioned step 202, in response to the hand gesture of the hand of the user changing to a predefined drag-release gesture, the current object position of the target object is determined as the target display position. The predefined drag-release gesture may be any predefined hand gesture action that signifies the end of “dragging”. In this embodiment, after the hand gesture of the user changing to the predefined drag-release gesture is determined, the current object position that follows the real-time displacement of the hand of the user may be determined as the target display position, thereby achieving the “drag” movement control of the target object visually.

In some optional embodiments, during the determination of the target display position in the aforementioned step 202, in response to the hand gesture of the hand of the user changing to a predefined fetch gesture, the current hand position of the hand of the user may be determined as the target display position.

When moving the target object according to the real-time movement displacement, in order to further indicate that the target object is in motion, the target object can be controlled to be display as in a moving state. The moving state may be any display state that identifies that the target object is in motion, such as a “higher transparency” display state.

When moving the target object following the real-time hand position of the hand of the user, in order to further indicate that the target object is in motion, the “drag” movement process may also be indicated by displaying an animation, etc., to further enhance the user experience regarding controlling.

In some optional embodiments, in a case that the predefined drag gesture is a first predefined drag gesture (which may be any predefined hand gesture that identifies a hand gesture for dragging the target object), when moving the target object following the real-time hand position of the hand of the user, a predefined association model of the target object is displayed at a corresponding real-time hand position by following the real-time hand position of the hand of the user. The predefined association model may be a “proportional reduction” model of the target object or any other predefined model corresponding to the target object, such as just a “sphere” model. The predefined association model has a visual correspondence with the target object, and the user may visually perceive the predefined association model as a “shadow” model of the target object.

To enhance the visual association between the predefined association model and the target object, an association animation for the predefined association model and the target object is displayed in real-time between the predefined association model and the target object during the entire “drag” movement process. The association animation may be flexibly set according to the scene, such as a “mapping projection” animation, a “bubble emission” animation, or the like.

To illustrate the entire movement control process of the target object using the first predefined drag gesture more intuitively, a specific application scenario is used as an example.

In this embodiment, as shown in FIG. 6A, in response to the hand gesture of the hand of the user being the selection gesture in the figure and the determined target object is a “cube 1”, when the first predefined drag gesture is a hand gesture where all fingers in the figure are kept together and all fingers of the user are detected to be kept together, the predefined association model of the target object is displayed at the corresponding real-time hand position by following the real-time hand position of the hand of the user. The predefined association model is a proportionally reduced a “cube 2” in the figure. An association animation is displayed between the “cube 1” and the “cube 2”, which is the “mapping animation” between the “cube 1” and the “cube 2” as shown in the figure. As shown, the transparency of the “cube 1” is higher in the moving state, and the “cube 1” is “highlighted” in the selected state.

After the hand gesture of the user is detected to change to the predefined drag-release gesture, the “mapping animation” and the “cube 2” are no longer displayed, and the “cube 1” is restored to the initial display state. The current position of the target object is determined as the target display position, thereby an experience of near control of the distant “cube 1” is achieved based on the display of the “cube 2”.

In some optional implementations, in a case that the predefined drag gesture is a second predefined drag gesture (which may be any other predefined hand gesture that identifies a gesture for dragging the target object), when moving the target object following the real-time hand position of the hand of the user, a first direction indication model is displayed in real-time between the hand of the user and the target object.

A starting point of the first direction indication model corresponds to a real-time hand position of the hand of the user. For example, the starting point of the first direction indication model is set at the real-time hand position of the hand of the user, or a position that is related to the real-time hand position of the hand of the user. An end point of the first direction indication model corresponds to a real-time object position of the target object. For example, the end point of the first direction indication model is set at the real-time object position of the target object, or a position that is related to the real-time object position of the target object.

Because the starting point of the first direction indication model corresponds to the real-time hand position of the hand of the user, and the end point of the first direction indication model corresponds to the real-time object position of the target object, the first direction indication model is always displayed between the hand of the user and the target object during the process of “dragging” the target object, achieving the visual effect of “pulling” to drag. The first direction indication model may be any predefined model that has the visual effect of “pulling” to drag, such as a “ray trajectory model”.

To illustrate the entire movement control process of the target object using the second predefined drag gesture more intuitively, a specific application scenario is used as an example.

In this embodiment, as shown in FIG. 6B, in response to the hand gesture of the hand of the user being the selection gesture in the figure and the determined target object is a “cube 3”, when the second predefined drag gesture is a hand gesture where the index finger and the thumb in the figure are pinched together and the index finger and the thumb of the user are detected to pinched together, the “ray trajectory model” is displayed following the real-time hand position and the real-time object position of the “cube 3”. The “ray trajectory model” is the aforementioned first direction indication model. The starting point of the “ray trajectory model” is located at the real-time hand position, and the end point of the “ray trajectory model” is located at the real-time object position. The transparency of the “cube 3” is higher in the moving state, and the “cube 3” is “highlighted” in the selected state.

After detecting that the hand gesture of the hand of the user changing to the predefined fetch gesture, the “3” is no longer controlled to display as in the moving state, and the “Cube 3” is restored to its initial display state. The current hand position is determined as the target display position, thereby an experience of near control of the distant the “Cube 3” is achieved.

It is to be understood that the predefined selection gesture, the predefined movement confirmation gesture (including the predefined fetch gesture or the predefined drag-release gesture), and the predefined drag gesture mentioned in the embodiments of the present disclosure may be flexibly set according to the needs of the scene.

In some possible embodiments, hand gestures are different from each other. To avoid misjudgment, a corresponding hand gesture operation has a requirement of an identification time order. When a corresponding hand gesture action is identified in a corresponding identification time order, it can be determined that the corresponding hand gesture is effectively identified. For example, the identification time order of the predefined movement confirmation gesture is after identifying the predefined selection gesture, that is, if the corresponding hand gesture action of the predefined movement confirmation gesture is identified after identifying the predefined selection gesture, it is determined that the predefined movement confirmation gesture has been effectively obtained. The identification time order of the predefined drag gesture is after the predefined selection gesture and before the predefined movement confirmation gesture (such as the predefined drag-release gesture), that is, after identifying the predefined selection gesture and before identifying the predefined movement confirmation gesture, the predefined drag gesture is determined to be effectively obtained if a hand gesture action corresponding to the predefined drag gesture is identified.

In some possible embodiments, some of the hand gestures may be the same. Similarly, in this embodiment, in order to avoid misjudgment, there is a need for identification time orders of corresponding hand gesture operations. When a corresponding hand gesture action is identified at the corresponding identification time order, it can be determined that the corresponding hand gesture is effectively identified. In this embodiment, in order to differentiate different hand gestures and improve an accuracy of object movement control, hand gestures at adjacent identification time orders are different. For example, in some application scenarios, the corresponding hand gestures at identification time orders from front to back are the predefined selection gesture, the predefined drag gesture, and the predefined movement confirmation gesture (including the predefined fetch gesture or the predefined drag-release gesture). The predefined selection gesture and the predefined movement confirmation gesture may be the same, the predefined drag gesture is different from the adjacent predefined selection gesture, and the predefined drag gesture is also different from the adjacent predefined movement confirmation gesture. The process of identifying the corresponding gesture in the corresponding identification time order may refer to the above embodiment and will not be elaborated here. The method of object movement control in the embodiments of the present disclosure may flexibly control the movement of the target object through hand gestures according to the scene requirements, which expands the control method of the object and improves the intelligent experience of control.

Based on the above embodiments, a hand gesture of the hand of the user is identified before controlling the movement according to the hand gesture, for example, in response to the hand gesture of the hand of the user being the predefined selection gesture, and in response to the hand gesture of the hand of the user changing to the predefined movement confirmation gesture.

It is to be understood that there are different ways of identifying the hand gesture of the user in different application scenarios, as shown in the following examples.

In some optional embodiments, a hand image for the hand of the user is captured, such as the hand image within the line of sight of the camera is captured through a virtual reality device, etc. The hand gesture is identified according to the hand image. In this embodiment, the hand gesture may be identified based on an image recognition method.

In some optional embodiments, hand key points are predefined. For example, as shown in FIG. 7, the hand key points are defined according to positions of joint points for the hand of the user. Positions of the hand key points for the hand of the user may be identified, and the hand gesture may be identified according to the positions of the hand key points.

A position relationship of the hand key points may be set according to a predefined hand gesture that indicates controlling the target object. As such, the hand gesture may be identified according to the positions of the hand key points.

In some possible embodiments, if the hand gesture that controls the target object is related to a degree of bending for a first predefined finger and an interphalangeal distance between the first predefined finger and a second predefined finger. For example, the first predefined finger is the index finger, and the second predefined finger is the thumb. As shown in FIG. 8, the predefined selection gesture is a hand gesture with a larger distance between a finger and the thumb, and a smaller degree of bending for the index finger; and the predefined drag gesture is a hand gesture with a large degree of bending for the index finger, and an overlap between a finger and the thumb. In this embodiment, the degree of bending for the first predefined finger of the hand of the user is determined according to the positions of hand key points. For example, as shown in FIG. 9, the angle between the line where a finger key point 0 and a finger key point 1 of the first predefined finger are located and the line where a finger key point 2 and a finger key point 3 of the first predefined finger are located may be the degree of bending for the first predefined finger (only the line where the finger key point 0 and the finger key point 1 is located and the line where the finger key point 2 and the finger key point 3 is located are shown in the figure).

In this embodiment, a key point distance between an interphalangeal key point of the first predefined finger and an interphalangeal key point of the second predefined finger is determined according to the positions of hand key points.

The method of object movement control in the embodiments of the present disclosure may flexibly choose a way of identifying a hand gesture according to the scene requirements, which further improves the flexibility of movement control based on the hand gesture.

To implement the above embodiments, an apparatus for object movement control is also proposed in the present disclosure.

FIG. 10 shows a structural schematic diagram of an apparatus for object movement control provided by the embodiments of the present disclosure. The apparatus may be implemented by a software and/or a hardware and is generally integrated in an electronic device to achieve the object movement control. As shown in FIG. 10, the apparatus includes an object determining module 1010, a display position determining module 1020, and a display controlling module 1030.

The object determining module 1010 is configured to determine a target object corresponding to the hand gesture in response to a hand gesture of a hand of a user being a predefined selection gesture.

The display position determining module 1020 is configured to determine a target display position in response to the hand gesture of the hand of the user changing to a predefined movement confirmation gesture.

The display controlling module 1030 is configured to control the target object to be displayed at the target display position.

The apparatus for object movement control provided by the embodiments of the present disclosure may execute the method of object movement control provided by any of the embodiments of the present disclosure and has functional modules and beneficial effects corresponding to the execution of the method, which will not be elaborated here.

In order to implement the above embodiments, a computer program product is also proposed by the present disclosure. The computer program product comprises a computer program/instructions which, when executed by a processor, implement the method of object movement control in the aforementioned embodiments.

FIG. 11 shows a structural schematic diagram of an electronic device provided by the embodiments of the present disclosure.

Referring specifically to FIG. 11, it shows a structural schematic diagram of an electronic device 1100 suitable for implementing the embodiments of the present disclosure. The electronic device 1100 in the embodiments of the present disclosure may include, but is not limited to, a mobile terminal such as a mobile phone, a laptop, a digital broadcast receiver, a personal digital assistant (PDA), a tablet, a portable multimedia player (PMP), and a vehicle-mounted terminal (such as a vehicle-mounted navigation terminal), as well as a fixed terminal such as a digital TV and a desktop computer. The electronic device shown in