^I.

In step 304, the IMU is read out and provides rotational movements around the Pitch, Yaw and Roll directions, i.e. X_IMU, Y_IMUand Z_IMUrespectively. Usually, IMU provides data on the angular acceleration which is to be integrated for determining the rotation angle. The IMU data may be used to calculate the undesired rotational motion of the device.

In step 306, a coordinate transformation is performed. The coordinate transformation is required because the STC's POV change caused by an undesired rotational motion of the device and the sensing of the undesired rotational motion occur in different coordinate systems.

A processing unit such as an AP or a MCU may be configured for performing the coordinate transformation (e.g. AP 440 of device 400 or device 480, or MCU 470 of device 400 in FIG. 4A). In some examples, an AP or MCU may solve the below equations analytically, or AP or MCU may use a polynomial fit or a linear fit for solving the equations approximately. In other examples, the AP or MCU may not perform calculations but use a Look Up Table (LUT) for coordinate transformation. In some examples and such as e.g. shown in FIG. 4A, the coordinate transformation may be performed by a MCU such as MCU 470 connected to the STC module 410.

In some examples, the transformation may be performed in order to express the coordinates of the STC in the coordinate system of the IMU. Device rotations and compensation motions may then be calculated in the IMU's coordinate system.

In some examples, a 2D chart such as chart 320 shown in FIG. 3B may be used to express the coordinates of the STC in the IMU's coordinate system. Chart 320 may resemble a calibration chart for calibrating the STC or for calibrating a dual-camera, e.g. including a Tele camera and a Wide camera. STC aperture 322 may be located at (0, 0, 0). The handheld device may be pointed towards chart 320 in “landscape” direction, i.e. with reference to the coordinate system of FIG. 3B, the long side of a smartphone as shown in FIG. IC may be parallel to the X axis and the short side parallel to the Y axis, with the STC aperture pointing towards the chart in Z direction. All POVs that the STC can reach are given by “POV vectors” or “camera pointing vector” P which are pointing to coordinates lying on chart 320. The coordinates of the zero state position may be (0, 0, EFLT) with EFLT being the EFL of the STC. At zero position, the coordinates of the IMU (and of the handheld device) overlap with the STC's coordinates.

If the STC is directed to a non-zero POV, a coordinate transform from the IMU's to the STC's coordinates must be performed. In some examples, Rodrigues' rotation formula may be used. The IMU's pitch/yaw/roll rotation values may be named “hnd_pitch”, “hnd_yaw” and “hnd_roll”. IMU provides hnd_pitch, hnd_yaw and hnd_roll in a coordinate system having the following unit vectors:

- Pitch unit vector R_P: R_P=(1, 0, 0),
- Yaw unit vector R_Y: R_Y=(0, 1, 0),
- Roll unit vector R_R: R_R=(0, 0, 1).

In general, OIS corrects small angles only. Therefore, in some situations and approximately, one may treat the pitch/yaw/roll rotations independently. For any (slight) rotation of the device, Rodrigues' rotation formula may be applied to pitch/yaw/roll rotations independently, wherein the (slight) rotation may be represented by the sum over the pitch/yaw/roll rotations. A hand motion only by hnd_pitch, or only by hnd_yaw or only by hnd_roll (in the IMU's coordinates R_P, R_Yand R_R) applied to any initial POV vector P^Imay result in the following final POV vector P^F(“cross (x, y)” indicates the cross product of vectors x and y, “dot (x, y)” indicates the dot product of vectors x and y):

- POV vector P^F_Pafter rotation by hnd_pitch around R_P(hnd_yaw, hnd_roll=0): P^F_P=(P^I·cos(hnd_pitch)+cross(P^I, R_P)·sin(hnd_pitch)+R_P·(dot(P^I, R_P)·(1−cos(hnd_pitch))));
- POV vector P^F_Yafter rotation by hnd_yaw around R_Y(hnd_pitch, hnd_roll=0): P^F_Y=(P^I·cos(hnd_yaw)+cross (P^I, R_Y)·sin (hnd_yaw)+R_Y·(dot (P^I, R^Y)·(1−cos (hnd_yaw)));
- POV vector P^F_Rafter rotation by hnd_roll around R_R(hnd_pitch, hnd_yaw=0): P^F_R=(P^I·cos(hnd_roll)+cross (P^I, R_R)·sin (hnd_roll)+R_R·(dot (P^I, R^R)·(1−cos (hnd_roll)))).
  
  For small angles, a final POV vector (before normalization) P^F′ that underwent both Pitch, Yaw and Roll rotations may be given by:

P

F
⁢
′

=

P
I

+

(

P
I

-

P
P
F

)

+

(

P
I

-

P
Y
F

)

+

(

P
I

-

P
R
F

)

A normalization may be performed in order to ensure that the final POV vector P^Fcomes to lie on chart 320. In some examples, P^Fmay be obtained by normalizing P^F′ with EFLT/P^F′_z, wherein P^F′_zis the z-component of P^F′, i.e.:

P
F

=

P

F
⁢
′

·

EFL
T

/

P
z

F
⁢
′

.

From the above equations it is evident that for compensating for undesired rotational hand motion in a STC, in contrast for a non-scanning camera like e.g. Wide camera