x
6

6

(
2
)

In an embodiment, the approximated exponentiation function e^xis a Maclaurin series expansion of the corresponding exponentiation function as given in equation (3).

e
x

=

1
+
x
+

x
2

2

+

x
3

6

+

x
4

2
⁢
4

+

x
5

1
⁢
2
⁢
0

(
3
)

In an embodiment, the approximated division by encrypted value function is obtained by generating an inverse of a denominator value and further multiplying by a numerator value, wherein the inverse is obtained using a Newton iterative method. The Newton iterative method is used for generating approximated values for real valued functions.

In an embodiment, the approximated softmax function e^xⁱ/Σ_i=1ⁿe^xⁱis obtained by initially computing the approximated exponentiation for e^xusing a corresponding Maclaurin series expansion. Further, an exponential sum Σ_x=1ⁿe^xis computed by adding of a plurality of exponentials. Further, an inverse of the exponential sum is computed using the Newton iterate method. Finally, the approximated softmax function is computed by multiplying the approximated exponentiation and the computed inverse. The approximated softmax function is given in equation (4).

approximated softmax=approx(e^x)*newton_iterate(Σ_i=1ⁿe^xⁱ) (4)

In an embodiment, the approximated sign(x) function is computed by as follows. Initially, a maximum value between ‘x’ and ‘0’ is obtained using max(x, 0) function, wherein ‘x’ is a non-zero element. Further, an inverse of x is computed using the Newton iterate method. Finally, the approximated sign(x) function is computed by multiplying the obtained maximum value and the computed inverse of x. The approximated sign(x) function is given in equation (5). Now referring to equation (5), max(x, 0) is the ReLU function on input ‘x’. Hence max(x, 0) can be computed using ReLU_approx polynomial approximation.

sign(x)=max(x,0)*newton_iterate_inverse(x)) (5)

In an embodiment, the approximated abs(x) function given in equation (6) is obtained using a corresponding ReLU approximation of ‘x’. In an embodiment, the ReLU approximation is obtained using the pseudocode 1 given in Table I. Now referring to Pseudocode 1, ‘a’, ‘b’ and ‘c’ are coefficients of the corresponding polynomial used for approximation. The resultant value or the approximated ReLU value is obtained using the expression a+b*x+c*x².

abs(x)=ReLU_approx(x)+ReLU_approx(x*−1) (6)

TABLE I

Pseudocode 1: ReLU_approx(x):

1.
x = x/7

2.
a = 0.125

3.
b = 0.5

4.
c = 0.25

5.
res = a + b*x + c*x²

6.
return res*7

In an embodiment, the approximated minimum function associated with a first value ‘x’ and a second value ‘y’ is obtained as follows. Initially, a corresponding ReLU approximation is computed based on a first difference (x−y) and a second difference (y−x). Further, a ReLU approximation sum value is computed based on the first difference (x−y) and the second difference (y−x). Further, an inverse of the ReLU approximation sum value is generated using the Newton iterate method. After that, a first coefficient and a second coefficient is computed based on the first difference and the second difference. Finally, the minimum approximation function corresponding to the first value and the second value is obtained by adding a first multiplicated value and a second multiplicative value. The first multiplicative value is obtained based on the first coefficient and the first value. The second multiplicative value is obtained based on the second coefficient and the second value. In an embodiment, the pseudocode for generating the approximated minimum/maximum function is given in Table II and an example computation of minimum value using approximated maximum value is given in Table III.

TABLE II

Pseudocode 2: Approximated minimum/maximum function

1.
z1 = ReLU_approx(y − x),

2.
z2 = ReLU_approx(x − y)

3.
z = z1 + z2

4.
z′ = newton_iterate_inverse(z)

5.
a = z1*z′; b = z2*z′

6.
min = a*x + b*y for min // max = b*x + a*y for max

TABLE III

Example: Approximated minimum function

1.
min_approx(x, y):

2.
min_approx(5, 2):

3.
z1 = relu_approx(y − x) = 0, z2 = relu_approx(x − y) = 3

4.
z = z1 + z2 = 3

5.
z′ = newton_iterate_inverse(z) = ⅓

6.
a = z1*z′ = 0; b = z2*z′ = 1

7.
min = a*x + b*y = 0*5 + 1*2 = 2

In an embodiment, the maximum approximation function is obtained by swapping the coefficients ‘a’ and ‘b’ associated with the minimum approximation function. For example, a max computation is given in Table IV. Now referring to Table IV, it is evident that the maximum value is obtained by swapping the coefficients at step 7

TABLE IV

Example: Approximated maximum function

1.
max_approx(x, y):

2.
max_approx(5, 2):

3.
z1 = relu_approx(y − x) = 0, z2 = relu_approx(x − y) = 3

4.
z = z1 + z2 = 3

5.
z′ = newton_iterate_inverse(z) = ⅓

6.
a = z1*z′ = 0; b = z2*z′ = 1

7.
max = b*x + a*y = 1*5 + 0*2 = 5

In an embodiment, the approximated line search function computes a loss function corresponding to each of a plurality of step sizes and selects a step size with minimum loss. The pseudocode 3 for line search function and the corresponding the approximated line search function (pseudocode 4) is given in Table V.

TABLE V

Pseudocode 4: Approximated

Pseudocode 3: Line search function
line search function

FUNCTION exact_line_search(fun, x,
line_search_approx( )

p):
Initialize min = losses[0]

FUNCTION fx(alpha):
min_alpha = alpha_list[0]

return fun(x + alpha*p)
For i = [1, ...

alpha_list = [list of Alpha
length(losses)−1]:

values]
i.
min, a, b =

losses = [Empty list]

min_approx(min,

i = 0

losses[i])

FOR 0 TO length(alpha_list):
ii.
min_alpha = a *

losses =

min_alpha + b *

append_this_to_list(fx(alpha_list[i]))

alpha_list[i]

i = i + 1
return min_alpha

index =
Example: Approximated line

index_of_min_loss(losses)
search function

return alpha_list[index]
let losses be : 1 , −1, 3

ENDFUNCTION
alpha_list be: 2, 5, 6

min element in losses is −1, index

1

line search needs to return

alpha_list[1] = 5.

we initialize min=losses[0] = 1

min_aplha = alpha_list[0] = 2

for i = 1 :

min, a, b =

min_approx(min, losses[i]) =

min_approx(1,−1)

min, a, b = −1, 0, 1

min_alpha = a*min_alpha

+ b*alpha_list[i] = 0*2 + 1*5 = 5

for i=2 :

min, a, b =

min_approx(min, losses[i]) =

min_approx(−1, 3)

min, a, b = −1, 1, 0

min_alpha = a*mn_alpha +

b*alpha_list[i] = 1*5 + 0*6 = 5

Now referring to Table V, the main challenge is to compute the index_of_min_loss (Pseudocode 3). Losses are the list of loss values computed in the for loop of pseudocode 3. The corresponding approximated version is given in pseudocode 4 of Table V. As the values are encrypted and hence it is challenging to identify the minimum value. Now referring to pseudocode 4, it is required to find the index of minimum value in the losses list, and then return alpha_list[index]. Here, the min_approx function is modified in such a way that it not only returns min(x,y) but also the coefficients a, b. These coefficients are further used to compute the required (minimum) element from the alpha list. In an embodiment, the approximated scalar function is obtained based on the approximated sign(x) function and the approximated abs(x) function. In an embodiment, the approximated scalar function is generated using the pseudocode 5 given in Table V. Now referring to Table VI, the epsilon and bound values are initialized and passed into Scalar Function class. The Scalar Function class includes two main functionalities to provide for BFGS algorithm including a “Generating scalar function for a given x0−fun(X0)” and a “Returns approx. derivative for the updated scalar function−grad(X0)”. In approx_derivative function Jacobian matrix is generated by calculating an absolute step from a relative step. In line 20 sign approximation and max approximation are used.

TABLE VI

Pseudocode 5: Approximated scalar function

1.
FUNCTION _prepare_scalar_function_approx(fun, x0):

2.
epsilon = eps

3.
bounds = (−infinity, +infinity) //make sure input is in the

bounds

4.
sf = ScalarFunction(fun, x0, bounds) //evaluate the

function

5.
return sf

6.
ENDFUNCTION

7.
CLASS ScalarFunction:

8.
FUNCTION fun(x0):

9.
return updated_function //evaluate the function after

every update

10.
ENDFUNCTION

11.

12.
FUNCTION grad(x0):

13.
return approx_derivative(updated_function, x0) // get

first order derivative to properly update in the next step

14.
ENDFUNCTION

15.
ENDCLASS

16.

17.
FUNCTION approx_derivative(fun, x0):

18.
rel_step = EPS**0.5

19.
f0 = [fun(x0)] //evaluate the current function value

20.
absolute_step (h) = rel_step * sign_x0 * max(1,

abs(x0)) //absolute value approximation

21.
x = x0 + h[0]

22.
dx = x[0] − x0[0]

23.
df = fun(x) − f0

24.
J_transposed[0] = df * newton_iterate(dx) //completes

the first order derivative computation

25.
return J_transposed

26.
ENDFUNCTION

At step 208 of the method 200, the ECE computation module 306 executed by the one or more hardware processors 102 is configured by the programmed instructions to compute the ECE value for the ML model based on a generated plurality of approximated functions using an approximated Broyden-Fletcher-Goldfarb-Shanno (BFGS) technique. The BFGS is a common technique used for computing the ECE value. However, generating an approximated BFGS capable of evaluating an encrypted ML model is a challenging task which is achieved by the present disclosure. In an embodiment, the approximated BFGS technique is obtained by replacing each of the plurality of evaluation functions associated with the BFGS with the corresponding approximated functions.

In an embodiment, the pseudocode 6 for the approximated BFGS is given in Table VII. Now referring to pseudocode 6, BFGS algorithm is used for Minimization of scalar function of one or more variables. It is obtained from the family of newton's method for second order optimization. In BFGS, both first order and second order approximations are leveraged which gives the information on decent direction (pk) and step size (alpha_k) respectively. Further, Hessian matrix (Hk) containing the second order information is iteratively calculated. Here, x(k) value is replaced with x(k+1) given that the new point x(k+1) is an optimal solution.

TABLE VII

Pseudocode 6: Approximated BFGS function

1.
FUNCTION minimize_bfgs( log_loss, x0)

2.
sf = prepare_scalar_function( log_loss, x0)

3.
f, fprime = sf.fun, sf.grad //returns

wrapped logloss function (sf.fun) and first order derivative

of logloss (sf.grad) respectively

4.
old_fval, gfk = f(x0), fprime(x0)

5.
k, N, I = 0, length(x0), Identity_matrix(N x N)

6.
Hk = I

7.
xk = x0

8.
WHILE k < maxiter

9.
pk = −Hk * gfk //pk gives us the direction to travel

along the slope

10.
alpha_k = exact_line_search(f, xk, pk)

11.
xkp1 = xk + alpha_k * pk

12.
gfkp1 = fprime(xkp1)

13.
sk = xkp1 − xk

14.
xk = xkp1

15.
IF gfkp1 = None

16.
THEN gfkp1 = myfprime(xkp1)

17.
ENDIF

18.
yk, gfk= gfkp1 − gfk, gfkp1

19.
k = k + 1

20.
rhok_inv = yk * sk

21.
rhok = newton_iterate(rhok_inv)

22.
A1 = I − sk[:] * yk[:] * rhok

23.
A2 = I − yk[:] * sk[:] * rhok

24.
Hk = A1 * ( Hk * A2) + (rhok * sk[:] * sk[:])

25.
ENDWHILE

26.
return xk

27.
ENDFUNCTION

28.
FUNCTION exact_line_search(wrapped_logloss, x, p):

29.
alpha_list = [list of Alpha values]

30.
losses = [Empty list]

31.
i = 0

32.
FOR 0 TO length(alpha_list):

33.
losses = append_this_to_list(wrapped_logloss(x +

alpha_list[i] * p))

34.
i = i + 1

35.
index = index_of_min_loss(losses)

36.
return alpha_list[index]

37.
ENDFUNCTION

38.

39.
FUNCTION _prepare_scalar_function_approx(log_loss,

x0):

40.
epsilon = eps

41.
bounds = (−infinite, +infinite) //make sure input is in the

bounds

42.
sf = ScalarFunction(log_loss, x0, bounds) //evaluate

the function

43.
return sf

44.
ENDFUNCTION

45.

46.
CLASS ScalarFunction:

47.
FUNCTION fun(x0):

48.
return wrapped_logloss //wrappes our log_loss

function

49.
ENDFUNCTION

50.

51.
FUNCTION grad(x0):

52.
return approx_derivative(wrapped_logloss, x0) // get

first order derivative to properly update in the next step

53.
ENDFUNCTION

54.
ENDCLASS

55.