基于同态加密的机器学习 (二)

AI Studio

847人浏览 · 2022-06-03 00:05:36

AI Studio · 2022-06-03 00:05:36 发布

1 项目说明
随着计算机技术的发展和人工智能的兴起，机器学习在许多领域的应用已经取得了巨大的成功。机器学习的性能表现通常与使用的数据量成正比，因此需要收集大量的数据。在大数据背景下，用户和服务商都面临着隐私泄露问题，此外还有一些攻击者通过一些手段来获取数据进行牟利。另一方面，也要防止机器学习中模型参数的泄露。因此解决机器学习中的隐私保护问题成为当务之急。在不影响机器学习性能的前提下，本项目使用神经网络的隐私保护训练方法，以有效的同态加密方案来保护数据的隐私性。
同时为了方便对比，本项目分为两部分，项目（一）未使用同态加密方案进行神经网络训练，项目（二）使用同态加密进行神将网络训练。

2 前期准备
2.1 环境和数据
环境配置以及数据处理都已经在项目（一）中详细叙述，项目（二）主要记录同态加密方面内容以及代码。
本项目同态加密方案是采用基于共轭搜索问题( Conjugacy Search Problem, CSP)进行构造，参考了文章Privacy preservation for machine learning training and classification based on homomorphic encryption schemes，但是上述文章所提方案不具备语义安全性，因此在此基础上重新构造了CSP加密方案。

2.2 同态加密
同态加密(Homomorphic encryption)使得用户能够直接在密文上直接进行运算，得到的结果经过解密后与在明文下的计算结果一致，是一项直接有效的保护数据隐私的技术。最近十年，在机器学习中应用FE来保护数据隐私的研究发展迅速。

2.3 CSP加密方案
给定C←HE.Enck(M)C←HE.Enc_k (M)C←HE.Enc
k

(M)在一个非阿贝尔交换代数结构ΨΨΨ上，其中C=HMH−1C=HMH^{-1}C=HMH
−1
，求解H∈ΨH∈ΨH∈Ψ是一个困难问题。这样可以提高数据的安全性。

2.3.1 数据编码
M←Encoding(m)M←Encoding(m)M←Encoding(m)：将数据mmm经过编码函数转化为MMM，转化的MMM是满足a1+a2=a3+a4=a5+a6=ma_1+a_2=a_3+a_4=a_5+a_6=ma
1

+a
2

=a
3

+a
4

=a
5

+a
6

=m的三对随机数(a1,a2),(a3,a4),(a5,a6)(a_1,a_2 ),(a_3,a_4 ), (a_5,a_6 )(a
1

,a
2

),(a
3

,a
4

),(a
5

,a
6

)，因此构造如下矩阵：

M1=(a1a2a2a1)M_{1}=\left(\begin{array}{ll}a_{1} & a_{2} \ a_{2} & a_{1}\end{array}\right)
M
1

=(
a
1

a
2

a
1

)

M2=(a3a4a4a3)M_{2}=\left(\begin{array}{ll}a_{3} & a_{4} \ a_{4} & a_{3}\end{array}\right)
M
2

=(
a
3

a
4

a
3

)

M3=(a5a6a6a5)M_{3}=\left(\begin{array}{ll}a_{5} & a_{6} \ a_{6} & a_{5}\end{array}\right)
M
3

=(
a
5

a
6

a
5

)

最终将M转换为

M=(M1R1R20M2R300M3)M=\left(\begin{array}{ccc}M_{1} & R_{1} & R_{2} \ 0 & M_{2} & R_{3} \ 0 & 0 & M_{3}\end{array}\right)
M=
⎝
⎛

M
1

0
0

R
1

M
2

R
2

R
3

M
3

⎠
⎞

2.3.2 密钥生成
k←HE.KeyGen(1κ)k←HE.KeyGen(1^κ)k←HE.KeyGen(1
κ
)：设置安全参数κκκ，生成加密密钥。当密钥生成时，矩阵均匀随机地从R(6×6)R^(6×6)R
(
6×6)中采样。它也可以表示为九个2×22×22×2矩阵的组合。

P=(P1P2P3P4P5P6P7P8P9),Pi=(Pi1Pi2Pi3Pi4),(i=1,2,…,9)P=\left(\begin{array}{lll}P_{1} & P_{2} & P_{3} \ P_{4} & P_{5} & P_{6} \ P_{7} & P_{8} & P_{9}\end{array}\right), P_{i}=\left(\begin{array}{ll}P_{i 1} & P_{i 2} \ P_{i 3} & P_{i 4}\end{array}\right),(i=1,2, \ldots, 9)
P=
⎝
⎛

P
1

P
4

P
7

P
2

P
5

P
8

P
3

P
6

P
9

⎠
⎞

,P
i

=(
P
i1

P
i3

P
i2

P
i4

),(i=1,2,…,9)

2.3.3 加密
C←HE.Enck(M)C←HE.Enc_k (M)C←HE.Enc
k

(M)：根据加密密钥和加密功能，对编码后的数据进行加密，获得密文数据CCC:

C=PMP−1=P(M1R1R20M2R300M3)P−1C=P M P^{-1}=P\left(\begin{array}{ccc}M_{1} & R_{1} & R_{2} \ 0 & M_{2} & R_{3} \ 0 & 0 & M_{3}\end{array}\right) P^{-1}
C=PMP
−1
=P
⎝
⎛

M
1

0
0

R
1

M
2

R
2

R
3

M
3

⎠
⎞

P
−1

2.3.4 解密
M←HE.Dec©M←HE.Dec©M←HE.Dec©：使用密钥对编码的消息进行解密M=P−1CPM=P^{-1}CPM=P
−1
CP，因此:

M1=(a1a2a2a1)M_{1}=\left(\begin{array}{ll}a_{1} & a_{2} \ a_{2} & a_{1}\end{array}\right)
M
1

=(
a
1

a
2

a
1

)

然后计算m=a1+a2m=a_1+a_2m=a
1

+a
2

，恢复明文。

2.3 小结
本项目主要是以构建项目为主，在安全性以及同态性质方面不做过多描述，有兴趣的小伙伴约阅读上面提过的文章。

3 代码改进
3.1 CSP同态方案构造代码
编码与解码

In [ ]
def Encode_source(self,m): #源数据编码,保证a1^>a2,a3^>a4,a5^>a6
a1=Fraction(randint(1,9))
a2=Fraction(m-a1)
a3=Fraction(randint(1,9))
a4=Fraction(m-a3)
a5=Fraction(randint(1,9))
a6=Fraction(m-a5)
M=sy.Matrix([[a1, a2, randint(1,9),randint(1,9),randint(1,9),randint(1,9)],
[a2, a1, randint(1,9),randint(1,9),randint(1,9),randint(1,9)],
[0, 0, 9, 4, randint(1,9),randint(1,9)],
[0, 0, 6, 3, randint(1,9),randint(1,9)],
[0, 0, 0, 0, 9, 4],
[0, 0, 0, 0, 6, 3]])
return M

def Decode_source(self,M):   #源数据解码
    return M[0,0]+M[0,1]

加密与解密

def Encryption(self,m):  #加密
    M=self.Encode_source(m)
    C=self.P*M*self.P_Inv
    return C

def Decryption(self,C):  #解密
    M=self.P_Inv*C*self.P
    m=self.Decode_source(M)
    return m

同态加法，同态乘法

def Hom_add(self,C1,C2):    #同态加法
    return C1+C2

def Hom_mul(self,C1,C2):    #同态乘法
    return C1*C2

完整同态方案代码：

In [ ]
import sys
sys.path.append(‘/home/aistudio/external-libraries’)
import sympy as sy
from fractions import Fraction
from random import *
import time

class CSP_1:
def init(self):
self.P= sy.randMatrix(6, 6, 1, 9)
self.P_Inv= self.P**-1
self.t = sy.Matrix([[0, 1, randint(1, 5), randint(1, 5), randint(1, 5), randint(1, 5)],
[1, 0, randint(1, 5), randint(1, 5), randint(1, 5), randint(1, 5)],
[0, 0, 0, 1, randint(1, 5), randint(1, 5)],
[0, 0, 1, 0, randint(1, 5), randint(1, 5)],
[0, 0, 0, 0, 0, 1],
[0, 0, 0, 0, 1, 0]])
self.T=self.Pself.tself.P_Inv

def Encode_source(self,m):   #源数据编码,保证a1^>a2^,a3^>a4^,a5^>a6^
    a1=Fraction(randint(1,9))
    a2=Fraction(m-a1)
    a3=Fraction(randint(1,9))
    a4=Fraction(m-a3)
    a5=Fraction(randint(1,9))
    a6=Fraction(m-a5)
    M=sy.Matrix([[a1, a2, randint(1,9),randint(1,9),randint(1,9),randint(1,9)],
                 [a2, a1, randint(1,9),randint(1,9),randint(1,9),randint(1,9)],
                 [0,  0,  9,            4,            randint(1,9),randint(1,9)],
                 [0,  0, 6,            3,            randint(1,9),randint(1,9)],
                 [0,  0,  0,             0,             9,           4],
                 [0,  0,  0,             0,             6,            3]])
    return M

def Decode_source(self,M):   #源数据解码
    return M[0,0]+M[0,1]

def Encryption(self,m):  #加密
    M=self.Encode_source(m)
    C=self.P*M*self.P_Inv
    return C

def Decryption(self,C):  #解密
    M=self.P_Inv*C*self.P
    m=self.Decode_source(M)
    return m

def Hom_add(self,C1,C2):    #同态加法
    return C1+C2

def Hom_mul(self,C1,C2):    #同态乘法
    return C1*C2

if name==“main”:
csp=CSP_1()
A = csp.Encryption(3)
B = csp.Encryption(2)
C = csp.Encryption(7)

T1 = csp.Hom_add(A,B)
T2 = csp.Hom_mul(B,C)
print('A:',3)
print('B:',2)
print('C:',7)
print("输出加密信息A，B相加，进行解密后的结果（同态加法）",csp.Decryption(T1))
print("输出加密信息B，C相加，进行解密后的结果（同态乘法）",csp.Decryption(T2))

A: 3
B: 2
C: 7
输出加密信息A，B相加，进行解密后的结果（同态加法） 5
输出加密信息B，C相加，进行解密后的结果（同态乘法） 14
3.2 同态加密神经网络代码
构造了同态加密方案后，需要对原来的神经网络的函数进行重载，因为同态加法和同态乘法与明文数据的运算不一样，先导入上面的同态方案:

from CSP_Scheme import CSP
3.2.1 前向传播
In [ ]
#正向传播
l1={}
l2={}
#l1 = self.sigmoid(np.dot(self.X, self.w1))
#l2 = self.sigmoid(np.dot(l1, self.w2))
#输入层和隐藏层, X:(random,input), w1:(input,hidden); -->l1(random,hidden)
temp_l1=self.tool.dict_dot(X_format, self.w1,self.size_X_r,self.input_nodes,self.hidden_nodes)
for i in temp_l1:
l1[i] = self.sigmoid(temp_l1[i])
#隐藏层和输出层, l1(random,hidden) w2:(hidden,output),–>l2(random,output)
temp_l2=self.tool.dict_dot(l1, self.w2,self.size_X_r,self.hidden_nodes,self.output_nodes)
for j in temp_l2:
l2[j] = self.sigmoid(temp_l2[j])
3.2.2 sigmoid函数
由于同态加密对于sigmoid函数太复杂了，因此我们采用分段多项式去拟合sigmoid函数，达到相同的效果：

y={0.571859+(0.392773)x+(0.108706)x2+(0.014222)x3+(0.000734)x4−∞<x≤−1.512+14x−148x3+1480x5−1.5<x<1.50.428141+(0.392773)x−(0.108706)x2+(0.014222)x3−(0.000734)x41.5≤x<∞y= \begin{cases}0.571859+(0.392773) x+(0.108706) x^{2}+ (0.014222) x^{3}+(0.000734) x^{4} & -\infty<x \leq-1.5 \ \frac{1}{2}+\frac{1}{4} x-\frac{1}{48} x^{3}+\frac{1}{480} x^{5} & -1.5<x<1.5 \ 0.428141+(0.392773) x-(0.108706) x^{2}+ (0.014222) x^{3}-(0.000734) x^{4} & 1.5 \leq x<\infty\end{cases}
y=
⎩
⎪
⎪
⎨
⎪
⎪
⎧

0.571859+(0.392773)x+(0.108706)x
2
+(0.014222)x
3
+(0.000734)x
4

2
1

+
4
1

x−
48
1

x
3
+
480
1

x
5

0.428141+(0.392773)x−(0.108706)x
2
+(0.014222)x
3
−(0.000734)x
4

−∞<x≤−1.5
−1.5<x<1.5
1.5≤x<∞

In [ ]
def sigmoid(self,X): #(1 / (1 + np.exp(-X))),使用多项式进行替换，A*X^3
if self.csp.Cipher_compare(X, self.con_2) <= 0: #x<-1.5
ans= self.A1 + self.tool.sigmoid_piece(self.A2,X,1) +
self.tool.sigmoid_piece(self.A3, X,2) + self.tool.sigmoid_piece(self.A4, X,3) +
self.tool.sigmoid_piece(self.A5, X,4)

elif self.csp.Cipher_compare(X,self.con_1) > 0:        #x>1.5
    ans= self.C1 + self.tool.sigmoid_piece(self.C2,X,1) - \
            self.tool.sigmoid_piece(self.C3, X,2) + self.tool.sigmoid_piece(self.C4, X,3) -\
            self.tool.sigmoid_piece(self.C5, X,4)
else:
    ans= self.B1 + self.tool.sigmoid_piece(self.B2, X, 1) - \
            self.tool.sigmoid_piece(self.B3, X, 3) + self.tool.sigmoid_piece(self.B4, X, 5)
if self.csp.Cipher_compare(ans,self.con_4) >= 0:
    ans=self.csp.Encryption(1)
if self.csp.Cipher_compare(ans,self.con_6) <= 0:
    ans=self.csp.Encryption(-1)
return ans

def sigmoid_prime(self, X): #sigmoid方程求导,X * (1 - X),此时参数X为加密矩阵
temp = self.csp.Hom_sub(self.sig_1, X)
return self.csp.Hom_mul(X,temp)
3.2.3 后向传播
In [ ]
error={}
for i in Y_format:
error[i] = self.csp.Hom_sub(Y_format[i],l2[i]) #索引问题

l2_delta = error * self.sigmoid_prime(l2)

l1_delta = l2_delta.dot(self.w2.T) * self.sigmoid_prime(l1),字典之间的运算

l2_delta={}
sig_l2={} #self.sigmoid_prime(l2)应该设置为字典
for i in l2: #l2(random,output)
sig_l2[i]=self.sigmoid_prime(l2[i]) #error和sig_l2具有一样的索引#
for i in error:
l2_delta[i] = self.csp.Hom_mul(error[i],sig_l2[i])

#l2_delta(random,output),W2.T(output,hidden);l1(random,hideen)
l1_delta={}
sig_l1={}
for i in l1:
sig_l1[i]=self.sigmoid_prime(l1[i])
w2_T=self.tool.dict_T(self.w2, self.hidden_nodes, self.output_nodes)
l1_delta_temp=self.tool.dict_dot(l2_delta, w2_T, self.size_X_r, self.output_nodes, self.hidden_nodes)
for i in l1_delta_temp:
l1_delta[i] = self.csp.Hom_mul(l1_delta_temp[i], sig_l1[i])
3.2.4 权值更新
In [ ]

self.w2 = np.add(self.w2, l1.T.dot(l2_delta) * self.learning_rate)

self.w1 = np.add(self.w1, self.X.T.dot(l1_delta) * self.learning_rate)

w2_update=self.tool.dict_dot(self.tool.dict_T(l1,self.size_X_r,self.hidden_nodes), l2_delta, self.hidden_nodes, self.size_X_r, self.output_nodes)
for i in w2_update:
w2_update[i]=w2_update[i]*self.learning_rate
self.w2 = self.tool.dict_add(self.w2, w2_update, self.hidden_nodes, self.output_nodes)
w1_update=self.tool.dict_dot(self.tool.dict_T(X_format,self.size_X_r,self.input_nodes),l1_delta ,self.input_nodes,self.size_X_r,self.hidden_nodes)
for i in w1_update:
w1_update[i]=w1_update[i]*self.learning_rate
self.w1 = self.tool.dict_add(self.w1, w1_update,self.input_nodes,self.hidden_nodes)
3.2.5 预测
将训练好的，模型进行预测

In [ ]
correct = 0
pred_list = []
X_format,Y_format = self.tool.data_En(self.X_test, self.Y_test,1)#参数为1时，对y_编码后使用分数表示
l1 = {} #l1 = self.sigmoid(np.dot(self.X_test, self.w1))
l2 = {} #l2 = self.sigmoid(np.dot(l1, self.w2))

输入层和隐藏层, X:(random,input), w1:(input,hidden); -->l1(random,hidden)

temp_l1 = self.tool.dict_dot(X_format, self.w1, self.size_Xt_r, self.input_nodes,self.hidden_nodes)
for i in temp_l1:
l1[i] = self.sigmoid(temp_l1[i])

隐藏层和输出层, l1(random,hidden) w2:(hidden,output),–>l2(random,output)

temp_l2 = self.tool.dict_dot(l1, self.w2, self.size_Xt_r, self.hidden_nodes,self.output_nodes)
for j in temp_l2:
l2[j] = self.sigmoid(temp_l2[j])

for i in l2: #比较进行加密
if self.csp.Cipher_compare(l2[i],self.con_3)>=0: # l2[i] >= 0.5
pred = self.con_4 # pred = 1
pred_lab=1
else:
pred = self.con_5 # pred = 0
pred_lab=0
pred_list.append(pred_lab)
#精准比较
if self.csp.Cipher_compare(Y_format[i],pred,1)==0: # pred == self.Y_test[i],
correct += 1
print('Test Accuracy : ', ((correct / self.size_Yt_r) * 100), ‘%’)
print(“真实值”, self.Y_test.reshape(1,self.size_Yt_r))
print(“预测值”, pred_list)
并且将结果绘制出来在这里插入图片描述
4 总结
项目（二）实现了同态加密的神经网络学习，用的是公开数据集，其中也有很多地方未来得及优化。
同态操作复杂度高，十分耗时间，因此项目（二）的训练时间大概在20h左右。下面附上完整的代码

In [ ]
import sys
sys.path.append(‘/home/aistudio/external-libraries’)
import numpy as np
import seaborn as sns
import time
import matplotlib.pyplot as plt
import pylab
import sympy as sy
from sklearn import datasets
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import confusion_matrix
from sklearn.metrics import precision_recall_fscore_support
from My_Tool import Tool
from CSP_Scheme import CSP_1

#导入数据
cancer = datasets.load_breast_cancer()
X = cancer.data
Y = cancer.target

#随机划分为训练子集和测试子集
X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.2)

#按行展开
Y_train = Y_train[:, None]
Y_test = Y_test[:, None]
#数据归一化
sc = StandardScaler()
X_train = sc.fit_transform(X_train)
X_test = sc.fit_transform(X_test)

#建立神经网络
class NeuralNetwork:
def init(self, X, Y, X_test, Y_test, hidden_nodes=5, learninig_rate=0.1, epochs=5):
self.csp=CSP_1()
self.tool = Tool(self.csp)
self.X = X
self.Y = Y #按行展开
self.X_test = X_test
self.Y_test = Y_test

    self.size_X_r=X.shape[0]   #训练集的行数,即random
    self.size_Y_r = Y.shape[0]
    self.size_Xt_r=X_test.shape[0]
    self.size_Yt_r=Y_test.shape[0]

    self.input_nodes = self.X.shape[1]   #输入层节点，30；训练数据中特征的数量
    self.hidden_nodes = hidden_nodes     #隐藏层节点，5
    self.output_nodes = self.Y.shape[1]  #输出层节点，1
    self.learning_rate = learninig_rate  #学习率0.1

    #sigmoid函数近似替代
    self.con_1 = self.csp.Encryption_target(1.5)  #sigmoid函数的分段点,加密目标函数，注意修复没有
    self.con_2 = self.csp.Encryption_target(-1.5)
    self.con_3 = self.csp.Encryption_target(0.5)
    self.con_4 = self.csp.Encryption_target(1)
    self.con_5 = self.csp.Encryption_target(0)
    self.con_6 = self.csp.Encryption_target(-1)

    self.sig_1=self.csp.Encryption(1)
    self.A1 = self.csp.Encryption(0.571859)
    self.A2 = 0.392773
    self.A3 = 0.108706
    self.A4 = 0.014222
    self.A5 = 0.000734

    self.B1 = self.csp.Encryption(0.5)
    self.B2 = 0.25
    self.B3 = 0.020833
    self.B4 = 0.002083

    self.C1 = self.csp.Encryption(0.428141)
    self.C2 = 0.392773
    self.C3 = 0.108706
    self.C4 = 0.014222
    self.C5 = 0.000734

    #初始化网络权值
    self.w1={}
    self.w2={}

    #初始化两个权重矩阵,进行加密
    r_w1=np.random.random((self.input_nodes, self.hidden_nodes))  #随机矩阵
    r_w2=np.random.random((self.hidden_nodes, self.output_nodes))  #随机矩阵

    w1_temp =self.tool.dict_toformat(r_w1)           #加上索引
    w2_temp =self.tool.dict_toformat(r_w2)           #同上

    for i in w1_temp:
        self.w1[i]=self.csp.Encryption(w1_temp[i])           #将权重函数进行操作，
    for i in w2_temp:
        self.w2[i]=self.csp.Encryption(w2_temp[i])

    self.train(epochs)  # 模型进行多次训练
    self.test()

#定义激活函数 ,参数X是键值对，是否需要修改，注意sigmoid_price是把X当成加密矩阵，考虑是否循环
def sigmoid(self,X): #(1 / (1 + np.exp(-X))),使用多项式进行替换，A*X^3
    if self.csp.Cipher_compare(X, self.con_2) <= 0:          #x<-1.5
        ans= self.A1 + self.tool.sigmoid_piece(self.A2,X,1) + \
             self.tool.sigmoid_piece(self.A3, X,2) + self.tool.sigmoid_piece(self.A4, X,3) +\
             self.tool.sigmoid_piece(self.A5, X,4)

    elif self.csp.Cipher_compare(X,self.con_1) > 0:        #x>1.5
        ans= self.C1 + self.tool.sigmoid_piece(self.C2,X,1) - \
             self.tool.sigmoid_piece(self.C3, X,2) + self.tool.sigmoid_piece(self.C4, X,3) -\
             self.tool.sigmoid_piece(self.C5, X,4)
    else:
        ans= self.B1 + self.tool.sigmoid_piece(self.B2, X, 1) - \
             self.tool.sigmoid_piece(self.B3, X, 3) + self.tool.sigmoid_piece(self.B4, X, 5)
    if self.csp.Cipher_compare(ans,self.con_4) >= 0:
        ans=self.csp.Encryption(1)
    if self.csp.Cipher_compare(ans,self.con_6) <= 0:
        ans=self.csp.Encryption(-1)
    return ans

def sigmoid_prime(self, X):  #sigmoid方程求导,X * (1 - X),此时参数X为加密矩阵   X如果为1，？？
    temp = self.csp.Hom_sub(self.sig_1, X)
    return self.csp.Hom_mul(X,temp)

def train(self, epochs):
    print('Train')
    X_format,Y_format=self.tool.data_En(self.X,self.Y)
    for e in range(epochs):
        print("\t===== epochs %d =====\t" % (e+1))
        #正向传播
        l1={}
        l2={}
        #l1 = self.sigmoid(np.dot(self.X, self.w1))
        #l2 = self.sigmoid(np.dot(l1, self.w2))
        temp_l1=self.tool.dict_dot(X_format, self.w1,self.size_X_r,self.input_nodes,self.hidden_nodes) #输入层和隐藏层, X:(random,input), w1:(input,hidden); -->l1(random,hidden)
        for i in temp_l1:
            l1[i] = self.sigmoid(temp_l1[i])
        temp_l2=self.tool.dict_dot(l1, self.w2,self.size_X_r,self.hidden_nodes,self.output_nodes)       #隐藏层和输出层, l1(random,hidden) w2:(hidden,output),-->l2(random,output)
        for j in temp_l2:
            l2[j] = self.sigmoid(temp_l2[j])

        #反向传播,误差(真实值-预测值)
        # 两加密后的字典，进行减法.,error = Y_format - l2;;;  erroe(519,1)
        error={}
        for i in Y_format:
            error[i] = self.csp.Hom_sub(Y_format[i],l2[i])   #索引问题
        #print('01:',error['0_0'])
        # l2_delta = error * self.sigmoid_prime(l2)
        # l1_delta = l2_delta.dot(self.w2.T) * self.sigmoid_prime(l1),,字典之间的运算
        l2_delta={}
        sig_l2={}         #self.sigmoid_prime(l2)应该设置为字典
        for i in l2:      #l2(random,output)
            sig_l2[i]=self.sigmoid_prime(l2[i])   #error和sig_l2具有一样的索引#
        #print(sig_l2)
        for i in error:
            l2_delta[i] = self.csp.Hom_mul(error[i],sig_l2[i])
            # print("*****************")
            print('02:',self.csp.Decryption(error[i]))
            # print('03:',self.csp.Decryption(sig_l2[i]))
            # print('04:',self.csp.Decryption(l2_delta[i]))

        #l2_delta(random,output),W2.T(output,hidden);l1(random,hideen)
        l1_delta={}
        sig_l1={}
        for i in l1:
            sig_l1[i]=self.sigmoid_prime(l1[i])
        w2_T=self.tool.dict_T(self.w2, self.hidden_nodes, self.output_nodes)
        l1_delta_temp=self.tool.dict_dot(l2_delta, w2_T, self.size_X_r, self.output_nodes, self.hidden_nodes)
        for i in l1_delta_temp:
            l1_delta[i] = self.csp.Hom_mul(l1_delta_temp[i], sig_l1[i])

        # self.w2 = np.add(self.w2, l1.T.dot(l2_delta) * self.learning_rate)
        # self.w1 = np.add(self.w1, self.X.T.dot(l1_delta) * self.learning_rate)
        w2_update=self.tool.dict_dot(self.tool.dict_T(l1,self.size_X_r,self.hidden_nodes), l2_delta, self.hidden_nodes, self.size_X_r, self.output_nodes)
        for i in w2_update:
            w2_update[i]=w2_update[i]*self.learning_rate
        self.w2 = self.tool.dict_add(self.w2, w2_update, self.hidden_nodes, self.output_nodes)

        w1_update=self.tool.dict_dot(self.tool.dict_T(X_format,self.size_X_r,self.input_nodes),l1_delta ,self.input_nodes,self.size_X_r,self.hidden_nodes)
        for i in w1_update:
            w1_update[i]=w1_update[i]*self.learning_rate
        self.w1 = self.tool.dict_add(self.w1, w1_update,self.input_nodes,self.hidden_nodes)


def test(self):
    print('Test')
    correct = 0
    pred_list = []

    X_format,Y_format = self.tool.data_En(self.X_test, self.Y_test,1)#参数为1时，对y_编码后使用分数表示
    l1 = {}    #l1 = self.sigmoid(np.dot(self.X_test, self.w1))
    l2 = {}    #l2 = self.sigmoid(np.dot(l1, self.w2))
    temp_l1 = self.tool.dict_dot(X_format, self.w1, self.size_Xt_r, self.input_nodes,self.hidden_nodes)  # 输入层和隐藏层, X:(random,input), w1:(input,hidden); -->l1(random,hidden)
    for i in temp_l1:
        l1[i] = self.sigmoid(temp_l1[i])

    temp_l2 = self.tool.dict_dot(l1, self.w2, self.size_Xt_r, self.hidden_nodes,self.output_nodes)  # 隐藏层和输出层, l1(random,hidden) w2:(hidden,output),-->l2(random,output)
    for j in temp_l2:
        l2[j] = self.sigmoid(temp_l2[j])

    for i in l2:    #比较进行加密
        if self.csp.Cipher_compare(l2[i],self.con_3)>=0:     # l2[i] >= 0.5
            pred = self.con_4                                # pred = 1
            pred_lab=1
        else:
            pred = self.con_5                                # pred = 0
            pred_lab=0
        pred_list.append(pred_lab)
            #精准比较
        if self.csp.Cipher_compare(Y_format[i],pred,1)==0:  # pred == self.Y_test[i],
            correct += 1
        #     true_list.append(pred_lab)
        # else:
        #     true_list.append(1-pred_lab)
        #pred_list.append(pred_lab)
    print('Test Accuracy : ', ((correct / self.size_Yt_r) * 100), '%')
    print("真实值", self.Y_test.reshape(1,self.size_Yt_r))
    print("预测值", pred_list)

    # 混淆矩阵
    #FN 00 预测为N,标签为P
    #FP 01 预测为P,标签为N
    #TN 10 预测为N,标签为N
    #FN 11 预测为P,标签为P
    # cm = confusion_matrix(Y_test, pred_list)
    # sns.heatmap(cm, annot=True)
    # plt.savefig('h.png')
    # plt.show()

#print(Y_test[0:20,:].reshape(1,20))
#nn = NeuralNetwork(X_train[0:10,:], Y_train[0:10,:], X_test[0:20,:], Y_test[0:20,:], 5, 0.1, 5)
nn = NeuralNetwork(X_train, Y_train, X_test, Y_test, 5, 0.1, 5)