# Logistic Regression as a Neural Network

## Logistic Regression Recap

### Computation Graph

• $u = bc$
• $v = a+u$
• $J = 3v$

• $\frac{dJ}{dv} = 3$
• $\frac{dJ}{du} = \frac{dJ}{dv} \times \frac{dv}{du} = 3 \times 1 = 3$
• $\frac{dJ}{da} = \frac{dJ}{dv} \times \frac{dv}{da} = 3 \times 1 = 3$
• $\frac{dJ}{db} = \frac{dJ}{dv} \times \frac{dv}{du} \times \frac{du}{db} = 3 \times 1 \times c = 3c$
• $\frac{dJ}{dc} = \frac{dJ}{dv} \times \frac{dv}{du} \times \frac{du}{dc} = 3 \times 1 \times b = 3b$

### Loss Function

• $da = \frac {dL(a,y)} {da} = - \frac{y}{a} + \frac{1-y}{1-a}$
• $dz = \frac {dL(a,y)} {dz} = \frac {dL(a,y)}{da} \times \frac {da}{dz} = a-y$
• $dw_1 = \frac {dL(a,y)} {dw_1} = x_1 \times dz$
• $dw_2 = \frac {dL(a,y)} {dw_2} = x_2 \times dz$

### Cost Function

J=0, dw1=0, dw2=0, db=0
for i=1 to m
z[i] = w.tx[i] + b
a[i] = sigmoid(z[i])
J += -y[i]*log(a[i]) + (1-y[i])log(1-a[i])
dz[i] = a[i] - y[i]
for j=1 to n
dw[j] += x[i][j] * dz[i] #第i组样本的第j个feature
db += dz[i]
# if n = 2
#dw1 += x[i][1] * dz[i]
#dw2 += x[i][2] * dz[i]
dw1 = dw1 / m
dw2 = dw2 / m
db  = db / m

w1 = w1 - alpha*dw1
w2 = w2 - alpha*dw2
b  = b - alpha*db


## 附录：Vectorization using Numpy

#for loop
z = 0
for i in range(1,n):
z += w[i] * x[i]
z+=b

# use numpy
# vectorized version of doing
# matrix multiplications
z = np.dot(w.T,x)+b


numpy数组的另一个特点是可以做element-wise的矩阵运算，这样让我们避开了for循环的使用

a = np.ones([1,2])  #[1,1]
a = a*2 #[2,2]


J=0,db=0
dw = np.zeros([n,1])
for i=1 to m
#z是1xm的
#x是nxm的
z[i] = w.tx[i] + b
a[i] = sigmoid(z[i])
J += -y[i]*log(a[i]) + (1-y[i])log(1-a[i])
dz[i] = a[i] - y[i]
# for j=1 to n
# dw[j] += x[i][j] * dz[i] #第i组样本的第j个feature
dw += x[i]*dz[i]
db += dz[i]
dw = dw/m


Z = np.dot(w.T,X) + b #b is a 1x1 number